[Hershey]

I have currently been working on my 2D simulations and my parallel UDF work quite well. Are there any compiler directives that need to be included when working in 3D?

I am only using two DEFINE_PROPERTY UDFs and one DEFINE_ADJUST UDF.

Thank you.

[blackmask]

There would be differences between 2D and 3D simulations, e.g., the area for 2d axisymmetric and 3d might differ. But as far as I am concerned, there is no parallel-specific difference.

[Hershey]

Thank you. Yes, my UDF are not specific to the actual dimension of the problem, just the boundary threads.

I seem to recall seeing a compiler directive #if RP_3D at some point and was not sure what it would be for. I guess your statement in dimensional dependent UDFs might explain its use.

[kornetka]

Hi
I have problem with different results for UDF computed in serial and parallel modes. I use parallel mode in order to utilize multicore processor on a single machine, if it makes any difference.
Attached mesh picture is 2D section of the middle plane of the canal - the case is 3D. Two other pictures presents distribution of the same UDM quantity in this middle plane. Both serial and parallel cases were computed with identical starting conditions, compiled UDF file is also exactly the same.
I have no idea why it is happening and how to deal with it. Can somebody explain it? Should I add some more information? Thanks in advance.
Regards,
kornetka

[Hershey]

Originally Posted by kornetka

Hi
I have problem with different results for UDF computed in serial and parallel modes. I use parallel mode in order to utilize multicore processor on a single machine, if it makes any difference.
Attached mesh picture is 2D section of the middle plane of the canal - the case is 3D. Two other pictures presents distribution of the same UDM quantity in this middle plane. Both serial and parallel cases were computed with identical starting conditions, compiled UDF file is also exactly the same.
I have no idea why it is happening and how to deal with it. Can somebody explain it? Should I add some more information? Thanks in advance.
Regards,
kornetka

You may want to post your UDF to the thread. The problem may be that you are not properly passing information between the cores if you are saying that the UDF is identical in parallel as it is in serial.

I use a UDF that does the same calculation in parallel, but it is parallelized to do so. Have you done this with your UDF?

[kornetka]

Thank you for your answer. As a matter of fact, up to now I had no idea about any parallelization so my UDF is probably not adjusted at all.
I think I'll read some manual and try to fix it myself for now. I'll come back here if (when ) I face some more troubles.
Regards,
kornetka

[kornetka]

Hi, unfortunately I'm back.
I've read the part of the UDF manual about parallelization, but I wasn't able to locate solution to my problem. As you can see, my UDF loops over a cell fluid cell thread and inside every cell does a loop over all faces. In the face loop it is supposed to take values from the adjacent cells (dudux1, ...) and perform simple operation (SRT1, Sface - still inside the face loop). The result of the face loop is tensor DS, but it's broken - the distribution of DS[0][0] is attached to my previous post, other elements of the DS tensor have the same problem.
I suppose the fishy part is the taking values from the adjacent cells? As stated earlier, due to the type of the tasks done by this code I'm not able see the problem in the parallelization itself, though the problem clearly exists. What am I overlooking? Or could it be some problem with the hardwawre/firewall/etc?
Regards, kornetka

DEFINE_ADJUST(adj_turb_var, domain) 
{
    int ii,jj,kk,n;
    real dudx,dudy,dudz,dvdx,dvdy,dvdz,dwdx,dwdy,dwdz;    
    real dudx1,dudy1,dudz1,dvdx1,dvdy1,dvdz1,dwdx1,dwdy1,dwdz1;

    real SRT[3][3], SRT1[3][3], Sface[3][3];
    real DS[3][3];


    #if !RP_Host
    face_t f=-1;
    cell_t c,c1=-1;
    Thread *t,*t1=NULL,*tf=NULL;

/*    ID=2 is fluid zone    */
    t = Lookup_Thread(domain, 2);


    thread_loop_c (t, domain) 
        begin_c_loop(c,t)
        {
/*    Strain rate tensor in cell centre    */
            dudx = C_U_G(c,t)[0]; 
            dudy = C_U_G(c,t)[1];
            dudz = C_U_G(c,t)[2];
            dvdx = C_V_G(c,t)[0];
            dvdy = C_V_G(c,t)[1];
            dvdz = C_V_G(c,t)[2];
            dwdx = C_W_G(c,t)[0];
            dwdy = C_W_G(c,t)[1];
            dwdz = C_W_G(c,t)[2];

            SRT[0][0] = 0.5*(dudx + dudx);
            SRT[0][1] = 0.5*(dudy + dvdx);
            SRT[0][2] = 0.5*(dudz + dwdx);

            SRT[1][0] = 0.5*(dvdx + dudy);
            SRT[1][1] = 0.5*(dvdy + dvdy); 
            SRT[1][2] = 0.5*(dvdz + dwdy);

            SRT[2][0] = 0.5*(dwdx + dudz);
            SRT[2][1] = 0.5*(dwdy + dvdz);
            SRT[2][2] = 0.5*(dwdz + dwdz);

            c_face_loop(c,t,n)
            {
                f = C_FACE(c,t,n);
                tf = C_FACE_THREAD(c,t,n);

/*    Strain rate tensor in adjacent cell, if it exists    */
                if PRINCIPAL_FACE_P(f,tf)
                {
                    if (!BOUNDARY_FACE_THREAD_P(tf))
                    {
/* Get cell on other side of face */
                        c1 = F_C1(f,tf); 
                        t1 = F_C1_THREAD(f,tf);

                        dudx1 = C_U_G(c1,t1)[0];
                        dudy1 = C_U_G(c1,t1)[1];
                        dudz1 = C_U_G(c1,t1)[2];
                        dvdx1 = C_V_G(c1,t1)[0];
                        dvdy1 = C_V_G(c1,t1)[1];
                        dvdz1 = C_V_G(c1,t1)[2];
                        dwdx1 = C_W_G(c1,t1)[0];
                        dwdy1 = C_W_G(c1,t1)[1];
                        dwdz1 = C_W_G(c1,t1)[2];
  
                        SRT1[0][0] = 0.5*(dudx1 + dudx1);
                        SRT1[0][1] = 0.5*(dudy1 + dvdx1);
                        SRT1[0][2] = 0.5*(dudz1 + dwdx1);

                        SRT1[1][0] = 0.5*(dvdx1 + dudy1);
                        SRT1[1][1] = 0.5*(dvdy1 + dvdy1); 
                        SRT1[1][2] = 0.5*(dvdz1 + dwdy1);

                        SRT1[2][0] = 0.5*(dwdx1 + dudz1);
                        SRT1[2][1] = 0.5*(dwdy1 + dvdz1);
                        SRT1[2][2] = 0.5*(dwdz1 + dwdz1);

/*    Strain rate tensor on the face    */
                        Sface[0][0] = 0.5*(SRT[0][0] + SRT1[0][0]);
                        Sface[0][1] = 0.5*(SRT[0][1] + SRT1[0][1]);
                        Sface[0][2] = 0.5*(SRT[0][2] + SRT1[0][2]);

                        Sface[1][0] = 0.5*(SRT[1][0] + SRT1[1][0]);
                        Sface[1][1] = 0.5*(SRT[1][1] + SRT1[1][1]);
                        Sface[1][2] = 0.5*(SRT[1][2] + SRT1[1][2]);

                        Sface[2][0] = 0.5*(SRT[2][0] + SRT1[2][0]);
                        Sface[2][1] = 0.5*(SRT[2][1] + SRT1[2][1]);
                        Sface[2][2] = 0.5*(SRT[2][2] + SRT1[2][2]);
                    }
                    else
                    {
                        Sface[0][0] = 0.0;
                        Sface[0][1] = 0.0;
                        Sface[0][2] = 0.0;

                        Sface[1][0] = 0.0;
                        Sface[1][1] = 0.0;
                        Sface[1][2] = 0.0;

                        Sface[2][0] = 0.0;
                        Sface[2][1] = 0.0;
                        Sface[2][2] = 0.0;
                    }

/*    Numerator for some other formula    */
                for(ii=0; ii<3; ii++)
                    for(jj=0; jj<3; jj++)
                    {
                        DS[ii][jj] += Sface[ii][jj];
                    }
                }    
            }    /*    Here ends the face loop    */
        }
        end_c_loop(c,t)
    #endif
}

[blackmask]

Please explicitly initialize the DS array, otherwise the results of DS is undefined after the function call.

[kornetka]

Thank you. I added the initialization at the beggining of the cell loop and it works now.
Regards, kornetka