compiling UDF for moving mesh in parallel mode
 

Urgent, will wait online for answers!

I have to compile the following UDF in parallel for a dynamic mesh.

#include "udf.h"
#include "stdlib.h"
#include "stdio.h"
#include<string.h>

DEFINE_GRID_MOTION(remesh,domain,dt,time,dtime)
{ #if RP_HOST

FILE *fd;
Thread *tf = DT_THREAD(dt);
face_t f;
Node *v;
real NV_VEC(vetcoord);
int n,i,time_step;
char nomefile[14]="tempo", numero_file[5], estensione_file[5]=".txt";

time_step=N_TIME+1;
Message("time_step:%d\n", time_step);

sprintf(numero_file, "%d", time_step);
strcat(nomefile, numero_file);
strcat(nomefile, estensione_file);
fd=fopen(nomefile, "r");

i=0;
if( fd==NULL )
{
perror("Errore in apertura del file");
exit(1);
}

SET_DEFORMING_THREAD_FLAG(THREAD_T0(tf));


begin_f_loop(f,tf)
{
f_node_loop(f,tf,n)
{
v = F_NODE(f,tf,n);

if (NODE_POS_NEED_UPDATE (v))
{
NODE_POS_UPDATED(v);
fscanf(fd, "%g %g %g", &vetcoord[0], &vetcoord[1], &vetcoord[2]);
NV_V(NODE_COORD(v), +=, vetcoord);
}

}
}
end_f_loop(f,tf);

fclose(fd);
#endif
}


In series, there are no problem, the UDF is compiled and the dynamic mesh works.
But when I compile it in parallel mode, there are no motion of the mesh.
what would it be the problem? Maybe I have to pass to the node (4 CPU) the information of the new updated coordinates?

(NB: The function read the displacement, to add to the coordinates node of the mesh every time step, from file.txt.)

My read of your UDF is that you are opening a file and pulling new location values in and assigning them to Nodes. That is fine in serial, because you have implicitly assumed that the order of the nodes generated by the face/node loops will be consistent. None of that is true any longer in parallel. The UDF, as written, will run on all four processors--each will open the displacements file, each will run the face/node loop but will only traverse the faces and nodes that are on its process. So, each process will read the first portion of the displacements file and will update all of those nodes with the same data. So, my guess is that all of the PEs are mapping their Node locations on top of each other.

The key problem here is that there is no way to assume the data ordering of the serial case will map seamlessly to a parallel case. You will need to either partition your displacement data file into separate lists that follow the face/node ordering on each PE. Or (more directly), you need to store the displacement data with node reference ids that are unique and consistent whether the job runs in serial or on any number of parallel nodes. Note that this latter approach will require that you sort and search the displacement file as part of the UDF. Unfortunately, there are no easy answers here, but these are the essential problems that arise in parallel unstructured CFD. Good luck.

Thanks for your answer.

My conclusions are the same. I'm thinking if there are some ways or some macros to pass all node information to the host and to compile all the function in the host.
Thus the node update can be done before the subdivision to the other CPU.

For example I found this command:

(rpsetver 'parallel parallel/fast-io? #t): fluent will read the whole mesh into the host machine before distributing to the compute node.

But it doesn't work. There are other command like this?

No. There is isn't. Handling data distribution across PEs always involves a lot of bookkeeping.

If the values in the displacement file are constant, you could store the values in node User-Defined Memory (macro is N_UDMI). You can read the data in from the file in serial, store them in the UDM, and then save the case/data files. Then when you launch in parallel, FLUENT will manage the distribution of the UDMs to the correct PEs and your UDF can just read the UDM data and apply the location changes. That is the fastest/easiest route to accomplishing your task.