CFD Online Logo CFD Online URL
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

parallel code: change solver so that it varies a mesh depending on the processor?

Register Blogs Members List Search Today's Posts Mark Forums Read

LinkBack Thread Tools Search this Thread Display Modes
Old   March 14, 2017, 16:15
Question parallel code: change solver so that it varies a mesh depending on the processor?
New Member
Laura Sumner
Join Date: Mar 2017
Posts: 13
Rep Power: 5
LauraSumner is on a distinguished road
i am trying to implement a problem in OpenFOAM 3.0.1. It is quite hard to explain so I hope this makes sense.

My problem consists of a 2D (x,y) channel, for simplicity it has cyclic conditions at the left and right wall. The top surface moves as a sin wave, according to a text file read in at each time step.
I.e if I have 100 time steps, I need 200 files (100 for the x coordinates and 100 for the y).

I am using a modified solver (modification of pimpleFoam) which finds these text files and sets the mesh to be at these coordinates at each time step. Hence by reading in the files at each time step it ends up with a travelling wave along the top surface. Ultimately this wave will be far more complicated than a simple sin, but for now I just want the simulation to work in parallel with an output that I know that I can trust.

The simulation works perfectly before decomposing.

The problem is, once I decompose the geometry, instead of reading in only the portion of the coordinates that correspond to the particular part a processor is working on, the solver just reads in the entire set of coordinates. For example this means that if I had a sin wave with one period along the top of the channel, once the code has been decomposed into n pieces, tun and then reconstructed, there are n periods of the wave. There are obvious work arounds for a symmetric sin wave, but my final wave will not be symmetric at all.

I suppose my question is really how the mpirun actually works, the online information I have found is very technical. I have added in pieces of code which check what processor is running, and then extracts the coordinates from the text file accordingly. So if I split into 2 pieces down the middle (vertically) I can take the first half of the coordinates from the text file and assign them to the surface displacement points of processor0 and take the second half of the coordinates and assign them as the processor1 displacement.

I thought this would mean that the particular processor would continue through the code with a different set of coordinates as input and i wouldn't have to change anything else. Because how does a particular processor know what geometry it is working with..?

I really don't know how to then tell the rest of the solver this information.

I hope this makes sense!! Any help is appreciated, including general mpi advice: I'm sure others must have done a similar thing before.

Many thanks

LauraSumner is offline   Reply With Quote

Old   March 15, 2017, 17:09
Senior Member
Sandeep Menon
Join Date: Mar 2009
Location: Amherst, MA
Posts: 399
Rep Power: 20
deepsterblue will become famous soon enough
First of all, it's a really bad idea to be reading a file in each time step. A far better approach would be store all information to a single file and read it once at the start of the simulation, possibly on all processors.

Next, how are your displacements defined? Is it a function of the x-coordinate? Something like this perhaps:

X-coordinate Displacement
========= ==========
0.01 0.0005
0.02 0.0010

If not, you probably want it in a form similar to this, saved to file. When you say "set the mesh", how are you doing this exactly? Are you moving the mesh points by the specified displacement? Care to provide some code? This clearly looks like a mesh motion problem, where you need to displace the boundary in a prescribed fashion, while the inner points of the mesh are re-positioned in a way to preserve mesh quality.

Once you have the information read in on each processor, you would simply loop though boundary points on the "top surface" patch, and displace each one based on you inputs. You might need to have a lookup-table of some sort, and potentially interpolate linearly between values when your patch points don't quite match up to you inputs.

Ideally, you shouldn't have to worry about the way the mesh is decomposed - that's quite arbitrary, particularly when you partition based on a graph-based approach like Metis / Scotch.
Sandeep Menon
University of Massachusetts Amherst
deepsterblue is offline   Reply With Quote

Old   March 16, 2017, 06:23
New Member
Laura Sumner
Join Date: Mar 2017
Posts: 13
Rep Power: 5
LauraSumner is on a distinguished road
Totaly agree about the text file issue, the edited solver I was supplied with already had this method of data entry included and I was keen to change as little as possible. I will definitely use one data file eventually however.
I actually solved the problem now quite simply:

//read in entire text file of y coordinates

for(int k = 0; k < 1000; ++k)
file_y >> data_readInY[k];

//determine the processor currently being worked on, and then assign the appropriate subset of coordinates depending on which processor (and therefore which part of the geometry) (deltaL is the length of the decomposed section so that procPoint[0] is the first x coordinate of the section)

for(int m=0; m<n; ++m)

{ procNumber = m;
for(int j = 0; j < 250; ++j)
y_displ[j]= data_readInY[j+ m*HCnumberOfCells];



Info << "File "<< Result_x.c_str() <<" not found! Abort" << endl;

unsure how readable this is but basically does the job! thanks again for your response!
LauraSumner is offline   Reply With Quote


dynamic mesh, mpi, parallel code, personalised solver

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to use "translation" in solidBodyMotionFunction in OpenFOAM rupesh_w OpenFOAM Running, Solving & CFD 5 August 16, 2016 05:27
[mesh manipulation] Importing Multiple Meshes thomasnwalshiii OpenFOAM Meshing & Mesh Conversion 18 December 19, 2015 19:57
Star cd es-ice solver error ernarasimman STAR-CD 2 September 12, 2014 01:01
decomposePar pointfield flying OpenFOAM Running, Solving & CFD 28 December 30, 2013 16:05
[snappyHexMesh] snappyHexMesh won't work - zeros everywhere! sc298 OpenFOAM Meshing & Mesh Conversion 2 March 27, 2011 22:11

All times are GMT -4. The time now is 12:21.