parallel error with cyclic BCs for pimpleDyMFoam and trouble in resuming running
 

Two problems found when using pimpleDyMFoam and waveDisplacement：

1. When I use pimpleDyMFoam together with cyclic boundary conditions,(actually I use waveDisplacement, you can try the movingCylinder case, with a pair of cyclic boundaries and oscillatingVelocity for the cylinder, the same thing happens) parallel running cracks: after about 2 or 3 timeSteps, running stopped with the message " face o area does not match neighbour by 0.0195% --possible face ordering problem", no matter what kind of decomposition method I use, the same problem always appears. But when I use a single core to run it, no such problem happens. So I assume there might be some bugs in parallel running of dynamic mesh cases with cyclic BCs. Has anyone encountered this before? If anyone is interested, just let me know, I can post the tutorial.

2. When I use waveDisplacement to simulate a imposed wave motion of the bottom surface, something even more strange happens: if I stop the running, and resume it, it just explodes with the error "Floating point exception (core dumped)"! This is serious cause I cannot assure the running keeps going until I get satisfactory results. Once it stops, it cannot be resumed! Again, if anyone is interested, I can post the tutorial.

do ur cases run fine in serial mode?

I got same problem (1) when using pimpleDyMfoam with cyclic bc in parallel run (2.1.0).
In my case (LES), serial run with above and parallel run with pimpleform with cyclic was ok. I tried to loose match torlence (0.9) and decompose with scotch and simple method, but always encountered same error within 2-3 time steps.

Could you post how to solve it?

check tutorials.. u might have to use this patch type instead "cyclicAMI"

What is serial mode? I'v never heard of that before, could you explain a little bit? And is it ok to replace cyclic bc with cyclicAMI since the latter stands for cyclic arbitrary moving interface? Sorry to be ignorant.
Best!
Sunliming

By the way:
I tried mpirun -np 2 gradPchannelDyMFoam, which I think is the so called "serial run", and it seems ok until now.
How could cyclicAMI be used for the inlet, outlet, front and back wall since there is no arbitrary moving interface?

Hi, sunliming

Serial run is just to use only one CPU for computation...as far as I know...
In my case, I used 4 CPUs (-np 4 pimpleDyMFoam with cyclic BCs) in my desktop but failed around 2000 time step. In case of 200 CPUs, failed within 2-3 time step.

Maybe, as posted by mauricio, it seems cyclicAMI is the only option for dynamic mesh problem. however, it looks quite tricky for me to use cyclicAMI ...:(
If you managed to solve the problem, please let me know..

Thanks

I think serial run is running with several cores one after another rather than parallel, so I run "mpirun -np 2 channelFoam (here -parallel is removed so that it becomes serial mode)". But I think this mode is much slower than the parallel mode, and I doubt wether it is any faster than the single core mode. Anyway, I still think there are some bugs for parallel running with cyclic BCs, but I've no idea how to handle at the moment.

hi everyone!


yes serial is when u launch only one process using 100% of a thread.

when u set -np x it means your case should have x domains and you'll have to launch x processes. If you have x threads available then you will have x process running at 100% cpu each. Else if x is less than the number of threads available the process you launched you will have to share cpu resources.

cyclicAMI is for dynamic meshes because it handles non-conforming meshes (http://www.openfoam.org/version2.1.0/ami.php). you'll have to be careful how u set ur domain patches, bcs and the settings in the system folder since its a transient solution method.

gl

parallel running error with cyclic BCs for pimpleDyMFoam
 

Problem remains unsolved, here is the case file, hope anyone can check it and give some help!
Just run the following applications:
blockMesh
decomposePar
mpirun -np 2 pimpleDyMFoam -parallel

I guess there must be something wrong with the updatecoeffs() in the source code of e.g. oscillatingDisplacement. So when running in parallel one processor updated the coordinates and set the switch to "updated already", then the other processors get the message and stop updating the coordinates which should be updated. As a result, two cyclic patches will which contradict due to the inconsistence of coordinates. Does anybody know whether I am right or not? Thank you in advance for comments.

I'm now more sure about the existence of bugs, since it runs well in parallel mode with the same BCs(2 pair of cyclics,i.e. channel395 case) for static mesh and runs well in serial mode but crashes in parallel for dynamic mesh. I tried ro use "createPatch" as someone mentioned in the forum, but it still runs well in serial and craches in parallel. So I think there must be someting wrong for cyclic BCs to go well with dynamic mesh. Yet, I've no idea how to fix it. Has anyone encountered this before?

Greetings to all!

I've used an example case I really like from this bug report: http://www.openfoam.org/mantisbt/view.php?id=566 - and I adapted it to run with 4 cores and using only "cyclic" patches.

I've tested with 2.1.1 and the same error occurs, where "obj" files are dumped and it then crashes.
I've also tested with 2.1.x and it interestingly locks up instead of crashing...

Then I did some searching and adapted to "cyclicAMI" patches. Basically you need to:

1. Change all references from "type cyclic;" to "type cyclicAMI;".
2. And in "blockMeshDict" you need this:
   
   Code:

   ---

   transform      translational;
separationVector (0.001 0 0);

   ---

   Well, you should adjust the last vector value accordingly to your case/mesh. And keep in mind that the other patch needs to use the respective negative position, namely:
   
   Code:

   ---

   transform      translational;
separationVector (-0.001 0 0);

   ---

Attached you will find the two examples.


Note: the cyclicAMI version ends up crashing at around 0.0038s :rolleyes:, so it is not a perfected example ;).


To run either one of the cases, simply run:
Best regards,
Bruno

cyclicAMI works for parallel running where cyclic cannot
 

OH MY GOD, it works!!! You've solved the problem that has bothered me for months! Great great thanks to you, Bruno! Come to China the othet day and let me treat you to dinner!:)
A few days ago, I tried cyclicAMI with translational transform, but I was not clear about the vector and got the wrong direction and size and ended up in another failure.
Now I'm quite clear! The separationVector conveys 2 of important factors: dierction and the distance between the two cyclic patches (These two "brothers" are great trouble-makers in my opinion!) in that direction.
By the way, cyclicAMI with tanslational transform can ensure the two patches have equal corresponding field variables as cyclic can do, right? If so, it's perfect for me!
Best wishes!
Nick (sunliming)

Hi Nick,

You're welcome! To complete the answer, there are some more hints in the tutorials, more specifically in the file "incompressible/pimpleDyMFoam/mixerVesselAMI2D/constant/polyMesh/blockMeshDict.m4", namely the commented code:
The translational detail came from the file "applications/utilities/mesh/manipulation/createPatch/createPatchDict".

Best regards,
Bruno

Mesh goes into a mess with cyclicAMI
 

Hi Bruno,
Thank you for the supplement!

I tried cyclicAMI for my case, and found the mesh intersects using cyclicAMI which will not happen with cyclic.

Attacted are my cyclic and cyclicAMI cases, cyclic case can run in serial mode without disturbing the mesh, but with cyclicAMI meshes all go wrong, could have a look at the files, run with the 'Allrun' script for each case and see what is wrong with cyclicAMI.

Thank you! Merry Christmas!

Nick

Hi Nick,

:eek: This is one crazy bug :D Attached are two iterations detailing the issue!

So, on both images you'll see on the left the mesh with "cyclic" patches and on the right with "cyclicAMI".
In the iteration 0.25, we start to see a slight difference. At 0.5, they are veeery different! This is why it crashes with "cyclicAMI", since there are cells overlapping each other!
You can see this effect on either one of the 4 bottom corners.

I suggest that you simplify a bit more the examples you've attached. More specifically:

• Make both cases only run up to 0.75. This way you'll get 3 more iterations, where the 3rd iteration wont be possible with AMI.
• The AMI case should run in serial as well. It's easier that way to diagnose.

Then create a new report at http://www.openfoam.org/mantisbt/ with these simplified examples and you can also include these 2 images I attached as well. This way it will make life a lot easier for the developers to diagnose the problem ;)

Best regards,
Bruno

Bug lies in waveDisplacement due to update the internalFields alone without other BCs
 

Hi Bruno,

Thank you for the quick reply and good advice!

I'll make a report to the bug centre later. I guess it is not a problem with the cyclicAMI BC, for when I replace 'cyclic' and/or 'cyclicAMI' with 'patch',(see attached case file) the mesh still goes wrong, so it is concluded that only 'cyclic' can be used for the motion type 'waveDisplacement' where the bug is supposed to be.

When I use 'patch' for all lateral walls, the mesh is good for the internal field but wrong at the inlet and outlet.(see attached pictures) So I checked the source code "opemfoam211/src/fvMotionSolver/pointPatchFields/derived/waveDisplacement/waveDisplacementPointPatchVectorField.C",the member function updateCoeffs() only updates the internalField and the moving boundary, excluding the other boundarys, see "const polyMesh& mesh = this->dimensionedInternalField().mesh()();".

I don't know how to include other boundary pathes for point motion, for I'm not quite familiar with OpenFOAM syntax.

Best wishes!
Nick

Two more pictures attached for the lower part of inlet at 4s and outlet at 4s.

Hi Nick,

The reason why I asked you to report on the bug tracker is also because I'm not experienced enough to quickly figure out what's the solution on this one.
But I can tell you that:

1. "dimensionedInternalField().mesh()()" is only used in this case for getting information about the current time.
2. It's the "waveNumber_ & patch().localPoints()" that does the actual point retrieval.
3. My guess is that when using "cyclicAMI", there's a step somewhere that is missing for updating the positions of the points for the previous vs next iteration.

But the "waveNumber" reminded me of something... have you looked into this :confused:: http://openfoamwiki.net/index.php/Contrib/waves2Foam
I'm not certain of the problem/solution you're after is related to the information on that page, but it sort of looks like it might be.

Best regards,
Bruno

Hi Bruno,

Actually, I'm doing research on wind-wave interaction. I'm a graduate student of wind engineering, my final goal is to study mean wind profiles over the ocean under strong wind and give some instructions of proper wind load on offshore structures.

So the most suitable model for my case should be two-phase flow, but I wanna start with a relatively easy model--imposed solid wave undulation, this is just what waveDisplacement can do. As for 'waves2Foam', I have studied it for a while, but later I found its focus is on the wave rather than wind field which is my concern. And I did some simple simulation, finding only near wave layer of the upper wind field has evident change. The wave cannot take into accout of wind energy transfered to it, thus modification must be made to satisfy my need. So this may be my next task providing current step can be accomplished.

Problems keep coming one after another, so there is still a lot to do.

Anyway, thank you giving so much help!

Best wishes!
Nick

ok... hi...

first when u use the tool meshes from OF u have to understand the software's bcs

first, you use cyclicami for actual mesh motion (different from mrf)
second, cycliami is an internal BCs, so its not an inlet or outlet

second, if your case is symmetric for a phi angle, then u use cyclic bcs if im not mistaken

third, make sure the bcs ur using are compatible with the current solver

fourth, symmetrical cases may be steady-state cases. cyclic ami is for transient solutions.

i advise you to read of's user guide and attempt a mesh created somewhere else.

gl

Greetings to all!

@Lev: If there is mesh motion, you probably will need cyclic AMI. Check post #12 for more information.

Best regards,
Bruno