CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM

no parallel computing using "dynamicBodyFvMesh"

Register Blogs Community New Posts Updated Threads Search

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   October 22, 2009, 16:16
Default no parallel computing using "dynamicBodyFvMesh"
  #1
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
Hi there

I'm using a slightly modified version of dynamicBodyFvMesh with OpenFOAM-1.5-dev, which gives me 2 translational DOF and 1 rotational DOF. I'm simulating a flapping wing in 2d (crosssection), but I am not able to use 2 or more processors. Decomposing works fine, mpirun starts as well but stops at the first line of the first time step (Where it says Time = 5e-6). Using top shows me, that all processor are working with 100% load, but even after 12 hours, theres no progress.

Anyone able to help me?

Thanks in advance, Bastian
Bastian is offline   Reply With Quote

Old   October 23, 2009, 03:57
Default
  #2
Member
 
Jean-Peer Lorenz
Join Date: Mar 2009
Location: Rostock, Germany
Posts: 33
Rep Power: 17
jploz is on a distinguished road
Hi,

have a look at the following thread. Maybe this helps.

http://www.cfd-online.com/Forums/ope...l-1-5-dev.html

I'm also using 1.5-dev and running dynamic mesh cases in parallel works
fine for me (2D and 3D). I'm using Metis as decomposition method.

HTH,
Jean-Peer
jploz is offline   Reply With Quote

Old   October 23, 2009, 07:40
Default
  #3
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
Yes, that solved my problem! It was the commsType in foam/etc/controlDict. Gotta set it to blocking


OptimisationSwitches
{
fileModificationSkew 10;
// commsType nonBlocking; //scheduled; //blocking;
commsType blocking; //scheduled; //blocking;
floatTransfer 0;// Floating transfer not realiable
nProcsSimpleSum 16;

nSquaredProjection 0;
}



Thanks a lot!!
Bastian is offline   Reply With Quote

Old   October 27, 2009, 11:05
Default
  #4
Member
 
Jean-Peer Lorenz
Join Date: Mar 2009
Location: Rostock, Germany
Posts: 33
Rep Power: 17
jploz is on a distinguished road
Unfortunately, I have to correct myself. Running 2D cases in parallel works indeed fine. 3D cases do not crash as they did in the beginning, however the motion solver diverges after some time when running in parallel. The same simulations run fine in serial. I don't no why it works in 2D but does not in 3D.

Keep posting.
Jean-Peer
jploz is offline   Reply With Quote

Old   October 27, 2009, 12:08
Default
  #5
Senior Member
 
Hrvoje Jasak
Join Date: Mar 2009
Location: London, England
Posts: 1,905
Rep Power: 33
hjasak will become famous soon enough
I know about it, but it needs to be fixed in the next merge. Retro-fitting is painful, because point interpolation is badly broken. In FEM, that means inconsistent constraints in parallel, which does indeed make the solver diverge.

If you really need it right now in parallel, the only option is 1.4.1-dev.

Apologies,

Hrv
__________________
Hrvoje Jasak
Providing commercial FOAM/OpenFOAM and CFD Consulting: http://wikki.co.uk
hjasak is offline   Reply With Quote

Old   October 27, 2009, 18:43
Default
  #6
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
Hi everybody,

I'm facing another problem now. My pressure stops to converge, initial values of 10e-6 start to slowly increase up to 0.1 and I don't know why. In the end it affects my Courant Number really badly, with values of about 10e+100, and (no wonder) my computation crashes.
I'm using PCG to solve the pressure, increasing the nonOrthogonalCorrectors helps a bit, but it seems to just postpone or slow down the problem.
The mesh looks a bit distorted (structured inner volume and a couple of cells at the 1.9e+00 scale), but right at the body it looks ok.
Also could anyone tell me what options I have for the diffusivity? Right now I'm using quadratic, and as a motion solver laplaceFaceDecomposition. Any fine tuning options there?

Thanks again

Bastian

P.S.: On the pictures you see a wing moving up, with no free stream. Its the last ouput (t=0.3185) before the computation crashes (t=0.31897)
Attached Images
File Type: jpg 0.3185.jpg (96.3 KB, 58 views)
File Type: jpg 0.3185_3.jpg (81.2 KB, 52 views)
Bastian is offline   Reply With Quote

Old   October 28, 2009, 03:56
Default
  #7
Member
 
Jean-Peer Lorenz
Join Date: Mar 2009
Location: Rostock, Germany
Posts: 33
Rep Power: 17
jploz is on a distinguished road
Hi Bastian,
is this a 2D case or is it in 3D? Do you run it in parallel? I've observed similar things, however only in parallel.

Regards.
Jean-Peer
jploz is offline   Reply With Quote

Old   October 28, 2009, 06:21
Default
  #8
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
Moin Jean,

it's the 2D case mentioned in the first post, and I run it in parallel. Yesterday I tried a very rough way to keep the pressure converging, set all the relTol to zero, tolerance to 1e-10, and a lot of nonOrthogonalCorrectors. I'll have access to the results in one hour, then I'll see if it helped.
What did you do when it occured?

Regards from Bremen

Bastian

P.S.: Suggestions on the motion solver or diffusivity? I had a hard time using try and error to see what options I have, but nothing else (except for laplaceFaceDecomposition and quadratic diffusivity) really worked for me, probably because some files in the case have been missing.

Last edited by Bastian; October 28, 2009 at 06:48.
Bastian is offline   Reply With Quote

Old   October 28, 2009, 08:35
Default
  #9
Member
 
Jean-Peer Lorenz
Join Date: Mar 2009
Location: Rostock, Germany
Posts: 33
Rep Power: 17
jploz is on a distinguished road
The computation of 2D cases in parallel and 3D cases in serial works for me. The problem occurred only in 3D parallel cases and cannot be solved at the moment (see post above). Obviously, there are severe issues related to the FEM mesh motion and parallelism in 1.5.-dev. So, I solved the problem by running in serial ;-)

Maybe you should try to run your case in serial in order to exclude whether your problems are related to parallelism or not. Moreover, have a look at your solver and review the changes you made. Maybe a mistake with the fluxes somewhere? (this also happened once for me-where codes is, there are bugs) For a proper case setup and a decent mesh motion issues like this should not arise. Check your Courant number and have a look at your deformed mesh before it diverges. Is there too much motion? Oh, you've posted pictures already. Does not look that bad. What says checkMesh for this mesh?

And yes, I also use laplaceFaceDecomposition and quadratic diffusivity since this worked best.

Good luck.
Jean-Peer
jploz is offline   Reply With Quote

Old   October 28, 2009, 10:40
Default
  #10
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
Hi

So increasing the nonOrthogonalCorrectors respectively decreasing the tolerances didn't help.
My solver worked already with other (more simple) cases (concerning the mesh), but I've never tried parallel computing there. And as I mentioned, the changes are really small, but I'll have a look at it just to be sure.

checkMesh -latest time

Checking geometry...
This is a 2-D mesh
Overall domain bounding box (-6 -5 -1.98584e-16) (12 5 0.01)
Mesh (non-empty) directions (1 1 0)
Mesh (non-empty, non-wedge) dimensions 2
All edges aligned with or perpendicular to non-empty directions.
Boundary openness (3.30823e-20 7.21796e-21 7.70782e-20) Threshold = 1e-06 OK.
***High aspect ratio cells found, Max aspect ratio: 1.89361e+06, number of cells 1 Threshold = 1000
<<Writing 1 cells with high aspect ratio to set highAspectRatioCells
Minumum face area = 1.21883e-08. Maximum face area = 0.00810153. Face area magnitudes OK.
***Zero or negative cell volume detected. Minimum negative volume: -1.21883e-10, Number of negative volume cells: 1
<<Writing 1 zero volume cells to set zeroVolumeCells
Mesh non-orthogonality Max: 70.8663 average: 10.8627 Threshold = 70
*Number of severely non-orthogonal faces: 1.
Non-orthogonality check OK.
<<Writing 1 non-orthogonal faces to set nonOrthoFaces
***Error in face pyramids: 5 faces are incorrectly oriented.
<<Writing 5 faces with incorrect orientation to set wrongOrientedFaces
Max skewness = 2.3677 OK.

Failed 3 mesh checks.


Not so good. Same for the writeInterval before. You think a more coarse mesh would help? I'm about to try serial run, maybe it works.

Cheers

Bastian
Bastian is offline   Reply With Quote

Old   October 28, 2009, 12:39
Default
  #11
Member
 
Jean-Peer Lorenz
Join Date: Mar 2009
Location: Rostock, Germany
Posts: 33
Rep Power: 17
jploz is on a distinguished road
It definitely makes the things a bit easier. You can also create a subset around the body and keep this constant. The motion is then accommodated by the surrounding coarser regions (sounds good but I've not done this before).

Nevertheless, my guess is something goes wrong with the mesh motion solver (negative cell volume).

HTH,
Jean-Peer
jploz is offline   Reply With Quote

Old   October 28, 2009, 12:58
Default
  #12
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
I just checked the results from the serial run, same error. I heard about subsets, never done it either.
I never touched the mesh motion solver, what I modified was just the way the body moves. I don't know if the negative volume is due to the rotation, it's actually not so big, 20° amplitude should hopefully work.
I'll see what I can find out about using subsets, and I have 2 more moving classes to try.

regards, Bastian
Bastian is offline   Reply With Quote

Old   October 29, 2009, 09:49
Default Flapping wings
  #13
Senior Member
 
maddalena's Avatar
 
maddalena
Join Date: Mar 2009
Posts: 436
Rep Power: 23
maddalena will become famous soon enough
Hello,
I run simulations for flapping wings using OF1.3, both in 2D and in 3D, so I do not know if there are problems connected with the new solver. However, I guess your problems are due to the mesh: you should definitely improve your mesh quality at the leading and trailing edge of the wing expecially. Try using some elliptic PDE methods to cluster cells without reduce their height.
Good luck!
maddalena is offline   Reply With Quote

Old   November 2, 2009, 19:06
Default
  #14
New Member
 
Join Date: Oct 2009
Posts: 11
Rep Power: 16
Bastian is on a distinguished road
Hi Maddalena, Hi Jean-Peer

I solved my problem using a coarser mesh and scale my volume so that my wing just moves in 40% of my volumes height. That way I didn't need to do anymore finetuning like using subsets and it is very convinient for now. To be frank, I never heard of the method Maddalena suggested, but that is for sure due to my experience, since this is for my Bachelor thesis and I'm working with CFD now for a bit more than half a year.
But I'll definitely keep it in my mind.

Have a good one!

Bastian
Bastian is offline   Reply With Quote

Reply

Tags
dynamicbodyfvmesh, moving mesh, parallel computing


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Diffusion equation solved using Parallel Computing Sachin Paramane Main CFD Forum 0 June 11, 2007 23:48
Parallel Computing on Multi-Core Processors Upgrading Hardware CFX 6 June 7, 2007 15:54
Parallel Computing peter Main CFD Forum 7 May 15, 2006 09:53
problem!! FLUENT 6.2 -SUSE parallel computing khanjaved FLUENT 1 August 15, 2005 22:00
Parallel Computing Classes at San Diego Supercomputer Center Jan. 20-22 Amitava Majumdar Main CFD Forum 0 January 5, 1999 12:00


All times are GMT -4. The time now is 02:46.