CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Programming & Development

dnsFoam in parallel - Has anyone tried this?

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   March 24, 2014, 00:24
Default dnsFoam in parallel - Has anyone tried this?
  #1
Member
 
Christian Butcher
Join Date: Jul 2013
Location: Japan
Posts: 84
Rep Power: 3
chrisb2244 is on a distinguished road
So it seems like the dnsFoam packaged with OF-2.2.2 doesn't support parallel running. Initially, it looks like the kMesh class is the one which creates some problems, but presumably these could be solved with not too much effort?

My question is essentially - is there some other (deeper) reason why dnsFoam does not work in parallel? I wanted to ask before altering lots of files and ending up with many slightly edited classes, just to find that the solver is for some (as now unknown to me) reason completely incompatible with parallel solving.

If the problem is only with the generation (and distribution?) of a kMesh, then could someone perhaps advise me on whether I should use some sort of

Code:
if (Pstream::master())
{
    // create a kMesh, and distribute to the other processes
    // possibly with some sort of scatter(..) function?
}
or alternatively, try and generate a kMesh for each subsection of a mesh, and only allow 1D decomposition (which I think should be ok?)

I am looking at this so that I can attempt to simulate turbulent pipe flow, and I'm just unsure as to the best way to go about it.

Any help or suggestions would be appreciated.
chrisb2244 is offline   Reply With Quote

Old   April 2, 2014, 14:48
Default
  #2
Senior Member
 
kmooney's Avatar
 
Kyle Mooney
Join Date: Jul 2009
Location: Amherst, MA USA - San Diego, CA USA
Posts: 268
Rep Power: 8
kmooney is on a distinguished road
Have you tried running this on the newest OpenFOAM2.3.x release? The problem just just disappear for you.
kmooney is offline   Reply With Quote

Old   April 2, 2014, 21:51
Default
  #3
Member
 
Christian Butcher
Join Date: Jul 2013
Location: Japan
Posts: 84
Rep Power: 3
chrisb2244 is on a distinguished road
Dear Kyle,

Thank you for your suggestion - I spent yesterday getting OF-2.3.0 built and installed, so hopefully I can test dnsFoam again today.

In the case that it doesn't work as hoped, I did find another thread in this set of forums which had V&V studies for an 'ico_DNS' solver, which used dns with a gradP forcing, rather than the kMesh implementation used by OF-2.2.2's (and presumably other OFs') dnsFoam.

I'll post back here when I know if the changes to parallel processing and mesh handling change anything, but for others reading this, the ico_DNS solver is pretty useful:

Everything you need to compute DNS in channel vs OF 2.1.0
chrisb2244 is offline   Reply With Quote

Old   July 16, 2014, 13:09
Default
  #4
Senior Member
 
Kent Wardle
Join Date: Mar 2009
Location: Illinois, USA
Posts: 195
Rep Power: 10
kwardle is on a distinguished road
Perhaps I am wrong, but I think it has been established here and elsewhere that the dnsFoam solver does not run in parallel due to the use of the fft for calculation of the turbulence forcing function. Others have correctly mentioned that DNS can be done for pressure driven flows (where turbulence forcing is not required) using other solvers, but this does not solve the problem if the goal is to simulate isotropic turbulence in a box---for which a parallel version of dnsFoam is needed unless some other turbulence forcing scheme can be used which does not rely on FFT (is there such a thing, even as an approximation to 'real' turbulence?). Alternatively, if an FFT library can be used which does not require a uniform 2^n mesh then perhaps the same forcing scheme can be employed and run in parallel.

So to my question, has anyone made any attempts to use the PFFT or PNFFT libraries available here https://www-user.tu-chemnitz.de/~mpi...ware.php#pnfft ? This would seem like a viable option and appears to be targeted to just this issue---calculation of FFT in parallel on irregularly spaced meshes (though perhaps still Cartesian, I haven't look close enough yet to determine the limitations of the algorithm).

Another possible alternative I can think of is to somehow compute the turbulence forcing FFT on a overlaid uniform mesh on the master node and then map this out to the processor meshes. This would perhaps create a bottleneck but might work OK if the cost of this step is a small fraction of the total computation.

Anyone out there tried to do something like either of these (PNFFT or overalid mesh hack)? I am trying to simulate droplet breakup dynamics for isotropic turbulence in a box using a DNSInterDyMFoam solver I have put together base on the turbulence forcing in dnsFoam. Unfortunately, I had not appreciated the serial limitation of the parent solver and am now trying to sort out how best to proceed...
kwardle is offline   Reply With Quote

Reply

Tags
dnsfoam, parallel

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Can not run OpenFOAM in parallel in clusters, help! ripperjack OpenFOAM Running, Solving & CFD 5 May 6, 2014 15:25
simpleFoam parallel AndrewMortimer OpenFOAM Running, Solving & CFD 10 November 12, 2013 18:03
simpleFoam in parallel issue plucas OpenFOAM Running, Solving & CFD 3 July 17, 2013 11:30
parallel Grief: BoundaryFields ok in single CPU but NOT in Parallel JR22 OpenFOAM Running, Solving & CFD 2 April 19, 2013 16:49
Is dnsFoam can run in parallel mamaly60 OpenFOAM Running, Solving & CFD 0 October 29, 2007 19:27


All times are GMT -4. The time now is 02:36.