CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

Parallel run with different solvers

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By gschaider

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 6, 2006, 09:27
Default Hi, I would like to know if i
  #1
New Member
 
Carlo Fabrizi
Join Date: Mar 2009
Location: Rome, Italy
Posts: 5
Rep Power: 17
carlo_fabrizi is on a distinguished road
Hi,
I would like to know if is possible to run a case in parallel with 2 different solvers for each block.
Thanks
carlo_fabrizi is offline   Reply With Quote

Old   June 6, 2006, 09:34
Default sorry, the post can be misunde
  #2
New Member
 
Carlo Fabrizi
Join Date: Mar 2009
Location: Rome, Italy
Posts: 5
Rep Power: 17
carlo_fabrizi is on a distinguished road
sorry, the post can be misunderstood..
I want to run in parallel a case and I would like to use for a block a simpleFoam solver and for the other a simpleFoam modified solver.. Is it possible?
carlo_fabrizi is offline   Reply With Quote

Old   June 7, 2006, 01:51
Default Hi, This thread is most clo
  #3
Member
 
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 17
ankgupta8um is on a distinguished road
Hi,

This thread is most closely related to the query I have, hence posting my question here:

For a parallel run, what change needs to be done in order to solve for flow only on decomposed domain 0 and some other scalar (say, temperature) only on domain 1 ?? Obviously, domain 0 and domain 1 don't have any common boundary, thus eliminating the need for communication across processors while solving the equations.
For clarity, the description of my model is as follows:
My computational domain consists of two non-connected blocks. After the domain decomposition, block 1 lies on processor 0 and block 2 lies on processor 1. I would like to solve for flow only on block 1 and some scalar equation only on block 2.
It seems to me that in OpenFOAM, the communication between the processors takes place when the equation is getting solved. Now in my case, as there is no common boundary between the two processors, it is conceptually possible to solve for flow field only on one processor. But, the existing equation solvers doesn't allow me to do so. I guess, there is something like MPI_Wait_all statement at the end of the equation-solver which waits for all the processors to solve the equation before moving any further. Now, as I am not solving the equation on all the processors, it just hangs eventually killing my run.
Please let me know if I am correct in my reasoning.
Also, how can I solve for the equations on specific processor(s)??

Thanks!
Regards,
Ankur
ankgupta8um is offline   Reply With Quote

Old   June 7, 2006, 03:43
Default > I want to run in parallel a
  #4
New Member
 
Andrea Palazzi
Join Date: Mar 2009
Posts: 15
Rep Power: 17
giampippetto is on a distinguished road
> I want to run in parallel a case and I would like to use for a block a simpleFoam solver and for the other a simpleFoam modified solver.. Is it possible?

Maybe you can split your domain in two or more zones and then write your own solver that uses different equations for different zones, so that you have your two solvers all in one program that can be run in parallel.

BTW, how do you decompose your domain for parallel processing? I don't think that decomposePar gives you so much control on how to split your domain, am I wrong?

Bye
giampippetto is offline   Reply With Quote

Old   June 7, 2006, 03:48
Default > My computational domain cons
  #5
New Member
 
Andrea Palazzi
Join Date: Mar 2009
Posts: 15
Rep Power: 17
giampippetto is on a distinguished road
> My computational domain consists of two non-connected blocks.

If your blocks are disconnected you can treat them as two different cases, can't you?

Bye
Andrea
giampippetto is offline   Reply With Quote

Old   June 7, 2006, 04:34
Default > decomposePar gives you so m
  #6
Senior Member
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26
mattijs is on a distinguished road
> decomposePar gives you so much control

'manual' option in decomposeParDict. Reads a file giving per cell what processor it should go to.
mattijs is offline   Reply With Quote

Old   June 7, 2006, 06:42
Default Hi Andrea! You said that it's
  #7
Senior Member
 
Francesco Del Citto
Join Date: Mar 2009
Location: Zürich Area, Switzerland
Posts: 237
Rep Power: 18
fra76 is on a distinguished road
Hi Andrea!
You said that it's possible to define two (or hopely more) different zones in my domain and use a different solvers for each zone, isn't it?

Well, how does OpenFOAM let me do this? What about the "communication" over the interface between the zones?

It' will be very useful for me writing an application with this feature...

Francesco
fra76 is offline   Reply With Quote

Old   June 7, 2006, 08:40
Default Well, I've stumbled in zones w
  #8
New Member
 
Andrea Palazzi
Join Date: Mar 2009
Posts: 15
Rep Power: 17
giampippetto is on a distinguished road
Well, I've stumbled in zones when I was trying to use the dynamic mesh features in OF, you can take a look at the dynamic mesh library and examples.
I'm not sure that it fits you needings, but my idea is to write your own solver that does:
- split of the domain into subdomains (zones), either by hard-coding the geometry into the solver or by other means (patches?)
- solve the subdomains each with the appropriate solving equation; it must also handle the interface between zones.

I've never looked deeply into the OF code and I'm not sure this is feasible, maybe some guru here have the right answer.

Bye
Andrea
giampippetto is offline   Reply With Quote

Old   June 7, 2006, 10:35
Default Hi, > If your blocks are di
  #9
Member
 
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 17
ankgupta8um is on a distinguished road
Hi,

> If your blocks are disconnected you can treat them as two different cases, can't you?

I need to patch flow solution from one block to other block. Thats why I need to solve the two blocks simultaneously on different processros. The problem I am facing with this is - though there are no connecting boundaries between the two blocks, while solving the equations where I am just solving flow on block 1 on processor 0, the equation solver just doesn't let me do so. The equation solver requires the equation to be solved on all processors. Thats is exactly where I am stuck.
Any ideas on how to make it possible to solve the equation on just one processor (in general, selected processors if the the domain decomposition is such that it doesn't require communication with other processors) ??

Thanks!
Regards,
Ankur
ankgupta8um is offline   Reply With Quote

Old   June 7, 2006, 12:23
Default At first: I think that running
  #10
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 4,225
Rep Power: 51
gschaider will become famous soon enoughgschaider will become famous soon enough
At first: I think that running different solvers on different domains in parallel would not be very efficient (if it was possible, which it isn't to my knowledge): 1. It is not very likely that both processors get equal load 2. Using more than one processor for one domain would be harder, hindering further parallelization.

@carlos initial request: how does the modified simpleFoam-solver differ from the original? If it's only by a source term you can apply the source term on the whole domain. Just multiply it with a field that is 1 wherever you want the source term to be active and 0 everywhere else (similar approaches for material properties and differential operators).

@ankur: if I understand your problem correctly the best solution would be to write ONE solver that does the following

1. solve equation1 on region1
2. write boundary information from region1 to region2
3. solve equation2 on region2

Examples of such solvers exist on the message board (a solver on coupling two heat transport equations) or the Wiki (a crude fluid-struct coupling).
Nostradamus likes this.
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   June 7, 2006, 13:46
Default Hi Bernhard, I have just on
  #11
Member
 
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 17
ankgupta8um is on a distinguished road
Hi Bernhard,

I have just one mesh in my computational domain, which consists of two blocks. The two blocks are not connected to each other thus I dont need to transfer boundary information from block 1 to block 2. For this configuration, I want to solve flow on just block 1 and some scalar on block 2. I decomposed my domain such that block 1 lies on processor 0 and block 2 lies on processor 1. So basically, in my problem, I would like to solve for Navier-Stokes only on processor 0, and on processor 1 I would like to solve some scalar equation.
Now, when I am trying to do this in OpenFOAM, it doesn't let me solve for N-S equation only on processor 0. Similarly, it doesn't allow me to solve for some scalar only on processor 1.
Any ideas on why this is happening and what can be done to get around this problem ?? I would appreciate any help on this!!

Thanks!
Regards,
Ankur
ankgupta8um is offline   Reply With Quote

Old   June 7, 2006, 14:49
Default You must excuse me, I'm a bit
  #12
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 4,225
Rep Power: 51
gschaider will become famous soon enoughgschaider will become famous soon enough
You must excuse me, I'm a bit slow today: Why would anyone want to solve two different equations in one run if they are not coupled?

My suggestion:
1. generate two cases, each with a different grid (== block1 & 2). You can do that
a. by using your preprocessor of choice
b. applying the cellSet and the subsetMesh-utilities to your mesh
2. run the right solver on each case
a. an NS-solver (like simpleFoam) on case1
b. another solver (laplacianFoam etc) on case2

Advantages are:
- no programming has to be done
- if you start simpleFoam on proc0 and laplacianFoam on proc1 the parallelization efficiency is 100% (because they are completly independent)
- you don't have to use the same set of field variables for the two simulations
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   June 7, 2006, 16:00
Default Hi Bernhard, The two blocks
  #13
Member
 
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 17
ankgupta8um is on a distinguished road
Hi Bernhard,

The two blocks I have in my domain are exactly similar. The reason I am going for such configuration is due to the different boundary conditions for flow and scalar. For flow I need to have cyclic boundary, while for scalar I require fixedValue and zero gradient boundaries. The base patch type for the boundary can either be cyclic or of "patch" type which can later be specified to be of "fixedValue" type while specifying boundary conditions.
Thats why I have two blocks solving two different equations. I hope I made myself clear.
Thanks Bernhard for keeping with me

Regards,
Ankur
ankgupta8um is offline   Reply With Quote

Old   June 7, 2006, 17:15
Default OK. I think I get it: you have
  #14
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 4,225
Rep Power: 51
gschaider will become famous soon enoughgschaider will become famous soon enough
OK. I think I get it: you have a cyclic flow field which transports a scalar which can't be transported across the cyclic boundary (you'll have your reason for that).

If the flow is stationary and not influenced by the scalar I have a solution:
1. Solve the flow field
2. Create a new case where you modify the base patch type
3. Copy the last U file there
4. Use scalarTransportFoam to calculate the propagation of the scalar

If this can't be done for your problem try to bait somebody with more knowledge about the internals with a new thread (title "Mixed base types for patches" or similar). I'm afraid I can't help you there.
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   June 7, 2006, 19:53
Default Hi Bernhard, I am doing LES
  #15
Member
 
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 17
ankgupta8um is on a distinguished road
Hi Bernhard,

I am doing LES so stationary flow is out of picture.
I already inquired about the possibility of implementating a "mixed base type" boundary and I came to know that it is highly non-trivial. So, I am looking for some workaround where I can model two blocks - one for flow and other for scalar.

Thanks again for all your inputs. Presently I am looking at the possibility of having two meshes.

Regards,
Ankur
ankgupta8um is offline   Reply With Quote

Old   June 8, 2006, 06:31
Default Here is what you do: Create
  #16
Senior Member
 
Eugene de Villiers
Join Date: Mar 2009
Posts: 725
Rep Power: 21
eugene is on a distinguished road
Here is what you do:

Create two identical meshes, say polyMesh1 and polyMesh2 except that mesh2 does not have cyclic boundaries.

On mesh1, solve your LES equations.
After each LES solver step, set the U.internalField() field on mesh2 equal to that on mesh1. You can create the mesh2 U field with all boundaries zeroGradient.

Now solve your scalar transport equations using U2.

This should work fine in parallel as well.
eugene is offline   Reply With Quote

Old   June 8, 2006, 09:38
Default to Bernhard, there is not dif
  #17
New Member
 
Carlo Fabrizi
Join Date: Mar 2009
Location: Rome, Italy
Posts: 5
Rep Power: 17
carlo_fabrizi is on a distinguished road
to Bernhard,
there is not difference of a source terms, the variables are the same, but in one block I resolve the simpleFoam equation in a inertial frame of reference, in the other I resolve it in a rotating frame of reference so with the the coriolis and centrifugal forces. Both write the solution in the inertial frame.
Have you any ideas?
Thanks!
carlo_fabrizi is offline   Reply With Quote

Old   June 8, 2006, 15:28
Default Hi Eugene/Mattijs, I checke
  #18
Member
 
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 17
ankgupta8um is on a distinguished road
Hi Eugene/Mattijs,

I checked the two mesh solver (SimpleFoam-twoMeshes) that Mattijs wrote about a year ago. It is working fine in parallel only when I include transportProperties and turbulenceProperties dictionary in processor?/constant/ directory. I was also required to include fvSchemes and fvSolutions in processor?/system/ directory.
For a parallel run with just one mesh, it is not required to place the above mentioned dictionaries in processor?/constant/ and processor?/system/ directories. Then why is it required for two-mesh domains??

When I didn't include the dictionaries, I came across errors like this:
----------------------------------------
--> FOAM FATAL IO ERROR : cannot open file

file: /home6/axg330/OpenFOAM/axg330-1.2/run/tutorials/simpleFoam/pitzDaily_test/proces sor0/constant/region1/transportProperties at line 0.
-------------------------------------------
So, what I inferred from the above error is that I need to place transportProperties in processor?/constant/region?/ directory, where region1 and region2 are my two meshes. When I placed 'transportProperties' in processor?/constant/region?/ directory, it worked fine. Later I tested a case where instead of placing it in processor?/constant/region?/ directory, I just placed it in processor?/constant/ directory and it worked fine in this case too.
So, how does OpenFoam work while looking for dictionaries ?? Does it first check constant/ directory and then /constant/region?/ directory??

Waiting eagerly for your valuable response(s).

Thanks!
Regards,
Ankur
ankgupta8um is offline   Reply With Quote

Old   June 8, 2006, 15:34
Default Hi Carlo! To me the Corioli
  #19
Assistant Moderator
 
Bernhard Gschaider
Join Date: Mar 2009
Posts: 4,225
Rep Power: 51
gschaider will become famous soon enoughgschaider will become famous soon enough
Hi Carlo!

To me the Coriolis force like it is implemented in http://www.cfd-online.com/OpenFOAM_D...ges/1/770.html is a source term. I'm not sure whether setting Omega to zero is a wise idea, but maybe the icoTopoFoam-case mentioned in http://www.cfd-online.com/OpenFOAM_D...es/1/1899.html might be of more help.
__________________
Note: I don't use "Friend"-feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request
gschaider is offline   Reply With Quote

Old   June 13, 2006, 05:57
Default Hi Ankur, the multi-region
  #20
Senior Member
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26
mattijs is on a distinguished road
Hi Ankur,

the multi-region handling is probably not perfect for dictionaries. In sinlge-region sims the 0 region is still there and called 'region0' but this name gets filtered out when reading/writing.

In similar situations I tend to just create a bunch of softlinks so I least I have to maintain only one version of the dictionaries.
mattijs is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Ignition fails in parallel run combustion solvers msha OpenFOAM Bugs 17 January 17, 2009 04:49
Understanding parallel solvers milkovp OpenFOAM Running, Solving & CFD 3 May 20, 2008 18:07
Parallel direct solvers khurram Main CFD Forum 0 January 17, 2007 17:45
Exporting problem from parallel solvers Kong FLUENT 6 May 4, 2005 12:55
Parallel solvers srik Main CFD Forum 3 October 17, 2003 03:46


All times are GMT -4. The time now is 16:25.