
[Sponsors] 
June 6, 2006, 08:27 
Hi,
I would like to know if i

#1 
New Member
Carlo Fabrizi
Join Date: Mar 2009
Location: Rome, Italy
Posts: 5
Rep Power: 10 
Hi,
I would like to know if is possible to run a case in parallel with 2 different solvers for each block. Thanks 

June 6, 2006, 08:34 
sorry, the post can be misunde

#2 
New Member
Carlo Fabrizi
Join Date: Mar 2009
Location: Rome, Italy
Posts: 5
Rep Power: 10 
sorry, the post can be misunderstood..
I want to run in parallel a case and I would like to use for a block a simpleFoam solver and for the other a simpleFoam modified solver.. Is it possible? 

June 7, 2006, 00:51 
Hi,
This thread is most clo

#3 
Member
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 10 
Hi,
This thread is most closely related to the query I have, hence posting my question here: For a parallel run, what change needs to be done in order to solve for flow only on decomposed domain 0 and some other scalar (say, temperature) only on domain 1 ?? Obviously, domain 0 and domain 1 don't have any common boundary, thus eliminating the need for communication across processors while solving the equations. For clarity, the description of my model is as follows: My computational domain consists of two nonconnected blocks. After the domain decomposition, block 1 lies on processor 0 and block 2 lies on processor 1. I would like to solve for flow only on block 1 and some scalar equation only on block 2. It seems to me that in OpenFOAM, the communication between the processors takes place when the equation is getting solved. Now in my case, as there is no common boundary between the two processors, it is conceptually possible to solve for flow field only on one processor. But, the existing equation solvers doesn't allow me to do so. I guess, there is something like MPI_Wait_all statement at the end of the equationsolver which waits for all the processors to solve the equation before moving any further. Now, as I am not solving the equation on all the processors, it just hangs eventually killing my run. Please let me know if I am correct in my reasoning. Also, how can I solve for the equations on specific processor(s)?? Thanks! Regards, Ankur 

June 7, 2006, 02:43 
> I want to run in parallel a

#4 
New Member
Andrea Palazzi
Join Date: Mar 2009
Posts: 15
Rep Power: 10 
> I want to run in parallel a case and I would like to use for a block a simpleFoam solver and for the other a simpleFoam modified solver.. Is it possible?
Maybe you can split your domain in two or more zones and then write your own solver that uses different equations for different zones, so that you have your two solvers all in one program that can be run in parallel. BTW, how do you decompose your domain for parallel processing? I don't think that decomposePar gives you so much control on how to split your domain, am I wrong? Bye 

June 7, 2006, 02:48 
> My computational domain cons

#5 
New Member
Andrea Palazzi
Join Date: Mar 2009
Posts: 15
Rep Power: 10 
> My computational domain consists of two nonconnected blocks.
If your blocks are disconnected you can treat them as two different cases, can't you? Bye Andrea 

June 7, 2006, 03:34 
> decomposePar gives you so m

#6 
Super Moderator
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 18 
> decomposePar gives you so much control
'manual' option in decomposeParDict. Reads a file giving per cell what processor it should go to. 

June 7, 2006, 05:42 
Hi Andrea!
You said that it's

#7 
Senior Member
Francesco Del Citto
Join Date: Mar 2009
Location: Zürich Area, Switzerland
Posts: 220
Rep Power: 11 
Hi Andrea!
You said that it's possible to define two (or hopely more) different zones in my domain and use a different solvers for each zone, isn't it? Well, how does OpenFOAM let me do this? What about the "communication" over the interface between the zones? It' will be very useful for me writing an application with this feature... Francesco 

June 7, 2006, 07:40 
Well, I've stumbled in zones w

#8 
New Member
Andrea Palazzi
Join Date: Mar 2009
Posts: 15
Rep Power: 10 
Well, I've stumbled in zones when I was trying to use the dynamic mesh features in OF, you can take a look at the dynamic mesh library and examples.
I'm not sure that it fits you needings, but my idea is to write your own solver that does:  split of the domain into subdomains (zones), either by hardcoding the geometry into the solver or by other means (patches?)  solve the subdomains each with the appropriate solving equation; it must also handle the interface between zones. I've never looked deeply into the OF code and I'm not sure this is feasible, maybe some guru here have the right answer. Bye Andrea 

June 7, 2006, 09:35 
Hi,
> If your blocks are di

#9 
Member
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 10 
Hi,
> If your blocks are disconnected you can treat them as two different cases, can't you? I need to patch flow solution from one block to other block. Thats why I need to solve the two blocks simultaneously on different processros. The problem I am facing with this is  though there are no connecting boundaries between the two blocks, while solving the equations where I am just solving flow on block 1 on processor 0, the equation solver just doesn't let me do so. The equation solver requires the equation to be solved on all processors. Thats is exactly where I am stuck. Any ideas on how to make it possible to solve the equation on just one processor (in general, selected processors if the the domain decomposition is such that it doesn't require communication with other processors) ?? Thanks! Regards, Ankur 

June 7, 2006, 11:23 
At first: I think that running

#10 
Assistant Moderator
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,994
Rep Power: 42 
At first: I think that running different solvers on different domains in parallel would not be very efficient (if it was possible, which it isn't to my knowledge): 1. It is not very likely that both processors get equal load 2. Using more than one processor for one domain would be harder, hindering further parallelization.
@carlos initial request: how does the modified simpleFoamsolver differ from the original? If it's only by a source term you can apply the source term on the whole domain. Just multiply it with a field that is 1 wherever you want the source term to be active and 0 everywhere else (similar approaches for material properties and differential operators). @ankur: if I understand your problem correctly the best solution would be to write ONE solver that does the following 1. solve equation1 on region1 2. write boundary information from region1 to region2 3. solve equation2 on region2 Examples of such solvers exist on the message board (a solver on coupling two heat transport equations) or the Wiki (a crude fluidstruct coupling).
__________________
Note: I don't use "Friend"feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request 

June 7, 2006, 12:46 
Hi Bernhard,
I have just on

#11 
Member
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 10 
Hi Bernhard,
I have just one mesh in my computational domain, which consists of two blocks. The two blocks are not connected to each other thus I dont need to transfer boundary information from block 1 to block 2. For this configuration, I want to solve flow on just block 1 and some scalar on block 2. I decomposed my domain such that block 1 lies on processor 0 and block 2 lies on processor 1. So basically, in my problem, I would like to solve for NavierStokes only on processor 0, and on processor 1 I would like to solve some scalar equation. Now, when I am trying to do this in OpenFOAM, it doesn't let me solve for NS equation only on processor 0. Similarly, it doesn't allow me to solve for some scalar only on processor 1. Any ideas on why this is happening and what can be done to get around this problem ?? I would appreciate any help on this!! Thanks! Regards, Ankur 

June 7, 2006, 13:49 
You must excuse me, I'm a bit

#12 
Assistant Moderator
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,994
Rep Power: 42 
You must excuse me, I'm a bit slow today: Why would anyone want to solve two different equations in one run if they are not coupled?
My suggestion: 1. generate two cases, each with a different grid (== block1 & 2). You can do that a. by using your preprocessor of choice b. applying the cellSet and the subsetMeshutilities to your mesh 2. run the right solver on each case a. an NSsolver (like simpleFoam) on case1 b. another solver (laplacianFoam etc) on case2 Advantages are:  no programming has to be done  if you start simpleFoam on proc0 and laplacianFoam on proc1 the parallelization efficiency is 100% (because they are completly independent)  you don't have to use the same set of field variables for the two simulations
__________________
Note: I don't use "Friend"feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request 

June 7, 2006, 15:00 
Hi Bernhard,
The two blocks

#13 
Member
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 10 
Hi Bernhard,
The two blocks I have in my domain are exactly similar. The reason I am going for such configuration is due to the different boundary conditions for flow and scalar. For flow I need to have cyclic boundary, while for scalar I require fixedValue and zero gradient boundaries. The base patch type for the boundary can either be cyclic or of "patch" type which can later be specified to be of "fixedValue" type while specifying boundary conditions. Thats why I have two blocks solving two different equations. I hope I made myself clear. Thanks Bernhard for keeping with me Regards, Ankur 

June 7, 2006, 16:15 
OK. I think I get it: you have

#14 
Assistant Moderator
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,994
Rep Power: 42 
OK. I think I get it: you have a cyclic flow field which transports a scalar which can't be transported across the cyclic boundary (you'll have your reason for that).
If the flow is stationary and not influenced by the scalar I have a solution: 1. Solve the flow field 2. Create a new case where you modify the base patch type 3. Copy the last U file there 4. Use scalarTransportFoam to calculate the propagation of the scalar If this can't be done for your problem try to bait somebody with more knowledge about the internals with a new thread (title "Mixed base types for patches" or similar). I'm afraid I can't help you there.
__________________
Note: I don't use "Friend"feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request 

June 7, 2006, 18:53 
Hi Bernhard,
I am doing LES

#15 
Member
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 10 
Hi Bernhard,
I am doing LES so stationary flow is out of picture. I already inquired about the possibility of implementating a "mixed base type" boundary and I came to know that it is highly nontrivial. So, I am looking for some workaround where I can model two blocks  one for flow and other for scalar. Thanks again for all your inputs. Presently I am looking at the possibility of having two meshes. Regards, Ankur 

June 8, 2006, 05:31 
Here is what you do:
Create

#16 
Senior Member
Eugene de Villiers
Join Date: Mar 2009
Posts: 725
Rep Power: 14 
Here is what you do:
Create two identical meshes, say polyMesh1 and polyMesh2 except that mesh2 does not have cyclic boundaries. On mesh1, solve your LES equations. After each LES solver step, set the U.internalField() field on mesh2 equal to that on mesh1. You can create the mesh2 U field with all boundaries zeroGradient. Now solve your scalar transport equations using U2. This should work fine in parallel as well. 

June 8, 2006, 08:38 
to Bernhard,
there is not dif

#17 
New Member
Carlo Fabrizi
Join Date: Mar 2009
Location: Rome, Italy
Posts: 5
Rep Power: 10 
to Bernhard,
there is not difference of a source terms, the variables are the same, but in one block I resolve the simpleFoam equation in a inertial frame of reference, in the other I resolve it in a rotating frame of reference so with the the coriolis and centrifugal forces. Both write the solution in the inertial frame. Have you any ideas? Thanks! 

June 8, 2006, 14:28 
Hi Eugene/Mattijs,
I checke

#18 
Member
Ankur Gupta
Join Date: Mar 2009
Posts: 38
Rep Power: 10 
Hi Eugene/Mattijs,
I checked the two mesh solver (SimpleFoamtwoMeshes) that Mattijs wrote about a year ago. It is working fine in parallel only when I include transportProperties and turbulenceProperties dictionary in processor?/constant/ directory. I was also required to include fvSchemes and fvSolutions in processor?/system/ directory. For a parallel run with just one mesh, it is not required to place the above mentioned dictionaries in processor?/constant/ and processor?/system/ directories. Then why is it required for twomesh domains?? When I didn't include the dictionaries, I came across errors like this:  > FOAM FATAL IO ERROR : cannot open file file: /home6/axg330/OpenFOAM/axg3301.2/run/tutorials/simpleFoam/pitzDaily_test/proces sor0/constant/region1/transportProperties at line 0.  So, what I inferred from the above error is that I need to place transportProperties in processor?/constant/region?/ directory, where region1 and region2 are my two meshes. When I placed 'transportProperties' in processor?/constant/region?/ directory, it worked fine. Later I tested a case where instead of placing it in processor?/constant/region?/ directory, I just placed it in processor?/constant/ directory and it worked fine in this case too. So, how does OpenFoam work while looking for dictionaries ?? Does it first check constant/ directory and then /constant/region?/ directory?? Waiting eagerly for your valuable response(s). Thanks! Regards, Ankur 

June 8, 2006, 14:34 
Hi Carlo!
To me the Corioli

#19 
Assistant Moderator
Bernhard Gschaider
Join Date: Mar 2009
Posts: 3,994
Rep Power: 42 
Hi Carlo!
To me the Coriolis force like it is implemented in http://www.cfdonline.com/OpenFOAM_D...ges/1/770.html is a source term. I'm not sure whether setting Omega to zero is a wise idea, but maybe the icoTopoFoamcase mentioned in http://www.cfdonline.com/OpenFOAM_D...es/1/1899.html might be of more help.
__________________
Note: I don't use "Friend"feature on this forum out of principle. Ah. And by the way: I'm not on Facebook either. So don't be offended if I don't accept your invitation/friend request 

June 13, 2006, 04:57 
Hi Ankur,
the multiregion

#20 
Super Moderator
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 18 
Hi Ankur,
the multiregion handling is probably not perfect for dictionaries. In sinlgeregion sims the 0 region is still there and called 'region0' but this name gets filtered out when reading/writing. In similar situations I tend to just create a bunch of softlinks so I least I have to maintain only one version of the dictionaries. 

Thread Tools  
Display Modes  


Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
Ignition fails in parallel run combustion solvers  msha  OpenFOAM Bugs  17  January 17, 2009 04:49 
Understanding parallel solvers  milkovp  OpenFOAM Running, Solving & CFD  3  May 20, 2008 17:07 
Parallel direct solvers  khurram  Main CFD Forum  0  January 17, 2007 17:45 
Exporting problem from parallel solvers  Kong  FLUENT  6  May 4, 2005 11:55 
Parallel solvers  srik  Main CFD Forum  3  October 17, 2003 02:46 