CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM

Cannot run the code in parallel. Help please (urgent).

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By crst15

Reply
 
LinkBack Thread Tools Display Modes
Old   February 9, 2014, 07:10
Default Cannot run the code in parallel. Help please (urgent).
  #1
New Member
 
Steven
Join Date: Dec 2013
Location: Perth
Posts: 15
Rep Power: 3
crst15 is on a distinguished road
Hi all,

I can run my simulation in series, but I have a problem every time I run my simulation in parallel. I am using OpenFOAM 2.1.1 to simulate a data center cooling performance with k-epsilon model and buoyantBoussinesqSimpleFoam as solver. I always get this following error message:


--> FOAM FATAL IO ERROR:
wrong token type - expected Scalar, found on line 3 the punctuation token '-'

file: /............/processor9/system/data::solverPerformance::epsilon at line 3.

From function operator>>(Istream&, Scalar&)
in file lnInclude/Scalar.C at line 91.

FOAM parallel run exiting



I have tried to search for system/data::solverPerformance::epsilon and also lnInclude/Scalar.C but I couldn't find those files.

Everytime I run the code, I type the command as suggested in tutorial, i.e.

mpirun -np <number of processors> <solver> -parallel > log

Do you have any idea about what's going on? I will appreciate your help so much.


Thanks
crst15 is offline   Reply With Quote

Old   February 10, 2014, 09:43
Default
  #2
Member
 
Join Date: Jan 2014
Posts: 62
Rep Power: 3
hxaxtma is on a distinguished road
You are giving too less information to say something about your case. But this line gives a hint:

Quote:
wrong token type - expected Scalar, found on line 3 the punctuation token '-'

I guess there must be something wrong in your Boundary Conditions. Is the simulation running in serial?
hxaxtma is offline   Reply With Quote

Old   February 10, 2014, 09:56
Default
  #3
New Member
 
Steven
Join Date: Dec 2013
Location: Perth
Posts: 15
Rep Power: 3
crst15 is on a distinguished road
I found no problem when the case was run in serial. The problem occurred when the case was run in parallel.

Previously, I had run exactly the same code in my own laptop (OpenFOAM 2.2.2) and got no problem (so I think my boundary conditions are ok). Now, I have to run using a supercomputer with OpenFOAM 2.1.1 and the problem occurs.

Yesterday, I tried to play with the number of processors used. For some cases, I got no error message only if using 12 processors, and for other cases I got no error message only if using 48 processors. This is so strange. I totally have no idea about this phenomenon. Any clue will be appreciated..

Cheers
pruthvi1991 likes this.
crst15 is offline   Reply With Quote

Old   February 10, 2014, 10:35
Default
  #4
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,110
Rep Power: 19
alexeym will become famous soon enoughalexeym will become famous soon enough
Hi,

What type of decomposition do you use? Or just post your decomposeParDict.
alexeym is offline   Reply With Quote

Old   February 10, 2014, 10:48
Default
  #5
New Member
 
Steven
Join Date: Dec 2013
Location: Perth
Posts: 15
Rep Power: 3
crst15 is on a distinguished road
Hi,

I use exactly similar decomposition as suggested in the tutorial. Here is my decomposeParDict.

Cheers
Attached Files
File Type: txt decomposeParDict.txt (1.2 KB, 17 views)
crst15 is offline   Reply With Quote

Old   February 10, 2014, 11:00
Default
  #6
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,110
Rep Power: 19
alexeym will become famous soon enoughalexeym will become famous soon enough
Tutorial is just an example not a suggestion I'd suggest you using scotch decomposition method

I once had similar problem when due to decomposition rather small fragment of mesh become separate domain.
alexeym is offline   Reply With Quote

Old   February 10, 2014, 11:32
Default
  #7
Member
 
Join Date: Jan 2014
Posts: 62
Rep Power: 3
hxaxtma is on a distinguished road
are you using cyclic BCs?
hxaxtma is offline   Reply With Quote

Old   February 10, 2014, 12:27
Default
  #8
New Member
 
Steven
Join Date: Dec 2013
Location: Perth
Posts: 15
Rep Power: 3
crst15 is on a distinguished road
Hi all,

Thanks for your reply. I have tried scotch decomposition method and still doesn't work. I got the following error message:


--> FOAM FATAL IO ERROR:
wrong token type - expected Scalar, found on line 3 the word 'nan'

file: /.............../processor46/system/data::solverPerformance::epsilon at line 3.

From function operator>>(Istream&, Scalar&)
in file lnInclude/Scalar.C at line 91.


I put in bold the difference with the previous error message, i.e. the punctuation token '-' (previously) and the word 'nan' (currently). Referring to the user guide, the two entries inside "scotchCoeffs" are optional. Hence, I made the processorWeights equally distributed and the strategy as default (attached file). I don't know if this induces the error.

@hxaxtma: No, I don't use any cyclic. I just use wall and patch.


Regards,

Steven
Attached Files
File Type: txt decomposeParDict.txt (1.3 KB, 10 views)
crst15 is offline   Reply With Quote

Old   February 10, 2014, 12:49
Default
  #9
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,110
Rep Power: 19
alexeym will become famous soon enoughalexeym will become famous soon enough
Well,

We can continue to play in "guess what's wrong".

Basically error means that during decomposition something goes wrong with epsilon calculation so instead of a number in decomposed variant of epsilon file 'nan' or '-' is written.

0. Does this error appear on the first iteration of solver?
1. Do you use standard solver?
2. Do you use standard BCs? (once I had similar error when BC was implemented incorrectly)
3. What BC do you use for epsilon?
4. What IC do you use?

The easiest way to answer these question is surely to post case files.
alexeym is offline   Reply With Quote

Old   February 12, 2014, 07:40
Default
  #10
New Member
 
Steven
Join Date: Dec 2013
Location: Perth
Posts: 15
Rep Power: 3
crst15 is on a distinguished road
Hi,

Sorry for the late reply. To answer for alexeym's question:

0. Does this error appear on the first iteration of solver?
Ans: No, it doesn't appear on the first iteration. The simulation can run up to several time steps but with an exponentially growing time step continuity errors (until 10^100 or even more for sum local, global, and cumulative errors) before terminates.

1. Do you use standard solver?
Ans: I am using buoyantBoussinesqSimpleFoam. Not sure whether it's a standard solver or not. But it is also used in heatTransfer tutorials.

2. Do you use standard BCs? (once I had similar error when BC was implemented incorrectly)
Ans: I found no problem when running the case in series. The only problem I get is when running the case in parallel. Thus, I don't think the BC is incorrect. Even for parallel run, there is 1 "magic" number of processors which can give me no error, while the other numbers always give me error. So strange, right?
Before I run the case in supercomputer, I tried to run it in my laptop (OpenFOAM 2.2.2) using 4 processors and everything was ok. Then, I tried to run it in supercomputer (OpenFOAM 2.1.1) using also 4 processors and I got the error message. I don't know whether it's due to version difference or due to "improper" OpenFOAM installation at the supercomputer or another possible factor.

3. What BC do you use for epsilon?
Ans: I use epsilonWallFunction for all walls, turbulentMixingLengthDissipationRateInlet for all flows going to simulation domain, and zeroGradient for all flows leaving simulation domain.

4. What IC do you use?
Ans: What do you mean IC? Sorry, I'm still new to OpenFOAM .


I have also posted the case files .

Cheers
Attached Files
File Type: zip dataCenter.zip (12.8 KB, 10 views)
crst15 is offline   Reply With Quote

Old   February 12, 2014, 10:47
Default
  #11
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,110
Rep Power: 19
alexeym will become famous soon enoughalexeym will become famous soon enough
Well, though I needed to correct boundary conditions to run it on my desktop with OF 2.2.2, it was running ok (at least for 300 time steps).

If you'd like to check if OF installation on the supercomputer is correct - you can install 2.1.1 on your laptop and test if it will blow up. Or you can try to install 2.2.2 in your home folder on supercomputer

But I guess the problem can be in (this is a code from epsilonWallFunctionFvPatchScalarField.C)
Code:
        epsilon[faceCellI] = Cmu75*pow(k[faceCellI], 1.5)/(kappa_*y[faceI]);

        G[faceCellI] =
            (nutw[faceI] + nuw[faceI])
           *magGradUw[faceI]
           *Cmu25*sqrt(k[faceCellI])
           /(kappa_*y[faceI]);
I've checked yPlus values and the grows rather rapidly

Code:
...
Patch 5 named frontWall y+ : min: 5477.84 max: 140055 average: 53592.3

Patch 6 named rack1Wall y+ : min: 11612.2 max: 440140 average: 109040

Patch 7 named rack2Wall y+ : min: 23519.2 max: 297508 average: 103464
...
The difference between 2.1.x and 2.2.x is that when you divide something by very large value 2.1.x will return you nan while 2.2.x will return you 0 (IIRC).

And just to clarify - IC stands for initial conditions
alexeym is offline   Reply With Quote

Old   February 17, 2014, 02:02
Default
  #12
New Member
 
Anirudh
Join Date: Jan 2012
Posts: 4
Rep Power: 5
ani1990 is on a distinguished road
Hi Steven! (and everyone else in this discussion )

I'm trying to learn more about performing a CFD analysis for data centers specifically using OF. If any one could suggest some reading material which would help me get on my way I'd be really grateful!

Cheers!

P.S: I'm also a relative newbie
ani1990 is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Unable to run OpenFOAM 1.6-ext in parallel with more than one machine mm.abdollahzadeh OpenFOAM Installation 14 January 27, 2014 10:40
dynamicMesh parallel run popcorn OpenFOAM Running, Solving & CFD 0 October 2, 2012 12:34
SnappyHexMesh OF-1.6-ext crashes on a parallel run norman1981 OpenFOAM Bugs 5 December 7, 2011 13:48
Parallel Run on dynamically mounted partition braennstroem OpenFOAM Running, Solving & CFD 14 October 5, 2010 14:43
is there any parallel code for the famous Lid Driven Cavity flow? gholamghar Main CFD Forum 0 August 1, 2010 01:55


All times are GMT -4. The time now is 02:36.