CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Running, Solving & CFD

trying floatTransfer=1

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   July 21, 2009, 04:42
Default trying floatTransfer=1
  #1
Senior Member
 
BastiL
Join Date: Mar 2009
Posts: 462
Rep Power: 10
bastil is on a distinguished road
Hi all,

I want to try to switch on floatTransfer for paralle calculations with 1.5-dev. I tried creating ~/.OpenFOAM-1.5-dev/controlDict and set it to 1 there - without effect.

Where does the local controlDict have to be placed?
~/.OpenFOAM-1.5-dev/etc/controlDict)
~/OpenFOAM/.OpenFOAM-1.5-dev/controlDict?
Is it dependent on Installation location?

Thanks.
bastil is offline   Reply With Quote

Old   July 21, 2009, 09:06
Default
  #2
Senior Member
 
Laurence R. McGlashan
Join Date: Mar 2009
Posts: 367
Rep Power: 13
l_r_mcglashan will become famous soon enough
The controlDict is in:

OpenFOAM-1.5-dev\etc
__________________
Laurence R. McGlashan :: Website
l_r_mcglashan is offline   Reply With Quote

Old   July 21, 2009, 09:22
Default
  #3
lin
Senior Member
 
Hua Zen
Join Date: Mar 2009
Posts: 106
Rep Power: 7
lin is on a distinguished road
~/OpenFOAM/OpenFOAM-dev/etc/controlDict
lin is offline   Reply With Quote

Old   July 21, 2009, 14:04
Default
  #4
Senior Member
 
BastiL
Join Date: Mar 2009
Posts: 462
Rep Power: 10
bastil is on a distinguished road
Thanks, I will try this out. Documentation of this is not very accurate.

Regards BastiL
bastil is offline   Reply With Quote

Old   July 22, 2009, 02:36
Default
  #5
Senior Member
 
Mark Olesen
Join Date: Mar 2009
Location: http://olesenm.github.io/
Posts: 774
Rep Power: 17
olesen will become famous soon enough
Quote:
Originally Posted by bastil View Post
I want to try to switch on floatTransfer for paralle calculations with 1.5-dev. I tried creating ~/.OpenFOAM-1.5-dev/controlDict and set it to 1 there - without effect.
From my (bad) experience, I would advise against using floatTransfer=1.
In some cases it is fine, but for a few particular problems (and perhaps particular decompositions thereof) I think rounding error issues become a problem. In my case, the solution ran well for 800-1000 iter or so and then the hit a very strange spike in the convergence behaviour.

Quote:
Where does the local controlDict have to be placed?
~/.OpenFOAM-1.5-dev/etc/controlDict
~/OpenFOAM/.OpenFOAM-1.5-dev/controlDict?
None of the above. The main controlDict is searched using the "~OpenFOAM/" text expansion semantics mentioned in the docs:

Code:
//- Search for @em name in the following hierarchy:
//  -# personal settings:
//    - ~/.OpenFOAM/\<VERSION\>/
//      <em>for version-specific files</em>
//    - ~/.OpenFOAM/
//      <em>for version-independent files</em>
//  -# site-wide settings:
//    - $WM_PROJECT_INST_DIR/site/\<VERSION\>
//      <em>for version-specific files</em>
//    - $WM_PROJECT_INST_DIR/site/
//      <em>for version-independent files</em>
//  -# shipped settings:
//    - $WM_PROJECT_DIR/etc/
You probably remember using something like ~/.OpenFOAM-<VERSION>, but this was for older versions.

Note: unlike other dictionaries, do not attempt to use #include or other directives within the controlDict. It is read before these mechanisms are available within OpenFOAM.

/mark
olesen is offline   Reply With Quote

Old   July 27, 2009, 02:53
Default
  #6
Senior Member
 
BastiL
Join Date: Mar 2009
Posts: 462
Rep Power: 10
bastil is on a distinguished road
Thanks Mark,

I found this code in src/OpenFOAM/include/OSspecific.H. Do I have to uncomment some of it? And where do I finally have to place my file to?
The reason why I would like to try floatTransfer is very poor scaling of OF on our Myrinet-Cluster. It seems there is nearly no speedup between 16 and 64 cores - ths behaviour is completely different to commerical codes.
bastil is offline   Reply With Quote

Old   July 27, 2009, 03:01
Default
  #7
Senior Member
 
Mark Olesen
Join Date: Mar 2009
Location: http://olesenm.github.io/
Posts: 774
Rep Power: 17
olesen will become famous soon enough
Quote:
Originally Posted by bastil View Post
I found this code in src/OpenFOAM/include/OSspecific.H. Do I have to uncomment some of it?
No.

Quote:
And where do I finally have to place my file to?
~/.OpenFOAM/1.5.x/etc/controlDict would be a good place.
Then set floatTransfer 1; in that file
olesen is offline   Reply With Quote

Old   July 27, 2009, 03:31
Default
  #8
Senior Member
 
BastiL
Join Date: Mar 2009
Posts: 462
Rep Power: 10
bastil is on a distinguished road
Thanks. However, that does not seem to be much quicker at all.... I have no idea:

16 Cores:
Time = 250

PBiCG: Solving for Ux, Initial residual = 0.000399256, Final residual = 3.3002e-05, No Iterations 2
PBiCG: Solving for Uy, Initial residual = 0.00462622, Final residual = 0.000422648, No Iterations 2
PBiCG: Solving for Uz, Initial residual = 0.00475788, Final residual = 0.000443842, No Iterations 1
GAMG: Solving for p, Initial residual = 0.0483258, Final residual = 0.000298896, No Iterations 3
time step continuity errors : sum local = 0.000588824, global = 1.06764e-05, cumulative = -0.000579853
PBiCG: Solving for epsilon, Initial residual = 4.90512e-05, Final residual = 1.28975e-07, No Iterations 1
bounding epsilon, min: -0.403354 max: 6.75487e+08 average: 41716.5
PBiCG: Solving for k, Initial residual = 5.70964e-05, Final residual = 2.91034e-07, No Iterations 1
ExecutionTime = 26228.3 s ClockTime = 26328 s

64 Cores:
Time = 250

PBiCG: Solving for Ux, Initial residual = 0.000395327, Final residual = 1.16597e-05, No Iterations 3
PBiCG: Solving for Uy, Initial residual = 0.00493349, Final residual = 0.000152124, No Iterations 3
PBiCG: Solving for Uz, Initial residual = 0.00514608, Final residual = 0.000143217, No Iterations 3
GAMG: Solving for p, Initial residual = 0.0501587, Final residual = 0.000316504, No Iterations 4
time step continuity errors : sum local = 0.000624036, global = -1.80558e-05, cumulative = -0.00034109
PBiCG: Solving for epsilon, Initial residual = 7.91971e-05, Final residual = 6.22651e-07, No Iterations 1
bounding epsilon, min: -0.42092 max: 1.81212e+09 average: 49460.6
PBiCG: Solving for k, Initial residual = 9.47916e-05, Final residual = 1.49895e-06, No Iterations 1
ExecutionTime = 24337.1 s ClockTime = 24453 s

This is a Myrinet CLuster, I use OpenMPI 1.26 as with OF1.5 compiled with mx turned on and I use the --mca mtl Option. Ideas?
bastil is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 19:29.