CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Programming & Development

Anyone succesful on static executable solver?

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   September 14, 2011, 11:58
Default Anyone succesful on static executable solver?
  #1
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Hi there,

I have found some stuff on the forum and also using google.
But so far I couldn't see any proper description of how to get a static executable...

Did anyone have any success?
I am just thinking it would be generally very nice to have one single executable even if it is some mbs large.
I am interested in that in order to run parallel computation on a cluster where the nodes have no OpenFOAM package installed.

Best regards,

sebastian
seboxx is offline   Reply With Quote

Old   September 15, 2011, 04:22
Default
  #2
Senior Member
 
Eugene de Villiers
Join Date: Mar 2009
Posts: 725
Rep Power: 12
eugene is on a distinguished road
So far the best results we have had is with a tool called "Magic Ermine". There are still some issues with MPI though and it is not free.

Unfortunately, there doesn't seem to be an easy way to do this.
eugene is offline   Reply With Quote

Old   September 15, 2011, 08:08
Default
  #3
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Greetings to all!

@Sebastian: I've been accompanying your posts on another thread, namely Compiling with exe option. It's been a year since my last posts on this subject.
OpenFOAM 2.0 has evolved in this direction to provide a quicker way to create static binaries, but it's only ready half way through. My experience on this subject has also increased a bit, but for now I'm only able to provide an incomplete hack: https://github.com/wyldckat/OpenFOAM...staticBuilding

Basically apply that patch and then run AllwmakeStatic instead of Allwmake.

Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   September 16, 2011, 05:59
Default
  #4
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Hi Bruno,

thank you for your quick reply.
I see that it is still not trivial to get static binaries...
I will have a look at your hack. It seems to be messing with the whole OpenFOAM installation doesn't it? So I better prepare a setup that is independant from my current workspace.
seboxx is offline   Reply With Quote

Old   September 16, 2011, 19:07
Default
  #5
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Sebastian,

I've tried to keep the hack as contained as possible to keep the dynamic version untainted, but I haven't tested if the dynamic version still builds as intended. So the answer to your question is: yes, you better keep this hacked version on a different work folder.

I've just updated my hacking on the branch I've posted before: https://github.com/wyldckat/OpenFOAM...staticBuilding

It's still a somewhat of an ugly hack, but now all applications build statically. I haven't tested executing them to confirm if they work as intended. They all link well, so I expect that they work!
If some of the binaries don't work, then the only other way is to build first the dynamic version and then the static version, based on the library link list provided by the dynamic version.

Let me know if this is working as intended or not, since I don't know when I'll get the chance to do some testing of my own.

I failed to mention on the previous post that this hack implies that both your cluster and workstation already have libScotch and a system MPI already installed and properly configured for building with OpenFOAM; therefore, these two libraries, along with the other system libraries, will be the only ones that are dynamically linked to the static OpenFOAM binaries.

Now, all that is left, is to:
  • Give this ugly hack a plastic operation...
  • Test, test, test...
  • Do a bit of benchmarking, just for the sake of figuring out how much additional performance can be pulled from this.
Best regards,
Bruno
wyldckat is offline   Reply With Quote

Old   September 18, 2011, 16:36
Default
  #6
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Hi Bruno,

I really appreciate your comments.
One thing I am considering at the moment...
I want to run my solver on a cluster, managed by condor, that has (up to now) no OpenFOAM package installed on the nodes.

What I am looking for is the easiest way to run my simulations on that cluster.

You have posted in another post:
Compiling with exe option
that the easiest way might be to ship the solver binary along with the dependant libraries found by
Code:
ldd mySolver
to the node.

I have done exactly that and also shipped the libfiniteVolume.so but I get this in the error logs:

Code:
error while loading shared libraries: libfiniteVolume.so: cannot open shared object file: No such file or directory
Can you point me in a direction to get this solved? Is the problem some environment variables?

I feel a bit sorry for not having tested your hack so far. But working on an actual master thesis, with a time-limit, and where the physics is in the foreground makes me look for easy solutions w.r.t. to the computations

Actually I don't need to decompose my domain to different cpus, I need to run a system on reasonably simple geometries with a lot of different sets of parameters. So one stand-alone simulation on a node would do...

If I get it to work on the condor cluster without having OpenFOAM installed to all the nodes, I promise to participate in writing a wiki article if desired

Sebastian
seboxx is offline   Reply With Quote

Old   September 19, 2011, 03:22
Default
  #7
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Sebastian,

OK, here's an example:
  1. You must build OpenFOAM with the system's MPI, not with the custom OpenMPI, otherwise this will fail big time!
  2. Go to the case you want to run; e.g.:
    Code:
    cd $FOAM_RUN/tutorials/incompressible/icoFoam/cavityGrade
  3. Copy the necessary binary; e.g. blockMesh:
    Code:
    cp `which blockMesh` .
  4. Copy the necessary libraries for running the binary; e.g. blockMesh:
    Code:
    ldd `which blockMesh` | grep OpenFOAM | grep -v libmpi | sed -e 's=.*\('$WM_PROJECT_INST_DIR'.*\) .*=\1=' | xargs -I {} cp {} .
  5. Copy OpenFOAM's etc folder:
    Code:
    cp -r $WM_PROJECT_DIR/etc .
  6. Now, somehow you must set in the job description for the cluster the following commands or analogous:
    Code:
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
    export WM_PROJECT_DIR=.
    These commands will add the relative current folder to the library search paths as well as defining the necessary variable for OpenFOAM to work.
I only tested these out on a single machine, where:
  • one terminal had a working OpenFOAM environment;
  • while the other terminal did not have a working OpenFOAM environment, only running the commands described in point 6.
As for having a working OpenFOAM environment only when I need it, see this blog post: Advanced tips for working with the OpenFOAM shell environment - point 3
edit: As for more information about running in parallel, here is a compilation of information on the subject: Notes about running OpenFOAM in parallel

As for Condor Cluster system, I've never used it and I don't have access to one, so I think this is as far as I can help you

Best regards and good luck!
Bruno

Last edited by wyldckat; September 19, 2011 at 03:25. Reason: see "edit:"
wyldckat is offline   Reply With Quote

Old   September 19, 2011, 11:28
Default
  #8
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Hey Bruno,

it is working nicely!
I am very happy!
It really runs nicely on all nodes of the condor cluster with just my local installation of OpenFOAM having recompiled it with the right MPI options...

You really helped me out a lot here and I will surely mention you in my Acknowledgement.
seboxx is offline   Reply With Quote

Old   September 19, 2011, 16:18
Default
  #9
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Sebastian,

Quote:
Originally Posted by seboxx View Post
You really helped me out a lot here and I will surely mention you in my Acknowledgement.
You're very welcome! I'm glad it worked as well as intended. And many thanks for the Acknowledgement

Although I feel a bit bad that I didn't manage to get around to this months ago.. oh well, can't do everything all at once .

Ah, one final note: sometimes there is also a need to include in the job package the libraries that are summoned by the controlDict for the case you're running! Such an example would be the somewhat famous libforces.so library.

Best regards and good luck!
Bruno
wyldckat is offline   Reply With Quote

Old   September 28, 2011, 10:08
Default
  #10
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Hi Bruno,

I am running into another Problem.
It used to work once, but now it seems like it doesnt work anymore.
Executing everything locally works as good as before. But on the cluster it stops after having executed the first time step each run...

What I get is the following error message:
Quote:
--> FOAM FATAL IO ERROR:
wrong token type - expected Scalar, found on line 3 the word 'nan'

file: /users/boie/simulations/flat_slope0.00005beta0.06/system/data::solverPerformance::u at line 3.

From function operator>>(Istream&, Scalar&)
in file lnInclude/Scalar.C at line 91.

FOAM exiting
and this in the log files
Time = 0.0001

GAMG: Solving for u, Initial residual = 0.000118434, Final residual = 1.68319e-16, No Iterations 1
diagonal: Solving for v, Initial residual = 0, Final residual = 0, No Iterations 0
GAMG: Solving for u, Initial residual = 1, Final residual = 1.06043e-17, No Iterations 1
diagonal: Solving for v, Initial residual = 0, Final residual = 0, No Iterations 0
wavesize= 0ExecutionTime = 0.22 s ClockTime = 0 s

Time = 0.0002

GAMG: Solving for u, Initial residual = 1, Final residual = nan, No Iterations 1000
diagonal: Solving for v, Initial residual = 0, Final residual = 0, No Iterations 0
GAMG: Solving for u, Initial residual = nan, Final residual = nan, No Iterations 1000

I find surprising that the Final residual in the second timestep is nan...

As mentioned this problem only occurs when running on cluster.
Any ideas?

EDIT:

I have done a bit more testing. And strangely I get different results when running locally and on the cluster. I have executed one small time step and printed out the results, once from cluster calculation and once locally.
When running locally my scalarField has mostly values of the order e-100 over the whole domain, which is reasonable since I only have a small fraction of the field not equal to zero and have a diffusion process involved. So this seems fine...

However once I compute it on the cluster I get a value of -1.80998e+97 on each grid point, even the same value. No wonder my calculations diverge immediately. Only I cant see why...

EDIT2:

I have also tried to run the solver on another local machine without openfoam, the result is similar. So I think it is some problem that already appears when compiling the solver... it is executable on any machine but doesnt give any decent results

EDIT3:

I found the root of the problem. I was declaring a scalar with double S;
and assigning a value and later using it in one of the equations. This seems not to be appropriate way of coding in OpenFOAM. Using a class out of the openfoam package instead of a simple variable declaration solved the problem! I dont know why it wouldnt handle the double properly... but anyway its working ;-)
Sorry for all the editing and posting ;-)

Last edited by seboxx; September 28, 2011 at 16:43.
seboxx is offline   Reply With Quote

Old   October 1, 2011, 11:58
Default
  #11
Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 8,318
Blog Entries: 34
Rep Power: 84
wyldckat is just really nicewyldckat is just really nicewyldckat is just really nicewyldckat is just really nice
Hi Sebastian,

Sorry for the late reply, but I'm really glad that you managed to sort things out on your own! And I do prefer that people edit their posts, instead of making new ones... unless it's really worth the a new post

Since you were seeing "nan" in results, I was going to suggest that you would check the following variables defined in OpenFOAM's "etc/bashrc":
Code:
#- Floating-point signal handling:
#    set or unset
export FOAM_SIGFPE=

#- memory initialisation:
#    set or unset
#export FOAM_SETNAN=
When these are defined in the environment, they do what the comments talk about. When they are not defined, which is the situation for your when running in the cluster, these would be the reasons why things were different! Very different indeed!


But that "double S" declaration was really a bad idea OpenFOAM likes and demands that things are done the right way, or else... there will be consequences!

Best regards and good luck!
Bruno
wyldckat is offline   Reply With Quote

Old   August 4, 2012, 20:06
Default
  #12
Member
 
HD
Join Date: Jul 2011
Posts: 56
Rep Power: 6
Rebecca513 is on a distinguished road
Hello,

I am trying to run OF on a cluster, and ran into the same error messages. I wonder whether you have solved the issue.

Thank you~

Best,

Hang
Rebecca513 is offline   Reply With Quote

Old   August 5, 2012, 05:17
Default
  #13
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Hey Hang,

yes using a cluster without openfoam on the individual nodes eventually worked for me. What error message do you get?

best,

seboxx
seboxx is offline   Reply With Quote

Old   August 5, 2012, 06:14
Default
  #14
Member
 
HD
Join Date: Jul 2011
Posts: 56
Rep Power: 6
Rebecca513 is on a distinguished road
Hi seboxx,

The error message I got is very similar to the one you posted:


--> FOAM FATAL IO ERROR:
wrong token type - expected Scalar, found on line 3 the word 'nan'

file: /scratch/gpfs/hangdeng/FOAM_Run/fracAfter/test2/system/data::solverPerform
ance: at line 3.

From function operator>>(Istream&, Scalar&)
in file lnInclude/Scalar.C at line 91.

FOAM exiting

Thank you~

Best,
Hang
Rebecca513 is offline   Reply With Quote

Old   August 5, 2012, 14:06
Default
  #15
Member
 
Join Date: Aug 2011
Posts: 33
Rep Power: 5
seboxx is on a distinguished road
Well for me the problem was a poorly implemented additional function.

I don't know where your calculations go wrong. Did you implement functions to the solver? If so maybe try to comment them out and see whether the same thing happens.

Does the simulation produce good results when you run them on a single machine?
seboxx is offline   Reply With Quote

Old   August 6, 2012, 23:18
Default
  #16
Member
 
HD
Join Date: Jul 2011
Posts: 56
Rep Power: 6
Rebecca513 is on a distinguished road
The solver actually works for one of the tutorial case (both parallel and non-parallel), but failed my mesh in both parallel and non-parallel cases.

So, I don't think it is because of the additional function. I guess there might be something wrong with my mesh, I will look into that~

Thank you~

Best,

Hang
Rebecca513 is offline   Reply With Quote

Old   May 9, 2013, 10:39
Default
  #17
New Member
 
Join Date: May 2013
Posts: 3
Rep Power: 4
mr-albert is on a distinguished road
@ Rebecca
Hi how are you?
i have problem similar yours

this is my Error
--> FOAM FATAL IO ERROR:
wrong token type - expected Scalar, found on line 3 the word 'nan'

file: /home/mohammadreza/Desktop/periodicSolitary/system/data::solverPerformance:corr at line 3.

From function operator>>(Istream&, Scalar&)
in file lnInclude/Scalar.C at line 91.

FOAM exiting


do you solve your problems?
could you help me?

king regard
mr-albert
mr-albert is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Creating New Solver: For particle-laden compressible jets sankarv OpenFOAM Running, Solving & CFD 17 December 3, 2014 20:41
Quarter Burner mesh with periosic condition SamCanuck FLUENT 2 August 31, 2011 11:34
Working directory via command line Luiz CFX 4 March 6, 2011 21:02
why the solver reject it? Anyone with experience? bearcat CFX 6 April 28, 2008 14:08
Error during Solver cfd guy CFX 4 May 8, 2001 06:04


All times are GMT -4. The time now is 20:00.