CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

OpenFOAM on IBM BlueGeneL

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   December 25, 2008, 23:51
Default Hi all, I am trying to buil
  #1
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Hi all,

I am trying to build OpenFOAM on the BlueGene. Unfortunately I hear that the BlueGene/L does not support shared libraries. My question is whether I can build OpenFOAM using static libraries (i.e. lib instead of libso). Has anyone implemented this successfully even on a GNU/Linux system?
msrinath80 is offline   Reply With Quote

Old   December 26, 2008, 07:01
Default Hi Srinath That's going to
  #2
Member
 
Michael Wild
Join Date: Mar 2009
Location: Bern, Switzerland
Posts: 79
Rep Power: 9
mwild is on a distinguished road
Hi Srinath

That's going to be a pretty tough one if that really is the case... Problem is that OpenFOAM pretty much relies on dynamically loading shared libraries (e.g. the grid motion solvers, the new forces library and the GAMG linear solvers are examples). To get around that, you would need to do some serious surgery and make sure that you link all required libraries at link-time and not at run-time as is the case now. Further you need to fix the build system to statically link the correct libPstream.a library (currently this is done by setting LD_LIBRARY_PATH, which you can't use for obvious reasons).

Here's what I would try to do:

- Fix up the wmake build system (i.e. replace the "libso" calls in the Allwmake scripts by "lib" (I hope I got the names right...)

- Make sure that the correct Pstream library gets compiled and linked. For safety, disable the compiling of the other implementations!

- In src/OpenFOAM/dlLibraryTable/ remove all the #include's for <dlfcn.h> and replace the definition of "bool Foam::dlLibraryTable::open(const fileName& functionLibName)" by a dummy implementation which fails with a FatalErrorIn and a very distinctive message, such that you know what is going on. Also replace "Foam::dlLibraryTable::~dlLibraryTable()" such that it doesn't call "dlclose" anymore.

- Compile the thing on a "normal" computer and try to link the code statically. You will probably get some undefined references along the way, which you have to fix by adding the appropriate libraries to the relevant Make/options files.

- Once you managed to compile, you will need to make sure that the programs you're going to need on BlueGene are running. For this, just start them up. If they fail with unresolved symbol errors, go back to the last step by adding required libraries, and re-linking.

- When you run the programs and see that FatalErrorIn you inserted in the dlLibraryTable class, figure out where the code is that called that "open" function and fix it. Also add the required library to the list of link-libraries.


I hope this helps.

Michael
mwild is offline   Reply With Quote

Old   December 27, 2008, 03:04
Default Thanks Michael. That sounds re
  #3
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Thanks Michael. That sounds really complicated though. Life would have been a lot simpler if IBM used a GNU/Linux based kernel on the BlueGene compute nodes.

I'll keep you posted on any developments.
msrinath80 is offline   Reply With Quote

Old   December 29, 2008, 14:08
Default Hi Srinath, What OS is the
  #4
New Member
 
Shivasubramanian Gopalakrishnan
Join Date: Mar 2009
Location: Amherst, Massachusetts, USA
Posts: 15
Rep Power: 9
shivasub is on a distinguished road
Hi Srinath,

What OS is the BlueGene/L machine running?

A while ago I had compiled OpenFOAM on an IBM AIX machine. I compiled both static and dynamic libraries there.

For the static version I compiled each library by hand using wmake lib, as Michael suggests, Subsequently compiling only apps and utils which I needed.

The main issue I had with IBM's was that I had to use the native linker rather than the GNU linker.

you have to pass additional linker flags.

LINKLIBSO = $(CC) $(c++FLAGS) -shared -Wl,-G -Wl,-bbigtoc -Wl,-brtl -Wl,-bnoentry

in wmake/rules/$ARCH .

This is on the AIX, if the BlueGene/L is running AIX

Hope this helps.

-Shiva
shivasub is offline   Reply With Quote

Old   December 29, 2008, 16:00
Default Hi Shiva, Thanks for your i
  #5
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Hi Shiva,

Thanks for your insights. I believe that the BlueGene/L head node runs Suse Enterprise Linux 9.x, while the actual compute (C) nodes run a stripped down version of AIX. We therefore cross compile the code on the head node using blrts_xl* compilers which are variants of the xl* compilers that are specifically built for the BlueGene.
msrinath80 is offline   Reply With Quote

Old   December 29, 2008, 16:46
Default Hi Srinath, Have you had an
  #6
New Member
 
Shivasubramanian Gopalakrishnan
Join Date: Mar 2009
Location: Amherst, Massachusetts, USA
Posts: 15
Rep Power: 9
shivasub is on a distinguished road
Hi Srinath,

Have you had any success with the XL compiler?

My solution was to use gcc with native linker. My attempt at compiling with XL threw up so many errors that it was not appetizing to go down that road.


-Shiva
shivasub is offline   Reply With Quote

Old   December 30, 2008, 01:59
Default Not really. I plan to use the
  #7
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Not really. I plan to use the GNU ones that were ported to the BlueGene by IBM. However, since the GNU compiler versions are 3.x at best, I am already anticipating issues with the build!
msrinath80 is offline   Reply With Quote

Old   December 30, 2008, 12:21
Default I had a similar issue where th
  #8
New Member
 
Shivasubramanian Gopalakrishnan
Join Date: Mar 2009
Location: Amherst, Massachusetts, USA
Posts: 15
Rep Power: 9
shivasub is on a distinguished road
I had a similar issue where the system gcc was 3.x. Compiled a 4.2 (I think or maybe 4.1.2) using 3.x and used the personal install.

You will need a custom wmake/rules for all the additional linker flags you need to pass to compile foam with native linker. I have attached the version used. I think I have the IBM manual somewhere, will dig that up and try to send that.

This is the rules tar file
shivasub is offline   Reply With Quote

Old   December 30, 2008, 12:24
Default okay sorry about the attachmen
  #9
New Member
 
Shivasubramanian Gopalakrishnan
Join Date: Mar 2009
Location: Amherst, Massachusetts, USA
Posts: 15
Rep Power: 9
shivasub is on a distinguished road
okay sorry about the attachment, its larger than the file size limit. Let me know if you need it, will email it to you.

regards
Shiva
shivasub is offline   Reply With Quote

Old   December 31, 2008, 23:04
Default Looks like I am going to need
  #10
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Looks like I am going to need this. I'd appreciate if you could email it to me. My email address is in my profile. Thanks.
msrinath80 is offline   Reply With Quote

Old   March 6, 2009, 14:54
Default Hello there, Thanks for all
  #11
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Hello there,

Thanks for all your help. I have given up on the BlueGene/L for now. This (cross-compiling with static libs) is the longest and most confusing maze I have ever wandered through. If anyone does manage to get OpenFOAM working on the BlueGene/L, please do post the instructions here.

Thanks,
Srinath
msrinath80 is offline   Reply With Quote

Old   March 6, 2009, 17:00
Default I think I will get my hands on
  #12
Senior Member
 
Hrvoje Jasak
Join Date: Mar 2009
Location: London, England
Posts: 1,783
Rep Power: 22
hjasak will become famous soon enough
I think I will get my hands onto this problem soon - will keep you posted.

Hrv
__________________
Hrvoje Jasak
Providing commercial FOAM/OpenFOAM and CFD Consulting: http://wikki.co.uk
hjasak is offline   Reply With Quote

Old   March 6, 2009, 17:06
Default Hello Srinath Did you manag
  #13
New Member
 
Shivasubramanian Gopalakrishnan
Join Date: Mar 2009
Location: Amherst, Massachusetts, USA
Posts: 15
Rep Power: 9
shivasub is on a distinguished road
Hello Srinath

Did you manage to compile 4.1.2 or higher Gcc for the AIX on the bluegene? Is it absolutely necessary to use static libs there?

As I mentioned earlier, I had compiled OpenFOAM on IBM SP4 machine running AIX. I do not have access to a Bluegene to give it a shot.

Best
-Shiva
shivasub is offline   Reply With Quote

Old   March 7, 2009, 13:30
Default Hi Shiva and Hrv, Thanks fo
  #14
Senior Member
 
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 13
msrinath80 is on a distinguished road
Hi Shiva and Hrv,

Thanks for your responses. No, I was unable to build gcc 4.x for the BlueGene as it required me to port some IBM patches for gcc 3.x to 4.x which isn't trivial (for me). As a result, I downgraded to OpenFOAM 1.1 as it was built at that time using gcc 3.x. However, I was unable to progress further because of lots of compiler issues afterward.

I will await Hrv's instructions. I will have access to this BlueGene most likely only till July.
msrinath80 is offline   Reply With Quote

Old   November 2, 2009, 19:46
Default
  #15
Member
 
matteo lombardi
Join Date: Apr 2009
Posts: 67
Rep Power: 9
matteoL is on a distinguished road
Hello,
I am thinkng to compile Openfoam on Bluegene/P as well, anyone has had any luck so far?
Any sugegstions where I should start?
Are you sure that bluegene doesn't support dinamically linked libraries?It seemed to me it could..(altough I must admit I haven't really looked into it..)

Thanks,
best regards,
matteo
matteoL is offline   Reply With Quote

Old   May 20, 2011, 19:38
Default
  #16
Member
 
Yong Wang
Join Date: Apr 2009
Posts: 34
Rep Power: 9
ywang is on a distinguished road
I am also working on the installation of OF on BlueGene/P......I am very confused.....
ywang is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
OpenFoam vs CFX5 mass balance in OpenFoam tangd OpenFOAM Running, Solving & CFD 33 May 23, 2010 16:36
CheckMesh error using a tutorial from OpenFOAM 114 with openFOAM 13 martapajon OpenFOAM Native Meshers: blockMesh 7 January 21, 2008 13:52
OpenFOAM users in Munich OpenFOAM benutzer in M%c3%bcnchen jaswi OpenFOAM 0 August 3, 2007 13:11
Anybody try GCC 420 on OpenFOAM yet connclark OpenFOAM Installation 1 July 26, 2007 17:14
A new Howto on the OpenFOAM Wiki Compiling OpenFOAM under Unix mbeaudoin OpenFOAM Installation 2 April 28, 2006 08:54


All times are GMT -4. The time now is 01:15.