CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Installation

[OpenFOAM.com] Multiple Installation Issue: Parallel Processing No Longer Works

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree1Likes
  • 1 Post By fertinaz

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 27, 2011, 22:33
Default Multiple Installation Issue: Parallel Processing No Longer Works
  #1
Senior Member
 
Daniel
Join Date: Jul 2009
Location: Montreal, Canada
Posts: 156
Rep Power: 17
dancfd is on a distinguished road
Hello all,

I have been using OF1.7.x for a number of cases, and when OF2 was released I eagerly installed it. Unfortunately, since I installed OF2, I can no longer run applications in parallel in OF1.7.x. Some more info:
-Parallel processing works fine in OF2
-There appears to be some kind of MPI problem (MPI installed with OF2 is newer than that installed with OF17x, did it delete the older version?)
-I have set up my .bashrc to call one of the two OF bashrc when I call the appropriate alias
-using Ubuntu 10.04
-I would like to continue using OF1.7.x for the cases that are in progress since it would take some effort to change the structure to be suitable for use in OF2.

-I get the following error:
Code:
--------------------------------------------------------------------------
Sorry!  You were supposed to get help about:
    opal_init:startup:internal-failure
But I couldn't open the help file:
    /home/dan/OpenFOAM/ThirdParty-1.7.x/platforms/linux64Gcc/openmpi-1.4.1/share/openmpi/help-opal-runtime.txt: No such file or directory.  Sorry!
--------------------------------------------------------------------------
[two-desktop:12987] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file ../../../orte/runtime/orte_init.c at line 77
[two-desktop:12987] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file ../../../../../orte/tools/orterun/orterun.c at line 541
Note that I re-ran foamInstallationTest, and I now have a critical failure:
Code:
-------------------------------------------------------------------------------
Environment_variable Set_to_file_or_directory                            Valid Path Crit
-------------------------------------------------------------------------------
$FOAM_LIBBIN         ...M/OpenFOAM-1.7.x/lib/linux64GccDPOpt     yes   yes   yes
$FOAM_SITE_LIBBIN    ...nFOAM/site/1.7.x/lib/linux64GccDPOpt     no            no
$FOAM_USER_LIBBIN    ...enFOAM/dan-1.7.0/lib/linux64GccDPOpt   yes  yes   no
$MPI_ARCH_PATH       ...x/platforms/linux64Gcc/openmpi-1.4.1      no           yes
-------------------------------------------------------------------------------
I would appreciate any help you could offer. I am not familiar with MPIs, but I have a feeling I need to make OF17x use the newer MPI. Just not sure how to do it.

Thanks,
Dan
dancfd is offline   Reply With Quote

Old   June 28, 2011, 20:01
Default
  #2
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Greetings Dan,

You forgot to mention how exactly do you differentiate between OpenFOAM versions!?
My guess is that your "$HOME/.bashrc" currently looks something like this:
Code:
source $HOME/OpenFOAM/OpenFOAM-1.7.x/etc/bashrc
source $HOME/OpenFOAM/OpenFOAM-2.0.x/etc/bashrc
This is a big no-no You should have something like this:
Code:
alias of17x='source $HOME/OpenFOAM/OpenFOAM-1.7.x/etc/bashrc'
alias of20x='source $HOME/OpenFOAM/OpenFOAM-2.0.x/etc/bashrc'
And whenever you start a new terminal, you must run the respective alias name. For example, to start up OpenFOAM 1.7.x environment, run:
Code:
of17x
OK, this only covers 50% of the problem. Now comes the other problem: whenever you launch a parallel run in OpenFOAM, you are probably simply using mpirun to launch it. Something like:
Code:
mpirun -np 4 simpleFoam -parallel
This only works because your $HOME/.bashrc file is sourced by all remote/parallel processes.
Now, since we are as of now relying on aliases, things will definitely not work, since there is no OpenFOAM environment available for them to use.

So, there are at least two solutions:
  • Run mpirun like this:
    Code:
    mpirun -np 4 `which foamExec` simpleFoam -parallel
    Or like this:
    Code:
    mpirun -np 4 $WM_PROJECT_DIR/bin/foamExec simpleFoam -parallel
  • Or use foamJob to launch the parallel run, like this:
    Code:
    foamJob -p -s simpleFoam
    Want to know more options or about these, run:
    Code:
    foamJob -help
Best regards,
Bruno
__________________
wyldckat is offline   Reply With Quote

Old   June 28, 2011, 22:29
Default
  #3
Senior Member
 
Daniel
Join Date: Jul 2009
Location: Montreal, Canada
Posts: 156
Rep Power: 17
dancfd is on a distinguished road
Hi Bruno,

Thank you for your detailed response. I have set up my .bashrc with aliases for each instance of OF, and I have only called one instance in each terminal window. I also run the solver in parallel using a different command - I use "runParallel simpleFoam 2" as is done in the "Allrun" files in the tutorials.

I tried using the "mpirun" command, and using foamJob, and I received the same error as above in both instances.

Any other ideas?

Thanks,
Dan
dancfd is offline   Reply With Quote

Old   July 3, 2011, 07:01
Default
  #4
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Hi Daniel,

I'm sorry, but so much got in the way that I forgot about your thread

The RunParallel function is only meant to be used in a single machine with a single installation, i.e. with the OpenFOAM environment activated automatically in ~/.bashrc.

This does look like something got damaged somewhere somehow, but further experimentation should be done.

OK, I've reviewed your first post and only now did something pop out to me, which you had already mentioned... so, to verify this, run with the 1.7.x environment active the following commands:
Code:
echo $WM_MPLIB
echo $MPI_ARCH_PATH
ls -l $MPI_ARCH_PATH
The last line should show you what exists in the installation folder of OpenMPI, if it exists at all.

If it doesn't find such folder, check if you have the folder "$WM_THIRD_PARTY_DIR/platforms/":
Code:
ls -l $WM_THIRD_PARTY_DIR/platforms/
And the contents therein. Check its sub-folders as well.

If you can't find something or nothing at all, then something got damaged along the way . Rebuilding OpenFOAM 1.7.x might be the safest bet.

Best regards and good luck!
Bruno
__________________
wyldckat is offline   Reply With Quote

Old   July 12, 2011, 22:37
Default
  #5
Senior Member
 
Daniel
Join Date: Jul 2009
Location: Montreal, Canada
Posts: 156
Rep Power: 17
dancfd is on a distinguished road
Hi Bruno,

Thanks again for your response. I tried a few things, but still no luck. As it turns out, making the cases compatible with OF2 was not as demanding as I had anticipated, so I am now using OF2 without any further difficulty. Thanks for your help,

Dan
dancfd is offline   Reply With Quote

Old   July 13, 2011, 17:03
Default
  #6
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Hi Daniel,

I'm glad you've got the hang of OF 2.0

And I know this is a bit uncalled for, but this phrase comes to mind: OpenFOAM 2.0 rules, 1.7 drools

Have fun!
Bruno
__________________
wyldckat is offline   Reply With Quote

Old   February 17, 2012, 05:21
Default Using multiple OF installations on one machine
  #7
Senior Member
 
Gerhard Holzinger
Join Date: Feb 2012
Location: Austria
Posts: 342
Rep Power: 28
GerhardHolzinger will become famous soon enoughGerhardHolzinger will become famous soon enough
Recently I installed OF2.0 and yesterday I installed OF 2.1.

As I was looking for information whether two different versions of OF can be run on a single machine I found this thread. The information above was very helpful.

I tried to alter the solution of wyldckat and it seems to work properly - but: no guarantee!

My ~/.bashrc looks like this:

Code:
source $HOME/OpenFOAM/OpenFOAM-2.1.x/etc/bashrc
alias of20x='source $HOME/OpenFOAM/OpenFOAM-2.0.x/etc/bashrc'
I appended these two lines. Sourcing the bashrc file of OF2.1 as if OF2.1 was the only version of OF on my machine.
The alias defined, sources the bashrc file of OF2.0. When I want to use OF2.0 I open a new terminal an run of20x as first command.

This way I managed to run the cavity tutorial, using domain decomposition and running icoFoam in parallel, using OF2.0 in one tab and OF2.1 in another tab of my Gnome-terminal.

Note: All versions were installed from sources using the git-repository.
GerhardHolzinger is offline   Reply With Quote

Old   February 17, 2012, 05:57
Default
  #8
Retired Super Moderator
 
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,981
Blog Entries: 45
Rep Power: 128
wyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to allwyldckat is a name known to all
Greetings Gerhard and welcome to the forum!

A word of warning: that method you are using only works because you have both installations of OpenFOAM on the same working area. If you were using 2.1.0 on "/opt" and 2.1.x on "$HOME/OpenFOAM", this would not work.

The advisable method is to use wmUNSET (available as of 2.0.0) to unload the current OpenFOAM version and then activate the proper version with the respective alias, such as of21x.

For more: Advanced tips for working with the OpenFOAM shell environment

Best regards,
Bruno
__________________
wyldckat is offline   Reply With Quote

Old   November 19, 2018, 18:54
Default
  #9
Member
 
Join Date: Sep 2010
Posts: 36
Rep Power: 16
anfho is on a distinguished road
I am getting the same error message as Dan above (with OpenFoam 2.4):

Code:
--------------------------------------------------------------------------
Sorry!  You were supposed to get help about:
    opal_init:startup:internal-failure
But I couldn't open the help file:
    /home/afh/Programs/OpenFOAM/ThirdParty-2.4.0/platforms/linux64Gcc/openmpi-1.8.5/share/openmpi/help-opal-runtime.txt: No such file or directory.  Sorry!
--------------------------------------------------------------------------
This is my "foamInstallationTest" output:
Checking the OpenFOAM env variables set on the LD_LIBRARY_PATH...
Code:
-------------------------------------------------------------------------------
Environment_variable Set_to_file_or_directory                Valid Path Crit
-------------------------------------------------------------------------------
$FOAM_LIBBIN         ...-2.4.0/platforms/linux64GccDPOpt/lib  yes  yes  yes
$FOAM_SITE_LIBBIN    .../2.4.0/platforms/linux64GccDPOpt/lib  no        no
$FOAM_USER_LIBBIN    ...-2.4.0/platforms/linux64GccDPOpt/lib  no        no
$MPI_ARCH_PATH       ...0/platforms/linux64Gcc/openmpi-1.8.5  no        yes
-------------------------------------------------------------------------------
$echo $MPI_ARCH_PATH returns:
Code:
echo $MPI_ARCH_PATH
/home/afh/Programs/OpenFOAM/ThirdParty-2.4.0/platforms/linux64Gcc/openmpi-1.8.5
but "/home/afh/Programs/OpenFOAM/ThirdParty-2.4.0/platforms/linux64Gcc" does not have a directory "openmpi-1.8.5"

I have followed these installation instructions: https://openfoam.org/download/2-4-0-source/
and have set WM_MPILIB=OPENMPI, but only after some initial execution of ./Allmake, then changed the setting and ran ./Allmake again. Should I have done an ./Allclean before running ./Allwmake again?

Thank you!
Andreas
anfho is offline   Reply With Quote

Old   November 20, 2018, 12:08
Default
  #10
Member
 
Join Date: Sep 2010
Posts: 36
Rep Power: 16
anfho is on a distinguished road
I was actually able to fix this issue by following this fix: https://bugs.openfoam.org/view.php?id=1770

and doing:
Code:
foam3rdParty
wget www.open-mpi.org/software/ompi/v1.8/downloads/openmpi-1.8.5.tar.gz
tar -xf openmpi-1.8.5.tar.gz
But when I am running a parallel application, I am now getting this error:
Code:
--> FOAM FATAL ERROR:
Trying to use the dummy Pstream library.
This dummy library cannot be used in parallel mode

    From function UPstream::init(int& argc, char**& argv)
    in file UPstream.C at line 37.

FOAM exiting
Has this been solved somewhere else already? (Again, its OpenFoam 2.4, Ubuntu 16.04 and I want to use openmpi-1.8.5 as downloaded above).
anfho is offline   Reply With Quote

Old   November 20, 2018, 12:48
Default
  #11
Member
 
Join Date: Sep 2010
Posts: 36
Rep Power: 16
anfho is on a distinguished road
I recompiled Pstream and that fixed the issue. Is it possible that the dependencies got somehow mixed up when re-installing openmpi, or what could be a possible explanation for Pstream needing to be recompiled in order for this to work?
anfho is offline   Reply With Quote

Old   November 20, 2018, 17:08
Default
  #12
Member
 
Fatih Ertinaz
Join Date: Feb 2011
Location: Istanbul
Posts: 64
Rep Power: 15
fertinaz is on a distinguished road
Just guessing, but it is probably because your environment was linked to "FOAM_LIBBIN/dummy/libPstream.so" instead of the one in the openmpi directory. So the easiest solution would have been either prepending the openmpi directory to the "LD_LIBRARY_PATH" or creating a symbolic link.
anfho likes this.
fertinaz is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Doubt about parallel processing vishwesh OpenFOAM Programming & Development 0 November 10, 2017 07:20
SU2v4.0.0 parallel installation error Mohit Singh SU2 Installation 2 February 19, 2016 14:17
Parallel Processing by region Swift OpenFOAM Running, Solving & CFD 0 July 30, 2015 09:42
Issue with running in parallel on multiple nodes daveatstyacht OpenFOAM 7 August 31, 2010 18:16
bubbly flow and parallel processing mvee FLUENT 0 September 12, 2007 06:08


All times are GMT -4. The time now is 15:28.