CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

Configuration of MPICH

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   September 7, 2005, 12:52
Default Hi, How can I set the MPICH e
  #1
mer
Member
 
merrouche djemai
Join Date: Mar 2009
Location: ain-oussera, djelfa, algeria
Posts: 46
Rep Power: 8
mer is on a distinguished road
Hi,
How can I set the MPICH env_variables:
MPICH_PATH, MPICH_ROOT, etc.
I'm using a Pc with linux red Hat 9.0
I need this, because I have received in test installation that MPICH is not set.
THANKS
mer is offline   Reply With Quote

Old   September 7, 2005, 23:06
Default Is this the foamInstallationTe
  #2
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Is this the foamInstallationTest script? This script is no longer valid for OpenFOAM 1.2 and the warning message is probably not relevant.

By default OpenFOAM uses lam and not mpich so none of the MPICH_ROOT etc. are set.
mattijs is offline   Reply With Quote

Old   September 8, 2005, 12:50
Default Hi, Then, what is the problem
  #3
mer
Member
 
merrouche djemai
Join Date: Mar 2009
Location: ain-oussera, djelfa, algeria
Posts: 46
Rep Power: 8
mer is on a distinguished road
Hi,
Then, what is the problem? When I compile the exemple turboFOAM with wmake and compare the results on my screen I can't find lines in compilation related to lam/MPI and mpich as expected by the user guide.
thanks
mer is offline   Reply With Quote

Old   September 8, 2005, 13:25
Default There might be no problem, the
  #4
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
There might be no problem, the text in the user guide might just describe a slightly different environment.

The question is does it run in parallel?
mattijs is offline   Reply With Quote

Old   September 9, 2005, 05:09
Default Hi, As you can see, I'm a new
  #5
mer
Member
 
merrouche djemai
Join Date: Mar 2009
Location: ain-oussera, djelfa, algeria
Posts: 46
Rep Power: 8
mer is on a distinguished road
Hi,
As you can see, I'm a new user of openFOAM and I try to execute this exemple in parallel with 2 Pcs on red hat 9.0 environment. When I got a result, I will be back.
thanks.
mer is offline   Reply With Quote

Old   May 18, 2006, 07:07
Default Hello, I would like to run
  #6
Member
 
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 8
anne is on a distinguished road
Hello,

I would like to run OpenFoam with Mpich and not LAM.
By default I know that the LAM MPI is used by OpenFoam.
I would like to change the variable WM_MPLIB to
MPICH, however I couldn't find the
file where you can modify the default
value of WM_MPLIB. I looked in wmake
directory but so many files that
I didn't find out it.

Thanks

Anne
anne is offline   Reply With Quote

Old   May 18, 2006, 07:37
Default Look in the file $WM_PROJECT_
  #7
theof
Guest
 
Posts: n/a
Look in the file
$WM_PROJECT_INST_DIR/OpenFOAM-$VERSION/.OpenFOAM/{cshrc , bashrc}
and set the WM_MPLIB.

Then go to
$WM_PROJECT_INST_DIR/OpenFOAM-$VERSION/.{cshrc, bashrc}
and set the path to your mpich installation.

theophane
  Reply With Quote

Old   May 18, 2006, 07:41
Default Look in: ~/OpenFOAM/OpenFOA
  #8
Senior Member
 
Hrvoje Jasak
Join Date: Mar 2009
Location: London, England
Posts: 1,761
Rep Power: 21
hjasak will become famous soon enough
Look in:

~/OpenFOAM/OpenFOAM-1.3/.cshrc or .bashrc towards the end.

Hrv
__________________
Hrvoje Jasak
Providing commercial FOAM/OpenFOAM and CFD Consulting: http://wikki.co.uk
hjasak is offline   Reply With Quote

Old   May 18, 2006, 08:58
Default Sorry, I am not very good i
  #9
Member
 
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 8
anne is on a distinguished road
Sorry,

I am not very good in shell.
What I can see is that in the .cshrc there are a lot of options of MPI (lam, mpich ..)
alll with a il/else if
Please, could you be
a little more explicit on what I have to write
to force MPICH.

Do I have just to write in the .csrhc:

setenv WM_MPILIB MPICH

Thanks,

Anne
anne is offline   Reply With Quote

Old   May 19, 2006, 02:49
Default Ok, look in ~/OpenFOAM/OpenFOA
  #10
theof
Guest
 
Posts: n/a
Ok, look in ~/OpenFOAM/OpenFOAM-1.3/.OpenFOAM/cshrc:
Go to section corresponding to your operating system
(linux, sun or silicon graphics workstation). Then comment the line corresponding to lam, and uncomment the line for mpich.

Then go to ~/OpenFOAM/OpenFOAM-1.3/.cshrc:
in the communications library section, set the variables corresponding to your installation of mpich.

theophane
  Reply With Quote

Old   May 19, 2006, 03:29
Default K, Thank you, I had just
  #11
Member
 
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 8
anne is on a distinguished road
K,

Thank you,

I had just missed the correct path of the first directory you wrote me so that I couldn't find the cshrc file and was looking always in the .cshrc.


Thanks a lot,

I will do it now,

Anne
anne is offline   Reply With Quote

Old   July 14, 2006, 03:48
Default I'm trying to link OpenFOAM 1.
  #12
Senior Member
 
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 193
Rep Power: 8
hani is on a distinguished road
I'm trying to link OpenFOAM 1.3 to a locally installed MPICH 1.2.7 and then run simpleFoam trough PBS (and actually also through Swegrid). I have compiled OpenFOAM locally with gcc 4.1.1 and I have produced a libPstream.so for MPICH using the procedure described below.

When I run simpleFoam it seems like MPICH starts the tasks (mpiexec: All 4 tasks started.), but then OpenFOAM aborts:
0] MPI Abort by user Aborting program !
[0] Aborting program!
p0_3256: p4_error: : 1
rm_l_3_30702: (0.004464) net_send: could not write to fd=9, errno = 32

Does anyone have a clue? I have tried to find the answer in the forum, without success.

Håkan.

--------------------------------------------
I have done the following to compile libPstream.so for MPICH:
In .OpenFOAM-1.3/cshrc:
setenv WM_MPLIB MPICH
In )openFOAM-1.3/.cshrc:
setenv MPICH_VERSION 1.2.7
setenv MPICH_PATH /usr/local/mpich
setenv MPICH_ARCH_PATH $MPICH_PATH
setenv MPICH_ROOT $MPICH_ARCH_PATH

AddLib $MPICH_ARCH_PATH/lib
AddPath $MPICH_ARCH_PATH/bin

setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/mpich-$MPICH_VERSION

Then in src/Pstream/mpi:
wclean
wmake libso

And I get a mpich-1.2.7/libPstream.so

There are no error messages when doing this except the warning: use of old-style cast

------------------------------------------
Result from ldd `which simpleFoam`:
libincompressibleTurbulenceModels.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libincompressibleTurb ulen
ceModels.so (0x40001000)
libincompressibleTransportModels.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libincompressibleTran sport
Models.so (0x401a7000)
libfiniteVolume.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libfiniteVolume.so (0x401d8000)
libmeshTools.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libmeshTools.so (0x40946000)
libOpenFOAM.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libOpenFOAM.so (0x40a96000)
libstdc++.so.6 => /users/unicc/hani/OpenFOAM/linux/gcc-4.1.1/lib/libstdc++.so.6 (0x40d64000)
libm.so.6 => /lib/i686/libm.so.6 (0x49c85000)
libgcc_s.so.1 => /users/unicc/hani/OpenFOAM/linux/gcc-4.1.1/lib/libgcc_s.so.1 (0x40e49000)
libpthread.so.0 => /lib/i686/libpthread.so.0 (0x49d16000)
libc.so.6 => /lib/i686/libc.so.6 (0x49b43000)
libPstream.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/mpich-1.2.7/libPstrea m.so (0x40e54000)
libtriSurface.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libtriSurface.so (0x40ec7000)
liblagrangian.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/liblagrangian.so (0x40f37000)
libfoamUser.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libfoamUser.so (0x40f3a000)
libz.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libz.so (0x40f3c000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00643000)
hani is offline   Reply With Quote

Old   July 14, 2006, 08:36
Default It seems like my compilation w
  #13
Senior Member
 
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 193
Rep Power: 8
hani is on a distinguished road
It seems like my compilation with MPICH was alright all the time since I am now able to run it in parallel locally. The problem is when I submit my job.

When running using mpiexec through PBS there are lots of arguments added. Does anyone know how to make OpenFOAM not care about those:

/users/unicc/hani/OpenFOAM/OpenFOAM-1.3/applications/bin/linuxGcc4DPOpt/simpleFo am /users/unicc/hani/OpenFOAM/hani-1.3/run/pitzDaily/.. pitzDaily -parallel -p4wd /users/unicc/hani/OpenFOAM/hani-1.3/run/pitzDaily -execer_id mpiexec -master_host node101 -my_hostname node101 -my_nodenum 0 -my_numprocs 1 -total_numnodes 4 -master_port 32854 -remote_info node063 1 node061 1 node041 1

Error message:
[0] --> FOAM FATAL ERROR : Wrong number of arguments, expected 2 found 15

Håkan.
hani is offline   Reply With Quote

Old   July 14, 2006, 12:41
Default Pstream/mpi/Pstream.C Pstre
  #14
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
Pstream/mpi/Pstream.C

Pstream::addValidParOptions
mattijs is offline   Reply With Quote

Old   July 17, 2006, 10:05
Default Hello Hakan, I have compi
  #15
Member
 
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 8
anne is on a distinguished road
Hello Hakan,


I have compiled Mpich_1.2.7. While I am
running an application: mpirun -np 2 `which application` ....


I have the following error as you get earlier (see below)
BUT it is because tehere is a problem with
a shared library that is not found.

I have just installed mpich-1.2.7 in Src of openFoam1-3, and then went to
Pstream/mpi and made wclean, wmake libso.

Did you also have an error with shared library ??
Also, I has the warning about " Pstream.C:279: warning: use of old-style cast" but I don't know how far it is important.
Did you succeed in runnning mpich ?


I am fighting a while with this. On which
station do you run it ??

Thanks for Your help,

Anne


--------------------------------------------
Exec : /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/applications/bin/linuxGcc4DPOpt/po is_ufixed /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/run case2D_par -p4pg /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/run/PI22250 -p4wd /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/run
/afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/applications/bin/linuxGcc4DPOpt/po is_ufixed: error while loading shared libraries: libfiniteVolume.so: cannot open shared object file: No such file or directory
p0_22339: p4_error: Child process exited while making connection to remote process on lince.ciemat.es: 0
p0_22339: (9.173624) net_send: could not write to fd=5, errno = 32
anne is offline   Reply With Quote

Old   July 18, 2006, 03:44
Default Hello Anne, I am able to ru
  #16
Senior Member
 
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 193
Rep Power: 8
hani is on a distinguished road
Hello Anne,

I am able to run the locally installed MPICH. You have to make sure that all processes/processors have the correct PATHs and LD_LIBRARY_PATHs. You also have to be sure that they all have access to the shared library. When submitting jobs through a que system it is also important to think about where your submit script will be run. It is best to give full paths to the <root>. It is not for sure that the processes inherit all the environment variables, paths etc from the submit process.

I don't think we have to bother about old-style cast. It's just a warning. I think that it has been discussed before.

Right now I'm at vacation, so I didn't have time to test the suggestion by Mattijs (in fact I don't understand it yet).

As I said, I am able to run MPICH locally with mpirun, but when I submit through our que system I still have problems. Will have a look at it in August.

Håkan.
hani is offline   Reply With Quote

Old   July 18, 2006, 04:00
Default Hello Hakan, Thank you for
  #17
Member
 
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 8
anne is on a distinguished road
Hello Hakan,

Thank you for your answer.

I will check again if the paths were correctly declared. How do you check that the paths is correctly declared for all processes.
When I check some of the variables it is OK, at least in the the home directory where I am working.
I dont't know the way to check this on the processes where I want to run my jobs.
The test I have done were locally, I didn't try through the queue system.

well, I know you are on holidays,
so enjoy your time.

Thanks again,

Anne
anne is offline   Reply With Quote

Old   August 22, 2006, 06:25
Default Thanks Mattijs for your answer
  #18
Senior Member
 
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 193
Rep Power: 8
hani is on a distinguished road
Thanks Mattijs for your answer on July 14, it almost solved my problem.

I still have problems with the final flag:

-remote_info node063 1 node061 1 node041 1

This flag lists information about the other nodes involved in the parallel run. As you can see there are more than one argument to the flag, in fact twice the number of other nodes so it will depend on how many nodes are involved. How can this flag be added as valid?

I understand that for instance
validParOptions.insert("p4wd", "directory");
makes the flag -p4wd valid and I guess that the directory listed after -p4wd may be found in OpenFOAM somehow using the keyword "directory".
In my case I am not interested in finding out the arguments, I just want to make the computation run.

I have tried different ways of adding more arguments without success. Do you or anyone else have any idea?

Håkan.
hani is offline   Reply With Quote

Old   August 23, 2006, 10:04
Default After some more research it se
  #19
Senior Member
 
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 193
Rep Power: 8
hani is on a distinguished road
After some more research it seems like when running OpenFOAM using MPICH under PBS and using mpiexec, the extra flags added should actually be used by MPI_Init inside OpenFOAM.

Being unable to make the flag -remote_info and its arguments valid using validParOptions, I tried removing the FatalError.exit(); at line 193 in src/OpenFOAM/global/argList/argList.C (OF1.3)

The parallel run then works properly. However, I have then lost the nice check of arguments.

I would still like to have hints on how to make -remote_info and its arguments valid in a correct way. Note that -remote_info has a number of arguments that depends on the number of nodes involved.

Håkan.
hani is offline   Reply With Quote

Old   August 24, 2006, 03:54
Default - maybe there is an option for
  #20
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 16
mattijs is on a distinguished road
- maybe there is an option for mpiexec to read arguments from a file? There is one for the LAM mpiexec.

- the correct way would be to modify argList.C to do two passes over the arguments. First pass checks for validParOptions and calls parRunControl_.runPar which should filter the command line args. Then the second pass works as before and checks for correct OF arguments.
mattijs is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
MPICH r2d2 OpenFOAM Installation 1 June 15, 2005 19:32
PVM or MPICH CFX User CFX 5 November 3, 2004 17:31
LAM/MPI or MPICH? Junseok Kim Main CFD Forum 0 January 15, 2004 08:10
PC configuration for CFD Fernando Sanz Main CFD Forum 17 December 19, 2001 08:29
MPICH Junseok Kim Main CFD Forum 1 November 5, 2000 23:07


All times are GMT -4. The time now is 20:57.