|
[Sponsors] |
September 7, 2005, 12:52 |
Hi,
How can I set the MPICH e
|
#1 |
Member
merrouche djemai
Join Date: Mar 2009
Location: ain-oussera, djelfa, algeria
Posts: 46
Rep Power: 17 |
Hi,
How can I set the MPICH env_variables: MPICH_PATH, MPICH_ROOT, etc. I'm using a Pc with linux red Hat 9.0 I need this, because I have received in test installation that MPICH is not set. THANKS |
|
September 7, 2005, 23:06 |
Is this the foamInstallationTe
|
#2 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
Is this the foamInstallationTest script? This script is no longer valid for OpenFOAM 1.2 and the warning message is probably not relevant.
By default OpenFOAM uses lam and not mpich so none of the MPICH_ROOT etc. are set. |
|
September 8, 2005, 12:50 |
Hi,
Then, what is the problem
|
#3 |
Member
merrouche djemai
Join Date: Mar 2009
Location: ain-oussera, djelfa, algeria
Posts: 46
Rep Power: 17 |
Hi,
Then, what is the problem? When I compile the exemple turboFOAM with wmake and compare the results on my screen I can't find lines in compilation related to lam/MPI and mpich as expected by the user guide. thanks |
|
September 8, 2005, 13:25 |
There might be no problem, the
|
#4 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
There might be no problem, the text in the user guide might just describe a slightly different environment.
The question is does it run in parallel? |
|
September 9, 2005, 05:09 |
Hi,
As you can see, I'm a new
|
#5 |
Member
merrouche djemai
Join Date: Mar 2009
Location: ain-oussera, djelfa, algeria
Posts: 46
Rep Power: 17 |
Hi,
As you can see, I'm a new user of openFOAM and I try to execute this exemple in parallel with 2 Pcs on red hat 9.0 environment. When I got a result, I will be back. thanks. |
|
May 18, 2006, 07:07 |
Hello,
I would like to run
|
#6 |
Member
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 17 |
Hello,
I would like to run OpenFoam with Mpich and not LAM. By default I know that the LAM MPI is used by OpenFoam. I would like to change the variable WM_MPLIB to MPICH, however I couldn't find the file where you can modify the default value of WM_MPLIB. I looked in wmake directory but so many files that I didn't find out it. Thanks Anne |
|
May 18, 2006, 07:37 |
Look in the file
$WM_PROJECT_
|
#7 |
Guest
Posts: n/a
|
Look in the file
$WM_PROJECT_INST_DIR/OpenFOAM-$VERSION/.OpenFOAM/{cshrc , bashrc} and set the WM_MPLIB. Then go to $WM_PROJECT_INST_DIR/OpenFOAM-$VERSION/.{cshrc, bashrc} and set the path to your mpich installation. theophane |
|
May 18, 2006, 07:41 |
Look in:
~/OpenFOAM/OpenFOA
|
#8 |
Senior Member
Hrvoje Jasak
Join Date: Mar 2009
Location: London, England
Posts: 1,905
Rep Power: 33 |
Look in:
~/OpenFOAM/OpenFOAM-1.3/.cshrc or .bashrc towards the end. Hrv
__________________
Hrvoje Jasak Providing commercial FOAM/OpenFOAM and CFD Consulting: http://wikki.co.uk |
|
May 18, 2006, 08:58 |
Sorry,
I am not very good i
|
#9 |
Member
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 17 |
Sorry,
I am not very good in shell. What I can see is that in the .cshrc there are a lot of options of MPI (lam, mpich ..) alll with a il/else if Please, could you be a little more explicit on what I have to write to force MPICH. Do I have just to write in the .csrhc: setenv WM_MPILIB MPICH Thanks, Anne |
|
May 19, 2006, 02:49 |
Ok, look in ~/OpenFOAM/OpenFOA
|
#10 |
Guest
Posts: n/a
|
Ok, look in ~/OpenFOAM/OpenFOAM-1.3/.OpenFOAM/cshrc:
Go to section corresponding to your operating system (linux, sun or silicon graphics workstation). Then comment the line corresponding to lam, and uncomment the line for mpich. Then go to ~/OpenFOAM/OpenFOAM-1.3/.cshrc: in the communications library section, set the variables corresponding to your installation of mpich. theophane |
|
May 19, 2006, 03:29 |
K,
Thank you,
I had just
|
#11 |
Member
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 17 |
K,
Thank you, I had just missed the correct path of the first directory you wrote me so that I couldn't find the cshrc file and was looking always in the .cshrc. Thanks a lot, I will do it now, Anne |
|
July 14, 2006, 03:48 |
I'm trying to link OpenFOAM 1.
|
#12 |
Senior Member
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 204
Rep Power: 18 |
I'm trying to link OpenFOAM 1.3 to a locally installed MPICH 1.2.7 and then run simpleFoam trough PBS (and actually also through Swegrid). I have compiled OpenFOAM locally with gcc 4.1.1 and I have produced a libPstream.so for MPICH using the procedure described below.
When I run simpleFoam it seems like MPICH starts the tasks (mpiexec: All 4 tasks started.), but then OpenFOAM aborts: 0] MPI Abort by user Aborting program ! [0] Aborting program! p0_3256: p4_error: : 1 rm_l_3_30702: (0.004464) net_send: could not write to fd=9, errno = 32 Does anyone have a clue? I have tried to find the answer in the forum, without success. Håkan. -------------------------------------------- I have done the following to compile libPstream.so for MPICH: In .OpenFOAM-1.3/cshrc: setenv WM_MPLIB MPICH In )openFOAM-1.3/.cshrc: setenv MPICH_VERSION 1.2.7 setenv MPICH_PATH /usr/local/mpich setenv MPICH_ARCH_PATH $MPICH_PATH setenv MPICH_ROOT $MPICH_ARCH_PATH AddLib $MPICH_ARCH_PATH/lib AddPath $MPICH_ARCH_PATH/bin setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/mpich-$MPICH_VERSION Then in src/Pstream/mpi: wclean wmake libso And I get a mpich-1.2.7/libPstream.so There are no error messages when doing this except the warning: use of old-style cast ------------------------------------------ Result from ldd `which simpleFoam`: libincompressibleTurbulenceModels.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libincompressibleTurb ulen ceModels.so (0x40001000) libincompressibleTransportModels.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libincompressibleTran sport Models.so (0x401a7000) libfiniteVolume.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libfiniteVolume.so (0x401d8000) libmeshTools.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libmeshTools.so (0x40946000) libOpenFOAM.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libOpenFOAM.so (0x40a96000) libstdc++.so.6 => /users/unicc/hani/OpenFOAM/linux/gcc-4.1.1/lib/libstdc++.so.6 (0x40d64000) libm.so.6 => /lib/i686/libm.so.6 (0x49c85000) libgcc_s.so.1 => /users/unicc/hani/OpenFOAM/linux/gcc-4.1.1/lib/libgcc_s.so.1 (0x40e49000) libpthread.so.0 => /lib/i686/libpthread.so.0 (0x49d16000) libc.so.6 => /lib/i686/libc.so.6 (0x49b43000) libPstream.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/mpich-1.2.7/libPstrea m.so (0x40e54000) libtriSurface.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libtriSurface.so (0x40ec7000) liblagrangian.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/liblagrangian.so (0x40f37000) libfoamUser.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libfoamUser.so (0x40f3a000) libz.so => /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/lib/linuxGcc4DPOpt/libz.so (0x40f3c000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00643000) |
|
July 14, 2006, 08:36 |
It seems like my compilation w
|
#13 |
Senior Member
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 204
Rep Power: 18 |
It seems like my compilation with MPICH was alright all the time since I am now able to run it in parallel locally. The problem is when I submit my job.
When running using mpiexec through PBS there are lots of arguments added. Does anyone know how to make OpenFOAM not care about those: /users/unicc/hani/OpenFOAM/OpenFOAM-1.3/applications/bin/linuxGcc4DPOpt/simpleFo am /users/unicc/hani/OpenFOAM/hani-1.3/run/pitzDaily/.. pitzDaily -parallel -p4wd /users/unicc/hani/OpenFOAM/hani-1.3/run/pitzDaily -execer_id mpiexec -master_host node101 -my_hostname node101 -my_nodenum 0 -my_numprocs 1 -total_numnodes 4 -master_port 32854 -remote_info node063 1 node061 1 node041 1 Error message: [0] --> FOAM FATAL ERROR : Wrong number of arguments, expected 2 found 15 Håkan. |
|
July 14, 2006, 12:41 |
Pstream/mpi/Pstream.C
Pstre
|
#14 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
Pstream/mpi/Pstream.C
Pstream::addValidParOptions |
|
July 17, 2006, 10:05 |
Hello Hakan,
I have compi
|
#15 |
Member
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 17 |
Hello Hakan,
I have compiled Mpich_1.2.7. While I am running an application: mpirun -np 2 `which application` .... I have the following error as you get earlier (see below) BUT it is because tehere is a problem with a shared library that is not found. I have just installed mpich-1.2.7 in Src of openFoam1-3, and then went to Pstream/mpi and made wclean, wmake libso. Did you also have an error with shared library ?? Also, I has the warning about " Pstream.C:279: warning: use of old-style cast" but I don't know how far it is important. Did you succeed in runnning mpich ? I am fighting a while with this. On which station do you run it ?? Thanks for Your help, Anne -------------------------------------------- Exec : /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/applications/bin/linuxGcc4DPOpt/po is_ufixed /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/run case2D_par -p4pg /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/run/PI22250 -p4wd /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/run /afs/ciemat.es/users/u5303/OpenFOAM/u5303-1.3/applications/bin/linuxGcc4DPOpt/po is_ufixed: error while loading shared libraries: libfiniteVolume.so: cannot open shared object file: No such file or directory p0_22339: p4_error: Child process exited while making connection to remote process on lince.ciemat.es: 0 p0_22339: (9.173624) net_send: could not write to fd=5, errno = 32 |
|
July 18, 2006, 03:44 |
Hello Anne,
I am able to ru
|
#16 |
Senior Member
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 204
Rep Power: 18 |
Hello Anne,
I am able to run the locally installed MPICH. You have to make sure that all processes/processors have the correct PATHs and LD_LIBRARY_PATHs. You also have to be sure that they all have access to the shared library. When submitting jobs through a que system it is also important to think about where your submit script will be run. It is best to give full paths to the <root>. It is not for sure that the processes inherit all the environment variables, paths etc from the submit process. I don't think we have to bother about old-style cast. It's just a warning. I think that it has been discussed before. Right now I'm at vacation, so I didn't have time to test the suggestion by Mattijs (in fact I don't understand it yet). As I said, I am able to run MPICH locally with mpirun, but when I submit through our que system I still have problems. Will have a look at it in August. Håkan. |
|
July 18, 2006, 04:00 |
Hello Hakan,
Thank you for
|
#17 |
Member
anne dejoan
Join Date: Mar 2009
Location: madrid, spain
Posts: 66
Rep Power: 17 |
Hello Hakan,
Thank you for your answer. I will check again if the paths were correctly declared. How do you check that the paths is correctly declared for all processes. When I check some of the variables it is OK, at least in the the home directory where I am working. I dont't know the way to check this on the processes where I want to run my jobs. The test I have done were locally, I didn't try through the queue system. well, I know you are on holidays, so enjoy your time. Thanks again, Anne |
|
August 22, 2006, 06:25 |
Thanks Mattijs for your answer
|
#18 |
Senior Member
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 204
Rep Power: 18 |
Thanks Mattijs for your answer on July 14, it almost solved my problem.
I still have problems with the final flag: -remote_info node063 1 node061 1 node041 1 This flag lists information about the other nodes involved in the parallel run. As you can see there are more than one argument to the flag, in fact twice the number of other nodes so it will depend on how many nodes are involved. How can this flag be added as valid? I understand that for instance validParOptions.insert("p4wd", "directory"); makes the flag -p4wd valid and I guess that the directory listed after -p4wd may be found in OpenFOAM somehow using the keyword "directory". In my case I am not interested in finding out the arguments, I just want to make the computation run. I have tried different ways of adding more arguments without success. Do you or anyone else have any idea? Håkan. |
|
August 23, 2006, 10:04 |
After some more research it se
|
#19 |
Senior Member
Håkan Nilsson
Join Date: Mar 2009
Location: Gothenburg, Sweden
Posts: 204
Rep Power: 18 |
After some more research it seems like when running OpenFOAM using MPICH under PBS and using mpiexec, the extra flags added should actually be used by MPI_Init inside OpenFOAM.
Being unable to make the flag -remote_info and its arguments valid using validParOptions, I tried removing the FatalError.exit(); at line 193 in src/OpenFOAM/global/argList/argList.C (OF1.3) The parallel run then works properly. However, I have then lost the nice check of arguments. I would still like to have hints on how to make -remote_info and its arguments valid in a correct way. Note that -remote_info has a number of arguments that depends on the number of nodes involved. Håkan. |
|
August 24, 2006, 03:54 |
- maybe there is an option for
|
#20 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
- maybe there is an option for mpiexec to read arguments from a file? There is one for the LAM mpiexec.
- the correct way would be to modify argList.C to do two passes over the arguments. First pass checks for validParOptions and calls parRunControl_.runPar which should filter the command line args. Then the second pass works as before and checks for correct OF arguments. |
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
MPICH | r2d2 | OpenFOAM Installation | 1 | June 15, 2005 19:32 |
PVM or MPICH | CFX User | CFX | 5 | November 3, 2004 16:31 |
LAM/MPI or MPICH? | Junseok Kim | Main CFD Forum | 0 | January 15, 2004 07:10 |
PC configuration for CFD | Fernando Sanz | Main CFD Forum | 17 | December 19, 2001 07:29 |
MPICH | Junseok Kim | Main CFD Forum | 1 | November 5, 2000 22:07 |