Unable to run OpenFOAM 1.6-ext in parallel with more than one machine
Dear all
I'm facing a problem running in parallel in multi node. I can run in one node without any problem. I am receiving this error all the time: Quote:
I should say that I didn't receive no error installing openfoam1.6 ext. http://sourceforge.net/p/openfoam-ex...ci/1.6.1/tree/ other users are using openfoam 2.x and they don't have my problem. will be too much thankful to have your helps mehdi |
Greetings mm.abdollahzadeh,
Not much information to work with... therefore I'll have to ask a few questions:
Bruno |
Many thanks Burono
I have installed the openfoam in my folder, which is sharded on all nodes. we are using ROCKs. and below is the commands that I use Quote:
and the instruction from http://sourceforge.net/p/openfoam-ex..._5.5_64bit.txt best Mehdi |
Hi Mehdi,
If my diagnosis is correct, the problem is that the cluster does not accept using the customized Open-MPI installation that was done by default by the 1.6-ext installation scripts/instructions. You must use the MPI that the cluster already has got. Go into the where 1.6-ext is installed and edit the file "etc/prefs.sh", search for these lines: Code:
#export WM_MPLIB=SYSTEMOPENMPI But now there is a difficult problem on this case... figuring out what is the folder for the Open-MPI installation the cluster is using, so that you can define it in the variable "OPENMPI_DIR", namely in the second line. I'm too tired right now to estimate what path it might be in, so I suggest that you ask someone you know that works with that cluster. Once you know the path to the MPI toolbox, replace "path_to_system_installed_openmpi" for the path, e.g. "/opt/openmpi-1.6.2", save and close the file. Then start a new terminal and run Allwmake again in the main 1.6-ext folder. Once it's finished, and hopefully it does it with success, try running in parallel once again. Best regards, Bruno |
Dear Burno
Its some how solved but not completely. Now my cases are start to run. however there are still problems. even if for example I choose 30 processors ( no matter if work with orte or mpi) it starts running the case in one machine ( which just has 12 processor) !! I mention that other users are still running with out problem. best mahdi |
Hi Mahdi,
:confused: I can't see what you're seeing, unless you share it somehow (text or pictures). And are you certain that everything is properly compiled? What do these commands give you: Code:
which mpirun Bruno |
2 Attachment(s)
Dear Bruno
May thanks for your replay. here is the out put of the commands: which mpirun Quote:
Quote:
Quote:
Quote:
Attachment 28267 it starts to run. I can see in master that Quote:
Attachment 28268 best Regards mahdi |
Hi Mahdi,
Do you have another version of OpenFOAM, such as 2.2.2 or any other? And are you able to use it yourself on more than one machine? In addition, are you certain you are using the job script properly? Some job scripts need to specify the machines where to run. Best regards, Bruno |
Dear Bruno
unfortunately, I don't have. but other users have openfoam 2.1.0 without problem. I think so that is correct. I have doubt that maybe the ARGS in my script are not consistent with openfoam extended?! best mahdi |
Hi Mahdi,
You need to ask the other fellow users, since I'm not familiar with the job scheduler you are using. The ARGS entry currently is only taking care of disabling (or enabling?) one connection port and enabling another. It's not taking care of the machines to be used. Try adding the following to the ARGS variable: Code:
-host compute-0-3-3,compute-0-7-7,compute-0-9-9 Code:
ARGS="--mca btl ^openib --mca btl_tcp_if_include eth0 -host compute-0-3-3,compute-0-7-7,compute-0-9-9" Bruno |
Dear Bruno
unfortunately it didnt work yet give me : Quote:
Best Regards Mahdi |
Hi Mahdi,
Quote:
Code:
-host compute-0-3-3,compute-0-7-7,compute-0-9-9 Quote:
Code:
echo $FOAM_LIBBIN/$FOAM_MPI Best regards, Bruno |
Dear Bruno
certainly I have tested the command that you suggest. but it didnt work. and I got: Quote:
Quote:
Mahdi |
Hi Mahdi,
Sorry, then I have absolutely no idea. You need to talk to the system's administrator to assess what the problem is. In theory you should now be using the correct MPI toolbox, therefore the problem should be somewhere in the job script. Best regards, Bruno |
Thank you Bruno
I add this lines to my /etc/bashrc Quote:
many thanks Bruno best regards mahdi |
All times are GMT -4. The time now is 08:19. |