[harly]

Hi,

I am having some trouble to get OpenFOAM to run over two or more PCs.

The problem is that I get the error:

bash: orted: command not found

and I already figured out, what the problem is:

My .bashrc is not sourced in the non-interactive bash shell when I use "mpirun".

The problem is, that I don't know how to fix the problem. I tried every hint I could find for that problem, that did not require root privileges. I don't know if there is an easy fix for that, but I would assume so, because in Ubuntu I just edited the .bashrc and it worked instantly.

If you need any further Information please ask.

Thank you for your help.
-harly

[dkingsley]

you need to check your .bashrc and see if you are outputting something to the console. If you are mpirun will stop sourcing the .bashrc on Fedora machines.

We solved this at my last place of employment by checking if the login was interactive in the .bashrc script. If it was not interactive we did not output to the console and that solved the mpirun issue. Unfortunatly I don't remember the syntax.

[harly]

Hi,

I tried the following:

if [ -z "$PS1" ]; then
PATH=$PATH:/home2/harlacher/OpenFOAM/
fi

just to check if it would work.

But "ssh client printenv |grep PATH" does not return "/home2/harlacher/OpenFOAM/"

Any ideas what else I could do ?

-harly

[vkrishna]

I am running OF 1.5 in paralled on two machines kidam and soorya. I get the following error :
Can somebody advice the reason for this

[soorya:05105] *** An error occurred in MPI_Waitall
[soorya:05105] *** on communicator MPI_COMM_WORLD
[soorya:05105] *** MPI_ERR_TRUNCATE: message truncated
[soorya:05105] *** MPI_ERRORS_ARE_FATAL (goodbye)
[soorya:05104] *** An error occurred in MPI_Waitall
[soorya:05104] *** on communicator MPI_COMM_WORLD
[soorya:05104] *** MPI_ERR_TRUNCATE: message truncated
[soorya:05104] *** MPI_ERRORS_ARE_FATAL (goodbye)
mpirun noticed that job rank 2 with PID 4565 on node kidambiHP219 exited on signal 15 (Terminated).

[gschaider]

Could you be a bit more specific WHEN this error appears (before/after the first timestep, didn't even see the OF-Logo .... )

The two most popular problems with MPI are:

- too small MPI_BUFFER_SIZE
- different OpenMPI-versions on the machines (I suspect that in your case: your "master" machine uses the OF-OpenMPI, the "slave" uses another - system installed - OpenMPI)

Berhard

[vkrishna]

this problem happens after the first time step
. Also both machines used the same version of OpenMPI.. the one supplied with OF

[gschaider]

Hi!

Which solver? That might be an indication. For instance: if it is dieselFoam then a search for MPI_ERR_TRUNCATE might have gotten you two very interesting result. One of them would lead to the question "How do I install these 1.5.x-Bugfixes"

Bernhard

[vkrishna]

the solver is sonicTurbFoam.
I am attaching the log for reference:

Exec : sonicTurbFoam -parallel
Date : Feb 19 2009
Time : 12:31:39
Host : soorya
PID : 5104
Case : /home/openfoam15/OpenFOAM/vijay-1.5/run/vayumach2clus
nProcs : 4
Slaves :
3
(
soorya.5105
kidambiHP219.4565
kidambiHP219.4566
)

Pstream initialized with:
floatTransfer : 1
nProcsSimpleSum : 0
commsType : nonBlocking

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Reading thermophysical properties

Selecting thermodynamics package hThermo<puremixture<consttransport<speciethermo<hc onstthermo<perfectgas>>>>>
1 additional process aborted (not shown)


---vijay

[vkrishna]

Also the is the error messages I get:
Thu Feb 19 12:31:36 IST 2009
nohup: appending output to `nohup.out'
[soorya:05105] *** An error occurred in MPI_Waitall
[soorya:05105] *** on communicator MPI_COMM_WORLD
[soorya:05105] *** MPI_ERR_TRUNCATE: message truncated
[soorya:05105] *** MPI_ERRORS_ARE_FATAL (goodbye)
[soorya:05104] *** An error occurred in MPI_Waitall
[soorya:05104] *** on communicator MPI_COMM_WORLD
[soorya:05104] *** MPI_ERR_TRUNCATE: message truncated
[soorya:05104] *** MPI_ERRORS_ARE_FATAL (goodbye)
mpirun noticed that job rank 2 with PID 4565 on node kidambiHP219 exited on signal 15 (Terminated).
Command exited with non-zero status 1
0.02user 0.01system 0:06.14elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
4752inputs+16outputs (27major+2408minor)pagefaults 0swaps

[DSpreitz]

Harly,

thanks for your post. It was the last piece in my puzzle.

Dominic

Originally Posted by harly

Hi,

I tried the following:

if [ -z "$PS1" ]; then
PATH=$PATH:/home2/harlacher/OpenFOAM/
fi

just to check if it would work.

But "ssh client printenv |grep PATH" does not return "/home2/harlacher/OpenFOAM/"

Any ideas what else I could do ?

-harly

[harly]

Would you mind sharing what you managed to do with the information ?

Because I was not able to get my setup running (I blame fedora 9) because with ubuntu it worked out of the box.

- harly

[DSpreitz]

Sorry it took me so long to reply. I put

if [ -z "$PS1" ]; then
source /home/user/OpenFOAM/OpenFOAM-1.5/etc/bashrc
fi

in my .bashrc. That way all the required OF environment variables are correctly loaded, when MPI logs in over SSH. At least that's my understanding of the concept with the inter- and non-interactive shells. Anybody correct me, if I am wrong here.

Hope this helps.

Originally Posted by harly

Would you mind sharing what you managed to do with the information ?

Because I was not able to get my setup running (I blame fedora 9) because with ubuntu it worked out of the box.

- harly