I finally able to run my case
 

I finally able to run my case in parallel. There was some problem in the gcc installation. Now its working fine,.

Thank you ..

Nishant

I thought my parallel case is
 

I thought my parallel case is running, but actually it was not. However I can see the program running on queue. Error file is saying that..
ERROR: A daemon on node comp26 failed to start as expected.
As i mentioned earlier my mpirun -hostfile machine <rooot> <case> -parallel command is running quite well on cluster directly. I mean to say that it's working fine if we run on master node (for us its kittyhawk.dcs.hull.ac.uk) but it fails on any other node ( like comp01/02/10/11 etc) I tried a hello mpi programm as well but that also failed to run using qsub and running quite well directly on master kittyhawk. my gcc compiler is unable to compile a program on any other node except master node kittyhawk. however they are using the right gcc (that is openfoam version of gcc)

Again, I am using cluster's version of MPICH as PE. (#$ mpich -pe 4), which is installed at /usr/.....**
The default PE environment here is >>score<< which we run using mpisub command.
Do I need to use a local version of mpich in order to run in parallel using qsub? Or could it be possible to run openfoam program using score?
can anybody suggst something?

regards,
Nishant

Running OpenFOAM in parallel with SGE
 

Hi,

to run openFOAM in parallel with SGE, you need to make sure that the following requisites are satisfied:

1) use an openmpi version >= 1.2.0
The reason is that any version prior to that isn't working with SGE.

2) Make sure that you define a parallel environment, for instance "orte", with the following definition (that's here for 8 parallel slots = 8 cores in parallel):
pe_name orte
slots 8
user_lists NONE
xuser_lists NONE
start_proc_args /bin/true
stop_proc_args /bin/true
allocation_rule $round_robin
control_slaves TRUE
job_is_first_task FALSE
urgency_slots min
3) Submit your job with (for example with a run on 8 cores):
qsub RUN.sh
where RUN.sh contains:
#!/bin/sh
#$ -V
### number of processors and parallel environment
#$ -pe orte 8
### Job name
#$ -N "mypartest"
### Start from current working directory
#$ -cwd
### Generate the hostfile
HOSTFILE=system/hostfile
awk '{print $1" cpu=1"}' ${PE_HOSTFILE} > ${PWD}/${HOSTFILE}
### Run application
SOLVER=icoFoam
${MPI_ARCH_PATH}/bin/mpirun -np ${NSLOTS} --hostfile ${PWD}/${HOSTFILE} ${FOAM_APPBIN}/${SOLVER} -parallel
exit $?

You will also find further information at:
http://www.open-mpi.org/faq/?categor...run-scheduling

Alternatively, you can try to compile MPICH from source. I've been able to run v1.2.7p1 without any dramas. This is quite straightforward if you take a look at the Allwmake scripts in $WM_THIRD_PARTY.

Hope this helps...

Problem with openFoam and SGE
 

Hello to all,

I'm trying to run openFoam on a SGE Sun Cluster. When running the job on parallel in a single node, everything works. But when trying to run the same job on different nodes, I get the following error message:

epsilon2.o31752:
To submmit the job I'm using the command qsub with the following script:

------------------------------------------------------------------------------
#!/bin/tcsh
# This is a simple example of a SGE batch script
#$-o /nfs/home/cardenas/Documents/OpenFOAM/Cases/Platte/Laenge120mm/Pulsierend/eps ilon2 -j y
#$-N epsilon2
#$-pe batch_64_2 2
#$-S /bin/tcsh
touch $HOME/.ssh/known_hosts
cd /nfs/home/cardenas/Documents/OpenFOAM/Cases/Platte/Laenge120mm/Pulsierend/epsilon2
touch -a ./*.*
touch -a ./system/*
source /nfs/home/cardenas/OpenFOAM/OpenFOAM-1.6.x/etc/cshrc
cat $PE_HOSTFILE |awk '{ print $1 " cpu=" $2}' > $HOME/mpi/machines.LINUX.$JOB_ID
sleep 10;
mpirun --hostfile $HOME/mpi/machines.LINUX.$JOB_ID -np 2 icoFoam -parallel >log

-----------------------------------------------------------------------------------------------

It seems that something with the Host Keys is not working properly, but since I'm not expirienced in SGE, I would appreciete any suggestions and hints. Thank you very much

Alejandro

There are a myriad of things that could be going wrong.
The very first thing it to determine if GridEngine support has been compiled into your openmpi.

Use the command "ompi_info" to list all the backends and grep for gridengine. If it's not there, you should recompile openmpi using the --with-sge configure option (see the third-party Allwmake).


^^^ what is this? Touching a file into existence doesn't make the hosts known!

If you have GridEngine and the openmpi is configured to use it, you should not be using--hostfile or -np. The GridEngine already knows how many slots you have (which would be $NSLOTS in your script), and it knows the host names too. It should also take care of inheriting the environment as well.

If the final backend uses rsh, ssh, or the GridEngine builtin transport will depend on what you have configured as the 'rsh_command' and 'rsh_daemon' in GridEngine.

BTW: your example is using cshell. Be certain that the queue is configured with the corresponding shell_start_mode. Be default this will be 'posix_compliant' (ie, use /bin/sh) and not 'unix_behavior' (ie, use #! to determine the shell/program).

Pending but not running
 

Hi,

I also tried to run OpenFoam in parallel with SGE. I use the following script to submit the job:
I get the following error:




I don´t know why the grid engine can´t find the command.


Does anybody have an idea why it doesn’t work? Or are there any settings I have to modify?

I´m thankful for any help

Stefan

If you change the submit script to have

it will become more general. Nice hack thx

what is the full path to the OpenFOAM dir? maybe just

/soft/OpenFOAM/OpenFOAM-2.0.0/etc/cshrc

without the initial . ?

solution to "Host key verification failed"
 

Hi!
If somebody is having problems with host-keys, adding a line
at the end of
-file and restarting ssh-daemon from the computing nodes should fix it. Of course SGE has to know, that you're using ssh for communication instead rsh. This can be done by specifying
in the run script.