|
[Sponsors] |
November 16, 2010, 23:36 |
Installing OpenFOAM-1.5-dev on a cluster
|
#1 |
New Member
Join Date: Mar 2010
Posts: 12
Rep Power: 16 |
Hello all,
It is a while that I am trying to install OpenFOAM-1.5-dev on a cluster without any success. I use gcc-4.3.5 by loading it as a module because system’s default gcc is 3.4.1. I set WM_COMPILER_DIR accordingly. We have HPMPI and set that in cshrc file. I install in a directory other than the standard ~ directory due to speed limitation of home. I cleaned before reinstalling. Applications are not made but foamInstallationTest does not report any problem except: Code:
Checking the OpenFOAM env variables set on the LD_LIBRARY_PATH... ------------------------------------------------------------------------------- Environment_variable Set_to_file_or_directory Valid Path Crit ------------------------------------------------------------------------------- $MPI_ARCH_PATH /opt/hpmpi yes no yes ------------------------------------------------------------------------------- I really appreciate if someone could have a look on the attached log file and advise me how to solve the problem. Many thanks in advance ZKM P.S. Sorry I had to delete parts of the file due to file size restriction. Last edited by ZKM; November 17, 2010 at 01:18. |
|
November 17, 2010, 04:59 |
|
#2 | |
Senior Member
Gijsbert Wierink
Join Date: Mar 2009
Posts: 383
Rep Power: 18 |
Hi ZKM,
On line 113 of your log it reads Quote:
I have ported OpenFOAM to a large Cray cluster recently. The cluster is a bit different than yours I guess, but you may find my report on it useful. There it is explained what to change and how. Hope it helps!
__________________
Regards, Gijs |
||
December 25, 2010, 11:48 |
|
#3 |
New Member
Join Date: Mar 2010
Posts: 12
Rep Power: 16 |
Hello all and thanks Gijs for your help
I am still in battle for installing it. I managed to find the source of the libPstream.so problem. For some reasons that I can't understand FOAM_MPI_LIBBIN is not set inside the switch ("$WM_MPLIB") in settings.csh. I set it manually again: Code:
setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/hpmpi Code:
+ cd applications + ./Allwmake + cd solvers + wmake all make[2]: Nothing to be done for `application'. make[2]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[2]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[4]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[4]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[4]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[3]: Nothing to be done for `application'. make[4]: Nothing to be done for `application'. make[4]: Nothing to be done for `application'. make[4]: Nothing to be done for `application'. Making dependency list for source file conjugateHeatFoam.C Making dependency list for source file potentialFoam.C SOURCE=potentialFoam.C ; g++ -m64 -Dlinux64 -DDP -DFOAM_DEV_REVISION_NUMBER=1861 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-40 -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/finiteVolume/lnInclude -IlnInclude -I. -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OpenFOAM/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OSspecific/Unix/lnInclude -fPIC -c $SOURCE -o Make/linux64GccDPOpt/potentialFoam.o SOURCE=conjugateHeatFoam.C ; g++ -m64 -Dlinux64 -DDP -DFOAM_DEV_REVISION_NUMBER=1861 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-40 -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/coupledMatrix/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/finiteVolume/lnInclude -IlnInclude -I. -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OpenFOAM/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OSspecific/Unix/lnInclude -fPIC -c $SOURCE -o Make/linux64GccDPOpt/conjugateHeatFoam.o g++ -m64 -Dlinux64 -DDP -DFOAM_DEV_REVISION_NUMBER=1861 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-40 -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/coupledMatrix/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/finiteVolume/lnInclude -IlnInclude -I. -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OpenFOAM/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OSspecific/Unix/lnInclude -fPIC Make/linux64GccDPOpt/conjugateHeatFoam.o -L/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt \ -lcoupledLduMatrix -lfiniteVolume -lOpenFOAM -liberty -ldl /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so -lm -o /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/applications/bin/linux64GccDPOpt/conjugateHeatFoam g++ -m64 -Dlinux64 -DDP -DFOAM_DEV_REVISION_NUMBER=1861 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-40 -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/finiteVolume/lnInclude -IlnInclude -I. -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OpenFOAM/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OSspecific/Unix/lnInclude -fPIC Make/linux64GccDPOpt/potentialFoam.o -L/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt \ -lfiniteVolume -llduSolvers -lOpenFOAM -liberty -ldl /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so -lm -o /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/applications/bin/linux64GccDPOpt/potentialFoam /usr/bin/ld: warning: libPstream.so, needed by /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libmpi.so.1, needed by /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so, not found (try using -rpath or -rpath-link) /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Finalize' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Buffer_detach' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlockattr_setkind_np' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_create' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_rdlock' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Bsend' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_init' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Probe' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_mutexattr_destroy' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Send' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_destroy' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_unlock' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Test' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Allreduce' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Buffer_attach' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Comm_size' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Recv' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Isend' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Get_count' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Abort' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Waitall' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Get_processor_name' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_wrlock' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlockattr_init' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Init' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Comm_rank' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_mutexattr_settype' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `MPI_Irecv' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `hpmp_sum' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlockattr_destroy' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_trywrlock' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_rwlock_tryrdlock' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_mutexattr_init' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `hpmp_byte' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `hpmp_double' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_mutex_trylock' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `hpmp_packed' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so: undefined reference to `hpmp_comm_world' /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `pthread_join' collect2: ld returned 1 exit status make[2]: *** [/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/applications/bin/linux64GccDPOpt/conjugateHeatFoam] Error 1 make[1]: *** [conjugateHeatFoam] Error 2 make[1]: Target `application' not remade because of errors. make: *** [conjugate] Error 2 /usr/bin/ld: warning: libPstream.so, needed by /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/libOpenFOAM.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libmpi.so.1, needed by /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so, not found (try using -rpath or -rpath-link) I found out that libmpi.so.1 is located in two locations /opt/hpmpi/lib/linux_amd64 and /opt/hpmpi/lib/linux_amd64/nomalloc/ I am going to do something like this manually: Code:
setenv LD_LIBRARY_PATH /opt/hpmpi/lib/linux_amd64:$LD_LIBRARY_PATH As for the other warning, that I think ends in those errors, again I don't know which libPstream.so is needed (in lib/linux64GccDPOpt/dummy or in lib/linux64GccDPOpt/hpmpi) and how to tell it to use from that path. Our admin advised me to add -lpthread after the "-ldl" in: Code:
g++ -m64 -Dlinux64 -DDP -DFOAM_DEV_REVISION_NUMBER=1861 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository -ftemplate-depth-40 -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/finiteVolume/lnInclude -IlnInclude -I. -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OpenFOAM/lnInclude -I/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/src/OSspecific/Unix/lnInclude -fPIC Make/linux64GccDPOpt/potentialFoam.o -L/sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt \ -lfiniteVolume -llduSolvers -lOpenFOAM -liberty -ldl /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/lib/linux64GccDPOpt/hpmpi/libPstream.so -lm -o /sfs/nobackup/z/z_kesha/OpenFOAM/OpenFOAM-1.5-dev/applications/bin/linux64GccDPOpt/potentialFoam Any help and comments are appreciated. Last edited by ZKM; December 25, 2010 at 14:46. |
|
December 25, 2010, 15:06 |
|
#4 |
Senior Member
Gijsbert Wierink
Join Date: Mar 2009
Posts: 383
Rep Power: 18 |
Hi ZKM,
Probably it is best to create your own wmake rules for your cluster compiler. What kind of cluster are you building on actually? I built OpenFOAM on different clusters and needed to do that. I guess you could export every rules, as you describe, but that'll become a mess quite easily I'm afraid. Here you can find a manual I wrote. It's for OpenFOAM-1.6.x on Cray cluster, but the general recipe should be useful.
__________________
Regards, Gijs |
|
December 25, 2010, 16:59 |
|
#5 |
New Member
Join Date: Mar 2010
Posts: 12
Rep Power: 16 |
So I am not the only one who works on Christmas
I have already read your document but couldn't solve the problem. Here is some information about the cluster: http://users.encs.concordia.ca/~cirrus/1.html Yes very true and I must implement all changes in wmake rules once I really understand how this thing works. How threading is done in OpenFoam? Is it through Pstream? If not I guess I should follow that advice to link pthread by putting -lpthread after -ldl in the "general" rule file (is the file correct?) ZKM |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[OpenFOAM] ParaFoam instalation in OpenFoam 1.5 dev | titio | ParaView | 2 | July 25, 2010 00:24 |
OpenFOAM 1.5 dev | LVDH | OpenFOAM | 98 | May 5, 2010 18:01 |
[OpenFOAM] Paraview/Parafoam in OpenFoam 1.5 dev | titio | ParaView | 0 | December 9, 2009 13:13 |
[OpenFOAM] Paraview/Parafoam in OpenFoam 1.5 dev | titio | ParaView | 0 | December 9, 2009 13:12 |
Problem installing OpenFOAM 1.5 installation on RHEL 4. | vwsj84 | OpenFOAM Installation | 4 | April 23, 2009 05:48 |