|
[Sponsors] |
May 5, 2008, 12:10 |
Dear All,
I am struggling w
|
#1 |
Senior Member
Daniel WEI (老魏)
Join Date: Mar 2009
Location: Beijing, China
Posts: 689
Blog Entries: 9
Rep Power: 21 |
Dear All,
I am struggling with porting OpenFoam to my IRIX with "-parallel" option on. That means I can run the solvers without "-parallel" very well, but as I mpirun my solver, it failed with annoying coredump. The output is at the bottom of my post. what I don't understand is these lines in argList.C: ------------------------------------------------- wordList machinesInfo(1); machinesInfo[0] = hostName(); labelList pids(1); pids[0] = pid(); if (parRunControl_.parRun()) { if (Pstream::master()) { machinesInfo.setSize(Pstream::nProcs()); pids.setSize(Pstream::nProcs()); // From all slaves for ( int slave=Pstream::firstSlave(); slave<=Pstream::lastSlave(); slave++ ) { label slavei = slave - Pstream::masterNo(); IPstream fromSlave(slave); fromSlave >> machinesInfo[slavei] >> pids[slavei]; } else { OPstream toMaster(Pstream::masterNo(), 0, false); toMaster << machinesInfo[0] << pids[0]; } } } ------------------------------------------------- 1. How can pids[slavei] get 4 (in my case) different pid()? The reason I raise the question is because I found my pids[1] is zero, and my machinesInfo[slavei] is empty! that's strange. 2. Is there a way to store the 3 slave (in my case) pids? (they are : 513638,513610,513608.) Please help! Thanks in advance! Daniel -------------------------------------------------- bash-2.05$ mpirun -np 4 `which icoFoam` $FOAM_RUN/tutorials/icoFoam cavity -parallel MPI Pstream initialized with: floatTransfer : 1 nProcsSimpleSum : 0 scheduledTransfer : 0 /*---------------------------------------------------------------------------*\ | ========= | | | \ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \ / O peration | Version: 1.4.1-dev | | \ / A nd | Web: http://www.openfoam.org | | \/ M anipulation | | \*---------------------------------------------------------------------------*/ Exec : /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam /disk4/usr/people/student3/OpenFOAM/student3-1.4.1-dev/run/tutorials/icoFoam cavity -parallel [0] Date : Apr 28 2008 [0] Time : 16:44:15 [0] Host : onyx-bridge [0] PID : 513629 [3] Date : Apr 28 2008 [3] Time : 16:44:15 [3] Host : onyx-bridge [3] PID : 513638 [1] Date : Apr 28 2008 [1] Time : 16:44:15 [1] Host : onyx-bridge [1] PID : 513610 [2] Date : Apr 28 2008 [2] Time : 16:44:15 [2] Host : onyx-bridge [2] PID : 513608 MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 3, Process 513638 received signal SIGBUS(10) MPI: --------stack traceback------- PC: 0x5ddb100 MPI_SGI_stacktraceback in /usr/lib32/libmpi.so PC: 0x5ddb544 first_arriver_handler in /usr/lib32/libmpi.so PC: 0x5ddb7d8 slave_sig_handler in /usr/lib32/libmpi.so PC: 0xfaee79c _sigtramp in /usr/lib32/libc.so.1 libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x60204334 in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 0 0x60197ac8 abs (a lot of messages the same as above) (......) 115 0x60197ac8 abs PC: 0x42d432c (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 0 0x60197ac8 abs (a lot of messages the same as above) (......) 6 0x60197ac8 abs libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x6020374c in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 7 0x60197ac8 abs (a lot of messages the same as above) (......) 115 0x60197ac8 abs PC: 0x42d3744 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x602064bc in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 0 0x60197ac8 abs (a lot of messages the same as above) (......) 115 0x60197ac8 abs PC: 0x42d64b4 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x601ee308 in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 0 0x60197ac8 abs (a lot of messages the same as above) (......) 115 0x60197ac8 abs PC: 0x42be300 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x601c262c in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 0 0x60197ac8 abs (a lot of messages the same as above) (......) 115 0x60197ac8 abs PC: 0x4292624 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x601b0904 in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so 0 0x60197ac8 abs (a lot of messages the same as above) (......) 115 0x60197ac8 abs PC: 0x42808fc (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so PC: 0x1002aeec main in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam MPI: dbx version 7.3.3 (78517_Dec16 MR) Dec 16 2001 07:45:22 MPI: Process 513638 (icoFoam) stopped at [__waitsys:24 +0x8,0xfa53338] MPI: Source (of /xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/proc/waitsys.s) not available for Process 513638 MPI: > 0 __waitsys(0x0, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/proc/waitsys.s":24, 0xfa53338] MPI: 1 _system(0x7ffb5150, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/stdio/system.c":116, 0xfa5f868] MPI: 2 MPI_SGI_stacktraceback(0x0, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv4/mpt/1.8/mpi/work/4.3/lib/libmpi/libmpi_n32_M4/adi/sig.c":242, 0x5ddb268] MPI: 3 first_arriver_handler(0xa, 0x71756974, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv4/mpt/1.8/mpi/work/4.3/lib/libmpi/libmpi_n32_M4/adi/sig.c":445, 0x5ddb544] MPI: 4 slave_sig_handler(0xa, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv4/mpt/1.8/mpi/work/4.3/lib/libmpi/libmpi_n32_M4/adi/sig.c":542, 0x5ddb7e0] MPI: 5 _sigtramp(0x0, 0x7d662, 0x0, 0x3, 0x0, 0x0, 0x0, 0xf8) ["/xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/signal/sigtramp.s":71, 0xfaee79c] MPI: 6 Foam::IPstream::readFromBuffer(this = <illegal>, t = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/Pstreams/IPread.C":2, 0x42d4334] MPI: 7 Foam::IPstream::read(this = <illegal>, l = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/Pstreams/IPread.C":27, 0x42d3744] MPI: 8 Foam::IPstream::read(this = <illegal>, t = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/Pstreams/IPreadToken.C":4 MPI: More (n if no)?, 0x42d64b4] MPI: 9 Foam::token::token(this = <illegal>, is = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/token/tokenIO.C":4, 0x42be300] MPI: 10 <unknown>(is = <illegal>, L = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/global/arg List/argList.C":40, 0x4292624] MPI: 11 <unknown>(this = <illegal>, argc = <illegal>, argv = <illegal>, checkArgs = <illegal>, checkOpts = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/global/arg List/argList.C":66, 0x42808fc] MPI: 12 ::main(The debugger has detected a dbxpcs server error (Internal Consistency Error). MPI: dbxpcs server attempting to continue MPI: ) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/solvers/in compressible/icoFoam/icoFoam.C":0, 0x1002aeec] MPI: 13 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x1002a668] MPI: -----stack traceback ends----- MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 3, Process 513638: Dumping core on signal SIGBUS(10) into directory /disk4/usr/people/student3/OpenFOAM/student3-1.4.1-dev/run/tutorials/icoFoam MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 1, Process 513610: Core dump on signal SIGBUS(10) suppressed. MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 2, Process 513608: Core dump on signal SIGBUS(10) suppressed. MPI: MPI_COMM_WORLD rank 1 has terminated without calling MPI_Finalize() MPI: aborting job MPI: Received signal 9
__________________
~ Daniel WEI ------------- Boeing Research & Technology - China Beijing, China |
|
May 21, 2008, 12:27 |
Now, I found the problem is ar
|
#2 |
Senior Member
Daniel WEI (老魏)
Join Date: Mar 2009
Location: Beijing, China
Posts: 689
Blog Entries: 9
Rep Power: 21 |
Now, I found the problem is around these line in argList.C
from line 339 -------------------------- else { // Collect the master's argument list IPstream fromMaster(Pstream::masterNo()); fromMaster >> args_ >> options_; } -------------------------- obviously, the program fail at line fromMaster >> args_ >> options_; Could anybody give me some help on what does these lines mean, in Istream.H. -------------------------- typedef Istream& (*IstreamManip)(Istream&); //- operator>> handling for manipulators without arguments inline Istream& operator>>(Istream& is, IstreamManip f) { return f(is); } //- operator>> handling for manipulators without arguments inline Istream& operator>>(Istream& is, IOstreamManip f) { f(is); return is; } ------------------------------- I doubt whether there is restriction on the length of filename on my IRIX, for when i insert --------------- Sout << args() << endl; --------------- like this --------------- else { // Collect the master's argument list Sout << args() << endl; IPstream fromMaster(Pstream::masterNo()); fromMaster >> args_ >> options_; } ---------------- output is /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32SP Debug/icoFoam is this too long? Any hints? Thanks Daniel
__________________
~ Daniel WEI ------------- Boeing Research & Technology - China Beijing, China |
|
May 23, 2008, 04:42 |
(following up from http://www.
|
#3 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
(following up from http://www.cfd-online.com/cgi-bin/OpenFOAM_Discus/show.cgi?tpc=126&post=23069#POST230 69)
- write a non-foam c++ mpi test program which does mpi_init and a few send/receive. - put a few print stmts into Pstream/mpi/*.C to show you what is being received/sent or link OF with a tracing library (mpe?) - begin() returns a pointer to the underlying storage. |
|
May 23, 2008, 06:47 |
> write a non-foam c++ mpi tes
|
#4 |
Senior Member
Daniel WEI (老魏)
Join Date: Mar 2009
Location: Beijing, China
Posts: 689
Blog Entries: 9
Rep Power: 21 |
> write a non-foam c++ mpi test program which does mpi_init and a few send/receive.
I did it before and everying works perfect. > put a few print stmts into Pstream/mpi/*.C to show you what is being received/sent or link OF with a tracing library (mpe?) I am doing that! Thank you, Mattijs! Daniel
__________________
~ Daniel WEI ------------- Boeing Research & Technology - China Beijing, China |
|
May 23, 2008, 13:28 |
Hi Mattijs!
I found somethi
|
#5 |
Senior Member
Daniel WEI (老魏)
Join Date: Mar 2009
Location: Beijing, China
Posts: 689
Blog Entries: 9
Rep Power: 21 |
Hi Mattijs!
I found something new. Look at the error messages I got! ...... MPI: dbx version 7.3.3 (78517_Dec16 MR) Dec 16 2001 07:45:22 MPI: Process 513638 (icoFoam) stopped at [__waitsys:24 +0x8,0xfa53338] MPI: Source (of /xlv42.../waitsys.s) not available for Process 513638 MPI: > 0 __waitsys(0x0...........................sys.s":24, 0xfa53338] MPI: 1 _system(0x7ffb5...................dio/system.c":116, 0xfa5f868] MPI: 2 MPI_SGI_stacktr......................adi/sig.c":242, 0x5ddb268] MPI: 3 first_arriver_h..............................c":44 5, 0x5ddb544] MPI: 4 slave_sig_handl..........bmpi_n32_M4/adi/sig.c":542, 0x5ddb7e0] MPI: 5 _sigtramp(0x0, ...............signal/sigtramp.s":71, 0xfaee79c] MPI: 6 Foam::IPstream::readFromBuffer(this = <illegal>, t = <illegal>) [".../src/OpenFOAM/db/IOstrea ms/Pstreams/IPread.C":2, 0x42d4334] MPI: 7 Foam::IPstream::read(this = <illegal>, l = <illegal>) [".../src/OpenFOAM/db/IOstreams/Pstreams/IPread.C":27, 0x42d3744] MPI: 8 Foam::IPstream::read(this = <illegal>, t = <illegal>) [".../src/OpenFOAM/db/IOstreams/Pstreams/IPreadToken.C":4 MPI: 9 Foam::token::token(this = <illegal>, is = <illegal>) [".../src/OpenFOAM/db/IOstreams/token/tokenIO.C":4, 0x42be300] MPI: 10 <unknown>(is = <illegal>, L = <illegal>) [".../src/OpenFOAM/global/arg List/argList.C":40, 0x4292624] MPI: 11 <unknown>(this = <illegal>, argc = <illegal>, argv = <illegal>, checkArgs = <illegal>, checkOpts = <illegal>) [".../src/OpenFOAM/global/arg List/argList.C":66, 0x42808fc] MPI: 12 ::main(The debugger has detected a dbxpcs server error (Internal Consistency Error). MPI: dbxpcs server attempting to continue MPI: ) [".../applications/solvers/in compressible/icoFoam/icoFoam.C":0, 0x1002aeec] MPI: 13 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x1002a668] ...... 1st, from "MPI: 12" to "MPI: 6", it seems that the "Foam::IPstream::readFromBuffer" (in ./src/OpenFOAM/IOstreams/Pstreams/IPread.C) is the arch-criminal, I GUESS! So, I tried to print out the "bufPosition_" & "buf_[bufPosition_]" as the following. Note, I used "Sout" instead of "Info", for the latter didn't work in slave process, that's strange. |...... |template<class> |inline void IPstream::readFromBuffer(T& t) |{ |````Sout<< "bufPosition_````````:" << bufPosition_ << nl````````````(Output is 1) |````````<< "buf_[bufPosition_]``:" << buf_[bufPosition_] << endl;```(Output is nothing! This is the same as buf_.begin() and ||||buf_.end()) |````t = reinterpret_cast<t&>(buf_[bufPosition_]);```````````````````(ERROR!!!!!!!!!!!) |````Sout<< "blahblahblahblahblahblahblahblahblah " << endl;```````` (Die with NO output) |````bufPosition_ += sizeof(T); |````checkEof(); | |````//readFromBuffer(&t, sizeof(T)); |} |...... 2nd, In error messages, there are so many "illegal"s, though everything works fine with single CPU of IRIX. So, I have 2 questions concerning line 87-106 in ./src/Pstream/mpi/Pstream.C |...... |#```ifndef SGIMPI |````string bufferSizeName = getEnv("MPI_BUFFER_SIZE"); | |````if (bufferSizeName.size()) |````{ |````````int bufferSize = atoi(bufferSizeName.c_str()); | |````````if (bufferSize) |````````{ |````````````MPI_Buffer_attach(new char[bufferSize], bufferSize); |````````} |````} |````else |````{ |````````FatalErrorIn("Pstream::init(int& argc, char**& argv)") |````````````<< "Pstream::init(int& argc, char**& argv) : " |````````````<< "environment variable MPI_BUFFER_SIZE not defined" |````````````<< Foam::abort(FatalError); |````} |#```endif |...... a) what if I use SGIMPI now, is that SGIMPI don't need MPI_Buffer_attach() anymore? b) in ./src/malloc, which one should I use, bsdmalloc or fbsdmalloc? Thanks and Best Regards, Daniel
__________________
~ Daniel WEI ------------- Boeing Research & Technology - China Beijing, China |
|
May 23, 2008, 14:13 |
- you can't use Info/Pout yet.
|
#6 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
- you can't use Info/Pout yet. Isn't initialised yet.
- check whether SGIMPI still does not like MPI_Buffer_attach. That exclusion is from long long time ago. |
|
July 15, 2008, 23:35 |
Hello Majjijs,
First, to Op
|
#7 |
Senior Member
Daniel WEI (老魏)
Join Date: Mar 2009
Location: Beijing, China
Posts: 689
Blog Entries: 9
Rep Power: 21 |
Hello Majjijs,
First, to OpenFOAM-1.5, congratulations! I am curious! Is the new version being tested on SGI? Or the endeavor has been given up? For It seems the wmake/rules/sgi* have been removed!! Another question where can I get OpenFOAM-1.5-dev? Regards, Daniel
__________________
~ Daniel WEI ------------- Boeing Research & Technology - China Beijing, China |
|
July 16, 2008, 08:27 |
Hi Daniel,
no we've not tes
|
#8 |
Senior Member
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,419
Rep Power: 26 |
Hi Daniel,
no we've not tested on SGI. Where possible we've split the OS specifics off to a separate library (OSspecific) to make porting easier. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Experience in building OpenFOAM on LinuxIA64 | Nils Smeds (Smeds) | OpenFOAM Installation | 43 | July 25, 2010 10:09 |
New building machine for OpenFoam | gerbervdgraaf | OpenFOAM Installation | 23 | December 9, 2009 03:39 |
Problems in Compiling OpenFOAM in IRIX | lakeat | OpenFOAM Installation | 0 | June 7, 2007 22:44 |
Building OpenFOAM 13 on solaris10Sun Opteron AMD64 | phil | OpenFOAM Installation | 0 | August 28, 2006 02:12 |
Building OpenFoAm on SGI Altix 64bits | anne | OpenFOAM Installation | 8 | June 15, 2006 10:27 |