CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

Building OpenFOAM on IRIX

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   May 5, 2008, 11:10
Default Dear All, I am struggling w
  #1
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 10
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Dear All,

I am struggling with porting OpenFoam to my IRIX with "-parallel" option on. That means I can run the solvers without "-parallel" very well, but as I mpirun my solver, it failed with annoying coredump. The output is at the bottom of my post.

what I don't understand is these lines in argList.C:
-------------------------------------------------

wordList machinesInfo(1);
machinesInfo[0] = hostName();

labelList pids(1);
pids[0] = pid();

if (parRunControl_.parRun())
{
if (Pstream::master())
{
machinesInfo.setSize(Pstream::nProcs());
pids.setSize(Pstream::nProcs());

// From all slaves
for
(
int slave=Pstream::firstSlave();
slave<=Pstream::lastSlave();
slave++
)
{
label slavei = slave - Pstream::masterNo();
IPstream fromSlave(slave);
fromSlave >> machinesInfo[slavei] >> pids[slavei];

}
else
{
OPstream toMaster(Pstream::masterNo(), 0, false);
toMaster << machinesInfo[0] << pids[0];
}
}
}
-------------------------------------------------

1. How can pids[slavei] get 4 (in my case) different pid()? The reason I raise the question is because I found my pids[1] is zero, and my machinesInfo[slavei] is empty! that's strange.
2. Is there a way to store the 3 slave (in my case) pids? (they are : 513638,513610,513608.)

Please help! Thanks in advance!

Daniel

--------------------------------------------------
bash-2.05$ mpirun -np 4 `which icoFoam` $FOAM_RUN/tutorials/icoFoam cavity -parallel
MPI Pstream initialized with:
floatTransfer : 1
nProcsSimpleSum : 0
scheduledTransfer : 0

/*---------------------------------------------------------------------------*\
| ========= | |
| \ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \ / O peration | Version: 1.4.1-dev |
| \ / A nd | Web: http://www.openfoam.org |
| \/ M anipulation | |
\*---------------------------------------------------------------------------*/

Exec : /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam /disk4/usr/people/student3/OpenFOAM/student3-1.4.1-dev/run/tutorials/icoFoam cavity -parallel
[0] Date : Apr 28 2008
[0] Time : 16:44:15
[0] Host : onyx-bridge
[0] PID : 513629
[3] Date : Apr 28 2008
[3] Time : 16:44:15
[3] Host : onyx-bridge
[3] PID : 513638
[1] Date : Apr 28 2008
[1] Time : 16:44:15
[1] Host : onyx-bridge
[1] PID : 513610
[2] Date : Apr 28 2008
[2] Time : 16:44:15
[2] Host : onyx-bridge
[2] PID : 513608
MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 3, Process 513638 received signal SIGBUS(10)


MPI: --------stack traceback-------
PC: 0x5ddb100 MPI_SGI_stacktraceback in /usr/lib32/libmpi.so
PC: 0x5ddb544 first_arriver_handler in /usr/lib32/libmpi.so
PC: 0x5ddb7d8 slave_sig_handler in /usr/lib32/libmpi.so
PC: 0xfaee79c _sigtramp in /usr/lib32/libc.so.1
libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x60204334 in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
0 0x60197ac8 abs
(a lot of messages the same as above)
(......)
115 0x60197ac8 abs
PC: 0x42d432c (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
0 0x60197ac8 abs
(a lot of messages the same as above)
(......)
6 0x60197ac8 abs
libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x6020374c in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
7 0x60197ac8 abs
(a lot of messages the same as above)
(......)
115 0x60197ac8 abs
PC: 0x42d3744 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x602064bc in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
0 0x60197ac8 abs
(a lot of messages the same as above)
(......)
115 0x60197ac8 abs
PC: 0x42d64b4 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x601ee308 in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
0 0x60197ac8 abs
(a lot of messages the same as above)
(......)
115 0x60197ac8 abs
PC: 0x42be300 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x601c262c in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
0 0x60197ac8 abs
(a lot of messages the same as above)
(......)
115 0x60197ac8 abs
PC: 0x4292624 (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
libexc(513638): ERROR __exc_get_fde_name: Couldn't find any name at 0x601b0904 in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
0 0x60197ac8 abs
(a lot of messages the same as above)
(......)
115 0x60197ac8 abs
PC: 0x42808fc (null) in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/lib/sgiN32GccDPDebug/libO penFOAM.so
PC: 0x1002aeec main in /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam


MPI: dbx version 7.3.3 (78517_Dec16 MR) Dec 16 2001 07:45:22
MPI: Process 513638 (icoFoam) stopped at [__waitsys:24 +0x8,0xfa53338]
MPI: Source (of /xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/proc/waitsys.s) not available for Process 513638
MPI: > 0 __waitsys(0x0, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/proc/waitsys.s":24, 0xfa53338]
MPI: 1 _system(0x7ffb5150, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/stdio/system.c":116, 0xfa5f868]
MPI: 2 MPI_SGI_stacktraceback(0x0, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv4/mpt/1.8/mpi/work/4.3/lib/libmpi/libmpi_n32_M4/adi/sig.c":242, 0x5ddb268]
MPI: 3 first_arriver_handler(0xa, 0x71756974, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv4/mpt/1.8/mpi/work/4.3/lib/libmpi/libmpi_n32_M4/adi/sig.c":445, 0x5ddb544]
MPI: 4 slave_sig_handler(0xa, 0x7d662, 0x7ffb5080, 0x3, 0x0, 0x7ffb4ec8, 0x0, 0xf8) ["/xlv4/mpt/1.8/mpi/work/4.3/lib/libmpi/libmpi_n32_M4/adi/sig.c":542, 0x5ddb7e0]
MPI: 5 _sigtramp(0x0, 0x7d662, 0x0, 0x3, 0x0, 0x0, 0x0, 0xf8) ["/xlv42/6.5.23m/work/irix/lib/libc/libc_n32_M4/signal/sigtramp.s":71, 0xfaee79c]
MPI: 6 Foam::IPstream::readFromBuffer(this = <illegal>, t = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/Pstreams/IPread.C":2, 0x42d4334]
MPI: 7 Foam::IPstream::read(this = <illegal>, l = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/Pstreams/IPread.C":27, 0x42d3744]
MPI: 8 Foam::IPstream::read(this = <illegal>, t = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/Pstreams/IPreadToken.C":4
MPI: More (n if no)?, 0x42d64b4]
MPI: 9 Foam::token::token(this = <illegal>, is = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/db/IOstrea ms/token/tokenIO.C":4, 0x42be300]
MPI: 10 <unknown>(is = <illegal>, L = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/global/arg List/argList.C":40, 0x4292624]
MPI: 11 <unknown>(this = <illegal>, argc = <illegal>, argv = <illegal>, checkArgs = <illegal>, checkOpts = <illegal>) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/src/OpenFOAM/global/arg List/argList.C":66, 0x42808fc]
MPI: 12 ::main(The debugger has detected a dbxpcs server error (Internal Consistency Error).
MPI: dbxpcs server attempting to continue
MPI: ) ["/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/solvers/in compressible/icoFoam/icoFoam.C":0, 0x1002aeec]
MPI: 13 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x1002a668]

MPI: -----stack traceback ends-----
MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 3, Process 513638: Dumping core on signal SIGBUS(10) into directory /disk4/usr/people/student3/OpenFOAM/student3-1.4.1-dev/run/tutorials/icoFoam
MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 1, Process 513610: Core dump on signal SIGBUS(10) suppressed.
MPI: Program /disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32Gc cDPDebug/icoFoam, Rank 2, Process 513608: Core dump on signal SIGBUS(10) suppressed.
MPI: MPI_COMM_WORLD rank 1 has terminated without calling MPI_Finalize()
MPI: aborting job
MPI: Received signal 9

__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   May 21, 2008, 11:27
Default Now, I found the problem is ar
  #2
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 10
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Now, I found the problem is around these line in argList.C
from line 339
--------------------------
else
{
// Collect the master's argument list
IPstream fromMaster(Pstream::masterNo());
fromMaster >> args_ >> options_;
}
--------------------------
obviously, the program fail at line
fromMaster >> args_ >> options_;

Could anybody give me some help on what does these lines mean, in Istream.H.
--------------------------
typedef Istream& (*IstreamManip)(Istream&);

//- operator>> handling for manipulators without arguments
inline Istream& operator>>(Istream& is, IstreamManip f)
{
return f(is);
}

//- operator>> handling for manipulators without arguments
inline Istream& operator>>(Istream& is, IOstreamManip f)
{
f(is);
return is;
}
-------------------------------

I doubt whether there is restriction on the length of filename on my IRIX, for when i insert
---------------
Sout << args() << endl;
---------------
like this
---------------
else
{
// Collect the master's argument list
Sout << args() << endl;
IPstream fromMaster(Pstream::masterNo());
fromMaster >> args_ >> options_;
}
----------------
output is
/disk4/usr/people/student3/OpenFOAM/OpenFOAM-1.4.1-dev/applications/bin/sgiN32SP Debug/icoFoam

is this too long?

Any hints? Thanks

Daniel
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   May 23, 2008, 03:42
Default (following up from http://www.
  #3
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 15
mattijs is on a distinguished road
(following up from http://www.cfd-online.com/cgi-bin/OpenFOAM_Discus/show.cgi?tpc=126&post=23069#POST230 69)

- write a non-foam c++ mpi test program which does mpi_init and a few send/receive.
- put a few print stmts into Pstream/mpi/*.C to show you what is being received/sent or link OF with a tracing library (mpe?)
- begin() returns a pointer to the underlying storage.
mattijs is offline   Reply With Quote

Old   May 23, 2008, 05:47
Default > write a non-foam c++ mpi tes
  #4
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 10
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
> write a non-foam c++ mpi test program which does mpi_init and a few send/receive.
I did it before and everying works perfect.

> put a few print stmts into Pstream/mpi/*.C to show you what is being received/sent or link OF with a tracing library (mpe?)
I am doing that!

Thank you, Mattijs!
Daniel
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   May 23, 2008, 12:28
Default Hi Mattijs! I found somethi
  #5
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 10
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Hi Mattijs!

I found something new. Look at the error messages I got!
......
MPI: dbx version 7.3.3 (78517_Dec16 MR) Dec 16 2001 07:45:22
MPI: Process 513638 (icoFoam) stopped at [__waitsys:24 +0x8,0xfa53338]
MPI: Source (of /xlv42.../waitsys.s) not available for Process 513638
MPI: > 0 __waitsys(0x0...........................sys.s":24, 0xfa53338]
MPI: 1 _system(0x7ffb5...................dio/system.c":116, 0xfa5f868]
MPI: 2 MPI_SGI_stacktr......................adi/sig.c":242, 0x5ddb268]
MPI: 3 first_arriver_h..............................c":44 5, 0x5ddb544]
MPI: 4 slave_sig_handl..........bmpi_n32_M4/adi/sig.c":542, 0x5ddb7e0]
MPI: 5 _sigtramp(0x0, ...............signal/sigtramp.s":71, 0xfaee79c]
MPI: 6 Foam::IPstream::readFromBuffer(this = <illegal>, t = <illegal>) [".../src/OpenFOAM/db/IOstrea ms/Pstreams/IPread.C":2, 0x42d4334]
MPI: 7 Foam::IPstream::read(this = <illegal>, l = <illegal>) [".../src/OpenFOAM/db/IOstreams/Pstreams/IPread.C":27, 0x42d3744]
MPI: 8 Foam::IPstream::read(this = <illegal>, t = <illegal>) [".../src/OpenFOAM/db/IOstreams/Pstreams/IPreadToken.C":4
MPI: 9 Foam::token::token(this = <illegal>, is = <illegal>) [".../src/OpenFOAM/db/IOstreams/token/tokenIO.C":4, 0x42be300]
MPI: 10 <unknown>(is = <illegal>, L = <illegal>) [".../src/OpenFOAM/global/arg List/argList.C":40, 0x4292624]
MPI: 11 <unknown>(this = <illegal>, argc = <illegal>, argv = <illegal>, checkArgs = <illegal>, checkOpts = <illegal>) [".../src/OpenFOAM/global/arg List/argList.C":66, 0x42808fc]
MPI: 12 ::main(The debugger has detected a dbxpcs server error (Internal Consistency Error).
MPI: dbxpcs server attempting to continue
MPI: ) [".../applications/solvers/in compressible/icoFoam/icoFoam.C":0, 0x1002aeec]
MPI: 13 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x1002a668]
......


1st,
from "MPI: 12" to "MPI: 6", it seems that the "Foam::IPstream::readFromBuffer" (in ./src/OpenFOAM/IOstreams/Pstreams/IPread.C) is the arch-criminal, I GUESS!
So, I tried to print out the "bufPosition_" & "buf_[bufPosition_]" as the following.
Note, I used "Sout" instead of "Info", for the latter didn't work in slave process, that's strange.
|......
|template<class>
|inline void IPstream::readFromBuffer(T& t)
|{
|````Sout<< "bufPosition_````````:" << bufPosition_ << nl````````````(Output is 1)
|````````<< "buf_[bufPosition_]``:" << buf_[bufPosition_] << endl;```(Output is nothing! This is the same as buf_.begin() and ||||buf_.end())
|````t = reinterpret_cast<t&>(buf_[bufPosition_]);```````````````````(ERROR!!!!!!!!!!!)
|````Sout<< "blahblahblahblahblahblahblahblahblah " << endl;```````` (Die with NO output)
|````bufPosition_ += sizeof(T);
|````checkEof();
|
|````//readFromBuffer(&t, sizeof(T));
|}
|......


2nd,
In error messages, there are so many "illegal"s, though everything works fine with single CPU of IRIX.
So, I have 2 questions concerning line 87-106 in ./src/Pstream/mpi/Pstream.C

|......
|#```ifndef SGIMPI
|````string bufferSizeName = getEnv("MPI_BUFFER_SIZE");
|
|````if (bufferSizeName.size())
|````{
|````````int bufferSize = atoi(bufferSizeName.c_str());
|
|````````if (bufferSize)
|````````{
|````````````MPI_Buffer_attach(new char[bufferSize], bufferSize);
|````````}
|````}
|````else
|````{
|````````FatalErrorIn("Pstream::init(int& argc, char**& argv)")
|````````````<< "Pstream::init(int& argc, char**& argv) : "
|````````````<< "environment variable MPI_BUFFER_SIZE not defined"
|````````````<< Foam::abort(FatalError);
|````}
|#```endif
|......


a) what if I use SGIMPI now, is that SGIMPI don't need MPI_Buffer_attach() anymore?
b) in ./src/malloc, which one should I use, bsdmalloc or fbsdmalloc?


Thanks and Best Regards, Daniel
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   May 23, 2008, 13:13
Default - you can't use Info/Pout yet.
  #6
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 15
mattijs is on a distinguished road
- you can't use Info/Pout yet. Isn't initialised yet.

- check whether SGIMPI still does not like MPI_Buffer_attach. That exclusion is from long long time ago.
mattijs is offline   Reply With Quote

Old   July 15, 2008, 22:35
Default Hello Majjijs, First, to Op
  #7
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 10
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Hello Majjijs,

First, to OpenFOAM-1.5, congratulations!

I am curious!
Is the new version being tested on SGI?
Or the endeavor has been given up? For It seems the wmake/rules/sgi* have been removed!!

Another question where can I get OpenFOAM-1.5-dev?

Regards,
Daniel
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   July 16, 2008, 07:27
Default Hi Daniel, no we've not tes
  #8
Super Moderator
 
Mattijs Janssens
Join Date: Mar 2009
Posts: 1,416
Rep Power: 15
mattijs is on a distinguished road
Hi Daniel,

no we've not tested on SGI. Where possible we've split the OS specifics off to a separate library (OSspecific) to make porting easier.
mattijs is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Experience in building OpenFOAM on LinuxIA64 Nils Smeds (Smeds) OpenFOAM Installation 43 July 25, 2010 09:09
New building machine for OpenFoam gerbervdgraaf OpenFOAM Installation 23 December 9, 2009 02:39
Problems in Compiling OpenFOAM in IRIX lakeat OpenFOAM Installation 0 June 7, 2007 21:44
Building OpenFOAM 13 on solaris10Sun Opteron AMD64 phil OpenFOAM Installation 0 August 28, 2006 01:12
Building OpenFoAm on SGI Altix 64bits anne OpenFOAM Installation 8 June 15, 2006 09:27


All times are GMT -4. The time now is 04:39.