|
[Sponsors] |
November 7, 2013, 13:53 |
simpleFoam parallel
|
#1 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
Hi,
I'm having a bit of trouble running simpleFoam in parallel. I am using the motorBike tutorial and trying to run it on 6 cores (processor is i7-4930k). I ran blockMesh, surfaceFeatureExtract & snappyHexMesh. I then commented out the functions part of the controlDict file (following a tutorial from a lecturer). Then I ran decomposePar, and viewed the individual meshes in paraFoam and everything seemed to have split up evenly. The next step I ran Code:
mpirun -np 6 simpleFoam -parallel Code:
/*---------------------------------------------------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: 2.2.2 | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ Build : 2.2.2-9240f8b967db Exec : simpleFoam -parallel Date : Nov 07 2013 Time : 18:47:22 Host : "andrew-pc" PID : 620 Case : /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel nProcs : 6 Slaves : 5 ( "andrew-pc.621" "andrew-pc.622" "andrew-pc.623" "andrew-pc.624" "andrew-pc.625" ) Pstream initialized with: floatTransfer : 0 nProcsSimpleSum : 0 commsType : nonBlocking polling iterations : 0 sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE). fileModificationChecking : Monitoring run-time modified files using timeStampMaster allowSystemOperations : Disallowing user-supplied system call operations // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Create time Create mesh for time = 3 Reading field p [0] [0] [0] --> FOAM FATAL IO ERROR: [0] cannot find file [0] [0] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor0/3/p at line 0. [0] [0] From function regIOobject::readStream() [0] in file db/regIOobject/regIOobjectRead.C at line 73. [0] FOAM parallel run exiting [0] [1] [1] [1] --> FOAM FATAL IO ERROR: [1] cannot find file [1] [1] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor1/3/p at line 0. [1] [1] From function regIOobject::readStream() [1] in file db/regIOobject/regIOobjectRead.C at line 73. [1] FOAM parallel run exiting [1] [2] [2] [2] --> FOAM FATAL IO ERROR: [2] cannot find file [2] [2] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor2/3/p at line 0. [2] [2] From function regIOobject::readStream() [2] in file db/regIOobject/regIOobjectRead.C at line 73. [2] FOAM parallel run exiting [2] -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 5 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- [3] [3] [3] --> FOAM FATAL IO ERROR: [3] cannot find file [3] [3] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor3/3/p at line 0. [3] [3] From function regIOobject::readStream() [3] in file db/regIOobject/regIOobjectRead.C at line 73. [3] FOAM parallel run exiting [3] [4] [4] [4] --> FOAM FATAL IO ERROR: [4] cannot find file [4] [4] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor4/3/p at line 0. [4] [4] From function regIOobject::readStream() [4] in file db/regIOobject/regIOobjectRead.C at line 73. [4] FOAM parallel run exiting [4] [5] [5] [5] --> FOAM FATAL IO ERROR: [5] cannot find file [5] [5] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor5/3/p at line 0. [5] [5] From function regIOobject::readStream() [5] in file db/regIOobject/regIOobjectRead.C at line 73. [5] FOAM parallel run exiting [5] -------------------------------------------------------------------------- mpirun has exited due to process rank 1 with PID 621 on node andrew-pc exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- [andrew-pc:00619] 5 more processes have sent help message help-mpi-api.txt / mpi-abort [andrew-pc:00619] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages andrew@andrew-pc:~/OpenFOAM/andrew-2.2.2/run/motorBikeParallel$ mpirun -np 6 simpleFoam -parallel > simpleFoamParallel.log [0] [0] [0] --> FOAM FATAL IO ERROR: [0] cannot find file [0] [0] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor0/3/p at line 0. [0] [0] From function regIOobject::readStream() [0] in file db/regIOobject/regIOobjectRead.C at line 73. [1] [0] FOAM parallel run exiting [0] [1] [1] --> FOAM FATAL IO ERROR: [1] cannot find file [1] [1] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor1/3/p at line 0. [1] [1] From function regIOobject::readStream() [1] in file db/regIOobject/regIOobjectRead.C[2] [2] [2] --> FOAM FATAL IO ERROR: [2] cannot find file [2] [2] at line 73. [1] FOAM parallel run exiting [1] [4] [4] [4] --> FOAM FATAL IO ERROR: [4] cannot find file [4] [4] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor4/3/p at line 0. [4] [4] From function file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor2/3/p at line 0. [2] [2] From function regIOobject::readStream() [2] in file db/regIOobject/regIOobjectRead.C at line 73. [2] FOAM parallel run exiting [2] regIOobject::readStream() [4] in file db/regIOobject/regIOobjectRead.C at line 73. [4] FOAM parallel run exiting [4] -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- [3] [3] [3] --> FOAM FATAL IO ERROR: [3] cannot find file [3] [3] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor3/3/p at line 0. [3] [5] [5] [5] --> FOAM FATAL IO ERROR: [5] cannot find file [5] [5] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor5/3/p at line 0. [5] [5] From function regIOobject::readStream() [3] From function regIOobject::readStream() [3] in file db/regIOobject/regIOobjectRead.C at line 73. [3] FOAM parallel run exiting [3] [5] in file db/regIOobject/regIOobjectRead.C at line 73. [5] FOAM parallel run exiting [5] -------------------------------------------------------------------------- mpirun has exited due to process rank 2 with PID 630 on node andrew-pc exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- [andrew-pc:00627] 5 more processes have sent help message help-mpi-api.txt / mpi-abort [andrew-pc:00627] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages andrew@andrew-pc:~/OpenFOAM/andrew-2.2.2/run/motorBikeParallel$ mpirun -np 6 simpleFoam -parallel > simpleFoamParallel.log [1] [1] [1] --> FOAM FATAL IO ERROR: [1] cannot find file [1] [1] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor1/3/p at line 0. [1] [1] From function regIOobject::readStream() [1] in file db/regIOobject/regIOobjectRead.C at line 73. [1] FOAM parallel run exiting [1] [2] [2] [2] --> FOAM FATAL IO ERROR: [2] cannot find file [2] [2] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor2/3/p at line 0. [2] [2] From function regIOobject::readStream() [2] in file [4] [4] [4] --> FOAM FATAL IO ERROR: [4] cannot find file [4] [4] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor4/3/p at line 0. [4] -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 5 in communicator MPI_COMM_WORLD with errorcode 1. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- [3] [3] [3] --> FOAM FATAL IO ERROR: [3] cannot find file [3] [3] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor3/3/p at line 0. [3] [3] From function regIOobject::readStream() [3] in file db/regIOobject/regIOobjectRead.C at line 73. [3] FOAM parallel run exiting [3] db/regIOobject/regIOobjectRead.C at line 73. [2] FOAM parallel run exiting [2] [5] [5] [5] --> FOAM FATAL IO ERROR: [5] cannot find file [5] [5] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor5/3/p at line 0. [5] [5] From function regIOobject::readStream() [5] in file db/regIOobject/regIOobjectRead.C at line 73. [5] FOAM parallel run exiting [5] [4] From function regIOobject::readStream() [4] in file db/regIOobject/regIOobjectRead.C at line 73. [4] FOAM parallel run exiting [4] [0] [0] [0] --> FOAM FATAL IO ERROR: [0] cannot find file [0] [0] file: /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/processor0/3/p at line 0. [0] [0] From function regIOobject::readStream() [0] in file db/regIOobject/regIOobjectRead.C at line 73. [0] FOAM parallel run exiting [0] -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 639 on node andrew-pc exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- [andrew-pc:00638] 5 more processes have sent help message help-mpi-api.txt / mpi-abort [andrew-pc:00638] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages Thanks for any help, Andrew |
|
November 8, 2013, 10:33 |
|
#2 |
Senior Member
Lieven
Join Date: Dec 2011
Location: Leuven, Belgium
Posts: 299
Rep Power: 22 |
Hi Andrew,
Is the p-file present in the 0-folder of the un-decomposed case? decomposePar simply takes all files that it can find and decomposes them. Could you post the output of decomposePar? Cheers, L |
|
November 8, 2013, 13:09 |
|
#3 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
Hi Lieven,
No there are no files in each processor directory other than the constant folder containing the polymesh directory with all of the dictionaries usually found within polymesh. This is the output from decomposePar: Code:
/*---------------------------------------------------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: 2.2.2 | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ Build : 2.2.2-9240f8b967db Exec : decomposePar Date : Nov 08 2013 Time : 18:07:53 Host : "andrew-pc" PID : 3455 Case : /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel2 nProcs : 1 sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE). fileModificationChecking : Monitoring run-time modified files using timeStampMaster allowSystemOperations : Disallowing user-supplied system call operations // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Create time Decomposing mesh region0 Create mesh Calculating distribution of cells Selecting decompositionMethod hierarchical Finished decomposition in 0.26 s Calculating original mesh data Distributing cells to processors Distributing faces to processors Distributing points to processors Constructing processor meshes Processor 0 Number of cells = 58130 Number of faces shared with processor 1 = 1993 Number of faces shared with processor 3 = 4080 Number of processor patches = 2 Number of processor faces = 6073 Number of boundary faces = 10495 Processor 1 Number of cells = 58130 Number of faces shared with processor 0 = 1993 Number of faces shared with processor 2 = 1310 Number of faces shared with processor 3 = 43 Number of faces shared with processor 4 = 3157 Number of processor patches = 4 Number of processor faces = 6503 Number of boundary faces = 12452 Processor 2 Number of cells = 58131 Number of faces shared with processor 1 = 1310 Number of faces shared with processor 4 = 38 Number of faces shared with processor 5 = 4835 Number of processor patches = 3 Number of processor faces = 6183 Number of boundary faces = 2017 Processor 3 Number of cells = 58130 Number of faces shared with processor 0 = 4080 Number of faces shared with processor 1 = 43 Number of faces shared with processor 4 = 1980 Number of processor patches = 3 Number of processor faces = 6103 Number of boundary faces = 10818 Processor 4 Number of cells = 58130 Number of faces shared with processor 1 = 3157 Number of faces shared with processor 2 = 38 Number of faces shared with processor 3 = 1980 Number of faces shared with processor 5 = 1390 Number of processor patches = 4 Number of processor faces = 6565 Number of boundary faces = 12158 Processor 5 Number of cells = 58131 Number of faces shared with processor 2 = 4835 Number of faces shared with processor 4 = 1390 Number of processor patches = 2 Number of processor faces = 6225 Number of boundary faces = 1921 Number of processor faces = 18826 Max number of cells = 58131 (0.00114685% above average 58130.3) Max number of processor patches = 4 (33.3333% above average 3) Max number of faces between processors = 6565 (4.61596% above average 6275.33) Time = 0 Processor 0: field transfer Processor 1: field transfer Processor 2: field transfer Processor 3: field transfer Processor 4: field transfer Processor 5: field transfer End. Andrew |
|
November 8, 2013, 17:10 |
|
#4 |
Senior Member
Lieven
Join Date: Dec 2011
Location: Leuven, Belgium
Posts: 299
Rep Power: 22 |
Ok, that doesn't explain a lot. Could you enter the following in the terminal and post the output:
Code:
ls /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/0/ L |
|
November 8, 2013, 17:39 |
|
#5 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
Apologies I mis-read your post. In the undecomposed case yes the k,nut,omega,p and U files are present as well as the include folder containing fixedInlet, frontBackUpperPatches,initialConditions
ls /home/andrew/openFOAM/andrew-2.2.2/run/motorBikeParallel/0.org gives; Code:
include k nut omega p U Code:
fixedInlet frontBackUpperPatches initialConditions Andrew |
|
November 8, 2013, 17:49 |
|
#6 |
Senior Member
Lieven
Join Date: Dec 2011
Location: Leuven, Belgium
Posts: 299
Rep Power: 22 |
Ah, ok, I think I understand. The 0.org is not recognized by decomposePar. Instead you should have a 0-folder. Try again with the following series of commands:
Code:
cd /home/andrew/OpenFOAM/andrew-2.2.2/run/motorBikeParallel/ cp -r 0.orig 0 decomposePar L |
|
November 8, 2013, 18:25 |
|
#7 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
Great! It is running now Many thanks for the help, it is quite frustrating when something so little is wrong and nothing will run.
Andrew |
|
November 8, 2013, 18:56 |
|
#8 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
I'm afraid I may have posted a bit prematurely.
After running mpirun -np 6 simpleFoam The time directories, 100,200,300 and 400 are not going into the processor folders. That is they are simply being put in the casefile folder. ls /home/andrew/openFOAM/andrew-2.2.2/run/motorBikeParallel gives: Code:
0 300 constant motorBikeParallel.foam processor2 processor5 system 100 400 log processor0 processor3 simpleFoam.log 200 500 mesh.run processor1 processor4 solver.run |
|
November 9, 2013, 03:35 |
|
#9 |
Senior Member
Lieven
Join Date: Dec 2011
Location: Leuven, Belgium
Posts: 299
Rep Power: 22 |
Hi Andrew,
Indeed, were not there yet but at least were getting closer ;-) You should add the option -parallel to the solver when you want to run in it parallel. So try Code:
mpirun -np 6 simpleFoam -parallel Try also to run the simulation with foamJob, a tool provided by OF: Code:
foamJob -p simpleFoam Cheers, L |
|
November 9, 2013, 12:32 |
|
#10 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
I'm not sure what the problem is now, but I keep getting a floating point error! This seems strange as it would suggest there's something wrong with the boundary conditions, but I am using the tutorial files. Also I spoke to a friend who has successfully ran it in parallel and I have used the exact same commands as him, the only difference being he has 8 cores and I have 6. As such I edited the decomposePar file to reflect this.
Both Code:
mpirun -np 6 simpleFoam -parallel Code:
foamJob -p parallel The log from this can be found here: https://www.dropbox.com/s/ol8git043gbz1ic/log (couldn't seem to upload it to the forums and too many characters to post) Cheers, Andrew |
|
November 12, 2013, 17:03 |
|
#11 |
New Member
Andrew Mortimer
Join Date: Oct 2013
Posts: 15
Rep Power: 12 |
Just a note to say I solved this problem by doing an mpirun of potentialFoam before running simpleFoam.
That is I ran: Code:
mpirun -np 6 potentialFoam -parallel |
|
August 7, 2015, 18:08 |
|
#12 |
Member
Mike
Join Date: Apr 2011
Location: Canada
Posts: 83
Rep Power: 15 |
Hi,
I am new to OpenFoam and I am having similar problem. The point is when I run decomposePar, it does not create folder "0" for each processor. I have done a test for 2D cases and anther 3D case and fir them decomposePar works OK. but now that I want to start running a real 3D cases, decomposePar does not do the job. I do not know what I am missing. BTW, I am running it on a public domain computer and here is the error message when I run decomposePar: Time = 0 --> FOAM FATAL IO ERROR: keyword type is undefined in dictionary "/global/home/saeedi/bluffbody-OP/Re300-AR4/0/p.boundaryField.outlet" file: /global/home/saeedi/bluffbody-OP/Re300-AR4/0/p.boundaryField.outlet from line 34 to line 34. From function dictionary::lookupEntry(const word&, bool, bool) const in file db/dictionary/dictionary.C at line 437. FOAM exiting |
|
August 7, 2015, 18:45 |
|
#13 |
Member
Mike
Join Date: Apr 2011
Location: Canada
Posts: 83
Rep Power: 15 |
I think I figured out the problem.
I did looked to my file "p" at the line which specifies outlet. I thing I missed a line for that (type fixed value). |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Foam::error::printStack(Foam::Ostream&) with simpleFoam -parallel | U.Golling | OpenFOAM Running, Solving & CFD | 52 | September 23, 2023 03:35 |
SimpleFoam run in Parallel | jayrup | OpenFOAM | 9 | July 26, 2019 00:00 |
simpleFoam in parallel issue | plucas | OpenFOAM Running, Solving & CFD | 3 | July 17, 2013 11:30 |
parallel Grief: BoundaryFields ok in single CPU but NOT in Parallel | JR22 | OpenFOAM Running, Solving & CFD | 2 | April 19, 2013 16:49 |
parallel simpleFoam freezes the whole system | vangelis | OpenFOAM Running, Solving & CFD | 14 | May 16, 2012 05:12 |