| supvato |
February 22, 2022 10:12 |
parallel run error cma-permission-denied
Hello.
I am having some trouble with a parallel run using twoPhaseEulerFoam, and I was hoping someone here could give me some guidance on how to solve the issue.
I want to run twoPhaseEulerFoam in parallel using Ubuntu in WSL with OpenFOAM v2006. I have a mesh with ~2.4M elements, which runs ok (but slow) in series. decomposePar runs without any errors. When I run twoPhaseEulerFoam in parallel
Code:
mpirun -np 8 twoPhaseEulerFoam -parallel
it runs, but it gives me warning:
Code:
WARNING: Linux kernel CMA support was requested via the
btl_vader_single_copy_mechanism MCA variable, but CMA support is
not available due to restrictive ptrace settings.
The vader shared memory BTL will fall back on another single-copy
mechanism if one is available. This may result in lower performance.
I then attempted to run the depthCharge3D tutorial and get this:
Code:
tutorial/multiphase/compressibleInterFoam/laminar/depthCharge3D$ mpirun -np 6 compressibleInterFoam -parallel
--------------------------------------------------------------------------
WARNING: Linux kernel CMA support was requested via the
btl_vader_single_copy_mechanism MCA variable, but CMA support is
not available due to restrictive ptrace settings.
The vader shared memory BTL will fall back on another single-copy
mechanism if one is available. This may result in lower performance.
Local host: DESKTOP-O9OILL4
--------------------------------------------------------------------------
/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: v2006 |
| \\ / A nd | Website: www.openfoam.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : _b45f8f6f58-20200629 OPENFOAM=2006
Arch : "LSB;label=32;scalar=64"
Exec : compressibleInterFoam -parallel
Date : Feb 11 2022
Time : 10:48:30
Host : DESKTOP-O9OILL4
PID : 953
I/O : uncollated
Case : /mnt/d/OpenFOAM/tutorial/multiphase/compressibleInterFoam/laminar/depthCharge3D
nProcs : 6
Hosts :
(
(DESKTOP-O9OILL4 6)
)
Pstream initialized with:
floatTransfer : 0
nProcsSimpleSum : 0
commsType : nonBlocking
polling iterations : 0
trapFpe: Floating point exception trapping enabled (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 5, maxFileModificationPolls 20)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Create mesh for time = 0.5
I looked around on google, and found that you could modify the ptrace configuration in
Code:
etc/sysctl.d/10-ptrace.conf
I went and changed the specific line dealing with ptrace, but nothing changed.
I also tried running the parallel test application
Code:
foamJob -p -s Test-parallel
and the execution goes for a while but quits with an error. I've pasted part of the output below.
Code:
user@DESKTOP:/mnt/d/OpenFOAM/case_x$ foamJob -p -s Test-parallel
Application : Test-parallel (8 processes)
Output : log
Executing : /home/.../OpenFOAM/ThirdParty-v2006/platforms/linux64Gcc63/openmpi-4.0.3/bin/mpirun -n 8 /home/.../OpenFOAM/OpenFOAM-v2006/bin/tools/foamExec Test-parallel -parallel
--------------------------------------------------------------------------
WARNING: Linux kernel CMA support was requested via the
btl_vader_single_copy_mechanism MCA variable, but CMA support is
not available due to restrictive ptrace settings.
The vader shared memory BTL will fall back on another single-copy
mechanism if one is available. This may result in lower performance.
Local host: DESKTOP
--------------------------------------------------------------------------
/*---------------------------------------------------------------------------*\
| ========= | |
| \\ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \\ / O peration | Version: v2006 |
| \\ / A nd | Website: www.openfoam.com |
| \\/ M anipulation | |
\*---------------------------------------------------------------------------*/
Build : _b45f8f6f58-20200629 OPENFOAM=2006
Arch : "LSB;label=32;scalar=64"
Exec : Test-parallel -parallel
Date : Feb 11 2022
Time : 10:25:35
Host : DESKTOP
PID : 23465
I/O : uncollated
Case : /mnt/d/OpenFOAM/case_x
nProcs : 8
Hosts :
(
(DESKTOP 8)
)
Pstream initialized with:
floatTransfer : 0
nProcsSimpleSum : 0
commsType : nonBlocking
polling iterations : 0
trapFpe: Floating point exception trapping enabled (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 5, maxFileModificationPolls 20)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
--> FOAM Warning :
From void Foam::Time::setControls()
in file db/Time/Time.C at line 244
Increasing the timePrecision from 6 to 18 to support the formatting of the current time directory 34.4500292216046304
[0] test transfer (N4Foam6VectorIdEE): (0 1 2)
[0]
[0] master receiving from slave 1
[0] (0 1 2)
[0] master receiving from slave 2
[0] (0 1 2)
[0] master receiving from slave 3
[0] (0 1 2)
[0] master receiving from slave 4
[0] (0 1 2)
[0] master receiving from slave 5
[0] (0 1 2)
[0] master receiving from slave 6
[0] (0 1 2)
[0] master receiving from slave 7
[0] (0 1 2)
[0] [1] complexData:
[1] 115
[1] (
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[1] (1 3(1 2 3))
[2] complexData:
[2] 106
[2] (
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[2] (2 3(1 2 3))
[5] complexData:
[5] 87
[5] (
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[5] (5 3(1 2 3))
[6] complexData:
[6] 106
[6] (
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[6] (6 3(1 2 3))
[7] complexData:
[7] 84
[7] (
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[7] (7 3(1 2 3))
[0] complexData:
[0] 99
[0] (
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] (0 3(1 2 3))
[0] test tokenized "a"
[0]
[0] master receiving from slave 1
[0] on line 0: word 'a'
[0] master receiving from slave 2
[0] on line 0: word 'a'
[0] master receiving from slave 3
[0] on line 0: word 'a'
[0] master receiving from slave 4
[0] on line 0: word 'a'
[0] master receiving from slave 5
[0] on line 0: word 'a'
[0] master receiving from slave 6
[0] on line 0: word 'a'
[0] master receiving from slave 7
[0] on line 0: word 'a'
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized "$"
[0]
[0] master receiving from slave 1
[0] on line 0: error
[0] master receiving from slave 2
[0] on line 0: error
[0] master receiving from slave 3
[0] on line 0: error
[0] master receiving from slave 4
[0] on line 0: error
[0] master receiving from slave 5
[0] on line 0: error
[0] master receiving from slave 6
[0] on line 0: error
[0] master receiving from slave 7
[0] on line 0: error
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized ""test string1""
[0]
[0] master receiving from slave 1
[0] on line 0: string "test string1"
[0] master receiving from slave 2
[0] on line 0: string "test string1"
[0] master receiving from slave 3
[0] on line 0: string "test string1"
[0] master receiving from slave 4
[0] on line 0: string "test string1"
[0] master receiving from slave 5
[0] on line 0: string "test string1"
[0] master receiving from slave 6
[0] on line 0: string "test string1"
[0] master receiving from slave 7
[0] on line 0: string "test string1"
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized "test string1"
[0]
[0] master receiving from slave 1
[0] on line 0: word 'teststring1'
[0] master receiving from slave 2
[0] on line 0: word 'teststring1'
[0] master receiving from slave 3
[0] on line 0: word 'teststring1'
[0] master receiving from slave 4
[0] on line 0: word 'teststring1'
[0] master receiving from slave 5
[0] on line 0: word 'teststring1'
[0] master receiving from slave 6
[0] on line 0: word 'teststring1'
[0] master receiving from slave 7
[0] on line 0: word 'teststring1'
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized "3.141 59"
[0]
[0] master receiving from slave 1
[0] on line 0: word '3.141 59'
[0] master receiving from slave 2
[0] on line 0: word '3.141 59'
[0] master receiving from slave 3
[0] on line 0: word '3.141 59'
[0] master receiving from slave 4
[0] on line 0: word '3.141 59'
[0] master receiving from slave 5
[0] on line 0: word '3.141 59'
[0] master receiving from slave 6
[0] on line 0: word '3.141 59'
[0] master receiving from slave 7
[0] on line 0: word '3.141 59'
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized "" a ""
[0]
[0] master receiving from slave 1
[0] on line 0: string " a "
[0] master receiving from slave 2
[0] on line 0: string " a "
[0] master receiving from slave 3
[0] on line 0: string " a "
[0] master receiving from slave 4
[0] on line 0: string " a "
[0] master receiving from slave 5
[0] on line 0: string " a "
[0] master receiving from slave 6
[0] on line 0: string " a "
[0] master receiving from slave 7
[0] on line 0: string " a "
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized " a "
[0]
[0] master receiving from slave 1
[0] on line 0: word 'a'
[0] master receiving from slave 2
[0] on line 0: word 'a'
[0] master receiving from slave 3
[0] on line 0: word 'a'
[0] master receiving from slave 4
[0] on line 0: word 'a'
[0] master receiving from slave 5
[0] on line 0: word 'a'
[0] master receiving from slave 6
[0] on line 0: word 'a'
[0] master receiving from slave 7
[0] on line 0: word 'a'
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized "" $ ""
[0]
[0] master receiving from slave 1
[0] on line 0: string " $ "
[0] master receiving from slave 2
[0] on line 0: string " $ "
[0] master receiving from slave 3
[0] on line 0: string " $ "
[0] master receiving from slave 4
[0] on line 0: string " $ "
[0] master receiving from slave 5
[0] on line 0: string " $ "
[0] master receiving from slave 6
[0] on line 0: string " $ "
[0] master receiving from slave 7
[0] on line 0: string " $ "
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized " $ "
[0]
[0] master receiving from slave 1
[0] on line 0: error
[0] master receiving from slave 2
[0] on line 0: error
[0] master receiving from slave 3
[0] on line 0: error
[0] master receiving from slave 4
[0] on line 0: error
[0] master receiving from slave 5
[0] on line 0: error
[0] master receiving from slave 6
[0] on line 0: error
[0] master receiving from slave 7
[0] on line 0: error
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized "" $$ ""
[0]
[0] master receiving from slave 1
[0] on line 0: string " $$ "
[0] master receiving from slave 2
[0] on line 0: string " $$ "
[0] master receiving from slave 3
[0] on line 0: string " $$ "
[0] master receiving from slave 4
[0] on line 0: string " $$ "
[0] master receiving from slave 5
[0] on line 0: string " $$ "
[0] master receiving from slave 6
[0] on line 0: string " $$ "
[0] master receiving from slave 7
[0] on line 0: string " $$ "
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[0] test tokenized " $$ "
[0]
[0] master receiving from slave 1
[0] on line 0: word '$$'
[0] master receiving from slave 2
[0] on line 0: word '$$'
[0] master receiving from slave 3
[0] on line 0: word '$$'
[0] master receiving from slave 4
[0] on line 0: word '$$'
[0] master receiving from slave 5
[0] on line 0: word '$$'
[0] master receiving from slave 6
[0] on line 0: word '$$'
[0] master receiving from slave 7
[0] on line 0: word '$$'
[0] master sending to slave 1
[0] master sending to slave 2
[0] master sending to slave 3
[0] master sending to slave 4
[0] master sending to slave 5
[0] master sending to slave 6
[0] master sending to slave 7
[DESKTOP:23460] 7 more processes have sent help message help-btl-vader.txt / cma-permission-denied
[DESKTOP:23460] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
I have uploaded the full output here, in case someone needs to look at it.
Does anyone have any ideas on how to solve this issue?
Thanks for any help. |
