[didomenico]

Hello everybody,

I would like to know if OpenFOAM 1.3 is able
to perform parallel calculations with cyclic
patches splitted across processors.
I have tried such a case for a coodles tutorial

testCyclic.tgz

blockMesh and decomposePar gave no errors, but when I run the case, I get the following error message:

mpirun -np 2 coodles . testCyclic -parallel
/*---------------------------------------------------------------------------*\
| ========= | |
| \ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \ / O peration | Version: 1.3 |
| \ / A nd | Web: http://www.openfoam.org |
| \/ M anipulation | |
\*---------------------------------------------------------------------------*/

/*---------------------------------------------------------------------------*\
| ========= | |
| \ / F ield | OpenFOAM: The Open Source CFD Toolbox |
| \ / O peration | Version: 1.3 |
| \ / A nd | Web: http://www.openfoam.org |
| \/ M anipulation | |
\*---------------------------------------------------------------------------*/

Exec : coodles . testCyclic -parallel
Exec : coodles . testCyclic -parallel
[1] Date : Feb 23 2007
[1] Time : 10:04:39
[0] Date : Feb 23 2007
[1] Host : vt-numerik13
[1] PID : 24169
[0] Time : 10:04:39
[0] Host : vt-numerik13
[0] PID : 24168
[1] Root : /home/didomax/OpenFOAM/didomax-1.3/run
[1] Case : testCyclic
[1] Nprocs : 2
[0] Root : /home/didomax/OpenFOAM/didomax-1.3/run
[0] Case : testCyclic
[0] Nprocs : 2
[0] Slaves :
[0] 1
[0] (
[0] vt-numerik13.24169
[0] )
[0]
Create time

Create mesh for time = 0

[0] [1]
[1]
[1] --> FOAM FATAL ERROR : Cannot find patch edge with vertices (111 846) on patch procBoundary1to0
Can only find edges
3
(
(296 111)
(111 112)
(248 111)
)
connected to first vertex
[1]
[1] From function processorPolyPatch::updateMesh()
[1] in file meshes/polyMesh/polyPatches/derivedPolyPatches/processorPolyPatch/processorPolyPatch.C at line 351.
[1]
FOAM parallel run aborting
[1]
[1] _ZN4Foam5error10printStackERNS_7OstreamE
[1] _ZN4Foam5error5abortEv
[1] _ZN4Foam18processorPolyPatch10updateMeshEv
[1] _ZN4Foam16polyBoundaryMesh10updateMeshEv
[1] _ZN4Foam8polyMeshC2ERKNS_8IOobjectE
[1] _ZN4Foam6fvMeshC1ERKNS_8IOobjectE
[1] coodles [0x805829c]
[1] __libc_start_main
[1] __gxx_personality_v0

[0]
[0] --> FOAM FATAL ERROR : Cannot find patch edge with vertices (247 983) on patch procBoundary0to1
Can only find edges
3
(
(247 430)
(111 247)
(247 246)
)
connected to first vertex
-----------------------------------------------------------------------------
One of the processes started by mpirun has exited with a nonzero exit
code. This typically indicates that the process finished in error.
If your process did not finish in error, be sure to include a "return
0" or "exit(0)" in your C code before exiting the application.

PID 24168 failed on node n0 (127.0.0.1) with exit status 1.
-----------------------------------------------------------------------------

For such a simple testcase I would be
able to find a domain decomposition which does not
split the cyclic patches (1 1 2), but this is not suited for the cases I want to simulate.
Are there any options in the decomposePar utility in order to avoid such an error? Is the suggestion in thread

http://www.cfd-online.com/OpenFOAM_D...ges/1/415.html

the only way to get the code running?

Regards,
Max

[mattijs]

The suggestion is still a good one (so make it a warning)

You can avoid it by either having the cyclic halves completely on separate processors or all on the same processor.

[cosimobianchini]

Hi Matijs, I also got some problems dealing with parallel calculations with cyclic. I found expecially very difficult to follow your advices for decomposition in cases such as turbine blades in which it is not easy to automatically decompose the domain not cutting the cyclic halves.
Is there a smarter way than doing manual decomposition, for solving such problem?

About the third suggestion you gave in the above mentioned link [cite: "just change the FatalErrorIn..abort(FatalError) into a WarningIn..endl"] I don't understand where I must do this change, can you give me any hint?.

Thank you very much for your help
Cosimo

[mattijs]

Probably there is a smarter way but haven't thought about it.

Find out the line number that aborts. It will be a

FatalErrorIn(...) ... << abort(FatalError)

construct (or with 'exit(FatalError)'). Replace 'FatalErrorIn' with 'WarningIn ' and 'abort(FatalError)' with 'endl'.

and recompile.

[cosimobianchini]

Thanks a lot Mattijs and sorry for giving no answers for such a long time but I could try your suggestions only this morning.
Converting the fatal error into a warning let my simulations go.
Results seems to be almost coincident with single processor ones, even in case decomposition is done across cyclic: I believe parallelization is working right.
I try reading the code of the processorPolyPatch::updateMesh() but I was not able to understand under which condition it is correct to consider the fatal error at line 351 a simple warning.
Can you please tell me the purpose of such a check? I mean why is it not adviced to "cut" cyclic patches on different processors?
Thanks a lot for your patience.
Cosimo