CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM Running, Solving & CFD (https://www.cfd-online.com/Forums/openfoam-solving/)
-   -   Some questions about a multi region case run in parallel (https://www.cfd-online.com/Forums/openfoam-solving/150211-some-questions-about-multi-region-case-run-parallel.html)

zfaraday March 17, 2015 13:41
Some questions about a multi region case run in parallel
 
Dear all,

I'm running some cases in parallel using chtMultiRegionSimpleFoam solver and during the whole process a few questions arose in my mind. First of all I will explain a little all the steps I'm taking during all the process:
  • Creation of the mesh using blockMesh and refinement of some zones with refineMesh
  • Decomposition (decomposePar) of the mesh in order to work in 4 processors
  • Creation of the cellZones with topoSet to split and create later all the regions (splitMeshRegions). Some other pre-processing utiliies are used at this stage, all of them are run, indeed, in parallel.
  • chtMultiRegionSimpleFoam is run in parallel.
  • wallHeatFlux is run, also, in parallel.
  • reconstructPar gives back the whole original geometry (including all the regions created in parallel)
  • The case is totally done!

Well, first of all, I use some function objects in order to obtain some values of the run. The most of these function objects calculate some values at patches. As the case is decomposed and run in parallel the following message is shown in the solver's log file:

Code:
--> FOAM Warning :
    From function patchFunctionObject::start()
    in file patch/patchFunctionObject/patchFunctionObject.C at line 88
    Patch procBoundary0to1 in "/home/cfd/OpenFOAM/cfd-2.3.x/run/PFC/Inicial/Final/geometryUpdated/extraInnerRefinement/parallel_3.2.1.3/tol-5/prova-2_2_1-full_decomp/system/controlDict.functions.Qr_air_patches" is coupled.
Disabling. If you want it enabled set 'allowCoupled true;'
What's the meaning of that? Is there any problem in using function objects to calculate some variables at patches that may belong to more than one processor? Should I be worried about this message (I get it more than once in a run)? In spite of this message all values seem to be calculated correctly(?) somehow.

Another problem I had to face during the process is that I can't reconstruct the case (for this reason this stage is marked in red in the list). This is the output from the utility:

Code:
/*---------------------------------------------------------------------------*\
| =========                |                                                |
| \\      /  F ield        | OpenFOAM: The Open Source CFD Toolbox          |
|  \\    /  O peration    | Version:  2.3.x                                |
|  \\  /    A nd          | Web:      www.OpenFOAM.org                      |
|    \\/    M anipulation  |                                                |
\*---------------------------------------------------------------------------*/
Build  : 2.3.x-ddb78f96a68b
Exec  : reconstructPar -allRegions
Date  : Mar 17 2015
Time  : 17:45:21
Host  : "linux-cfd"
PID    : 11950
Case  : /home/cfd/OpenFOAM/cfd-2.3.x/run/PFC/Inicial/Final/geometryUpdated/extraInnerRefinement/parallel_3.2.1.3/tol-5/prova-2_2_1-full_decomp
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Reconstructing for all regions in regionProperties



Reconstructing fields for mesh air



--> FOAM FATAL ERROR:
Cannot find file "points" in directory "air/polyMesh" in times 0 down to constant

    From function Time::findInstance(const fileName&, const word&, const IOobject::readOption, const word&)
    in file db/Time/findInstance.C at line 203.

FOAM exiting
Of course there is no file "points" under "constant/air/polyMesh" because regions have been created in parallel so there are 4 files called "points" under "processor*/constant/air/polyMesh".

How can I run reconstrucPar successfully in my case?

Besides that, I realised that foamLog doesn't detect regions. It would be very useful to plot residuals for each region. I also found out that pyFoam doesn't hande multi region cases when it comes to residual outputs. Is there a way that I'm missing to plot residuals for each region separately?

Many thanks in advance!

Best regards,

Alex

zfaraday March 21, 2015 11:48
No answers by now. I don't believe that thare aren't anyone that don't know how should I proceed to reconstruct my mesh properly. Can anyone, please, give me a hint on how to correctly reconstruct my multi region mesh?

I will be very grateful if somebody sheds a light on it! :)

alexeym March 21, 2015 12:41
Hi,

Quote:
Originally Posted by zfaraday (Post 536827)
Well, first of all, I use some function objects in order to obtain some values of the run. The most of these function objects calculate some values at patches. As the case is decomposed and run in parallel the following message is shown in the solver's log file:

Code:
--> FOAM Warning :
    From function patchFunctionObject::start()
    in file patch/patchFunctionObject/patchFunctionObject.C at line 88
    Patch procBoundary0to1 in "/home/cfd/OpenFOAM/cfd-2.3.x/run/PFC/Inicial/Final/geometryUpdated/extraInnerRefinement/parallel_3.2.1.3/tol-5/prova-2_2_1-full_decomp/system/controlDict.functions.Qr_air_patches" is coupled.
Disabling. If you want it enabled set 'allowCoupled true;'
If you take a look at the code (http://sourceforge.net/p/openfoam-ex...nctionObject.C), you see that in case of warning patch just will be ignored by function object. The reason, I guess, values of the patch field can differ at adjacent coupled patches if solver was not fully converged (see for example http://www.openfoam.org/mantisbt/view.php?id=625).

Quote:
Originally Posted by zfaraday (Post 536827)
Another problem I had to face during the process is that I can't reconstruct the case (for this reason this stage is marked in red in the list).
It would be great to have small example case. Multi-region tutorial case runs OK and can be reconstructed.

zfaraday March 21, 2015 13:54
Dear Alexey,

Thanks for your answer.
Quote:
Originally Posted by alexeym (Post 537601)
It would be great to have small example case. Multi-region tutorial case runs OK and can be reconstructed.
Yes, they do, but the procedure in which they are run is a little different from the one I'm using. Check carefully the procedure list I posted on my first post (I attach it here so that you don't even need to scroll up :P):
  • Creation of the mesh using blockMesh and refinement of some zones with refineMesh
  • Decomposition (decomposePar) of the mesh in order to work in 4 processors
  • Creation of the cellZones with topoSet to split and create later all the regions (splitMeshRegions). Some other pre-processing utiliies are used at this stage, all of them are run, indeed, in parallel.
  • chtMultiRegionSimpleFoam is run in parallel.
  • wallHeatFlux is run, also, in parallel.
  • reconstructPar gives back the whole original geometry (including all the regions created in parallel)
  • The case is totally done!

However, tutorials are run with a slight difference in the step list. This would be the list I would use if I follow the tutorial's procedure:
  • Creation of the mesh using blockMesh and refinement of some zones with refineMesh
  • Creation of the cellZones with topoSet to split and create later all the regions (splitMeshRegions). Some other pre-processing utiliies are used at this stage, all of them are NOT run, indeed, in parallel.
  • Decomposition (decomposePar -allRegions) of the mesh in order to work in 4 processors
  • chtMultiRegionSimpleFoam is run in parallel.
  • wallHeatFlux is run, also, in parallel.
  • reconstructPar -allRegions gives back the whole original geometry (including all the regions created in parallel)
  • The case is totally done!

I also ran the case successfully following the steps as they are shown in the tutorials, but I wanted to try the alternative method because I wanted to first decompose the mesh and manipulate it afterwards in parallel instead of only running the solver in parallel and do the other pre-processing steps in only one processor. Can you see the difference between both methods now? I don't know if I explained it well enough... Have you any suggestion on how to properly reconstruct my mesh?

Many thanks in advance!

sabago October 28, 2015 12:10
Dear OFers,
I'm having a similar issue (to the one quoted below) when I try the mpirun after decomposePar

Part of error is below
FOAM FATAL ERROR:
[4] Cannot find file "points" in directory "anode/polyMesh" in times 0 down to constant

"anode" is a sub-folder in the "constant" folder (along with the overall "polyMesh") with its own "polyMesh" but when I check the processors0-7, they do not have the "anode" folder which I think is where the problem is coming from.

Any help/ideas on how to fix thus?

Best,
Sandra

Quote:
Originally Posted by zfaraday (Post 536827)
Dear all,

I'm running some cases in parallel using chtMultiRegionSimpleFoam solver and during the whole process a few questions arose in my mind. First of all I will explain a little all the steps I'm taking during all the process:
  • Creation of the mesh using blockMesh and refinement of some zones with refineMesh
  • Decomposition (decomposePar) of the mesh in order to work in 4 processors
  • Creation of the cellZones with topoSet to split and create later all the regions (splitMeshRegions). Some other pre-processing utiliies are used at this stage, all of them are run, indeed, in parallel.
  • chtMultiRegionSimpleFoam is run in parallel.
  • wallHeatFlux is run, also, in parallel.
  • reconstructPar gives back the whole original geometry (including all the regions created in parallel)
  • The case is totally done!

Well, first of all, I use some function objects in order to obtain some values of the run. The most of these function objects calculate some values at patches. As the case is decomposed and run in parallel the following message is shown in the solver's log file:

Code:
--> FOAM Warning :
    From function patchFunctionObject::start()
    in file patch/patchFunctionObject/patchFunctionObject.C at line 88
    Patch procBoundary0to1 in "/home/cfd/OpenFOAM/cfd-2.3.x/run/PFC/Inicial/Final/geometryUpdated/extraInnerRefinement/parallel_3.2.1.3/tol-5/prova-2_2_1-full_decomp/system/controlDict.functions.Qr_air_patches" is coupled.
Disabling. If you want it enabled set 'allowCoupled true;'
What's the meaning of that? Is there any problem in using function objects to calculate some variables at patches that may belong to more than one processor? Should I be worried about this message (I get it more than once in a run)? In spite of this message all values seem to be calculated correctly(?) somehow.

Another problem I had to face during the process is that I can't reconstruct the case (for this reason this stage is marked in red in the list). This is the output from the utility:

Code:
/*---------------------------------------------------------------------------*\
| =========                |                                                |
| \\      /  F ield        | OpenFOAM: The Open Source CFD Toolbox          |
|  \\    /  O peration    | Version:  2.3.x                                |
|  \\  /    A nd          | Web:      www.OpenFOAM.org                      |
|    \\/    M anipulation  |                                                |
\*---------------------------------------------------------------------------*/
Build  : 2.3.x-ddb78f96a68b
Exec  : reconstructPar -allRegions
Date  : Mar 17 2015
Time  : 17:45:21
Host  : "linux-cfd"
PID    : 11950
Case  : /home/cfd/OpenFOAM/cfd-2.3.x/run/PFC/Inicial/Final/geometryUpdated/extraInnerRefinement/parallel_3.2.1.3/tol-5/prova-2_2_1-full_decomp
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Reconstructing for all regions in regionProperties



Reconstructing fields for mesh air



--> FOAM FATAL ERROR:
Cannot find file "points" in directory "air/polyMesh" in times 0 down to constant

    From function Time::findInstance(const fileName&, const word&, const IOobject::readOption, const word&)
    in file db/Time/findInstance.C at line 203.

FOAM exiting
Of course there is no file "points" under "constant/air/polyMesh" because regions have been created in parallel so there are 4 files called "points" under "processor*/constant/air/polyMesh".

How can I run reconstrucPar successfully in my case?

Besides that, I realised that foamLog doesn't detect regions. It would be very useful to plot residuals for each region. I also found out that pyFoam doesn't hande multi region cases when it comes to residual outputs. Is there a way that I'm missing to plot residuals for each region separately?

Many thanks in advance!

Best regards,

Alex

Zeppo February 23, 2017 10:25
Quote:
Originally Posted by zfaraday (Post 537614)
I also ran the case successfully following the steps as they are shown in the tutorials, but I wanted to try the alternative method because I wanted to first decompose the mesh and manipulate it afterwards in parallel instead of only running the solver in parallel and do the other pre-processing steps in only one processor. Can you see the difference between both methods now? I don't know if I explained it well enough... Have you any suggestion on how to properly reconstruct my mesh?
What advantages does the first method has over the second one? Performance?


All times are GMT -4. The time now is 12:06.