CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

directMappedFixedValue in parallel foam-extend 3.2

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 19, 2020, 10:47
Default directMappedFixedValue in parallel foam-extend 3.2
  #1
Member
 
Nat K
Join Date: Oct 2017
Posts: 60
Rep Power: 4
nskelly is on a distinguished road
Hi,

I'm looking to use directMappedFixedValue for LES simulations in foam-extend 3.2. However, I have come across a problem when working in parallel (which has been discussed before).

The mapped boundary condition doesn't decompose properly. I have tried decomposing with different methods (scotch, metis, hierarchial) and with different number of processors.

It seems to decompose up to 6, but anything beyond this it gives me the following error.

Code:
--> FOAM FATAL ERROR:
Did not find sample (-0.0205 0.000156695 0) on any processor of region region0

    From function directMappedPatchBase::findSamples(const pointField&, labelList&, labelList&, pointField&)
    in file directMapped/directMappedPolyPatch/directMappedPatchBase.C at line 381.

FOAM aborting

I am looking to use at least 60 or 120 processors, so at the moment it is not looking promising.

Any help would be great!

N.B. I have also tried on foam-extend 4.1 and it does the same.
nskelly is offline   Reply With Quote

Old   June 19, 2020, 13:03
Default
  #2
Senior Member
 
Santiago Lopez Castano
Join Date: Nov 2012
Posts: 282
Rep Power: 10
Santiago is on a distinguished road
Quote:
Originally Posted by nskelly View Post
Hi,

I'm looking to use directMappedFixedValue for LES simulations in foam-extend 3.2. However, I have come across a problem when working in parallel (which has been discussed before).

The mapped boundary condition doesn't decompose properly. I have tried decomposing with different methods (scotch, metis, hierarchial) and with different number of processors.

It seems to decompose up to 6, but anything beyond this it gives me the following error.

Code:
--> FOAM FATAL ERROR:
Did not find sample (-0.0205 0.000156695 0) on any processor of region region0

    From function directMappedPatchBase::findSamples(const pointField&, labelList&, labelList&, pointField&)
    in file directMapped/directMappedPolyPatch/directMappedPatchBase.C at line 381.

FOAM aborting

I am looking to use at least 60 or 120 processors, so at the moment it is not looking promising.

Any help would be great!

N.B. I have also tried on foam-extend 4.1 and it does the same.

put the patch in question in one single processor. You can do that in decomposeParDict.
Santiago is offline   Reply With Quote

Old   June 19, 2020, 14:21
Default
  #3
Member
 
Ardalan
Join Date: Jul 2012
Location: Toronto, Canada
Posts: 68
Rep Power: 10
Ardali is on a distinguished road
Quote:
Originally Posted by Santiago View Post
put the patch in question in one single processor. You can do that in decomposeParDict.
OpenFoam extended is an issue. Those versions are older than 5 years.
Try OFplus versions.
Ardali is offline   Reply With Quote

Old   June 19, 2020, 14:28
Default
  #4
Senior Member
 
Santiago Lopez Castano
Join Date: Nov 2012
Posts: 282
Rep Power: 10
Santiago is on a distinguished road
Quote:
Originally Posted by Ardali View Post
OpenFoam extended is an issue. Those versions are older than 5 years.
Try OFplus versions.
Why foam extend is an issue? latest version of extend is one year old, and its constantly updated.
Santiago is offline   Reply With Quote

Old   June 19, 2020, 14:34
Default
  #5
Member
 
Ardalan
Join Date: Jul 2012
Location: Toronto, Canada
Posts: 68
Rep Power: 10
Ardali is on a distinguished road
Quote:
Originally Posted by Santiago View Post
Why foam extend is an issue? latest version of extend is one year old, and its constantly updated.
When I was at Chalmers I used to publish with OF-extended3.2. It goes back to 2015-16.
Entended uses many codes and many funtionalities which are not fully debugged, like immersed boundary, and many dynamic meshes.

The BC that you are using may work in extended version but for sure it comes from foundation version.
Ardali is offline   Reply With Quote

Old   June 19, 2020, 16:35
Default
  #6
Member
 
Nat K
Join Date: Oct 2017
Posts: 60
Rep Power: 4
nskelly is on a distinguished road
I have used other OpenFOAM versions (OF5) and used the mapped condition on that version without any issues.

However, I am using specific solvers only in foam-extend 3.2, hence why I am using this over the 'standard' versions.
nskelly is offline   Reply With Quote

Old   June 19, 2020, 16:42
Default
  #7
Member
 
Nat K
Join Date: Oct 2017
Posts: 60
Rep Power: 4
nskelly is on a distinguished road
Quote:
Originally Posted by Santiago View Post
put the patch in question in one single processor. You can do that in decomposeParDict.
I'm too sure what you mean.
nskelly is offline   Reply With Quote

Old   June 19, 2020, 18:30
Default
  #8
Senior Member
 
Santiago Lopez Castano
Join Date: Nov 2012
Posts: 282
Rep Power: 10
Santiago is on a distinguished road
Quote:
Originally Posted by nskelly View Post
I'm too sure what you mean.
i assume you wanted to say "not sure"... what I mean is that foam-extend uses non-blocking mpi communications that, when reading data from a file may not find the chunck of data "on time"''. This is easily fixed by just putting the "directMapped" patch in a single processor. you can do it by adding your patch in the preservePatches entry in decomposeParDict dictionary...

Remember, IO during runtime is better done by one single process. In OpenFOAM (foundation, etc) they have created buffered commnications, which avoid race conditions, but make the code slower. In extend there us no such thing, so it puts a bit more responsibility on the user...
Santiago is offline   Reply With Quote

Old   June 19, 2020, 18:40
Default
  #9
Senior Member
 
Santiago Lopez Castano
Join Date: Nov 2012
Posts: 282
Rep Power: 10
Santiago is on a distinguished road
Quote:
Originally Posted by Ardali View Post
Entended uses many codes and many funtionalities which are not fully debugged, like immersed boundary, and many dynamic meshes.
I have not found those "many bugs" that you mention, but i do agree that extend is not as foolproof as OF.
Santiago is offline   Reply With Quote

Old   June 20, 2020, 04:58
Default
  #10
Member
 
Nat K
Join Date: Oct 2017
Posts: 60
Rep Power: 4
nskelly is on a distinguished road
Hi Santiago,

Quote:
Originally Posted by Santiago View Post
i assume you wanted to say "not sure"...
Yes (sorry it was a long day yesterday!)

Quote:
Originally Posted by Santiago View Post
what I mean is that foam-extend uses non-blocking mpi communications that, when reading data from a file may not find the chunck of data "on time"''. This is easily fixed by just putting the "directMapped" patch in a single processor. you can do it by adding your patch in the preservePatches entry in decomposeParDict dictionary...

Remember, IO during runtime is better done by one single process. In OpenFOAM (foundation, etc) they have created buffered commnications, which avoid race conditions, but make the code slower. In extend there us no such thing, so it puts a bit more responsibility on the user...
Thanks for this information, I am actually doing some benchmarking on a new HPC system and this is useful information to know!

I will definitely give it a try, and get back to you.
nskelly is offline   Reply With Quote

Old   June 25, 2020, 12:02
Default
  #11
New Member
 
Artem
Join Date: Apr 2014
Posts: 24
Rep Power: 8
Kombinator is on a distinguished road
Quote:
Originally Posted by nskelly View Post
Hi,

I'm looking to use directMappedFixedValue for LES simulations in foam-extend 3.2. However, I have come across a problem when working in parallel (which has been discussed before).

The mapped boundary condition doesn't decompose properly. I have tried decomposing with different methods (scotch, metis, hierarchial) and with different number of processors.

It seems to decompose up to 6, but anything beyond this it gives me the following error.

Code:
--> FOAM FATAL ERROR:
Did not find sample (-0.0205 0.000156695 0) on any processor of region region0

    From function directMappedPatchBase::findSamples(const pointField&, labelList&, labelList&, pointField&)
    in file directMapped/directMappedPolyPatch/directMappedPatchBase.C at line 381.

FOAM aborting

I am looking to use at least 60 or 120 processors, so at the moment it is not looking promising.

Any help would be great!

N.B. I have also tried on foam-extend 4.1 and it does the same.
Hi Nat,

I had recently exactly the same problem. There can be three reasons for that:

1. The mapped direction should be always decomposed in one core. So, if you mapped in x-direction then decompose parameters should be like that:

simpleCoeffs
{
n ( 1 y z);
delta 1e-16;
}

where y and z are varied parameters.

2. Reducing the delta parameter to e-16 can also help.

3. Sometimes if the division result of the number of cores in a specific direction by the amount of the mesh nodes in that direction is not an integer you will get the error. To cut a long story short - if you split z-direction into 6 cores and the amount of mesh nodes in the z-direction is 10 then it is possible to get errors (the amount of cores should be 2 or 5 in that case).

I hope it helps.

Regards,
Artem
Kombinator is offline   Reply With Quote

Old   June 26, 2020, 07:13
Default
  #12
Member
 
Nat K
Join Date: Oct 2017
Posts: 60
Rep Power: 4
nskelly is on a distinguished road
Hi Artem,

Thank you for your suggestions. I gave them all a go, but still no luck with get it to decompose properly.

None of your suggestions seem to work for both the pitzDailyDirectMapped tutorial case and my own cases.

Thanks anyway

Nat
nskelly is offline   Reply With Quote

Old   June 26, 2020, 07:20
Default
  #13
New Member
 
Artem
Join Date: Apr 2014
Posts: 24
Rep Power: 8
Kombinator is on a distinguished road
Quote:
Originally Posted by nskelly View Post
Hi Artem,

Thank you for your suggestions. I gave them all a go, but still no luck with get it to decompose properly.

None of your suggestions seem to work for both the pitzDailyDirectMapped tutorial case and my own cases.

Thanks anyway

Nat
Hi Nat,

I just checked pitzDailyDirectMappedtutorial and decomposed it for 4 domains without any trouble. I am using foam-extend 4.0
Tutorial decomposition seems to be okay for me..
Did you change something in the tutorial?

regards,
Artem

Last edited by Kombinator; June 26, 2020 at 10:28.
Kombinator is offline   Reply With Quote

Old   July 11, 2020, 03:52
Default
  #14
Member
 
Nat K
Join Date: Oct 2017
Posts: 60
Rep Power: 4
nskelly is on a distinguished road
Hi Artem,

Yes that tutorial works fine decomposing up to 6 processors. However, I will be using over 60 CPUs so it looks like this will not work for my cases.

Thanks
nskelly is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
[surface handling] surfaceFeatureExtract in Foam Extend 3.2 ashish.svm OpenFOAM Meshing & Mesh Conversion 2 October 12, 2017 08:54
decomposePar is missing a library whk1992 OpenFOAM Pre-Processing 8 March 7, 2015 07:53
[blockMesh] non-orthogonal faces and incorrect orientation? nennbs OpenFOAM Meshing & Mesh Conversion 7 April 17, 2013 05:42
[Commercial meshers] Using starToFoam clo OpenFOAM Meshing & Mesh Conversion 33 September 26, 2012 04:04
[blockMesh] BlockMesh FOAM warning gaottino OpenFOAM Meshing & Mesh Conversion 7 July 19, 2010 14:11


All times are GMT -4. The time now is 09:01.