Parallelizing decomposePar utility

dogukanteber · July 14, 2022, 11:49

Hello, all.

I am a software engineering student and am pretty novice in terms of OpenFOAM and CFD. I am doing an internship to gain experience in HPC and parallel computation. My responsibility is to develop a utility that does the exact same thing as decomposePar does but in parallel so that the decomposition of meshes is performed as quickly as possible. Also, the initial version will only support simple decomposition method.

I know how to handle the parallelization stuff but I cannot understand the structure of a mesh. Since I do not understand a mesh, I do not know how to decompose a mesh. I have been reading the source code of decomposePar to understand how decomposition is done but without comprehending the concept of mesh, the code does not mean too much to me.

So my question is how can I decompose a mesh? In addition to that, can you recommend resources to understand the structure of a mesh expect this?

Thank you

olesen · July 14, 2022, 17:38

The simplest would be to use "redistributePar -decompose", which sounds much like what you want. If there is something extra you need for it, follow up here or (if you have funding and a project) on develop.openfoam.com

I think you will find understanding the redistributePar code much more difficult than decomposePar. Not a realistic task as an internship at all.

BTW: redistributePar has been updated in the latest (OPENFOAM-V2206) release to support point fields and finiteArea fields.

dogukanteber · July 18, 2022, 07:51

Thank you for your reply olesen and sorry for the late response. I wanted to be sure before I wrote this post.

I have talked to my mentor, who said that he had already tried other options but on each try, the performance did not make so much difference compared to decomposePar. That is why we are developing this utility.

I am open to other suggestions though.

olesen · July 18, 2022, 09:22

I noticed that you didn't mention which version of OpenFOAM you were using. See, for example, (https://develop.openfoam.com/Develop.../-/issues/1211) - which avoids triggering costly intermediate topological rebuilds. Might make sense to field through https://www.openfoam.com/products-se...ftware-support as a project to diagnose where your bottlenecks arise and what can be done about them. As I already mentioned, this is quite unrealistic as a project for an intern without prior and extensive OpenFOAM knowledge.

July 14, 2022, 11:49	Parallelizing decomposePar utility	#1
dogukanteber New Member Doğukan Teber Join Date: Jul 2022 Location: Izmir, Türkiye Posts: 13 Rep Power: 3	Hello, all. I am a software engineering student and am pretty novice in terms of OpenFOAM and CFD. I am doing an internship to gain experience in HPC and parallel computation. My responsibility is to develop a utility that does the exact same thing as decomposePar does but in parallel so that the decomposition of meshes is performed as quickly as possible. Also, the initial version will only support simple decomposition method. I know how to handle the parallelization stuff but I cannot understand the structure of a mesh. Since I do not understand a mesh, I do not know how to decompose a mesh. I have been reading the source code of decomposePar to understand how decomposition is done but without comprehending the concept of mesh, the code does not mean too much to me. So my question is how can I decompose a mesh? In addition to that, can you recommend resources to understand the structure of a mesh expect this? Thank you

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Other] Contribution a new utility: refine wall layer mesh based on yPlus field	lakeat	OpenFOAM Community Contributions	58	December 23, 2021 02:36
Something doens't work with wallHeatFlux utility or externalWallHeatFluxTemperat BC!!	zfaraday	OpenFOAM Post-Processing	0	February 5, 2015 16:47
DecomposePar utility	hsing	OpenFOAM Running, Solving & CFD	11	January 19, 2015 04:09
[Other] How to create uneven load for the processors using decomposePar utility.	shinde.gopal	OpenFOAM Meshing & Mesh Conversion	1	May 24, 2014 08:49
About the problem of decomposePar utility	zou_mo	OpenFOAM Running, Solving & CFD	3	July 27, 2005 18:18

July 14, 2022, 17:38		#2
olesen Senior Member Mark Olesen Join Date: Mar 2009 Location: https://olesenm.github.io/ Posts: 1,689 Rep Power: 40	The simplest would be to use "redistributePar -decompose", which sounds much like what you want. If there is something extra you need for it, follow up here or (if you have funding and a project) on develop.openfoam.com I think you will find understanding the redistributePar code much more difficult than decomposePar. Not a realistic task as an internship at all. BTW: redistributePar has been updated in the latest (OPENFOAM-V2206) release to support point fields and finiteArea fields.

July 18, 2022, 07:51		#3
dogukanteber New Member Doğukan Teber Join Date: Jul 2022 Location: Izmir, Türkiye Posts: 13 Rep Power: 3	Thank you for your reply olesen and sorry for the late response. I wanted to be sure before I wrote this post. I have talked to my mentor, who said that he had already tried other options but on each try, the performance did not make so much difference compared to decomposePar. That is why we are developing this utility. I am open to other suggestions though.

July 18, 2022, 09:22		#4
olesen Senior Member Mark Olesen Join Date: Mar 2009 Location: https://olesenm.github.io/ Posts: 1,689 Rep Power: 40	I noticed that you didn't mention which version of OpenFOAM you were using. See, for example, (https://develop.openfoam.com/Develop.../-/issues/1211) - which avoids triggering costly intermediate topological rebuilds. Might make sense to field through https://www.openfoam.com/products-se...ftware-support as a project to diagnose where your bottlenecks arise and what can be done about them. As I already mentioned, this is quite unrealistic as a project for an intern without prior and extensive OpenFOAM knowledge.