Large problem partitioner

tzy · February 6, 2018, 20:53

Hi all,

I am currently using cfx on 18.1 and trying to model a steady state flow from a large pipe with diameter of 20mm to a small channel with hydraulic diameter of 0.6mm. As such, the model ended up with a lot of elements (131mil elements, 48mil nodes).

I am facing problems trying to partition the model to allow it to run in parallel.
While examining the output, I see that cfx recommended me to run a large problem partitioner. However, even after turning on the large problem partitioner in the defining run screen, it still shows that I did not turn on the large problem partitioner.

I have checked the box for "Large Problem" under the global setting, checked the "Override Default Large Problem Setting" in the partitioner tab and the "Large Problem" which appears. I can see that the input for "large problem" under the global setting is correct, but I cannot seem to get the input for the "large problem" under the partitioner to work correctly. Is there any settings which I have missed out (I suspect its the CFX Command Language Upgrade but I have no idea where to change it)?

Out file after the failed run has been attached as a txt file due to it being too long.

ghorrocks · February 7, 2018, 05:14

An alternate approach is to move to a simpler partitioning algorithm. If you use recursive bisection or one of those simpler methods they use much less memory and you might not need the large problem partitioner.

But before you do this - are you sure you need such a huge mesh? It is challenging to mesh geometries with large features and small features, but in these cases it is often best to chop the geometry up into segments and use sweeps and block meshes where you can put some mesh bias in but still have high quality elements. For instance in pipe sections an extruded mesh which is elongated in the pipe length direction will be MUCH better than a tet mesh with inflation layers.

tzy · February 7, 2018, 12:18

Dear Glenn,

Thank you for your advice, I will try reduce the mesh by trying out extruded mesh. However, I suspect that it will not solve my problem as the part with a constant cross-section is only a short segment that accounts for 5mil elements (at most) by my estimation.

Just a further explanation of where the majority of the mesh is. Within the 0.6mm channel, there are about 200 small protrusions with an airfoil geometry which I have done an edge sizing and curvature turned on (normal angle=9). Without curvature turned on, the airfoil leading edge is basically a triangle and the trailing edge is truncated (I now realized I should not have used airfoils but its a little too late into my project). Without the edge sizing, the number of elements would probably be lesser than 10mil (based on previous experience with my own geometry). I am not sure what would be a more efficient way to mesh the protrusion. I am also currently trying to reduce the mesh by increasing the curvature angle and element size for the edge sizing.

I have tried various partitioning methods (even simple assignment which is said to require the least memory) none of them seems to work. I have also tried assigning different values of integer memory to be allocated (15000m seems to be around the maximum I can input as you can see from the out file) and it has failed as well.

I am unsure why the large problem partitioner is turned off even though I have selected it. Is there anyway to ensure that it remain on?

I do have access to HPC to run which I believe can solve this issue, but I hope to rely on it only as a last resort due to costs and I think it is better for me to learn how to work without it in case I do not have access to a HPC.

ghorrocks · February 7, 2018, 17:11

Please make sure you are using 64 bit CFX on a 64 bit OS. If you are running 32 bit you are going to be limiting the size simulation you can handle.

Before you do this simulation, have you done any checks to see what mesh resolution you actually need? All your comments appear to be about what mesh you can generate and nothing about what mesh is actually required. If you have just guessed the mesh size it is bound to be wrong.

I recommend you take a small section of your geometry and do a mes refinement study. Run a range of meshes (where you change the element edge length by a factor of 2 each time) and see what mesh is required to give the accuracy you require. Once you have determined the mesh you require on a small section of geometry you can extrapolate out to the full geometry and see how big a mesh you require for the entire thing.

February 7, 2018, 05:14		#2
ghorrocks Super Moderator Glenn Horrocks Join Date: Mar 2009 Location: Sydney, Australia Posts: 17,703 Rep Power: 143	An alternate approach is to move to a simpler partitioning algorithm. If you use recursive bisection or one of those simpler methods they use much less memory and you might not need the large problem partitioner. But before you do this - are you sure you need such a huge mesh? It is challenging to mesh geometries with large features and small features, but in these cases it is often best to chop the geometry up into segments and use sweeps and block meshes where you can put some mesh bias in but still have high quality elements. For instance in pipe sections an extruded mesh which is elongated in the pipe length direction will be MUCH better than a tet mesh with inflation layers. __________________ Note: I do not answer CFD questions by PM. CFD questions should be posted on the forum.

February 7, 2018, 17:11		#4
ghorrocks Super Moderator Glenn Horrocks Join Date: Mar 2009 Location: Sydney, Australia Posts: 17,703 Rep Power: 143	Please make sure you are using 64 bit CFX on a 64 bit OS. If you are running 32 bit you are going to be limiting the size simulation you can handle. Before you do this simulation, have you done any checks to see what mesh resolution you actually need? All your comments appear to be about what mesh you can generate and nothing about what mesh is actually required. If you have just guessed the mesh size it is bound to be wrong. I recommend you take a small section of your geometry and do a mes refinement study. Run a range of meshes (where you change the element edge length by a factor of 2 each time) and see what mesh is required to give the accuracy you require. Once you have determined the mesh you require on a small section of geometry you can extrapolate out to the full geometry and see how big a mesh you require for the entire thing. __________________ Note: I do not answer CFD questions by PM. CFD questions should be posted on the forum.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Fluent Parallelization Problem After AC Power Dropped	pawl	Hardware	5	November 13, 2016 06:08
Problem with large time scale and minute length scale	mech_engg_swag	OpenFOAM	0	August 23, 2016 07:39
Problem with porous flow at multi-material interface with large permeability diff.	Hisham	OpenFOAM Programming & Development	1	June 3, 2016 10:51
Problem with an old Simulation	FrankW	CFX	3	February 8, 2016 04:28
Adiabatic and Rotating wall (Convection problem)	ParodDav	CFX	5	April 29, 2007 19:13

February 7, 2018, 12:18		#3
tzy New Member Join Date: Feb 2018 Posts: 2 Rep Power: 0	Dear Glenn, Thank you for your advice, I will try reduce the mesh by trying out extruded mesh. However, I suspect that it will not solve my problem as the part with a constant cross-section is only a short segment that accounts for 5mil elements (at most) by my estimation. Just a further explanation of where the majority of the mesh is. Within the 0.6mm channel, there are about 200 small protrusions with an airfoil geometry which I have done an edge sizing and curvature turned on (normal angle=9). Without curvature turned on, the airfoil leading edge is basically a triangle and the trailing edge is truncated (I now realized I should not have used airfoils but its a little too late into my project). Without the edge sizing, the number of elements would probably be lesser than 10mil (based on previous experience with my own geometry). I am not sure what would be a more efficient way to mesh the protrusion. I am also currently trying to reduce the mesh by increasing the curvature angle and element size for the edge sizing. I have tried various partitioning methods (even simple assignment which is said to require the least memory) none of them seems to work. I have also tried assigning different values of integer memory to be allocated (15000m seems to be around the maximum I can input as you can see from the out file) and it has failed as well. I am unsure why the large problem partitioner is turned off even though I have selected it. Is there anyway to ensure that it remain on? I do have access to HPC to run which I believe can solve this issue, but I hope to rely on it only as a last resort due to costs and I think it is better for me to learn how to work without it in case I do not have access to a HPC.