Rule of Thumb for Selecting cores

AS_Aero · March 14, 2017, 07:17

Hi All

I am doing a Flow simulation using CFX and LES .
Number of elements for my mesh is around 18E6 (18million) so I wanted to know if anyone have a suggestion on selecting the number of cores to conduct this simulation, as the available licence on my cluster is very limited. So is there any kind of rule of thumb or from your experience.

evcelica · March 14, 2017, 14:19

I have seen very good scaling with CFX, even when I used a lot of cores for a very small problem. I've used 10 cores for a few thousand cells, and it was much faster than serial. With 18Mcells, I doubt you could use too many cores.

ghorrocks · March 14, 2017, 16:45

Erik is right, CFX scales well to zillions of partitions. The limitation is likely to be:
1) The amount of parallel licenses you have
2) Large parallel simulations require careful cluster design including high speed interconnects and many subtle motherboard issues.

AS_Aero · March 15, 2017, 05:41

Dear Erik and Glenn

So you mean to say that I can use as much as I want depending on the number of licenses I get from the University ? So utilizing the maximum licenses available which means using the maximum possible cores right !!

ghorrocks · March 15, 2017, 05:49

If you have lots of parallel licenses then you are probably going to be restricted by the quality of the compute cluster you have access to. I would test a sweep of partition counts on a benchmark simulation to find the best speed for your cluster. Try 1,2,4,8,16,32, etc until you run out of licenses or the performance starts dropping off.

In other words, do not assume that adding more partitions will always run faster.

highorder_cfd · March 15, 2017, 08:58

If I can make a comment here, with R17 and R18 ANSYS has made huge steps forwards to improve the scalability, so if you use most recent releases with HPC your simulation will be faster. I think this depends from the architecture of the machine where you are running and from your type of problem, but at the top of my head there should be an optimal number of cells for process, which it should be something around 15k per core (tests done on AWS).

AS_Aero · March 15, 2017, 09:20

Hi All

Thanks a lot for your valuable comments !!

March 14, 2017, 07:17	Rule of Thumb for Selecting cores	#1
AS_Aero Senior Member Join Date: Mar 2013 Location: Germany Posts: 357 Rep Power: 14	Hi All I am doing a Flow simulation using CFX and LES . Number of elements for my mesh is around 18E6 (18million) so I wanted to know if anyone have a suggestion on selecting the number of cores to conduct this simulation, as the available licence on my cluster is very limited. So is there any kind of rule of thumb or from your experience.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[ICEM] Mesh sizing - Rule of thumb !!!	Far	ANSYS Meshing & Geometry	6	February 22, 2013 03:59
general rule of thumb for bi-exponential meshing	hippiekyle	Mesh Generation & Pre-Processing	1	February 11, 2013 03:44
[swak4Foam] Installing swak4Foam to OpenFOAM in mac	Kaquesang	OpenFOAM Community Contributions	22	January 21, 2013 11:51
RPM (+/-) thumb rule	studentCFD	CFX	1	November 6, 2007 05:19
gcc and executable file from Mac to Linux	simone Marras	Main CFD Forum	0	April 8, 2007 15:49

March 14, 2017, 14:19		#2
evcelica Senior Member Erik Join Date: Feb 2011 Location: Earth (Land portion) Posts: 1,167 Rep Power: 23	I have seen very good scaling with CFX, even when I used a lot of cores for a very small problem. I've used 10 cores for a few thousand cells, and it was much faster than serial. With 18Mcells, I doubt you could use too many cores.

March 14, 2017, 16:45		#3
ghorrocks Super Moderator Glenn Horrocks Join Date: Mar 2009 Location: Sydney, Australia Posts: 17,716 Rep Power: 143	Erik is right, CFX scales well to zillions of partitions. The limitation is likely to be: 1) The amount of parallel licenses you have 2) Large parallel simulations require careful cluster design including high speed interconnects and many subtle motherboard issues.

March 15, 2017, 05:41		#4
AS_Aero Senior Member Join Date: Mar 2013 Location: Germany Posts: 357 Rep Power: 14	Dear Erik and Glenn So you mean to say that I can use as much as I want depending on the number of licenses I get from the University ? So utilizing the maximum licenses available which means using the maximum possible cores right !!

March 15, 2017, 05:49		#5
ghorrocks Super Moderator Glenn Horrocks Join Date: Mar 2009 Location: Sydney, Australia Posts: 17,716 Rep Power: 143	If you have lots of parallel licenses then you are probably going to be restricted by the quality of the compute cluster you have access to. I would test a sweep of partition counts on a benchmark simulation to find the best speed for your cluster. Try 1,2,4,8,16,32, etc until you run out of licenses or the performance starts dropping off. In other words, do not assume that adding more partitions will always run faster.

March 15, 2017, 08:58		#6
highorder_cfd Member Join Date: Jan 2015 Posts: 63 Rep Power: 11	If I can make a comment here, with R17 and R18 ANSYS has made huge steps forwards to improve the scalability, so if you use most recent releases with HPC your simulation will be faster. I think this depends from the architecture of the machine where you are running and from your type of problem, but at the top of my head there should be an optimal number of cells for process, which it should be something around 15k per core (tests done on AWS).

March 15, 2017, 09:20		#7
AS_Aero Senior Member Join Date: Mar 2013 Location: Germany Posts: 357 Rep Power: 14	Hi All Thanks a lot for your valuable comments !!