CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > ANSYS > CFX

Parallel vs Serial Run-Solver manager

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree3Likes
  • 1 Post By -Maxim-
  • 2 Post By flotus1

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   March 4, 2016, 05:10
Default Parallel vs Serial Run-Solver manager
  #1
New Member
 
Rohit S Iyer
Join Date: Oct 2015
Posts: 16
Rep Power: 10
rohit_turbo is on a distinguished road
Dear all,
I work on a DELL Inspiron T7810 with 128GB Ram and Intel Xeon 8 core processor. I am working my project in ANSYS CFX. My model contains about 14.5 L elements.
A very basic question off course which I would like to put up here is that, what difference would take place in results if I run all my simulations in Intel mpi local parallel with about 15 partitions and If I run the same simulation in serial run.
How do I know what amount of partitions is correct enough for the run.
(For e.g. 10,20 or what is the partition number maximum that can be given to run with accurate solutions)
Although it runs faster than a serial run , but I have a small hitch whether it will give me the same results as serial, since I lack time here to recheck and re simulate it, I seek help.
I know a very basic question to be put up, but due to the time constraint If somebody could be kind enough to explain the details I would be really glad.


Thanks in advance.
-Rohit
rohit_turbo is offline   Reply With Quote

Old   March 4, 2016, 05:41
Default
  #2
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,382
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
You will have to trust the software developers at Ansys that the results are the same no matter how many partitions were used. This assumption is not too far-fetched, checking this is part of very basic software v&v. Of course you could ask them for documentation.
When it comes to the maximum speedup, using more partitions than cores available is usually a bad idea. In your case, 8 or 7 partitions (in case you want to do other stuff with your workstation while it is solving) should hit the sweetspot.
flotus1 is offline   Reply With Quote

Old   March 4, 2016, 08:20
Default
  #3
New Member
 
Rohit S Iyer
Join Date: Oct 2015
Posts: 16
Rep Power: 10
rohit_turbo is on a distinguished road
Thanks flotus1, I usually use my workstation to only simulate my runs, so simultaneously I run about 8-10 runs at a time some with even a partition of 10 runs and some with a serial run.The system rarely hangs but I have one more grudge that how should I be making the fullest potential usage of my workstation.Even when I give 10 runs at a time and simulate it the task manager alsways shows the performance usage of the system at about 70-80% and the ram usage is never beyond 35GB.Am I doing the right way to find out answers or my system can work better than this!!.Thanks again
Rohit
rohit_turbo is offline   Reply With Quote

Old   March 4, 2016, 08:53
Default
  #4
Senior Member
 
Maxim
Join Date: Aug 2015
Location: Germany
Posts: 415
Rep Power: 12
-Maxim- is on a distinguished road
As far as I know, the RAM usage corresponds mostly to the size of your mesh. Also, transient simulations use more RAM in my experience.
I wouldn't worry about maxing out the RAM too much since the CPU is usually the bottleneck (running at 100%). As flotus said, I would recommend using 6-8 partitions. When you work with several partitions, they always have some overlap ('areas' that are calculated by two partitions next to each other). So more partitions actually slow your simulation down!
I would not run 10 simulations at the same time (how many licences do you have?!). Just write a batch and then let CFX solve your cases one by one...
rohit_turbo likes this.
-Maxim- is offline   Reply With Quote

Old   March 4, 2016, 09:01
Default
  #5
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,382
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
with 10 serial runs at a time and task manager showing only 60-70% CPU usage you obviously have SMT turned on (intel refers to it as hyperthreading) which gives you 16 virtual cores.
It is usually advised to turn SMT off for CFD workstations because it has no benefit in these workloads and can even decrease overall performance.
If some of the 10 simulations you have running simultaneously are multi-threaded (I did not quite understand if this was the case here) and the overall CPU usage is 60-70% there is quite obviously a bottleneck other than CPU performance. Usually it is memory bandwidth and the communication overhead introduced by mpi.

So my advice for optimal performance would be:
Turn off SMT in the bios.
If you have 8 or more simulations you can run simultaneously, run a maximum of 8 at a same time, all of them in serial.
If you have less than 8 simulations to run simultaneously, distribute the number of partitions so it does not exceed the number of cores available. E.g.
1 simulation - 8 partitions
2 simulations 4 partitions each... you get the picture
-Maxim- and rohit_turbo like this.
flotus1 is offline   Reply With Quote

Old   March 4, 2016, 09:34
Default
  #6
New Member
 
Rohit S Iyer
Join Date: Oct 2015
Posts: 16
Rep Power: 10
rohit_turbo is on a distinguished road
Thank you both of you!! That kind of relieves my pressure of rerunning and checking whether the results are correct or not!!I think I shall take into account what flotus said!!6-8 partitions per run is good I believe and it does run faster as compared to the serial one. I was thinking of increasing the partitions more than 16!! I have run with 16 partitions and it does give results in 1/4th of the time what serial gives and one more thing I always had a doubt about was why cant we run more than 10 runs simultaneously??Is it a barrier that Ansys created or my system isnt accepting??
rohit_turbo is offline   Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem during mpi in server: expected Scalar, found on line 0 the word 'nan' muth OpenFOAM Running, Solving & CFD 3 August 27, 2018 05:18
Some questions about a multi region case run in parallel zfaraday OpenFOAM Running, Solving & CFD 5 February 23, 2017 11:25
parallel run is slower than serial run (pimpleFoam) !!! mechy OpenFOAM 18 August 17, 2016 18:19
simpleFoam in parallel issue plucas OpenFOAM Running, Solving & CFD 3 July 17, 2013 12:30
Serial run OK parallel one fails r2d2 OpenFOAM Running, Solving & CFD 2 November 16, 2005 13:44


All times are GMT -4. The time now is 10:56.