CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   CFX (https://www.cfd-online.com/Forums/cfx/)
-   -   Parallel vs Serial Run-Solver manager (https://www.cfd-online.com/Forums/cfx/167552-parallel-vs-serial-run-solver-manager.html)

rohit_turbo March 4, 2016 04:10

Parallel vs Serial Run-Solver manager
 
Dear all,
I work on a DELL Inspiron T7810 with 128GB Ram and Intel Xeon 8 core processor. I am working my project in ANSYS CFX. My model contains about 14.5 L elements.
A very basic question off course which I would like to put up here is that, what difference would take place in results if I run all my simulations in Intel mpi local parallel with about 15 partitions and If I run the same simulation in serial run.
How do I know what amount of partitions is correct enough for the run.
(For e.g. 10,20 or what is the partition number maximum that can be given to run with accurate solutions)
Although it runs faster than a serial run , but I have a small hitch whether it will give me the same results as serial, since I lack time here to recheck and re simulate it, I seek help.
I know a very basic question to be put up, but due to the time constraint If somebody could be kind enough to explain the details I would be really glad.


Thanks in advance.
-Rohit

flotus1 March 4, 2016 04:41

You will have to trust the software developers at Ansys that the results are the same no matter how many partitions were used. This assumption is not too far-fetched, checking this is part of very basic software v&v. Of course you could ask them for documentation.
When it comes to the maximum speedup, using more partitions than cores available is usually a bad idea. In your case, 8 or 7 partitions (in case you want to do other stuff with your workstation while it is solving) should hit the sweetspot.

rohit_turbo March 4, 2016 07:20

Thanks flotus1, I usually use my workstation to only simulate my runs, so simultaneously I run about 8-10 runs at a time some with even a partition of 10 runs and some with a serial run.The system rarely hangs but I have one more grudge that how should I be making the fullest potential usage of my workstation.Even when I give 10 runs at a time and simulate it the task manager alsways shows the performance usage of the system at about 70-80% and the ram usage is never beyond 35GB.Am I doing the right way to find out answers or my system can work better than this!!.Thanks again
Rohit

-Maxim- March 4, 2016 07:53

As far as I know, the RAM usage corresponds mostly to the size of your mesh. Also, transient simulations use more RAM in my experience.
I wouldn't worry about maxing out the RAM too much since the CPU is usually the bottleneck (running at 100%). As flotus said, I would recommend using 6-8 partitions. When you work with several partitions, they always have some overlap ('areas' that are calculated by two partitions next to each other). So more partitions actually slow your simulation down!
I would not run 10 simulations at the same time (how many licences do you have?!:eek:). Just write a batch and then let CFX solve your cases one by one...

flotus1 March 4, 2016 08:01

with 10 serial runs at a time and task manager showing only 60-70% CPU usage you obviously have SMT turned on (intel refers to it as hyperthreading) which gives you 16 virtual cores.
It is usually advised to turn SMT off for CFD workstations because it has no benefit in these workloads and can even decrease overall performance.
If some of the 10 simulations you have running simultaneously are multi-threaded (I did not quite understand if this was the case here) and the overall CPU usage is 60-70% there is quite obviously a bottleneck other than CPU performance. Usually it is memory bandwidth and the communication overhead introduced by mpi.

So my advice for optimal performance would be:
Turn off SMT in the bios.
If you have 8 or more simulations you can run simultaneously, run a maximum of 8 at a same time, all of them in serial.
If you have less than 8 simulations to run simultaneously, distribute the number of partitions so it does not exceed the number of cores available. E.g.
1 simulation - 8 partitions
2 simulations 4 partitions each... you get the picture

rohit_turbo March 4, 2016 08:34

Thank you both of you!! That kind of relieves my pressure of rerunning and checking whether the results are correct or not!!I think I shall take into account what flotus said!!6-8 partitions per run is good I believe and it does run faster as compared to the serial one. I was thinking of increasing the partitions more than 16!! I have run with 16 partitions and it does give results in 1/4th of the time what serial gives and one more thing I always had a doubt about was why cant we run more than 10 runs simultaneously??Is it a barrier that Ansys created or my system isnt accepting??


All times are GMT -4. The time now is 06:51.