CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   SU2 (https://www.cfd-online.com/Forums/su2/)
-   -   Computational resource requirements of file updates (https://www.cfd-online.com/Forums/su2/221133-computational-resource-requirements-file-updates.html)

Situla October 6, 2019 10:05

Computational resource requirements of file updates
 
Hello everyone!

Before I start explaining, I just want to clarify that this is not technically about a problem I am having, but rather something interesting I have noticed.

How I came across this (can be skipped (I want to put this into a spoiler wrapper but do not know if that works here))
I run a MATLAB script that executes SU2 in a loop, (several instances at the same time, starting a new one when a previous one finishes). This led to many command prompts being open and idling after finishing. To avoid that, I decided to have MATLAB execute a batch script that runs SU2 and exits when it finishes. In doing so, I noticed that SU2 was actually about 30% faster (0.37 instead of 0.52 seconds per iteration). However, I ran into the problem that the history file was not being written (and actually, no other file that I was aware of), meaning that it did not work properly. Upon closer examination, I found that my COMODO Internet Security was actually sandboxing the batch script, preventing it from writing to any real location on my computer. After setting up an exception, everything worked, but the computation time was also roughly the same as without the batch script, so no gains there.
To summarise: a sandboxed SU2 is much faster than a normal SU2, but does not write any of the vital files.

Further investigation
  • It seems that writing the history file (and potentially other files I do not know of) makes the program slow down from approx. 0.37 to 0.52 seconds per iteration when running on three of my cores, or from approx. 1.1 to 1.4 seconds per iteration when running on one of my cores, which comes up to approximately one quarter of the computational time per iteration (numbers vary from case to case and do not seem to scale in an obvious manner with number of cores).
  • Changing the convergence history writing interval from 1 to 100 did not make a noticeable difference.

Questions
  • Is it really reasonable that writing the files takes about 25% of the computational time? That seems to be a lot. Are contributors aware of this? Maybe it could be optimised.. Unfortunately, my own knowledge of C++ is very limited.
  • What do you think could be the real reason behind the speed difference? Clearly it is not the actual history file. Does SU2 store information in a temporary file if the writing interval is larger than one? If no, how does it work?
  • As far as I am aware, sandboxed processes still write files normally, just in a temporary container on the file system. Could it be that that is actually faster somehow? Why? I do have an HDD, but all files related to this observation are on my SSD, which I think includes the sandbox.

I am curious to read what more experienced and knowledgeable people think about this.

pcg October 16, 2019 13:03

The last time I profiled the code (few months back) the output routines barely register (<< 1%)
Is it possible you are writing the output files on every iteration?


All times are GMT -4. The time now is 17:04.