Very long time for saving the output (sim) file ...
everytime I use our cluster, it takes a very long time to save the XYZ@xyz.sim output file. Today my simulation finished at 4 am and now, 6 hours later the sim file is still 0 KB an the job is running. If I cancel the job, the file will be xyz GB and the next job will start. Btw, CPU usage at the cluster is 0% all the time.
Where should I search for the problem? At STAR-CCM+ or MS Windows HPC Server 2008R2? The STAR-CCM+ Support never heard about this problem ...
On stopping some jobs that I have been running interactively I have had the client be hundreds of iterations behind the server. Thus on stopping it can take a long time before it actually gives you back control (so that you can save for example). It slowly updates the client until it is at the same iteration as the server.
No idea why it only seems to happen on some models but it is annoying.
When you cancel the job and there is an output file with a non-zero file size, it indicates, the file is just written slowly.
That might be due to hardware problems (like hard disks, network infrastucture etc) or due to software problems. Or due to utilisation of the system. Hard to say what causes the problem. Ever did a benchmark about disk performance? Especially when the disk is nearly full, the write performance drops dramatically.
Also did you ever check whether the system is swapping?
@RobertB: It is not recommended to stay connected to a server all the time when it's just running. That costs performance for updating the client - even when the client doesn't respond that fast, the server needs to send out the data. Best way is either to start simulations in batch mode or to disconnect when it runs. You still can connect from time to time to check whether it's still behaving like expected or not.
|All times are GMT -4. The time now is 23:30.|