reconstructPar and a high number of snapshots
Dear Foam Developers,
I am currently experiencing a little problem with reconstructPar. The statistics of OpenFoam (LES) are limited to <Ui> and <Ui'Uj'>. For most of the eningeering purposes this is enough. But for meteorological applications one needs correlations, high order statistics and more. The most used method is to use a high number of snapshots (>900). I have done so and my case (11 million grid points) runs very good on 640 cores of a AMD Bulldozer Cluster. I saved 1800 snapshots and now they are stored in the processor directories: one file for each field and time and processor. One reconstructPar job for all times is much to slow for all of the 1800 snapshots, one reconstructPar job for each time is a big problem for the file system and does not work also. Limiting the number of reconstructPar jobs to 32 (still one job for one time) gives a annoying behavior: some went through within 1 hour, some not within 12 hours. I have to say, I am not alone on this machine and I have to share it. Therefore exists a queuing system which limits the execution time. Most of the other codes I know uses one result file per field even in parallel runs. The fields of the processors are combined during runtime and than stored. May be it is possible to include such a feature into OpenFoam? This would reduce the amount of time for postprocessing very much? Kind regards, Fabian |
Greetings Fabian,
Wow, that's a whole lot of data. And I think you're not alone when it comes to post-processing this amount of OpenFOAM data from clusters, so until someone else more experience can give his/her very welcome hints, here are mine... OK, what kind of post-processing are you going to need?
Last but not least, if you want the official OpenFOAM developers to see your idea, either:
Bruno |
Thx Bruno for your detailed answer. I am using OF now for 3 years always on HPCs and I am still very happy with it. I am with you that visiting a workshop or buying support would be the best solution. Nevertheless I am working at the university (TU Dresden) and as you might know money for support, licences and workshops is always a big problem. Therefore I have to help myself or ask the forum, which helped me a lot in the past.
Quote:
Quote:
Quote:
Now something new: I found a workaround yesterday. Our queuing-system allows array jobs and one can specify the number of jobs running at the same time. Also reconstructPar support time ranges. Combining such a time range with an array job with limited jobs running parallel reduces the load for the file system and the jobs seems to have not such a high difference in the runtime. kind regards, Fabian |
All times are GMT -4. The time now is 14:09. |