pvbatch - How can I check if it is truly running in parallel?
Hi all,
I am trying to run pvbatch in parallel on an HPC system to post-process my case. The data is decomposed into 16 processor directories. I submit the job via: Code:
mpiexec -n 16 pvbatch --use-offscreen-rendering myScript.py myScript.py looks like this: Code:
from paraview.simple import * Is there some way I can better troubleshoot this problem? Thanks! -Nuc Edit: Do I need to do something special since I am post-processing a decomposed case? |
The easiest way would be to log in to the node that is executing the job, and then use top or similar tools to examine the amount of instances running, together with CPU loads and things like that.
|
Thanks Bernard.
Hmmm... I've done this before by identifying the nodes with pbstop, but pbstop is not installed on this system. Is there an easy way to identify the nodes? Edit: I am seeing the following error for the other processors - Code:
Error: mtl_mxm.c:180 - ompi_mtl_mxm_module_init() Failed to generate jobid |
How is pvbatch intended to work?
I believe I might have misunderstood how pvbatch is supposed to work!
I was hoping that a parallel pvbatch job would render single frames in parallel, decreasing the time per frame. However, I just ran a test script that I found in parallel: Code:
from paraview.simple import * Code:
mpiexec -n 16 pvbatch test.py So, do I need to instead setup pvbatch to run more like a job array, where each pvbatch instance renders a different set of frames? I was really hoping to increase the start-up time; loading my state file takes each pvbatch instance about 10 minutes... Thanks -Nuc |
Greetings to all!
@Nucleophobe: It took me a while to get around to have a better look into this thread, than I had originally expected. I believe that the primary problem you're having is that you are not loading the data with a data reader that is able to handle parallel data processing. It would help if we could know what is the file/data format you are using for loading the data into ParaView, since that would make it a lot easier to ascertain if the problem is on the loading side or not. In addition, before you use pvbatch, you should double-check if these steps work for you:
Best regards, Bruno |
I have the same question. I am using an EnSightReader for my data and i don't get any performance increase (in fact total time increases slightly) when running
mpirun -n 4 pvbatch my_script.py where my_script.py contains the stuff reading the ensight file and doing the post processing. Im not quite sure I understand how the command above will result in a faster execution, doesn't the program need some instructions about decomposing my case into different domains which each process will handle? At the moment it seems like pvbatch is supposed to perform some magic to get the case to run faster in parallel. many thanks! |
All times are GMT -4. The time now is 21:38. |