[Nucleophobe]

Hi all,

I am trying to run pvbatch in parallel on an HPC system to post-process my case. The data is decomposed into 16 processor directories.

I submit the job via:

mpiexec -n 16 pvbatch --use-offscreen-rendering myScript.py

(I am using a version of paraview compiled with osmesa)


myScript.py looks like this:

from paraview.simple import *
servermanager.LoadState("myStateFile.pvsm")
SetActiveView(GetRenderView())

WriteAnimation('animation/ani.png', Magnification=1, Quality=2, FrameRate=15.000000)

The job does work, at least at first (it crashes after a few frames). The strange part is I don't appear to be getting any performance benefit from running the job in parallel!

Is there some way I can better troubleshoot this problem?

Thanks!
-Nuc

Edit: Do I need to do something special since I am post-processing a decomposed case?

[Bernhard]

The easiest way would be to log in to the node that is executing the job, and then use top or similar tools to examine the amount of instances running, together with CPU loads and things like that.

[Nucleophobe]

Thanks Bernard.

Hmmm... I've done this before by identifying the nodes with pbstop, but pbstop is not installed on this system. Is there an easy way to identify the nodes?

Edit: I am seeing the following error for the other processors -

Error: mtl_mxm.c:180 - ompi_mtl_mxm_module_init() Failed to generate jobid
--------------------------------------------------------------------------
Error obtaining unique transport key from ORTE (orte_precondition_transports not present in
the environment).

I Googled this a bit, but not sure what to make of it.

[Nucleophobe]

I believe I might have misunderstood how pvbatch is supposed to work!

I was hoping that a parallel pvbatch job would render single frames in parallel, decreasing the time per frame.

However, I just ran a test script that I found in parallel:

from paraview.simple import *
cone = Cone(Resolution=32)
cone.Center = [1, 2, 3]
shrinkFilter = Shrink(cone)
shrinkFilter.UpdatePipeline()
Show(shrinkFilter)
Render()
WriteImage('test2.png')

With this command:

 mpiexec -n 16 pvbatch test.py

The result is 16 rendering windows opening to render the object!

So, do I need to instead setup pvbatch to run more like a job array, where each pvbatch instance renders a different set of frames? I was really hoping to increase the start-up time; loading my state file takes each pvbatch instance about 10 minutes...

Thanks
-Nuc

[wyldckat]

Greetings to all!

@Nucleophobe: It took me a while to get around to have a better look into this thread, than I had originally expected.

I believe that the primary problem you're having is that you are not loading the data with a data reader that is able to handle parallel data processing.
It would help if we could know what is the file/data format you are using for loading the data into ParaView, since that would make it a lot easier to ascertain if the problem is on the loading side or not.

In addition, before you use pvbatch, you should double-check if these steps work for you:

1. Launch manually the pvserver in parallel at the cluster, in a similar way to how you did for pvbatch.
2. Then connect with ParaView from your workstation to the pvserver port on the cluster.
3. Now open the file/data located at the cluster; pvserver will give you remote access to the files at the cluster.
4. Now try doing manually the animation/movie you planned to do with the Python script.

If this works well, only then can you move onto the idea of running pvbatch directly in parallel.

Best regards,
Bruno

[Jack001]

I have the same question. I am using an EnSightReader for my data and i don't get any performance increase (in fact total time increases slightly) when running

mpirun -n 4 pvbatch my_script.py

where my_script.py contains the stuff reading the ensight file and doing the post processing.

Im not quite sure I understand how the command above will result in a faster execution, doesn't the program need some instructions about decomposing my case into different domains which each process will handle? At the moment it seems like pvbatch is supposed to perform some magic to get the case to run faster in parallel.

many thanks!