CFD Online Discussion Forums

CFD Online Discussion Forums (http://www.cfd-online.com/Forums/)
-   ParaView (http://www.cfd-online.com/Forums/paraview/)
-   -   pvbatch - How can I check if it is truly running in parallel? (http://www.cfd-online.com/Forums/paraview/127466-pvbatch-how-can-i-check-if-truly-running-parallel.html)

Nucleophobe December 12, 2013 01:59

pvbatch - How can I check if it is truly running in parallel?
 
Hi all,

I am trying to run pvbatch in parallel on an HPC system to post-process my case. The data is decomposed into 16 processor directories.

I submit the job via:
Code:

mpiexec -n 16 pvbatch --use-offscreen-rendering myScript.py
(I am using a version of paraview compiled with osmesa)


myScript.py looks like this:
Code:

from paraview.simple import *
servermanager.LoadState("myStateFile.pvsm")
SetActiveView(GetRenderView())

WriteAnimation('animation/ani.png', Magnification=1, Quality=2, FrameRate=15.000000)

The job does work, at least at first (it crashes after a few frames). The strange part is I don't appear to be getting any performance benefit from running the job in parallel!

Is there some way I can better troubleshoot this problem?

Thanks!
-Nuc

Edit: Do I need to do something special since I am post-processing a decomposed case?

Bernhard December 12, 2013 03:01

The easiest way would be to log in to the node that is executing the job, and then use top or similar tools to examine the amount of instances running, together with CPU loads and things like that.

Nucleophobe December 12, 2013 09:37

Thanks Bernard.

Hmmm... I've done this before by identifying the nodes with pbstop, but pbstop is not installed on this system. Is there an easy way to identify the nodes?

Edit: I am seeing the following error for the other processors -
Code:

Error: mtl_mxm.c:180 - ompi_mtl_mxm_module_init() Failed to generate jobid
--------------------------------------------------------------------------
Error obtaining unique transport key from ORTE (orte_precondition_transports not present in
the environment).

I Googled this a bit, but not sure what to make of it.

Nucleophobe December 13, 2013 00:08

How is pvbatch intended to work?
 
I believe I might have misunderstood how pvbatch is supposed to work!

I was hoping that a parallel pvbatch job would render single frames in parallel, decreasing the time per frame.

However, I just ran a test script that I found in parallel:
Code:

from paraview.simple import *
cone = Cone(Resolution=32)
cone.Center = [1, 2, 3]
shrinkFilter = Shrink(cone)
shrinkFilter.UpdatePipeline()
Show(shrinkFilter)
Render()
WriteImage('test2.png')

With this command:
Code:

mpiexec -n 16 pvbatch test.py
The result is 16 rendering windows opening to render the object!

So, do I need to instead setup pvbatch to run more like a job array, where each pvbatch instance renders a different set of frames? I was really hoping to increase the start-up time; loading my state file takes each pvbatch instance about 10 minutes...

Thanks
-Nuc

wyldckat February 1, 2014 18:21

Greetings to all!

@Nucleophobe: It took me a while to get around to have a better look into this thread, than I had originally expected.

I believe that the primary problem you're having is that you are not loading the data with a data reader that is able to handle parallel data processing.
It would help if we could know what is the file/data format you are using for loading the data into ParaView, since that would make it a lot easier to ascertain if the problem is on the loading side or not.

In addition, before you use pvbatch, you should double-check if these steps work for you:
  1. Launch manually the pvserver in parallel at the cluster, in a similar way to how you did for pvbatch.
  2. Then connect with ParaView from your workstation to the pvserver port on the cluster.
  3. Now open the file/data located at the cluster; pvserver will give you remote access to the files at the cluster.
  4. Now try doing manually the animation/movie you planned to do with the Python script.
If this works well, only then can you move onto the idea of running pvbatch directly in parallel.

Best regards,
Bruno


All times are GMT -4. The time now is 13:35.