Paraview parallel processing with pvbatch
 

Hello all,

we've found numerous posts on how to run paraview in parallel, but somehow we are still stuck.

We have python scripts that open OpenFOAM data and process it into expored x3d files. It works fine with pvpython. Paraview was installed with the MPI option enabled.

If we then switch to pvbatch and run this command:
mpirun -np 4 pvbatch script.py -parallel
it runs 4 parallel processes but no speed ups are noticeable, and we get the following error (just after startup):
[[12489,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
Host: ip-172-31-29-146

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
[ip-172-31-29-146:27841] 3 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
[ip-172-31-29-146:27841] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

It seems as if the same process is started 4 times instead of splitting it up in 4 smaller parts (which would match earlier reportings on this forum).


If we then execute the following (based on other posts)
mpiexec -np 4 -mca btl pvbatch script.py -parallel
no output x3d files are generated and we get the following error 4 times (once for each process we assume):

Error: Could not import vtkCommonComputationalGeometry
Error: Could not import vtkCommonComputationalGeometry
Error: Could not import vtkCommonComputationalGeometry
Error: Could not import vtkCommonComputationalGeometry
Traceback (most recent call last):
File "/home/ubuntu/ParaViewBuild/lib/site-packages/paraview/vtk/vtkCommonCore.py", line 5, in <module>
from .vtkCommonCorePython import *
ImportError: No module named 'paraview.vtk.vtkCommonCorePython'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/airshaper/progress/runFiles/primalRun/MakeStreams3D.py", line 6, in <module>
from paraview.simple import *
File "/home/ubuntu/ParaViewBuild/lib/site-packages/paraview/simple.py", line 43, in <module>
from paraview import servermanager
File "/home/ubuntu/ParaViewBuild/lib/site-packages/paraview/servermanager.py", line 53, in <module>
from paraview import vtk
File "/home/ubuntu/ParaViewBuild/lib/site-packages/paraview/vtk/__init__.py", line 7, in <module>
from paraview.vtk.vtkCommonCore import *
File "/home/ubuntu/ParaViewBuild/lib/site-packages/paraview/vtk/vtkCommonCore.py", line 9, in <module>
from vtkCommonCorePython import *
ImportError: No module named 'vtkCommonCorePython'
Traceback (most recent call last):
File "/home/ubuntu/ParaViewBuild/lib/site-packages/paraview/vtk/vtkCommonCore.py", line 5, in <module>
from .vtkCommonCorePython import *
ImportError: No module named 'paraview.vtk.vtkCommonCorePython'



Our goal is to run this script in parallel using pvbatch.
Does anyone have any clue to what we are doing wrong?

Thanks!

Wouter.

Hey Wouter,

Did you get answer for this type of problem. If so, kindly share how to run paraview in parallel using Local computer.

Paraview in parallel
 

Hi,

no we didn't find a solution:
it seemed like none of the functions were actually running in parallel.
We did manage to launch multiple processes, but it didn't speed up - it seemed to just run the same process in full on each processor.

But I don't think this is a final conclusion, perhaps we should give it another look :) but unfortunately, no answer for now...