[General] speed up paraview

naffrancois · December 17, 2016, 06:04

Dear all,

I did not know whether opening this thread here or in the hardware subsection. I visualize transient data sets of about 1Gb size per time steps and plan to increase the number of dofs. These files come from a parallel solver which gathers the parallel raw output in a single binary vtk file per time step. Typical paraview usage includes, loading next time step and filters such as gradient computation or isovolume display. I set up paraview using the binary version on my local machine which has:

- 2x xeon 8core=>16 cores
- 64gb ram
- fx5800 ultra
- 300gb sas hdd

The parallel solver is host on a bigger beowulf cluster without good gpus capabilities (nvidia nvs 315). How could I take advantage of the existing hardware I have in hands to speed-up loading files and volume processing time ?

I know paraview can natively be run in parallel but I do not find easy tutorial to set it up. On the hardware side, what may probably be the bottleneck in my current local machine ? Should I change something, like ssd drive, better gpu ? Or should I use the cluster in client-server mode even if it does not have proper gpu capabilities ?

Many thanks for your advice

chegdan · December 22, 2016, 18:09

This may or may not work in your case, but running a pvserver on your cluster with a version of ParaView compiled to leverage offscreen rendering with OSmesa may help. Then, you can connect in client-server mode with a strategy similar to the one explained in this link. I have done this to visualize about 500 GB of transient data on anywhere from 48 to 256 cores at a time. For now, I do this regularly with paraView-4.1.0 since i have it working just well enough to use in client-server mode.

naffrancois · December 23, 2016, 02:32

many thanks for your advice and for the link.Then I should compile paraview from source on the cluster master node and all the compute nodes. I tried it some time ago and it did not go so well. I will have a new try

I do not know however whether to expect a gain or not because I would get much more cpu working but the parallel filesystem on the cluster is quite slow, nfs on a raid5 sata disks. That is why I was wondering which hardware part is most important using paraview.

chegdan · December 23, 2016, 10:03

In reality it is exactly as you have states i.e. it all depends on problem size and hardware....but this may work. Good luck!

Dan

December 17, 2016, 06:04	speed up paraview	#1
naffrancois Senior Member Join Date: Oct 2011 Posts: 239 Rep Power: 16	Dear all, I did not know whether opening this thread here or in the hardware subsection. I visualize transient data sets of about 1Gb size per time steps and plan to increase the number of dofs. These files come from a parallel solver which gathers the parallel raw output in a single binary vtk file per time step. Typical paraview usage includes, loading next time step and filters such as gradient computation or isovolume display. I set up paraview using the binary version on my local machine which has: - 2x xeon 8core=>16 cores - 64gb ram - fx5800 ultra - 300gb sas hdd The parallel solver is host on a bigger beowulf cluster without good gpus capabilities (nvidia nvs 315). How could I take advantage of the existing hardware I have in hands to speed-up loading files and volume processing time ? I know paraview can natively be run in parallel but I do not find easy tutorial to set it up. On the hardware side, what may probably be the bottleneck in my current local machine ? Should I change something, like ssd drive, better gpu ? Or should I use the cluster in client-server mode even if it does not have proper gpu capabilities ? Many thanks for your advice

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[OpenFOAM.org] Paraview 5.4 in shell environment of5x - Segmentation fault (core dumped)	dslbkxd	OpenFOAM Installation	1	February 3, 2018 00:56
[General] speed up ParaView on workstation	Omid999	ParaView	0	March 21, 2017 06:11
[General] Paraview launch & crash problems	dancfd	ParaView	3	January 17, 2013 12:04
[General] paraview ignores SPACING in STRUCTURED_POINTS vtk data?	jaffar	ParaView	0	November 27, 2012 09:36
paraview installation woes	vex	OpenFOAM Installation	15	January 30, 2011 07:11

December 22, 2016, 18:09		#2
chegdan Senior Member Daniel P. Combest Join Date: Mar 2009 Location: St. Louis, USA Posts: 621 Rep Power: 0	This may or may not work in your case, but running a pvserver on your cluster with a version of ParaView compiled to leverage offscreen rendering with OSmesa may help. Then, you can connect in client-server mode with a strategy similar to the one explained in this link. I have done this to visualize about 500 GB of transient data on anywhere from 48 to 256 cores at a time. For now, I do this regularly with paraView-4.1.0 since i have it working just well enough to use in client-server mode.

December 23, 2016, 02:32		#3
naffrancois Senior Member Join Date: Oct 2011 Posts: 239 Rep Power: 16	many thanks for your advice and for the link.Then I should compile paraview from source on the cluster master node and all the compute nodes. I tried it some time ago and it did not go so well. I will have a new try I do not know however whether to expect a gain or not because I would get much more cpu working but the parallel filesystem on the cluster is quite slow, nfs on a raid5 sata disks. That is why I was wondering which hardware part is most important using paraview.

December 23, 2016, 10:03		#4
chegdan Senior Member Daniel P. Combest Join Date: Mar 2009 Location: St. Louis, USA Posts: 621 Rep Power: 0	In reality it is exactly as you have states i.e. it all depends on problem size and hardware....but this may work. Good luck! Dan