CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Post-Processing

Postprocessing large data sets in parallel

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   June 10, 2011, 01:14
Default
  #21
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 12
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Quote:
Originally Posted by arjun View Post
just an update to my comment.


I implemented iso surface generation part. It is working well. So now for generating animation of Q criteria for 3billion cells case, I only need 1 processor paraview. Post processing files are very small and thus, i can save them with high frequency and it also means quality of animation would be much better than what I would get from saved calculation data.


Sweet success. :-D
Congrats! You are man!

Well, mind sharing your code, it must be a great code, how did you do that..
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   June 10, 2011, 01:16
Default
  #22
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 12
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Quote:
Originally Posted by sandy View Post
Hi lakeat, I get the error information again. Maybe I should try:

Danie -help

?
"Danie -help" would result an error msg, but "Daniel -help" will not
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   June 10, 2011, 02:22
Default
  #23
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 373
Rep Power: 10
arjun is on a distinguished road
I can outline what I did, so if someone wants to impliment it in openFOAM he can do it too.

first of all it is not difficult thing to do. Implementation does depend on data structure of solver so efficiency might be different from solver to solver.

First, lets assume that solver is has control volumes and each control volume is made of some faces. Each face in turn is made up of some nodes spread in 3d space. Face could be polygonal with nf nodes numbered as n1 n2 .... nf.

Take variable phi whose iso surface you want to generate. Now first step is to transfer this variable to nodes. That is calculate the values of this variable on nodes. This you anyway do if you export data for post processing , as most of post processors want node based data.

Second step is to loop through all the cells one by one. Lets say you are at cell c. For this cell c collect all the faces. Since each face is just a polygon , check its edges ie n1->n2, n2->n3 , nf-1 -> nf , where the iso value falls if it falls. Could be done with the help of equation of line. If iso value happens, store this new node (xi,yi,zi) .

so after you loop through all the faces of cell c, you will have say m new nodes generated. They make a polygon that you have to visualise. Store it and finally when all the cells are looped through export this in a vtk file or some format that your program can understand.


For parallel run, each process would generate a part of this file. So for each iteration you would have MAX Np files (Np = number of processors).

Write a small code that will read all these files, merge them and create a single file in any format that post processor can understand. I am using vtk polygons.


Side Note: For cell , m nodes are generated , you would have to arrange them in an order that when it is drawn by post processor it does not look mangled.


Even for very large calculations, each file is only few thousand polygons, and for this reason 1 processor post processor is good enough to create animations.

I compared results with fieldview and they look very similar to it.


I will be adding this into parallel unstructured grid based solver that i am working on. Will be good addition, I guess.
arjun is offline   Reply With Quote

Old   June 10, 2011, 21:04
Default
  #24
Senior Member
 
Sandy Lee
Join Date: Mar 2009
Posts: 207
Rep Power: 9
sandy is on a distinguished road
stunned .... what means ... pls make clear ...
sandy is offline   Reply With Quote

Old   June 10, 2011, 22:42
Default
  #25
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 373
Rep Power: 10
arjun is on a distinguished road
Quote:
Originally Posted by sandy View Post
stunned .... what means ... pls make clear ...
ooops.........
arjun is offline   Reply With Quote

Old   June 20, 2011, 21:58
Default
  #26
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 373
Rep Power: 10
arjun is on a distinguished road
In the middle of sept we are going to present our calculation results at http://www.j-focus.or.jp/ along with other presentors. The animations would be from the method i mentioned in this thread.

Before us there is a presentation where they will present a calculation involving 2 billion cells using 10000 cores (software is called frontflow).

This got me thinking, is anyone aware of largest calculation that is done with openFOAM?? And how many processors it used. Is there any guess for it?? Is it 100million 500 million or 1 billion or so??
arjun is offline   Reply With Quote

Old   June 21, 2011, 14:09
Default
  #27
Senior Member
 
lakeat's Avatar
 
Daniel WEI (老魏)
Join Date: Mar 2009
Location: South Bend, IN, USA
Posts: 688
Blog Entries: 9
Rep Power: 12
lakeat is on a distinguished road
Send a message via Skype™ to lakeat
Quote:
Originally Posted by arjun View Post
In the middle of sept we are going to present our calculation results at http://www.j-focus.or.jp/ along with other presentors. The animations would be from the method i mentioned in this thread.

Before us there is a presentation where they will present a calculation involving 2 billion cells using 10000 cores (software is called frontflow).

This got me thinking, is anyone aware of largest calculation that is done with openFOAM?? And how many processors it used. Is there any guess for it?? Is it 100million 500 million or 1 billion or so??
I recall csc of Finland has done some huge jobs..
__________________
~
Daniel WEI
-------------
NatHaz Modeling Laboratory
Department of Civil & Environmental Engineering & Earth Sciences
University of Notre Dame, USA
Email || My Personal CFD Blog
lakeat is offline   Reply With Quote

Old   June 22, 2011, 03:34
Default
  #28
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 373
Rep Power: 10
arjun is on a distinguished road
Quote:
Originally Posted by lakeat View Post
I recall csc of Finland has done some huge jobs..

thanks for this. Their website talks about testing openfoam for 8million , 13.8 million and 22million cells. So I gather they have not run anything higher than 100million cells .

few months ago, someone mentioned about running a case with 80million cells openfoam forum, i wonder how it went.

anyway thanks again.
arjun is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Smallest binary file format to save large data Zonexo Main CFD Forum 2 June 2, 2008 20:25
Large data file to manage fluid FLUENT 0 December 13, 2007 13:46
PostChannel in parallel maka OpenFOAM Post-Processing 1 May 30, 2006 05:37
Postprocessing Pressure data spanwise over a wing Rick FLUENT 1 February 3, 2006 10:28
How to update polyPatchbs localPoints liu OpenFOAM Running, Solving & CFD 6 December 30, 2005 18:27


All times are GMT -4. The time now is 15:37.