CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Main CFD Forum

GPU based CFD code

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   July 14, 2010, 07:19
Default GPU based CFD code
  #1
New Member
 
Harry
Join Date: Mar 2009
Posts: 27
Rep Power: 7
harry is on a distinguished road
is there some researches regarding the development of FVM based codes under GPU computational environments?
harry is offline   Reply With Quote

Old   July 14, 2010, 08:59
Default Gpu cfd
  #2
Senior Member
 
gocarts's Avatar
 
Richard Smith
Join Date: Mar 2009
Location: Enfield, NH, USA
Posts: 138
Blog Entries: 4
Rep Power: 7
gocarts is on a distinguished road
Yes, check out SpeedIT Toolkit 0.9 for OpenFOAM.
__________________
Symscape, Computational Fluid Dynamics for all
gocarts is offline   Reply With Quote

Old   July 14, 2010, 09:13
Default
  #3
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 353
Rep Power: 9
arjun is on a distinguished road
Quote:
Originally Posted by gocarts View Post
have you guys implimented BiCGstab method??
If yes how was the performance.

From the link you posted , a quick glance only show matrix vector product timings.

How about AMG in GPU???


PS: I recently tried BiCGstab but did not observe much speed up. (but my gpu only has 110 cores, it is gts 240)
arjun is offline   Reply With Quote

Old   July 14, 2010, 12:07
Default
  #4
Member
 
Nuno Gomes
Join Date: May 2009
Location: Portugal
Posts: 39
Rep Power: 7
Dinocrack is on a distinguished road
Sorry for my question, but what are the main differences between FVM based codes under GPU computational environments and FVM based codes under CPU computational environments?

If we want to write a code based under GPU enviroment what are the main points to take in mind?

There is some literature?

Thanks very many

Nuno
Dinocrack is offline   Reply With Quote

Old   July 14, 2010, 19:42
Default
  #5
New Member
 
Harry
Join Date: Mar 2009
Posts: 27
Rep Power: 7
harry is on a distinguished road
Quote:
Originally Posted by gocarts View Post
this is a very interesting work! It is not clear what kind of systems this linear equation solver is based on, linux/windows, or both.
Hope we can see its complete release soon.

Cheers,
Harry
harry is offline   Reply With Quote

Old   July 14, 2010, 21:19
Default
  #6
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 353
Rep Power: 9
arjun is on a distinguished road
Quote:
Originally Posted by Dinocrack View Post
Sorry for my question, but what are the main differences between FVM based codes under GPU computational environments and FVM based codes under CPU computational environments?

If we want to write a code based under GPU enviroment what are the main points to take in mind?

There is some literature?

Thanks very many

Nuno

Here are the basic rules of this game:

A) You have many cores that take the task and do it. You divide them into troops and each troop take a part of the job and work on it.
Within this troop you have worker, each worker can work on something.

B) all the workers will do the same thing. Imagine that you want to calculate
C[i] = A[i] + B[i];
each worker will do this , only difference is it will use different i, but the action is same.


C) each worker should not interfere with other worker's work. Or in other words , the work each worker is doing is independent of others.

For example c[i] = a[i] + b[i] is independent of other's values so could be programmed by GPU.

but for c[i] = c[ i ] + c[ i-1] + c[ i - 2 ]

c[ i] is dependent on i-1 and i-2 values, imagine that when worker on i is working on , the worker on i-1 and i-2 did not finish their job. This will cause error in results. So not directly parallelizable.

There are some more basics, but this is main idea behind it.
arjun is offline   Reply With Quote

Old   September 14, 2010, 11:53
Default
  #7
Senior Member
 
Andrea Pasquali
Join Date: Sep 2009
Location: Germany
Posts: 137
Rep Power: 6
andrea.pasquali is on a distinguished road
Hi,
I'm interesting to GPU for OpenFOAM.
I installed the SpeedIT Classic version but I have a problem you can see here:

SpeedIT Classic and OpenFOAM

Could anyone help me?

Thanks


Andrea
__________________
Andrea Pasquali
andrea.pasquali is offline   Reply With Quote

Old   May 22, 2012, 10:12
Default
  #8
Member
 
Lukasz Miroslaw
Join Date: Dec 2009
Location: Poland
Posts: 54
Rep Power: 6
Lukasz is on a distinguished road
Send a message via Skype™ to Lukasz
Quote:
Originally Posted by arjun View Post
have you guys implimented BiCGstab method??
If yes how was the performance.

From the link you posted , a quick glance only show matrix vector product timings.

How about AMG in GPU???
We have just released SpeedIT 2.1 with AMG preconditioner.
See our benchmarks at vratis.com/blog & speed-it.vratis.com for details.
Lukasz is offline   Reply With Quote

Old   May 22, 2012, 20:23
Default
  #9
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 353
Rep Power: 9
arjun is on a distinguished road
Quote:
Originally Posted by Lukasz View Post
We have just released SpeedIT 2.1 with AMG preconditioner.
See our benchmarks at vratis.com/blog & speed-it.vratis.com for details.

Just one small comment. I think your comparison with GAMG is not fair. Smoothed aggregation is very fast compared to GAMG and hence if you run the both on CPUs also you will see that solver with smoothed aggregation is fast.
arjun is offline   Reply With Quote

Old   May 23, 2012, 03:49
Default
  #10
Member
 
Lukasz Miroslaw
Join Date: Dec 2009
Location: Poland
Posts: 54
Rep Power: 6
Lukasz is on a distinguished road
Send a message via Skype™ to Lukasz
Could you be more specific and provide some examples where indeed CG+AMG is faster than GAMG? We did some tests with icoFoam and cavity3D, simpleFoam for Ahmedbody and Cabin cases: http://vratis.com/blog/?page_id=2
and in all these examples GAMG seemed to outperform other methods. Although we could only compare with CG+DIC/diagonal on CPU.
Our multi GPU solution with AMG seems to be about 60% faster than OpenFOAM (1.6x acceleration of nGPU vs. N CPU, where n is the number of CPU cores and N is the number of GPU cards, measured for various cases for 10 first iterations).
Lukasz is offline   Reply With Quote

Old   May 23, 2012, 04:20
Default
  #11
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 353
Rep Power: 9
arjun is on a distinguished road
Quote:
Originally Posted by Lukasz View Post
Could you be more specific and provide some examples where indeed CG+AMG is faster than GAMG? We did some tests with icoFoam and cavity3D, simpleFoam for Ahmedbody and Cabin cases: http://vratis.com/blog/?page_id=2
and in all these examples GAMG seemed to outperform other methods. Although we could only compare with CG+DIC/diagonal on CPU.
Our multi GPU solution with AMG seems to be about 60% faster than OpenFOAM (1.6x acceleration of nGPU vs. N CPU, where n is the number of CPU cores and N is the number of GPU cards, measured for various cases for 10 first iterations).

I can not at the moment (because i do not use openfoam) but I have implemented smoothed aggregation (both single matrix and coupled matrix (u,v,w and p) (iNavier uses smoothed aggregation preconditioned BiCGStab for press by default). It is my experience that BiCGStab preconditioned with smoothed aggregation is more than 2 times faster compared to simple AMG.
the gap is much more bigger when mesh sizes increases. (It could even be more than 5 times). If you really want to see the difference try 5 million or more cells.

This is based on my experience with smoothed aggregation, classical AMG and Bi CGSTab.

Note: CG preconditioned with AMG is not that fast. (sound strange but it is true in practice).



Edited to add: You are looking for timing of Navier stokes by changing Solvers. Which i think is not linearly related matrix solvers timing. You should be only comparing pressure equations convergence and time taken by matrix solvers.
arjun is offline   Reply With Quote

Old   May 24, 2012, 13:36
Default
  #12
Member
 
Lukasz Miroslaw
Join Date: Dec 2009
Location: Poland
Posts: 54
Rep Power: 6
Lukasz is on a distinguished road
Send a message via Skype™ to Lukasz
This is interesting and we will remember this for future.
FYI, we have never used BCGstab+AMG as most of the time of our solver was spent in solving the pressure equation with CG. Solving u,v,w with BCGstab took much less time: a few iterations per time step comparing to hundreds to solve pressure equation.

BTW, we are just writing a paper about ARAEL, our new Navier-Stokes solver that we completely implemented on GPU. Profiling analysis will be added there as well for several tests. if you are interested I could send you a camera-ready version once this is ready.
Lukasz is offline   Reply With Quote

Old   May 24, 2012, 13:42
Default
  #13
Senior Member
 
cfdnewbie
Join Date: Mar 2010
Posts: 546
Rep Power: 10
cfdnewbie is on a distinguished road
This is interesting, Lukasz. May I ask what type of solver is it? compressible/incompressible? FV or something else?
cfdnewbie is offline   Reply With Quote

Old   May 24, 2012, 13:49
Default
  #14
Member
 
Lukasz Miroslaw
Join Date: Dec 2009
Location: Poland
Posts: 54
Rep Power: 6
Lukasz is on a distinguished road
Send a message via Skype™ to Lukasz
Incompressible for steady-state and transient flows. Take a look at this presentation for more details.
Lukasz is offline   Reply With Quote

Old   May 24, 2012, 13:52
Default
  #15
Senior Member
 
cfdnewbie
Join Date: Mar 2010
Posts: 546
Rep Power: 10
cfdnewbie is on a distinguished road
Thanks a lot, very impressive.
cfdnewbie is offline   Reply With Quote

Old   May 24, 2012, 16:36
Default
  #16
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 353
Rep Power: 9
arjun is on a distinguished road
Quote:
Originally Posted by Lukasz View Post
This is interesting and we will remember this for future.
FYI, we have never used BCGstab+AMG as most of the time of our solver was spent in solving the pressure equation with CG. Solving u,v,w with BCGstab took much less time: a few iterations per time step comparing to hundreds to solve pressure equation.
I never solve u,v, w with AMG. It is not very efficient.

Quote:
Originally Posted by Lukasz View Post
BTW, we are just writing a paper about ARAEL, our new Navier-Stokes solver that we completely implemented on GPU. Profiling analysis will be added there as well for several tests. if you are interested I could send you a camera-ready version once this is ready.
I wrote full navier stokes in GPU in year 2010 but the code is for single GPU. I did not release that version of iNavier because I could not find time to create GPU version for rest of the things (like turbulence etc).


PS: I can not do AMG creation part in GPU. I know how CUSP lib does though.

Please do send it to me. I am definitely very interested. Thank you.
arjun is offline   Reply With Quote

Old   May 24, 2012, 16:37
Default
  #17
Senior Member
 
Arjun
Join Date: Mar 2009
Location: Nurenberg, Germany
Posts: 353
Rep Power: 9
arjun is on a distinguished road
Quote:
Originally Posted by Lukasz View Post
Incompressible for steady-state and transient flows. Take a look at this presentation for more details.

I am going to.
arjun is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
A simple CFD code for teaching basic CFD? Christoph Lund Main CFD Forum 13 September 14, 2005 04:36
ASME CFD Symposium Chris Kleijn Main CFD Forum 0 August 22, 2001 06:41
Customer Services Patrick Godon Main CFD Forum 32 August 23, 1999 06:55
Commercial CFD code Hanson G. He Main CFD Forum 1 October 15, 1998 08:49
CFD Symposium (Call for Papers) Chris R. Kleijn Main CFD Forum 0 October 5, 1998 10:25


All times are GMT -4. The time now is 07:23.