CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Installation

extend-3.0 cuda installation

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree8Likes

Reply
 
LinkBack Thread Tools Display Modes
Old   February 19, 2014, 11:24
Default Fix pushed to repository
  #21
New Member
 
Dominik Christ
Join Date: Mar 2009
Posts: 28
Rep Power: 8
dominik_christ is on a distinguished road
Hi all,

I fixed the issues mentioned in the posts above (Allwake logic, messed up header sections), tested and pushed to the repository (commit number: 55a148542d). To get the fix, run "git pull", clean and recompile.

Best regards,
Dominik
rkc.cfd likes this.
dominik_christ is offline   Reply With Quote

Old   February 19, 2014, 11:30
Default
  #22
New Member
 
Join Date: Aug 2011
Posts: 20
Rep Power: 5
Kojote is on a distinguished road
Quote:
Originally Posted by dominik_christ View Post
Hi all,

I fixed the issues mentioned in the posts above (Allwake logic, messed up header sections), tested and pushed to the repository (commit number: 55a148542d). To get the fix, run "git pull", clean and recompile.

Best regards,
Dominik
Hi

Sounds good, i will give it try. Thx again for your hard work.

Br

Christian
Kojote is offline   Reply With Quote

Old   April 3, 2014, 23:30
Default
  #23
New Member
 
Join Date: Dec 2012
Location: Tokyo, Japan
Posts: 5
Rep Power: 4
Frivole is on a distinguished road
Hi everyone!

I am also trying to use cuda with the ext3.0.
I have a working installation of cuda (in opt/cuda), and installed 3.0.
However when I try to run the Allwmake.firstInstall script, I get the error described by Dominik :
Quote:
$CUDA_ARCH is required by nvcc compiler but not set.
Check section '-gpu-architecture' in 'man nvcc' for details.

Proceed without compiling cudaSolvers? [Y/n]
I have in my etc/prefs.sh :
# System installed CUDA
export CUDA_SYSTEM=1
export CUDA_DIR=/opt/cuda
export CUDA_BIN_DIR=$CUDA_DIR/bin
export CUDA_LIB_DIR=$CUDA_DIR/lib
export CUDA_INCLUDE_DIR=$CUDA_DIR/include
export CUDA_ARCH=sm_20

I used the CUDA_ARCH setting present in etc/presh.sh-EXAMPLE.
The device I currently use is an old Quadro FX 570.
Best regards,

Pierre
Frivole is offline   Reply With Quote

Old   April 4, 2014, 04:01
Default
  #24
New Member
 
Dominik Christ
Join Date: Mar 2009
Posts: 28
Rep Power: 8
dominik_christ is on a distinguished road
Hi Pierre,
the message pops up only if CUDA_ARCH is not set. However, you say that is specified in prefs.sh . Can you please verify that prefs.sh is actually setting the variable?

Steps:
1) Source the bashrc:
. etc/bashrc
(when you are in installation root directory, e.g. ~/foam/foam-extend-3.0/)

2) Check whether CUDA_ARCH is set:
env | grep CUDA*

Please post the output of step 2)

Best regards,
Dominik
dominik_christ is offline   Reply With Quote

Old   April 7, 2014, 01:18
Default
  #25
New Member
 
Join Date: Dec 2012
Location: Tokyo, Japan
Posts: 5
Rep Power: 4
Frivole is on a distinguished road
Dear Dominik,

I'm very sorry for the late answer.
It appeared that I had an issue with my swak4foam installation. After reinstalling it I manage to run the Allwmake.firstInstall script without getting the previous error.
However I still cannot compile the cuda solver. When I run the Allmake I get the following result :
Code:
Found nvcc  --  enabling CUDA support.
make: *** No rule to make target `cudaCG/cgDiag.dep', needed by `Make/linux64Gcc46DPOpt/dependencies'.  Stop.
make: *** Waiting for unfinished jobs....
Making dependency list for source file cudaSolver/cudaSolver.C
Also the result of the second step gives :
Code:
CUDA_BIN_DIR=/opt/cuda/bin
CUDA_DIR=/opt/cuda
CUDA_SYSTEM=1
CUDA_ARCH=sm_20
CUDA_LIB_DIR=/opt/cuda/lib
CUDA_INCLUDE_DIR=/opt/cuda/include
Best regards,

Pierre
Frivole is offline   Reply With Quote

Old   April 7, 2014, 05:41
Default
  #26
New Member
 
zhong
Join Date: Oct 2013
Location: Dublin, Ireland
Posts: 3
Rep Power: 3
simulation is on a distinguished road
Hi everyone,

I am also trying to use the cudaSolver of foam-extend-3.0.
I am quite new to openfoam, so I set WM_COMPILE_OPTION:=Debug in the foam-extend-3.0etc/bashrc because I may need to debug to understand the code better, including the cudaSolver.
I got some errors related to the nvcc options during compilation, as follows:

Code:
+ cd cudaSolvers
+ ./Allwmake
Found nvcc -- enabling CUDA support.
SOURCE=cudaCG/cgDiag.cu ; nvcc -m64 -arch sm_20 -DNoRepository -Xcompiler -fPIC -I/home/zzhong/cuda_4.2/include -IlnInclude -I. -I/home/zzhong/foam/foam-extend-3.0/src/foam/lnInclude -I/home/zzhong/foam/foam-extend-3.0/src/OSspecific/POSIX/lnInclude -Dlinux64 -DWM_DP -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -c $SOURCE -o Make/linux64GccDPDebug/cgDiag.o
SOURCE=cudaCG/cgAinv.cu ; nvcc -m64 -arch sm_20 -DNoRepository -Xcompiler -fPIC -I/home/zzhong/cuda_4.2/include -IlnInclude -I. -I/home/zzhong/foam/foam-extend-3.0/src/foam/lnInclude -I/home/zzhong/foam/foam-extend-3.0/src/OSspecific/POSIX/lnInclude -Dlinux64 -DWM_DP -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -c $SOURCE -o Make/linux64GccDPDebug/cgAinv.o
SOURCE=cudaBiCGStab/bicgDiag.cu ; nvcc -m64 -arch sm_20 -DNoRepository -Xcompiler -fPIC -I/home/zzhong/cuda_4.2/include -IlnInclude -I. -I/home/zzhong/foam/foam-extend-3.0/src/foam/lnInclude -I/home/zzhong/foam/foam-extend-3.0/src/OSspecific/POSIX/lnInclude -Dlinux64 -DWM_DP -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -c $SOURCE -o Make/linux64GccDPDebug/bicgDiag.o
SOURCE=cudaCG/cgAmg.cu ; nvcc -m64 -arch sm_20 -DNoRepository -Xcompiler -fPIC -I/home/zzhong/cuda_4.2/include -IlnInclude -I. -I/home/zzhong/foam/foam-extend-3.0/src/foam/lnInclude -I/home/zzhong/foam/foam-extend-3.0/src/OSspecific/POSIX/lnInclude -Dlinux64 -DWM_DP -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -c $SOURCE -o Make/linux64GccDPDebug/cgAmg.o
SOURCE=cudaBiCGStab/bicgAinv.cu ; nvcc -m64 -arch sm_20 -DNoRepository -Xcompiler -fPIC -I/home/zzhong/cuda_4.2/include -IlnInclude -I. -I/home/zzhong/foam/foam-extend-3.0/src/foam/lnInclude -I/home/zzhong/foam/foam-extend-3.0/src/OSspecific/POSIX/lnInclude -Dlinux64 -DWM_DP -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -c $SOURCE -o Make/linux64GccDPDebug/bicgAinv.o
nvcc fatal : Unknown option 'fdefault-inline'
make: *** [Make/linux64GccDPDebug/cgDiag.o] Error 255
make: *** Waiting for unfinished jobs....
nvcc fatal : Unknown option 'fdefault-inline'
make: *** [Make/linux64GccDPDebug/cgAinv.o] Error 255
nvcc fatal : Unknown option 'fdefault-inline'
make: *** [Make/linux64GccDPDebug/cgAmg.o] Error 255
nvcc fatal : Unknown option 'fdefault-inline'
make: *** [Make/linux64GccDPDebug/bicgDiag.o] Error 255
nvcc fatal : Unknown option 'fdefault-inline'
make: *** [Make/linux64GccDPDebug/bicgAinv.o] Error 255
I an not very clear on how the whole project is compiled and where I can remove/modify those options for nvcc.

I have in my etc/prefs.sh :
export CUDA_SYSTEM=1
export CUDA_DIR=/home/zzhong/cuda_4.2
export CUDA_BIN_DIR=$CUDA_DIR/bin
export CUDA_LIB_DIR=$CUDA_DIR/lib64
export CUDA_INCLUDE_DIR=$CUDA_DIR/include
export CUDA_ARCH=sm_20

The GPU attached to the node is Tesla C2050.

There are no errors when compiling with WM_COMPILE_OPTION:=Opt, and the libcudaSolvers.so is generated into lib/linux64GccDPOpt.

Best regards
Zhong
simulation is offline   Reply With Quote

Old   April 23, 2014, 07:42
Default
  #27
New Member
 
Dominik Christ
Join Date: Mar 2009
Posts: 28
Rep Power: 8
dominik_christ is on a distinguished road
Dear Zhong,
Does the compilation work for you with Opt instead of Debug?
Best regards,
Dominik
dominik_christ is offline   Reply With Quote

Old   April 23, 2014, 08:20
Default
  #28
New Member
 
zhong
Join Date: Oct 2013
Location: Dublin, Ireland
Posts: 3
Rep Power: 3
simulation is on a distinguished road
Hi Dominik,

Yes, the compilation with Opt went well without errors.

Best regards
Zhong
simulation is offline   Reply With Quote

Old   April 23, 2014, 16:06
Default
  #29
Member
 
xxxxx
Join Date: Feb 2013
Posts: 34
Rep Power: 4
xxxx is on a distinguished road
Dear Zhong,

now that you have the cuda libraries working, do you find any advantage running the cases with cuda?
Can you please tell us about your experience?
Thank you.

F.F.
__________________
Federico
xxxx is offline   Reply With Quote

Old   April 24, 2014, 09:17
Default
  #30
New Member
 
zhong
Join Date: Oct 2013
Location: Dublin, Ireland
Posts: 3
Rep Power: 3
simulation is on a distinguished road
Hi Federico,

To be honest, I haven't done any serious test yet. Sorry for that.
I will let you know once I have some results.

Actually I am currently still experimenting with cufflink library rather than the cudaSolver included in the foam-extend-3.0, because at this moment I am interested in running the GPU solver in parallel, but I am not sure if the cudaSolver in foam-extend-3.0 can run in parallel.

I am trying to figure out what domain decomposition method is used and how it is implemented in foam, obviously some information (interfaces?) are exchanged between sub-domains during each iteration of the linear solver. Any ideas?

Best Regards
Zhong
simulation is offline   Reply With Quote

Old   June 7, 2014, 10:16
Default
  #31
Senior Member
 
mahdi abdollahzadeh
Join Date: Mar 2011
Location: Covilha,Portugal
Posts: 149
Rep Power: 6
mm.abdollahzadeh is on a distinguished road
Quote:
Originally Posted by rkc.cfd View Post
Hi,

Sorry for the late reply, some how I was not subscribed to the thread and did not get the message.

The link to my github is

https://github.com/mechysolv/openfoa...extend-3.0.git

I have downloaded the cusp solvers to

Code:
$WM_THIRD_PARTY_DIR/LocalDev
from

https://github.com/cusplibrary/cusplibrary

Hope that helps,

RKC
Dear Kirishna

I had installed everything successfully.
but when I try to run a case I am getting a problem.

Quote:
terminate called after throwing an instance of 'thrust::system::system_error' what (): invalid device function
Aborted (core dumped)
did u have similar problem ? or do u know how to resolve it?

many thanks

best
mahdi
mm.abdollahzadeh is offline   Reply With Quote

Old   June 9, 2014, 19:03
Default
  #32
New Member
 
Krishna
Join Date: Jun 2012
Posts: 27
Blog Entries: 1
Rep Power: 4
rkc.cfd is on a distinguished road
Hi Mahdi,

This looks like an error when you have not installed the Nvidia drivers properly. I have no experience in installing Cuda. Check if you have installed nvidia cuda drivers properly.

http://forums.udacity.com/questions/...ror-in-maincpp

Cheers
mm.abdollahzadeh likes this.
rkc.cfd is offline   Reply With Quote

Old   June 10, 2014, 04:09
Default
  #33
Senior Member
 
mahdi abdollahzadeh
Join Date: Mar 2011
Location: Covilha,Portugal
Posts: 149
Rep Power: 6
mm.abdollahzadeh is on a distinguished road
Quote:
Originally Posted by rkc.cfd View Post
Hi Mahdi,

This looks like an error when you have not installed the Nvidia drivers properly. I have no experience in installing Cuda. Check if you have installed nvidia cuda drivers properly.

http://forums.udacity.com/questions/...ror-in-maincpp

Cheers

Dear Krishna

I had solved that. problem was related to gcc version when compiling cuda solvers. I was using gcc 4.6 for installing cuda driver, but i didn't switch afterwards to gcc 4.8.

however I have another question? My graphic card is gforce 8400 Gs. is there any reason that I was not able to use another cuda arch like sm_20? just sm_10 worked in my case.

and Im wondering that is it possible to select some how the number of the gpus or run in parallel with gpus? since i guess foam extend is using cufftlink-library it should be possible, but i was not able to test even the cufftlink-library sepratly to test it as I guess my ubuntu and nvidia driver are more recent than the one it was tested for.


best
mahdi
mm.abdollahzadeh is offline   Reply With Quote

Old   June 10, 2014, 14:19
Default
  #34
New Member
 
Krishna
Join Date: Jun 2012
Posts: 27
Blog Entries: 1
Rep Power: 4
rkc.cfd is on a distinguished road
Hi,

I have not used multi GPU option, currently I am using only CPU. I have google and found this, hope it should help.

https://code.google.com/p/cufflink-l...ingStartedPage

Cheers
rkc.cfd is offline   Reply With Quote

Old   July 25, 2014, 05:34
Default
  #35
New Member
 
Join Date: Aug 2011
Posts: 20
Rep Power: 5
Kojote is on a distinguished road
Hi All

After a long time i am now able to run fe31 with cuda. Now its really easy. Install cuda and put the cusp folder into the cuda/include folder.
Setup the cuda paths in of. Done.

Should we start a thread "How to use cuda?"

Thx to all

Christian
Kojote is offline   Reply With Quote

Old   August 6, 2014, 06:09
Default
  #36
Member
 
Aleksey Rukhlenko
Join Date: Nov 2009
Location: Moscow
Posts: 49
Rep Power: 7
Aleksey_R is on a distinguished road
Send a message via ICQ to Aleksey_R
Dear colleagues,

did anyone succeed in with compilation of cudaSolvers for foam-extend-3.1?

During compilation I get that there's no rule for 'cudaCG/cgDiag.dep'. Indeed, there's no such file, there's only cgDiag.cu.

CUDA variables in prefs.sh are set properly.


Best regards,
Aleksey.
Aleksey_R is offline   Reply With Quote

Old   August 10, 2014, 10:48
Default
  #37
Member
 
Aleksey Rukhlenko
Join Date: Nov 2009
Location: Moscow
Posts: 49
Rep Power: 7
Aleksey_R is on a distinguished road
Send a message via ICQ to Aleksey_R
I've found out how to compile cudaSolvers. It appeared that there's no nvcc rules for gcc46. nvcc rules exist only for gcc. So I've compiled foam by system compiler.

1. I've downloaded cusplibrary and modified options file as it was mentioned before in this thread.

2. In all source files of cudaSolvers I've changed "#include <cusp/blas.h>" on "#include <cusp/blas/blas.h>"

3. I changed path for cuda libs from lib to lib64 folder in prefs.sh

After that everything compiled OK.

Best regards,
Aleksey.
Aleksey_R is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
A smooth installation of foam-3.0-extend on OpenSUSE 13.1 64-bits cookcaptain OpenFOAM Installation 6 March 12, 2014 00:05
what should we do for the application of CUDA in CFD? aerosjc Main CFD Forum 6 February 25, 2013 12:01
CFX11 + Fortran compiler ? Mohan CFX 20 March 30, 2011 18:56
64bitrhel5 OF installation instructions mirko OpenFOAM Installation 2 August 12, 2008 18:07
Installation problems shellbell1999 OpenFOAM Installation 9 April 6, 2006 13:29


All times are GMT -4. The time now is 01:29.