CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Programming & Development

Adding an external library which uses openmp to a solver

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   November 26, 2020, 14:40
Default Adding an external library which uses openmp to a solver
  #1
New Member
 
Join Date: Feb 2019
Posts: 23
Rep Power: 4
dolfus is on a distinguished road
Hello all,

I've added a dynamic external library '.so' to the interFoam solver. The library is parallelized using openmp and compiled using the flag -fopenmp in gcc compiler.

When I use this library on a C program written out of OpenFOAM, things go right. However, using it within OpenFOAM, the cpu-time increases by a factor of 2 running the same simulation (compared to the C program). It looks like OpenFOAM is not taking on account this parallelization and only uses a single core (checked with top command).

Regarding the OpenFOAM solver, I've compiled it adding to the make file the flag -fopenmp, and also writing the #include omp.h statement at interFoam.C file. No matter what value of OMP_NUM_THREADS I use for the OpenFOAM simualtion, the cpu-time is almost the same.

I don't know if anyone has experienced a similar problem.


Thank you

Last edited by dolfus; November 26, 2020 at 14:55. Reason: typo
dolfus is offline   Reply With Quote

Old   November 27, 2020, 12:26
Default
  #2
Senior Member
 
Mark Olesen
Join Date: Mar 2009
Location: http://olesenm.github.io/
Posts: 1,144
Rep Power: 28
olesen will become famous soon enougholesen will become famous soon enough
You might need to have COMP_OPENMP and LINK_OPENMP in your wrapping code in order to have proper linkage. Not really sure.


https://develop.openfoam.com/Develop...ral/Gcc/openmp


EDIT: I mean to have them actually used in the corresponding Make/options file.
olesen is offline   Reply With Quote

Old   November 27, 2020, 13:41
Default
  #3
New Member
 
Join Date: Feb 2019
Posts: 23
Rep Power: 4
dolfus is on a distinguished road
Thank you for your reply.

I've added those flags to the compiler, but it is still not working. In fact, if I run the simulation with OMP_NUM_THREADS=1, CPU-time is a 10% lesser that in the case with 2 threads, which makes no sense.
I'm not sure what's going on.
dolfus is offline   Reply With Quote

Old   November 27, 2020, 15:09
Default
  #4
Senior Member
 
Mark Olesen
Join Date: Mar 2009
Location: http://olesenm.github.io/
Posts: 1,144
Rep Power: 28
olesen will become famous soon enougholesen will become famous soon enough
Probably need to test something really basic, like threads with an "printf" to see if anything is actually running with threads when linked with openfoam.
olesen is offline   Reply With Quote

Old   November 30, 2020, 07:48
Default
  #5
New Member
 
Join Date: Feb 2019
Posts: 23
Rep Power: 4
dolfus is on a distinguished road
I've checked it out with a printf on my library and, surprisingly, it actually runs using threads when linked with openFOAM.
So it should work fine but it's not. Probably I'm still missing something, or maybe this is the best way openFOAM can handle openmp on an external library.
I also noticed that when running with 2 threads execution time is almost twice the clock time, but when running with 1 thread they are quite similar.

Last edited by dolfus; November 30, 2020 at 11:11.
dolfus is offline   Reply With Quote

Old   December 1, 2020, 13:55
Default
  #6
New Member
 
Join Date: Feb 2019
Posts: 23
Rep Power: 4
dolfus is on a distinguished road
After several tests I think your solution fixed my problem of communication with openMP. Thank you very much.

Despite that, I would like to add something I realized, in case someone find it usefull. Turns out that runTime.elapsedCpuTime() sums up the times consumed by all threads. That's why I was obtaining such different times between my code and OF. Using omp_get-wtime() instead of OF's solved this issue. However, using OF with MPI on a single PC with 2 processors there is no such difference between CPU-time and clock-time.
dolfus is offline   Reply With Quote

Reply

Tags
compilation, openfoam, openmp, shared libraries

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Duplicate library entries when running a solver with custom library francescomarra OpenFOAM Programming & Development 0 May 8, 2020 09:02
Coupling to external code using MPI calls from within a solver Regis_ OpenFOAM Programming & Development 0 October 31, 2017 11:50
Running external application in solver dariodario32 OpenFOAM Programming & Development 2 July 10, 2017 09:52
External aerodynamics of an aircraft: pressure based solver or density based solver? shereez234 FLUENT 2 April 30, 2015 16:30
adding Porous media to lagrangian Solver kalyan OpenFOAM Programming & Development 0 September 19, 2014 08:41


All times are GMT -4. The time now is 05:59.