CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM Installation (https://www.cfd-online.com/Forums/openfoam-installation/)
-   -   Installation OpenFOAM-1.7.x with wmakeScheduler (https://www.cfd-online.com/Forums/openfoam-installation/81344-installation-openfoam-1-7-x-wmakescheduler.html)

Tobi October 24, 2010 16:13

Installation OpenFOAM-1.7.x with wmakeScheduler
 
Hey I am Tobi and a new community member,

first of all. Hello everybody.


I am working with Ubuntu 10.04 an had installed OpenFoam 1.7.1 with the Ubuntu/Debian Pack without problems.

So a good CFD - eng said, that the only version is up to date is the git responsitory.

I had downloaded everything and did all steps in the introductions.
Got so much mistakes and issues with compiling OpenFoam, but hope i have fixed it now. The only thing i can 't fix until now is using wmakeScheduler for compiling OF with my two cores :/

-> export WM_SCHEDULER=wmakeScheduler
-> export WM_HOSTS="shorty:2"
-> export WM_NCOMPPROCS=$($WM_SCHEDULER -count)

after typing echo $WM_NCOMPPROCS I got --> 2

After starting ./Allwmake it will stop at the beginning of line

"SOURCE=..."
"SOURCE=..."

what did i wrong? Sorry but i am new on Ubuntu - Distributions and so on.

Hope someone can help me. Couse i need the new Solver added in the new update.

Thanks
Tobi

wyldckat October 24, 2010 16:28

Greetings Tobi and welcome to the forum!

Mmm, they should really put up a warning that for single machines, the wmakeScheduler isn't necessary. You can simply just do:
Code:

export WM_NCOMPPROCS=2
Or equal to the number of cores that your machine has. The other two variables aren't necessary.

edit: Wait... but WM_NCOMPPROCS did get set properly, since you said that 'export WM_HOSTS="shorty:2"' !! There was nothing wrong there, as far as I can tell!

Additionally, there are a few packages that you'll need to install to ensure that OpenFOAM will build: Building OpenFOAM1.7.0 from source post #4 and onward
In that thread you'll find the packages needed for building OpenFOAM and the code documentation (a few posts down).

Finally, to keep a log of the building process, run Allwmake like this:
Code:

./Allwmake > make.log 2>&1
You might need it after it's done, since something could go wrong :(

Best regards and good luck!
Bruno

Tobi October 24, 2010 17:47

Hi Bruno,

I have read your thread that you have linked above a hour befor and added all the packages. Think that was my real problem to compile it without issues.


Just, i compile with 1 core :/ although i set WM_NCOMPPROS=2
Don 't know why it don' t work.

Will install OF on my new computer in three day's (6 cores)
I will test it with your Code

Code:

./Allwmake > make.log 2>&1
Thx for answering :)
Couse i need the chtMultiRegionLiquidFoam solver :)


Tobi

wyldckat October 24, 2010 18:13

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 280558)
Just, i compile with 1 core :/ although i set WM_NCOMPPROS=2
Don 't know why it don' t work.

Ooooo, so close... but if you really used that name, it's no wonder it didn't work :( It's:
Code:

export WM_NCOMPPROCS=2
You missed a C before the last S :(

If you want a more self-adapting code for setting that variable, you can use this:
Quote:

Originally Posted by $WM_THIRD_PARTY_DIR/tools/makeThirdPartyFunctions
Code:

    if [ -r /proc/cpuinfo ]
    then
        WM_NCOMPPROCS=$(egrep "^processor" /proc/cpuinfo | wc -l)
        [ $WM_NCOMPPROCS -le 8 ] || WM_NCOMPPROCS=8
    else
        WM_NCOMPPROCS=1
    fi

    echo "Building on $WM_NCOMPPROCS cores"


Add it to OpenFOAM's bashrc file, or to your own "~/.bashrc" file :)

Best regards,
Bruno

Tobi October 24, 2010 18:19

Quote:

Originally Posted by wyldckat (Post 280559)

Ooooo, so close... but if you really used that name, it's no wonder it didn't work :( It's:
Code:

export WM_NCOMPPROCS=2
You missed a C before the last S :(

:) no I got the right var. Why it don 't work - don 't know.
Will try your last post and set it into my .bashrc.

Thx. Will make an answer how it work.

Tobi October 25, 2010 03:44

Okay :)

OpenFoam-1.7.x (git) is now working (jehaaa) (:

if i install it on my new machine i will try to set the "WM_HOSTS" variable to

Code:

export WM_HOSTS="localhost:6"

Maybe this will work.

Greetings Tobi

Tobi October 25, 2010 06:41

Hey again,


included your code to my ~/.bashrc

Code:

shorty@shorty:~/OpenFOAM/OpenFOAM-1.7.x$ . $HOME/.bashrc
Building on 2 cores
shorty@shorty:~/OpenFOAM/OpenFOAM-1.7.x$ ./Allwmake

But he worked just with 1 core. Don 't understand why.
Got Ubuntu 10.04. A CFD-eng. said today, that the compiling with more cores doen 't work. He has linux and there it works but on his ubuntu don 't work the Scheduler.

I tested everything. If i build on 2 cores he is working until that code end

Code:

Done ThirdParty Allwmake
========================================

+ wmakePrintBuild -check
same version as previous build
+ wmakeLnInclude OpenFOAM
+ wmakeLnInclude OSspecific/POSIX
+ Pstream/Allwmake
+ wmake libso dummy
'/home/shorty/OpenFOAM/OpenFOAM-1.7.x/lib/linux64GccDPOpt/dummy/libPstream.so' is up to date.
+ set +x

Note: ignore spurious warnings about missing mpicxx.h headers
+ WM_OPTIONS=linux64GccDPOptOPENMPI
+ wmake libso mpi
'/home/shorty/OpenFOAM/OpenFOAM-1.7.x/lib/linux64GccDPOpt/openmpi-1.4.1/libPstream.so' is up to date.
+ wmake libo OSspecific/POSIX
'/home/shorty/OpenFOAM/OpenFOAM-1.7.x/lib/linux64GccDPOpt/libOSspecific.o' is up to date.
+ wmake libso OpenFOAM
SOURCE=meshes/primitiveMesh/primitiveMesh.C ; wmakeScheduler g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/shorty/OpenFOAM/ThirdParty-1.7.x/zlib-1.2.3 -IlnInclude -I. -I/home/shorty/OpenFOAM/OpenFOAM-1.7.x/src/OpenFOAM/lnInclude -I/home/shorty/OpenFOAM/OpenFOAM-1.7.x/src/OSspecific/POSIX/lnInclude  -fPIC -c $SOURCE -o Make/linux64GccDPOpt/primitiveMesh.o

Hmm :/

Any ideas ?

Can make a log if this will help.
Tobi

wyldckat October 25, 2010 07:08

Hi Tobi,

Only when building libscotch and OpenMPI, it will only use a single core (the people at OpenCFD forgot about it :( ).

What does this output:
Code:

egrep "^processor" /proc/cpuinfo | wc -l
Additionally, are you building in a real or virtual machine?

-------------------- EDIT:
I think I figured it out - run this:
Code:

echo $HOST
If nothing outputs, then that is the problem! wmakeScheduler assumes that this variable exists, but apparently it depends on the Linux distribution used.
Try running this before you run Allwmake:
Code:

export HOST=$HOSTNAME
And use:
Code:

export WM_HOSTS="$HOSTNAME:6"
Instead of localhost!
Otherwise, you will have to make your ssh sessions passwordless. For example: How to connect with ssh without using a password

EDIT2: Ironically, this has already been fixed a few days ago in OpenFOAM 1.7.x.

Best regards,
Bruno

Tobi October 26, 2010 04:10

Hi Bruno,

first of all. Thx for your help.
I 'll got a new mainboard etc. so I have to wait until all is received.

And then I install everything new.

Question: Ubuntu 10.04 or CAE Linux Ubuntu 10.04?


After the installation i will check out your commands and post the output here.

Your question, if i build in a real or virtual machine, i think a real :( but I am not sure what you mean.

Build on my personal computer so i think its real.

Hope my english skills aren 't as bad as i think :(

Best reguards.
Tobi


PS: Think i will start compiling at saturdays.

wyldckat October 26, 2010 17:53

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 280742)
first of all. Thx for your help.

You're welcome and I'm glad I could be of help to you! Although this is almost as far as I can help when it comes to OpenFOAM itself, because I still don't have much experience with CFD and OpenFOAM+CFD :D

Quote:

Originally Posted by Tobi (Post 280742)
Question: Ubuntu 10.04 or CAE Linux Ubuntu 10.04?

Personally I've never used CAE Linux, but it can save you several hours in trying to install all of those applications that already comes with it! But if you only need one or two of them, maybe you're better off using the standard Ubuntu 10.04. But it depends very much on what tools you will need!

Quote:

Originally Posted by Tobi (Post 280742)
Your question, if i build in a real or virtual machine, i think a real :( but I am not sure what you mean.

If you don't know, then it's the real machine ;) Here is an example of a virtual machine: Install Windows XP inside ubuntu using Virtualbox

Quote:

Originally Posted by Tobi (Post 280742)
Hope my english skills aren 't as bad as i think :(

Mmm, perhaps you were still sleepy? ;)

Quote:

Originally Posted by Tobi (Post 280742)
PS: Think i will start compiling at saturdays.

With a bit of practice, with 6 cores you can let it compile while you lunch! Or while you have a snack :D Or go for a jog. My AMD 1055T built yesterday OpenFOAM 1.5-dev in about 36 minutes. So with 1.7.x should do it in roughly 40min.

Best regards,
Bruno

Tobi October 27, 2010 04:04

Quote:

With a bit of practice, with 6 cores you can let it compile while you lunch! Or while you have a snack :D Or go for a jog. My AMD 1055T built yesterday OpenFOAM 1.5-dev in about 36 minutes. So with 1.7.x should do it in roughly 40min.

hihi, but only if you can build on 2 - 3 - 4 - 6 cores etc.
You build OF - 1.5 dev on Linux or Ubuntu?

A friend told me that if you use Linux, its very easy to build with more cores couse you just have to type the WM_HOSTS compared with ubuntu.

I hope it will work with more cores in a few days ;)


Tobi

wyldckat October 27, 2010 07:00

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 280936)
hihi, but only if you can build on 2 - 3 - 4 - 6 cores etc.
You build OF - 1.5 dev on Linux or Ubuntu?

A friend told me that if you use Linux, its very easy to build with more cores couse you just have to type the WM_HOSTS compared with ubuntu.

I hope it will work with more cores in a few days ;)

Ubuntu is a Linux distribution! What happened is like I wrote in another post: OpenFOAM's 1.7.1 wmakeScheduler has a bug which is already fixed in 1.7.x. And if you only do:
Code:

export WM_NCOMPPROCS=6
it will build with multiple cores!

Best regards,
Bruno

Tobi October 28, 2010 10:30

Hey Bruno,

i know that Ubuntu is a distribution of Linux (:

So far. I will get the latest OpenFoam Version with git and hope that you are right (but i know that you are right (: )

But one problem still exists.
I am still waiting for my new mainbaord and cpu :/
But tomorrow - yea tomorrow it will be mine ;)


Thx a lot.
If any errors occur i ll post it.

Tobi

Tobi October 28, 2010 10:42

Wrong entry :(

Sorry for that post.

Tobi October 29, 2010 20:55

Hi Bruno,

Quote:

Originally Posted by wyldckat (Post 280620)
Hi Tobi,

What does this output:
Code:

egrep "^processor" /proc/cpuinfo | wc -l

Output: 6

Quote:

run this:
Code:

echo $HOST

without "export HOST="..." " there is no output like you said.

After setting HOST the variable exists and the output is: "shorty" (my computer name) but you said, that it is fixed now. I ll see.

I will try it now and if it doesn 't work i ll try it with SSH



Bye Tobi

Tobi October 29, 2010 21:12

Hey Bruno,

its me Tobi. It 's working with all 6 cores. Nice - very nice.

So last question. Is a process that is in background faster than a process which is shown on the terminal?

Tobi

wyldckat October 30, 2010 07:06

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 281442)
its me Tobi. It 's working with all 6 cores. Nice - very nice.

I'm glad it's finally working :)

Quote:

Originally Posted by Tobi (Post 281442)
So last question. Is a process that is in background faster than a process which is shown on the terminal?

I've never timed that in Linux :( I know that in Windows 95 in a 486, it was better to output to a file or simply hide the command window. Nowadays, I'm not certain.
In theory, outputting to the screen will require additional processing, so it will depend if it's the graphics card job to fully interpret what needs to be shown, therefore done in parallel, without holding back the CPU. If that's done properly, it should not weight very much. Otherwise, you're better off outputting to a file, since it will only require CPU+memory operations, some of which already necessary, and then the occasional flush to the hard drive.

I know that psychologically, I always feel that outputting to a file is faster :D

Best regards,
Bruno

socon009 May 15, 2012 17:25

Quote:

Originally Posted by wyldckat (Post 280552)
Greetings Tobi and welcome to the forum!

Mmm, they should really put up a warning that for single machines, the wmakeScheduler isn't necessary. You can simply just do:
Code:

export WM_NCOMPPROCS=2
Or equal to the number of cores that your machine has. The other two variables aren't necessary.

edit: Wait... but WM_NCOMPPROCS did get set properly, since you said that 'export WM_HOSTS="shorty:2"' !! There was nothing wrong there, as far as I can tell!

Additionally, there are a few packages that you'll need to install to ensure that OpenFOAM will build: Building OpenFOAM1.7.0 from source post #4 and onward
In that thread you'll find the packages needed for building OpenFOAM and the code documentation (a few posts down).

Finally, to keep a log of the building process, run Allwmake like this:
Code:

./Allwmake > make.log 2>&1
You might need it after it's done, since something could go wrong :(

Best regards and good luck!
Bruno

Hi,

I have followed numerous threads here on this topic.

Bruno, thank you for all of your input - it's been very helpful for me.

Currently, I am installing in RHEL 6.2 on multiple cores using the following command:

Code:

./Allwmake > make.log 2>&1
It has suppressed the output that I usually see while installing OpenFOAM.

Could you please offer an explanation of the command above?

In particular,

Code:

> make.log 2>&1
How does bash interpret this?

Lastly, where is the log file written to?

Thank you,
Shaun

wyldckat May 15, 2012 17:33

Greetings Shaun and welcome to the forum!

The "make.log" file is placed in the same folder where you ran Allwmake.
  • The first ">" redirects the output from Allwmake to "make.log"
  • The "2>&1" specifies that the stderr (2) output is to be redirected to stdout (1). In other words, both the errors and normal output are sent to the same file. If this isn't used, only the normal output is sent to the log file, while the errors would be shown on screen.
Best regards,
Bruno

socon009 May 15, 2012 17:59

Ok thanks! Great explanation.

Unfortunately, the log file created indicates that the installation process flopped. :(

I would like to figure out what is going on, but I am not sure how I should go about seeking help. :confused:

- Should I start another thread so as not to hi-jack this one?
- How can I use this log file to figure out what went wrong?

(All I can tell right now is that there are a heck of a lot of "undefined references" and nearly every one has something to do with one of a myriad of libraries.)

Thanks for any thoughts and suggestions :),
Shaun

Tobi May 16, 2012 02:10

Hi Shaun,

well you can use the following command:
Code:

./Allwmake 1> make.log 2> error.log
Its the same like befor but the stderr (2) is send to the error.log file. After the compiling you should get just the errors in the error.log.

Upload it and we can help you :)

Tobi

You installed the necessary packages ?
Are you compiling 1.7.x?

Quote:

Originally Posted by socon009 (Post 361304)
Ok thanks! Great explanation.

Unfortunately, the log file created indicates that the installation process flopped. :(

I would like to figure out what is going on, but I am not sure how I should go about seeking help. :confused:

- Should I start another thread so as not to hi-jack this one?
- How can I use this log file to figure out what went wrong?

(All I can tell right now is that there are a heck of a lot of "undefined references" and nearly every one has something to do with one of a myriad of libraries.)

Thanks for any thoughts and suggestions :),
Shaun


socon009 May 16, 2012 11:22

Hey again,

Thanks for the response, Tobi. :D

I am installing OpenFOAM-2.1.x (Repository Release)

Currently, it is OpenFOAM-2.1.0

Seeing as it's not 1.7.0, I created a new thread to post to.

In reference to installing the necessary packages:

I have installed all of the packages that were brought to my attention. However, there may be some that were not mentioned since the closest Linux distro to RHEL 6.2 on the OpenFOAM installation tutorial is Fedora 15/16.

(I mention these in more detail in the new thread.)

Therefore, I simply installed as if my distro was Fedora 15 or 16 and downloaded the packages described as necessary for installation on these distros.

Finally I uploaded a portion of the error log in the new thread. Check it out if you'd like. Feel free to let me know any ways I can clean it up to make it easier to read/debug.

Thanks again,
Shaun

wyldckat May 16, 2012 16:01

Hi Shaun and Tobi,

@Tobi:
Quote:

Originally Posted by Tobi (Post 361349)
well you can use the following command:
Code:

./Allwmake 1> make.log 2> error.log
Its the same like befor but the stderr (2) is send to the error.log file. After the compiling you should get just the errors in the error.log.

I always state the usage of "2>&1" for a very simple reason: I don't like having to guess when and where the error occurs :mad: :rolleyes:

And it's really simple when we have a single log file: search for the first or second occurrence of "Error " and voilą, reason for the problem found.

@Shaun: I'm heading now to the other thread: http://www.cfd-online.com/Forums/ope...hel-6-2-a.html

Best regards,
Bruno

Chris_DSMC June 18, 2014 07:34

Hello there,

first of all my name is Christopher and I stumbled across this thread while searching for an answer for the parallel compilation of OpenFoam.

I am using OpenFoam 2.3.0 on Ubuntu 14.04. I followed the description of the official support of using the wmakeScheduler but compilation only works on one processor.

Also the hints in this thread of setting up the hostname does not fix the problem.

Do you have any up-to-date hint for me? Is there any known bug?

Best Regards
Chris

wyldckat June 21, 2014 16:07

Greetings Chris and welcome to the forum!

Using wmakeScheduler is only useful when using more than one machine in parallel.
Clearer instructions for installing OpenFOAM 2.3.0 on Ubuntu 14.04 are given here: http://openfoamwiki.net/index.php/In...u#Ubuntu_14.04 - there you've find how to build with multiple cores with a single machine.

Best regards,
Bruno

Chris_DSMC June 24, 2014 05:06

Hi wyldckat,

this perfectly answered my question.

Thank you very much. Nice Forum!

Chris


All times are GMT -4. The time now is 06:55.