CFD Online Discussion Forums

CFD Online Discussion Forums (https://www.cfd-online.com/Forums/)
-   OpenFOAM Installation (https://www.cfd-online.com/Forums/openfoam-installation/)
-   -   Installation OpenFOAM-1.7.x with wmakeScheduler (https://www.cfd-online.com/Forums/openfoam-installation/81344-installation-openfoam-1-7-x-wmakescheduler.html)

Tobi October 24, 2010 16:13

Installation OpenFOAM-1.7.x with wmakeScheduler
 
Hey I am Tobi and a new community member,

first of all. Hello everybody.


I am working with Ubuntu 10.04 an had installed OpenFoam 1.7.1 with the Ubuntu/Debian Pack without problems.

So a good CFD - eng said, that the only version is up to date is the git responsitory.

I had downloaded everything and did all steps in the introductions.
Got so much mistakes and issues with compiling OpenFoam, but hope i have fixed it now. The only thing i can 't fix until now is using wmakeScheduler for compiling OF with my two cores :/

-> export WM_SCHEDULER=wmakeScheduler
-> export WM_HOSTS="shorty:2"
-> export WM_NCOMPPROCS=$($WM_SCHEDULER -count)

after typing echo $WM_NCOMPPROCS I got --> 2

After starting ./Allwmake it will stop at the beginning of line

"SOURCE=..."
"SOURCE=..."

what did i wrong? Sorry but i am new on Ubuntu - Distributions and so on.

Hope someone can help me. Couse i need the new Solver added in the new update.

Thanks
Tobi

wyldckat October 24, 2010 16:28

Greetings Tobi and welcome to the forum!

Mmm, they should really put up a warning that for single machines, the wmakeScheduler isn't necessary. You can simply just do:
Code:

export WM_NCOMPPROCS=2
Or equal to the number of cores that your machine has. The other two variables aren't necessary.

edit: Wait... but WM_NCOMPPROCS did get set properly, since you said that 'export WM_HOSTS="shorty:2"' !! There was nothing wrong there, as far as I can tell!

Additionally, there are a few packages that you'll need to install to ensure that OpenFOAM will build: Building OpenFOAM1.7.0 from source post #4 and onward
In that thread you'll find the packages needed for building OpenFOAM and the code documentation (a few posts down).

Finally, to keep a log of the building process, run Allwmake like this:
Code:

./Allwmake > make.log 2>&1
You might need it after it's done, since something could go wrong :(

Best regards and good luck!
Bruno

Tobi October 24, 2010 17:47

Hi Bruno,

I have read your thread that you have linked above a hour befor and added all the packages. Think that was my real problem to compile it without issues.


Just, i compile with 1 core :/ although i set WM_NCOMPPROS=2
Don 't know why it don' t work.

Will install OF on my new computer in three day's (6 cores)
I will test it with your Code

Code:

./Allwmake > make.log 2>&1
Thx for answering :)
Couse i need the chtMultiRegionLiquidFoam solver :)


Tobi

wyldckat October 24, 2010 18:13

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 280558)
Just, i compile with 1 core :/ although i set WM_NCOMPPROS=2
Don 't know why it don' t work.

Ooooo, so close... but if you really used that name, it's no wonder it didn't work :( It's:
Code:

export WM_NCOMPPROCS=2
You missed a C before the last S :(

If you want a more self-adapting code for setting that variable, you can use this:
Quote:

Originally Posted by $WM_THIRD_PARTY_DIR/tools/makeThirdPartyFunctions
Code:

    if [ -r /proc/cpuinfo ]
    then
        WM_NCOMPPROCS=$(egrep "^processor" /proc/cpuinfo | wc -l)
        [ $WM_NCOMPPROCS -le 8 ] || WM_NCOMPPROCS=8
    else
        WM_NCOMPPROCS=1
    fi

    echo "Building on $WM_NCOMPPROCS cores"


Add it to OpenFOAM's bashrc file, or to your own "~/.bashrc" file :)

Best regards,
Bruno

Tobi October 24, 2010 18:19

Quote:

Originally Posted by wyldckat (Post 280559)

Ooooo, so close... but if you really used that name, it's no wonder it didn't work :( It's:
Code:

export WM_NCOMPPROCS=2
You missed a C before the last S :(

:) no I got the right var. Why it don 't work - don 't know.
Will try your last post and set it into my .bashrc.

Thx. Will make an answer how it work.

Tobi October 25, 2010 03:44

Okay :)

OpenFoam-1.7.x (git) is now working (jehaaa) (:

if i install it on my new machine i will try to set the "WM_HOSTS" variable to

Code:

export WM_HOSTS="localhost:6"

Maybe this will work.

Greetings Tobi

Tobi October 25, 2010 06:41

Hey again,


included your code to my ~/.bashrc

Code:

shorty@shorty:~/OpenFOAM/OpenFOAM-1.7.x$ . $HOME/.bashrc
Building on 2 cores
shorty@shorty:~/OpenFOAM/OpenFOAM-1.7.x$ ./Allwmake

But he worked just with 1 core. Don 't understand why.
Got Ubuntu 10.04. A CFD-eng. said today, that the compiling with more cores doen 't work. He has linux and there it works but on his ubuntu don 't work the Scheduler.

I tested everything. If i build on 2 cores he is working until that code end

Code:

Done ThirdParty Allwmake
========================================

+ wmakePrintBuild -check
same version as previous build
+ wmakeLnInclude OpenFOAM
+ wmakeLnInclude OSspecific/POSIX
+ Pstream/Allwmake
+ wmake libso dummy
'/home/shorty/OpenFOAM/OpenFOAM-1.7.x/lib/linux64GccDPOpt/dummy/libPstream.so' is up to date.
+ set +x

Note: ignore spurious warnings about missing mpicxx.h headers
+ WM_OPTIONS=linux64GccDPOptOPENMPI
+ wmake libso mpi
'/home/shorty/OpenFOAM/OpenFOAM-1.7.x/lib/linux64GccDPOpt/openmpi-1.4.1/libPstream.so' is up to date.
+ wmake libo OSspecific/POSIX
'/home/shorty/OpenFOAM/OpenFOAM-1.7.x/lib/linux64GccDPOpt/libOSspecific.o' is up to date.
+ wmake libso OpenFOAM
SOURCE=meshes/primitiveMesh/primitiveMesh.C ; wmakeScheduler g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/shorty/OpenFOAM/ThirdParty-1.7.x/zlib-1.2.3 -IlnInclude -I. -I/home/shorty/OpenFOAM/OpenFOAM-1.7.x/src/OpenFOAM/lnInclude -I/home/shorty/OpenFOAM/OpenFOAM-1.7.x/src/OSspecific/POSIX/lnInclude  -fPIC -c $SOURCE -o Make/linux64GccDPOpt/primitiveMesh.o

Hmm :/

Any ideas ?

Can make a log if this will help.
Tobi

wyldckat October 25, 2010 07:08

Hi Tobi,

Only when building libscotch and OpenMPI, it will only use a single core (the people at OpenCFD forgot about it :( ).

What does this output:
Code:

egrep "^processor" /proc/cpuinfo | wc -l
Additionally, are you building in a real or virtual machine?

-------------------- EDIT:
I think I figured it out - run this:
Code:

echo $HOST
If nothing outputs, then that is the problem! wmakeScheduler assumes that this variable exists, but apparently it depends on the Linux distribution used.
Try running this before you run Allwmake:
Code:

export HOST=$HOSTNAME
And use:
Code:

export WM_HOSTS="$HOSTNAME:6"
Instead of localhost!
Otherwise, you will have to make your ssh sessions passwordless. For example: How to connect with ssh without using a password

EDIT2: Ironically, this has already been fixed a few days ago in OpenFOAM 1.7.x.

Best regards,
Bruno

Tobi October 26, 2010 04:10

Hi Bruno,

first of all. Thx for your help.
I 'll got a new mainboard etc. so I have to wait until all is received.

And then I install everything new.

Question: Ubuntu 10.04 or CAE Linux Ubuntu 10.04?


After the installation i will check out your commands and post the output here.

Your question, if i build in a real or virtual machine, i think a real :( but I am not sure what you mean.

Build on my personal computer so i think its real.

Hope my english skills aren 't as bad as i think :(

Best reguards.
Tobi


PS: Think i will start compiling at saturdays.

wyldckat October 26, 2010 17:53

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 280742)
first of all. Thx for your help.

You're welcome and I'm glad I could be of help to you! Although this is almost as far as I can help when it comes to OpenFOAM itself, because I still don't have much experience with CFD and OpenFOAM+CFD :D

Quote:

Originally Posted by Tobi (Post 280742)
Question: Ubuntu 10.04 or CAE Linux Ubuntu 10.04?

Personally I've never used CAE Linux, but it can save you several hours in trying to install all of those applications that already comes with it! But if you only need one or two of them, maybe you're better off using the standard Ubuntu 10.04. But it depends very much on what tools you will need!

Quote:

Originally Posted by Tobi (Post 280742)
Your question, if i build in a real or virtual machine, i think a real :( but I am not sure what you mean.

If you don't know, then it's the real machine ;) Here is an example of a virtual machine: Install Windows XP inside ubuntu using Virtualbox

Quote:

Originally Posted by Tobi (Post 280742)
Hope my english skills aren 't as bad as i think :(

Mmm, perhaps you were still sleepy? ;)

Quote:

Originally Posted by Tobi (Post 280742)
PS: Think i will start compiling at saturdays.

With a bit of practice, with 6 cores you can let it compile while you lunch! Or while you have a snack :D Or go for a jog. My AMD 1055T built yesterday OpenFOAM 1.5-dev in about 36 minutes. So with 1.7.x should do it in roughly 40min.

Best regards,
Bruno

Tobi October 27, 2010 04:04

Quote:

With a bit of practice, with 6 cores you can let it compile while you lunch! Or while you have a snack :D Or go for a jog. My AMD 1055T built yesterday OpenFOAM 1.5-dev in about 36 minutes. So with 1.7.x should do it in roughly 40min.

hihi, but only if you can build on 2 - 3 - 4 - 6 cores etc.
You build OF - 1.5 dev on Linux or Ubuntu?

A friend told me that if you use Linux, its very easy to build with more cores couse you just have to type the WM_HOSTS compared with ubuntu.

I hope it will work with more cores in a few days ;)


Tobi

wyldckat October 27, 2010 07:00

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 280936)
hihi, but only if you can build on 2 - 3 - 4 - 6 cores etc.
You build OF - 1.5 dev on Linux or Ubuntu?

A friend told me that if you use Linux, its very easy to build with more cores couse you just have to type the WM_HOSTS compared with ubuntu.

I hope it will work with more cores in a few days ;)

Ubuntu is a Linux distribution! What happened is like I wrote in another post: OpenFOAM's 1.7.1 wmakeScheduler has a bug which is already fixed in 1.7.x. And if you only do:
Code:

export WM_NCOMPPROCS=6
it will build with multiple cores!

Best regards,
Bruno

Tobi October 28, 2010 10:30

Hey Bruno,

i know that Ubuntu is a distribution of Linux (:

So far. I will get the latest OpenFoam Version with git and hope that you are right (but i know that you are right (: )

But one problem still exists.
I am still waiting for my new mainbaord and cpu :/
But tomorrow - yea tomorrow it will be mine ;)


Thx a lot.
If any errors occur i ll post it.

Tobi

Tobi October 28, 2010 10:42

Wrong entry :(

Sorry for that post.

Tobi October 29, 2010 20:55

Hi Bruno,

Quote:

Originally Posted by wyldckat (Post 280620)
Hi Tobi,

What does this output:
Code:

egrep "^processor" /proc/cpuinfo | wc -l

Output: 6

Quote:

run this:
Code:

echo $HOST

without "export HOST="..." " there is no output like you said.

After setting HOST the variable exists and the output is: "shorty" (my computer name) but you said, that it is fixed now. I ll see.

I will try it now and if it doesn 't work i ll try it with SSH



Bye Tobi

Tobi October 29, 2010 21:12

Hey Bruno,

its me Tobi. It 's working with all 6 cores. Nice - very nice.

So last question. Is a process that is in background faster than a process which is shown on the terminal?

Tobi

wyldckat October 30, 2010 07:06

Hi Tobi,
Quote:

Originally Posted by Tobi (Post 281442)
its me Tobi. It 's working with all 6 cores. Nice - very nice.

I'm glad it's finally working :)

Quote:

Originally Posted by Tobi (Post 281442)
So last question. Is a process that is in background faster than a process which is shown on the terminal?

I've never timed that in Linux :( I know that in Windows 95 in a 486, it was better to output to a file or simply hide the command window. Nowadays, I'm not certain.
In theory, outputting to the screen will require additional processing, so it will depend if it's the graphics card job to fully interpret what needs to be shown, therefore done in parallel, without holding back the CPU. If that's done properly, it should not weight very much. Otherwise, you're better off outputting to a file, since it will only require CPU+memory operations, some of which already necessary, and then the occasional flush to the hard drive.

I know that psychologically, I always feel that outputting to a file is faster :D

Best regards,
Bruno

socon009 May 15, 2012 17:25

Quote:

Originally Posted by wyldckat (Post 280552)
Greetings Tobi and welcome to the forum!

Mmm, they should really put up a warning that for single machines, the wmakeScheduler isn't necessary. You can simply just do:
Code:

export WM_NCOMPPROCS=2
Or equal to the number of cores that your machine has. The other two variables aren't necessary.

edit: Wait... but WM_NCOMPPROCS did get set properly, since you said that 'export WM_HOSTS="shorty:2"' !! There was nothing wrong there, as far as I can tell!

Additionally, there are a few packages that you'll need to install to ensure that OpenFOAM will build: Building OpenFOAM1.7.0 from source post #4 and onward
In that thread you'll find the packages needed for building OpenFOAM and the code documentation (a few posts down).

Finally, to keep a log of the building process, run Allwmake like this:
Code:

./Allwmake > make.log 2>&1
You might need it after it's done, since something could go wrong :(

Best regards and good luck!
Bruno

Hi,

I have followed numerous threads here on this topic.

Bruno, thank you for all of your input - it's been very helpful for me.

Currently, I am installing in RHEL 6.2 on multiple cores using the following command:

Code:

./Allwmake > make.log 2>&1
It has suppressed the output that I usually see while installing OpenFOAM.

Could you please offer an explanation of the command above?

In particular,

Code:

> make.log 2>&1
How does bash interpret this?

Lastly, where is the log file written to?

Thank you,
Shaun

wyldckat May 15, 2012 17:33

Greetings Shaun and welcome to the forum!

The "make.log" file is placed in the same folder where you ran Allwmake.
  • The first ">" redirects the output from Allwmake to "make.log"
  • The "2>&1" specifies that the stderr (2) output is to be redirected to stdout (1). In other words, both the errors and normal output are sent to the same file. If this isn't used, only the normal output is sent to the log file, while the errors would be shown on screen.
Best regards,
Bruno

socon009 May 15, 2012 17:59

Ok thanks! Great explanation.

Unfortunately, the log file created indicates that the installation process flopped. :(

I would like to figure out what is going on, but I am not sure how I should go about seeking help. :confused:

- Should I start another thread so as not to hi-jack this one?
- How can I use this log file to figure out what went wrong?

(All I can tell right now is that there are a heck of a lot of "undefined references" and nearly every one has something to do with one of a myriad of libraries.)

Thanks for any thoughts and suggestions :),
Shaun


All times are GMT -4. The time now is 17:26.