[Other] MPI won't work anymore after installing OpeFOAM.com alongside OpenFOAM.org under WSL

NablaDyn · May 10, 2021, 11:17

Dear FOAMers,
I could not find a similar problem in previous posts so I hope you can help me.

I have been using mainly OpenFOAM.com (ESI) distributions in the past until I recently switched to OpenFOAM.org (OpenFOAM Foundation) for select purposes. Unfortunately, after again installing the latest .com version (v2012), I cannot get to run my .org versions (7, 8 and dev) in parallel again. Everytime I try, I get an error (after regularly sourcing the 7/8/dev bashrc) when I call OpenFOAM in parallel:

Code:

[NB-0485:29833] *** Process received signal ***
[NB-0485:29833] Signal: Segmentation fault (11)
[NB-0485:29833] Signal code: Address not mapped (1)
[NB-0485:29833] Failing at address: (nil)
[NB-0485:29833] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7f6d82446210]
[NB-0485:29833] *** End of error message ***
[1]+  Segmentation fault      (core dumped) mpirun -np 5 ~/OpenFOAM/lichtmes-dev/platforms/linux64GccDPInt32Opt/bin/myReactingParticleFoam -parallel > log

Please note that my application worked perfectly before and the error pops up for regular OpenFOAM applications (7/8/dev) as well.

I hope someone can give me a hint on how to fix this. I tried multiple reinstalls (incl. --purge) of all versions and different sequences of it but the error won't go away. I am stuck...

Bests,

NablaDyn · May 10, 2021, 11:55

ok, seems I managed to get it running again although I don't have any clue what exactly did the trick. In summary, I uninstalled all my .org versions and, respectively, deleted all .com installation directories completely. Afterwards, I only reinstalled the .org versions and restarted WSL. Now I don't get the error anymore but I am reluctant to reinstall again any .com version.
So I guess it would still be nice to get an idea about what went wrong before and how to avoid such behaviour in the future...

olesen · May 11, 2021, 02:56

Quote:

Originally Posted by NablaDyn

ok, seems I managed to get it running again although I don't have any clue what exactly did the trick. In summary, I uninstalled all my .org versions and, respectively, deleted all .com installation directories completely. Afterwards, I only reinstalled the .org versions and restarted WSL. Now I don't get the error anymore but I am reluctant to reinstall again any .com version.
So I guess it would still be nice to get an idea about what went wrong before and how to avoid such behaviour in the future...

How have you been switching environments or setting things up? It sounds like you have some old LD_LIBRARY_PATH cruft. Not sure what the openfoam.org version uses for openmpi, but the openfoam.com version will just use the standard one from the system (ie, no additional thirdparty). If you've used the regular Ubuntu installer, there is no reason that they cannot co-exist. Can consider the openfoam-selector for making the switching easier, or the shell session.

https://develop.openfoam.com/Develop...enfoam-package

NablaDyn · May 23, 2021, 00:38

Quote:

Originally Posted by olesen

How have you been switching environments or setting things up? It sounds like you have some old LD_LIBRARY_PATH cruft. Not sure what the openfoam.org version uses for openmpi, but the openfoam.com version will just use the standard one from the system (ie, no additional thirdparty). If you've used the regular Ubuntu installer, there is no reason that they cannot co-exist. Can consider the openfoam-selector for making the switching easier, or the shell session.

https://develop.openfoam.com/Develop...enfoam-package

Hi Mark,

thanks for your hint, a library path issue sounds reasonable to me although I haven't checked it yet.

I used the "standard" WSL version for .com, meaning I just got the archive and unpacked it as suggested in the sense of the developer-recommended default setup. For the org. versions I followed their (openfoam.org) respective default setup for WSL accordingly, which is installation of the Ubuntu pack via apt.

Yann · May 24, 2021, 03:50

Hi NablaDyn,

I have OpenFOAM-8 and OpenFOAM-v2012 installed alongside on Ubuntu 20.04 in WSL on my laptop. I don't use it extensively but I can run tutorials in parallel from both versions without any trouble with MPI. I just tried now to switch from one version to another and I could not reproduce your issue.

I know it's not really helping to solve your problem but I thought it might be worth sharing, at least to let you know it can work.

Yann

NablaDyn · May 25, 2021, 07:13

Quote:

Originally Posted by Yann

Hi NablaDyn,

I have OpenFOAM-8 and OpenFOAM-v2012 installed alongside on Ubuntu 20.04 in WSL on my laptop. I don't use it extensively but I can run tutorials in parallel from both versions without any trouble with MPI. I just tried now to switch from one version to another and I could not reproduce your issue.

I know it's not really helping to solve your problem but I thought it might be worth sharing, at least to let you know it can work.

Yann

Hi Yann,
thank you for your descriptions. In fact, I still haven't checked a possible library problem as suggested by Mark, however, I reinstalled the .com version after the .org ones (8 and dev) with both .org versions up and running flawlessly before. And again, after sourcing the bashrc of the .com version just once, I can no longer run parallel .org computations. But I noticed that when I source e.g. the OpenFOAM 8 bashrc file, I get the following error which I currently don't know how to resolve:

Code:

Cannot open configuration file /home/pawan/OpenFOAM/ThirdParty-v2006/platforms/linux64Gcc63/openmpi-4.0.3/share/openmpi/mpicc-wrapper-data.txt
Error parsing data file mpicc: Not found

Maybe someone already has a quick hint on where to look for conflicts (libraries etc.) before I dig into this and let it possibly take days to find out what's wrong.

Regards and thanks for your comments...

Yann · May 25, 2021, 09:47

Hello NablaDyn,

After reading your message I reproduced these steps :

source OpenFOAM-v2012
run a parallel tutorial => Working fine
source OpenFOAM-8
run a parallel tutorial => Crash (Signal: Segmentation fault (11))
run the same tutorial in serial => Working fine

If I source OpenFOAM-8 for a second time I get the exact same error as you do:

Code:

Cannot open configuration file /home/pawan/OpenFOAM/ThirdParty-v2006/platforms/linux64Gcc63/openmpi-4.0.3/share/openmpi/mpicc-wrapper-data.txt
Error parsing data file mpicc: Not found

It felt a bit odd to get a path pointing to ThirdParty-v2006 as I never installed OpenFOAM-v2006 on my system. After checking the installation directory, it seems to be related to the compilation of the ThirdParty package located in "/opt/OpenFOAM/OpenFOAM-v2012/ThirdParty". In previous versions, ThirdParties used to be located in a specific directory (/opt/OpenFOAM/ThirdParty-v2012/) and it's still how it's done with OpenFOAM-V2012 when you compile the code by yourself.

As Mark suggested, something seems to be wrong with the environment variables, as there is no reason to have a path leading to ThirdParty-v2006 in OpenFOAM-8.

Here is what I get in LD_LIBRARY_PATH:

Code:

/opt/ThirdParty-8/platforms/linux64Gcc/gperftools-svn/lib:
/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/paraview-5.6:
/opt/paraviewopenfoam56/lib:/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/openmpi-system:
/opt/ThirdParty-8/platforms/linux64GccDPInt32/lib/openmpi-system:
/home/yann/OpenFOAM/yann-8/platforms/linux64GccDPInt32Opt/lib:
/opt/site/8/platforms/linux64GccDPInt32Opt/lib:
/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib:
/opt/ThirdParty-8/platforms/linux64GccDPInt32/lib:
/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/dummy:
/opt/OpenFOAM/OpenFOAM-v2012/ThirdParty/platforms/linux64Gcc63/openmpi-4.0.3/lib64:
 /usr/lib/x86_64-linux-gnu/openmpi/lib

I guess the line referring to OpenFOAM-v2012 should not be there.

Workaround:

Since this is all related to environment variables, here is what you can do:
Edit your user bashrc (~/.bashrc) to source OpenFOAM-8:

Code:

source /opt/openfoam8/etc/bashrc

When starting a new session, your default OpenFOAM version will be OpenFOAM-8 and it will run properly in parallel.
If you need to use OpenFOAM-v2012, source it manually in your terminal.
If you need to get back to OpenFOAM-8, just start a new session and it will be fine.
(Actually, this is my setup on WSL and this is why I never had this issue until now)

This is just a workaround, but at least it should allow you to work with both versions.

I'm not familiar enough with OpenFOAM environment variables to fix it but I'm pretty sure Mark or somebody else will have a better solution for this.

Cheers,
Yann

NablaDyn · May 31, 2021, 10:11

Quote:

Originally Posted by Yann

Hello NablaDyn,

After reading your message I reproduced these steps :

source OpenFOAM-v2012
run a parallel tutorial => Working fine
source OpenFOAM-8
run a parallel tutorial => Crash (Signal: Segmentation fault (11))
run the same tutorial in serial => Working fine

If I source OpenFOAM-8 for a second time I get the exact same error as you do:

Code:

Cannot open configuration file /home/pawan/OpenFOAM/ThirdParty-v2006/platforms/linux64Gcc63/openmpi-4.0.3/share/openmpi/mpicc-wrapper-data.txt
Error parsing data file mpicc: Not found

It felt a bit odd to get a path pointing to ThirdParty-v2006 as I never installed OpenFOAM-v2006 on my system. After checking the installation directory, it seems to be related to the compilation of the ThirdParty package located in "/opt/OpenFOAM/OpenFOAM-v2012/ThirdParty". In previous versions, ThirdParties used to be located in a specific directory (/opt/OpenFOAM/ThirdParty-v2012/) and it's still how it's done with OpenFOAM-V2012 when you compile the code by yourself.

As Mark suggested, something seems to be wrong with the environment variables, as there is no reason to have a path leading to ThirdParty-v2006 in OpenFOAM-8.

Here is what I get in LD_LIBRARY_PATH:

Code:

/opt/ThirdParty-8/platforms/linux64Gcc/gperftools-svn/lib:
/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/paraview-5.6:
/opt/paraviewopenfoam56/lib:/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/openmpi-system:
/opt/ThirdParty-8/platforms/linux64GccDPInt32/lib/openmpi-system:
/home/yann/OpenFOAM/yann-8/platforms/linux64GccDPInt32Opt/lib:
/opt/site/8/platforms/linux64GccDPInt32Opt/lib:
/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib:
/opt/ThirdParty-8/platforms/linux64GccDPInt32/lib:
/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/dummy:
/opt/OpenFOAM/OpenFOAM-v2012/ThirdParty/platforms/linux64Gcc63/openmpi-4.0.3/lib64:
 /usr/lib/x86_64-linux-gnu/openmpi/lib

I guess the line referring to OpenFOAM-v2012 should not be there.

Workaround:

Since this is all related to environment variables, here is what you can do:
Edit your user bashrc (~/.bashrc) to source OpenFOAM-8:

Code:

source /opt/openfoam8/etc/bashrc

When starting a new session, your default OpenFOAM version will be OpenFOAM-8 and it will run properly in parallel.
If you need to use OpenFOAM-v2012, source it manually in your terminal.
If you need to get back to OpenFOAM-8, just start a new session and it will be fine.
(Actually, this is my setup on WSL and this is why I never had this issue until now)

This is just a workaround, but at least it should allow you to work with both versions.

I'm not familiar enough with OpenFOAM environment variables to fix it but I'm pretty sure Mark or somebody else will have a better solution for this.

Cheers,
Yann

Hi Yann,
firstly, sorry for the delay, I couldn't find the time to answer last week. Secondly, thank you very much for looking into this and your thorough descriptions. I will happily use your workaround for now. Come v2106, I will simply compile everything from source and see if that fixes it completely - if not already done so by the devs.

all the best,
Martin

isulovsky · March 30, 2022, 08:46

Hi NablaDyn,

Have You resolved this issue? I am also stuck configuring OFv2012 and OF9 in WSL and I get the exact same error while running cases in parallel (/home/pawan....wrapper-data.txt). Workaround suggested from Yann doesn't work for me so I am interested if You have succeded?

Kind regards,

Ivan

NablaDyn · March 31, 2022, 00:17

Hi Ivan,
In fact, this problem did not bother me anymore with the release of v2106. I have a habbit of always using the latest version, so now I am already using v2112. However, I also did not use v9 recently so I cannot say for sure whether this problem persists for the newer versions.
Martin

dasith0001 · February 23, 2023, 01:13

Hi,

Not sure why it is happening but found out an easy way to run both v2012 and of10.

I am running of on widows with WSL2 using Ubuntu 22.04.02.

the fix is, whenever you want to switch to of10 from v2012, close the ubuntu app and reopen then use the alias to get to of10.

you dont have to close the app when you shift from of10 to v2012.

it's a pain, please let me know if anyone finds a better solution

On the other hand, v2206 and OF10 works perfectly fine

Dasith

May 10, 2021, 11:17	MPI won't work anymore after installing OpeFOAM.com alongside OpenFOAM.org under WSL	#1
NablaDyn Senior Member Join Date: Oct 2015 Location: Germany Posts: 100 Rep Power: 10	Dear FOAMers, I could not find a similar problem in previous posts so I hope you can help me. I have been using mainly OpenFOAM.com (ESI) distributions in the past until I recently switched to OpenFOAM.org (OpenFOAM Foundation) for select purposes. Unfortunately, after again installing the latest .com version (v2012), I cannot get to run my .org versions (7, 8 and dev) in parallel again. Everytime I try, I get an error (after regularly sourcing the 7/8/dev bashrc) when I call OpenFOAM in parallel: Code: [NB-0485:29833] * Process received signal * [NB-0485:29833] Signal: Segmentation fault (11) [NB-0485:29833] Signal code: Address not mapped (1) [NB-0485:29833] Failing at address: (nil) [NB-0485:29833] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7f6d82446210] [NB-0485:29833] * End of error message * [1]+ Segmentation fault (core dumped) mpirun -np 5 ~/OpenFOAM/lichtmes-dev/platforms/linux64GccDPInt32Opt/bin/myReactingParticleFoam -parallel > log Please note that my application worked perfectly before and the error pops up for regular OpenFOAM applications (7/8/dev) as well. I hope someone can give me a hint on how to fix this. I tried multiple reinstalls (incl. --purge) of all versions and different sequences of it but the error won't go away. I am stuck... Bests,

May 25, 2021, 09:47		#7
Yann Senior Member Yann Join Date: Apr 2012 Location: France Posts: 1,076 Rep Power: 26	Hello NablaDyn, After reading your message I reproduced these steps : source OpenFOAM-v2012 run a parallel tutorial => Working fine source OpenFOAM-8 run a parallel tutorial => Crash (Signal: Segmentation fault (11)) run the same tutorial in serial => Working fine If I source OpenFOAM-8 for a second time I get the exact same error as you do: Code: Cannot open configuration file /home/pawan/OpenFOAM/ThirdParty-v2006/platforms/linux64Gcc63/openmpi-4.0.3/share/openmpi/mpicc-wrapper-data.txt Error parsing data file mpicc: Not found It felt a bit odd to get a path pointing to ThirdParty-v2006 as I never installed OpenFOAM-v2006 on my system. After checking the installation directory, it seems to be related to the compilation of the ThirdParty package located in "/opt/OpenFOAM/OpenFOAM-v2012/ThirdParty". In previous versions, ThirdParties used to be located in a specific directory (/opt/OpenFOAM/ThirdParty-v2012/) and it's still how it's done with OpenFOAM-V2012 when you compile the code by yourself. As Mark suggested, something seems to be wrong with the environment variables, as there is no reason to have a path leading to ThirdParty-v2006 in OpenFOAM-8. Here is what I get in LD_LIBRARY_PATH: Code: /opt/ThirdParty-8/platforms/linux64Gcc/gperftools-svn/lib: /opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/paraview-5.6: /opt/paraviewopenfoam56/lib:/opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/openmpi-system: /opt/ThirdParty-8/platforms/linux64GccDPInt32/lib/openmpi-system: /home/yann/OpenFOAM/yann-8/platforms/linux64GccDPInt32Opt/lib: /opt/site/8/platforms/linux64GccDPInt32Opt/lib: /opt/openfoam8/platforms/linux64GccDPInt32Opt/lib: /opt/ThirdParty-8/platforms/linux64GccDPInt32/lib: /opt/openfoam8/platforms/linux64GccDPInt32Opt/lib/dummy: /opt/OpenFOAM/OpenFOAM-v2012/ThirdParty/platforms/linux64Gcc63/openmpi-4.0.3/lib64: /usr/lib/x86_64-linux-gnu/openmpi/lib I guess the line referring to OpenFOAM-v2012 should not be there. Workaround: Since this is all related to environment variables, here is what you can do: Edit your user bashrc (~/.bashrc) to source OpenFOAM-8: Code: source /opt/openfoam8/etc/bashrc When starting a new session, your default OpenFOAM version will be OpenFOAM-8 and it will run properly in parallel. If you need to use OpenFOAM-v2012, source it manually in your terminal. If you need to get back to OpenFOAM-8, just start a new session and it will be fine. (Actually, this is my setup on WSL and this is why I never had this issue until now) This is just a workaround, but at least it should allow you to work with both versions. I'm not familiar enough with OpenFOAM environment variables to fix it but I'm pretty sure Mark or somebody else will have a better solution for this. Cheers, Yann NablaDyn likes this.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Error installing SU2 binaries with mpi support on windows	Creg	SU2	3	May 31, 2022 09:57
SU2-7.0.1 on ubuntu 18.04	hyunko	SU2 Installation	7	March 16, 2020 04:37
Sgimpi	pere	OpenFOAM	27	September 24, 2011 07:57
Error using LaunderGibsonRSTM on SGI ALTIX 4700	jaswi	OpenFOAM	2	April 29, 2008 10:54
Is Testsuite on the way or not	lakeat	OpenFOAM Installation	6	April 28, 2008 11:12

May 10, 2021, 11:55		#2
NablaDyn Senior Member Join Date: Oct 2015 Location: Germany Posts: 100 Rep Power: 10	ok, seems I managed to get it running again although I don't have any clue what exactly did the trick. In summary, I uninstalled all my .org versions and, respectively, deleted all .com installation directories completely. Afterwards, I only reinstalled the .org versions and restarted WSL. Now I don't get the error anymore but I am reluctant to reinstall again any .com version. So I guess it would still be nice to get an idea about what went wrong before and how to avoid such behaviour in the future...

May 24, 2021, 03:50		#5
Yann Senior Member Yann Join Date: Apr 2012 Location: France Posts: 1,076 Rep Power: 26	Hi NablaDyn, I have OpenFOAM-8 and OpenFOAM-v2012 installed alongside on Ubuntu 20.04 in WSL on my laptop. I don't use it extensively but I can run tutorials in parallel from both versions without any trouble with MPI. I just tried now to switch from one version to another and I could not reproduce your issue. I know it's not really helping to solve your problem but I thought it might be worth sharing, at least to let you know it can work. Yann

March 30, 2022, 08:46		#9
isulovsky New Member Ivan Join Date: Dec 2021 Posts: 6 Rep Power: 4	Hi NablaDyn, Have You resolved this issue? I am also stuck configuring OFv2012 and OF9 in WSL and I get the exact same error while running cases in parallel (/home/pawan....wrapper-data.txt). Workaround suggested from Yann doesn't work for me so I am interested if You have succeded? Kind regards, Ivan

March 31, 2022, 00:17		#10
NablaDyn Senior Member Join Date: Oct 2015 Location: Germany Posts: 100 Rep Power: 10	Hi Ivan, In fact, this problem did not bother me anymore with the release of v2106. I have a habbit of always using the latest version, so now I am already using v2112. However, I also did not use v9 recently so I cannot say for sure whether this problem persists for the newer versions. Martin

February 23, 2023, 01:13		#11
dasith0001 Senior Member Desh Join Date: Mar 2021 Location: Sydney Posts: 108 Rep Power: 5	Hi, Not sure why it is happening but found out an easy way to run both v2012 and of10. I am running of on widows with WSL2 using Ubuntu 22.04.02. the fix is, whenever you want to switch to of10 from v2012, close the ubuntu app and reopen then use the alias to get to of10. you dont have to close the app when you shift from of10 to v2012. it's a pain, please let me know if anyone finds a better solution On the other hand, v2206 and OF10 works perfectly fine Dasith