Python/Foam error with import_array

adrian.delser · November 22, 2021, 09:21

Hi everyone,

I'm experimenting with passing arrays between openfoam9 and python 3.8 using the numpy C api (inside a custom solver called myPisoFoam). The application compiles fine with wmake, but when run, FOAM exits due to a floating point exception and almost 70 lines of errors which I do not understand. I have narrowed down the issue to the use of "import_array()" to initialise the numpy array API.

**EDIT**

I can now see that upon compilation (only if import_array(), or its component function PyImport_ImportModule("numpy.core._multiarray_umat h") is called, wmake will compile successfully but with the warnings :

Code:

could not open file ndarrayobject.h for source file myPisoFoam.C due to No such file or directory
could not open file npy_interrupt.h for source file myPisoFoam.C due to No such file or directory
could not open file noprefix.h for source file myPisoFoam.C due to No such file or directory

Clearly it is struggling to find the required numpy libraries, so perhaps I am not linking to them properly inside Make/options?

-----------------------------------------------------------------------------------------------------------------------

The C code (on top of the normal solver code) looks like this:

Code:

/* Initialize Python and add current directory to path*/
Py_Initialize ();
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append (".")");

/* initialize numpy array library */
import_array1(-1);


/* -----More code after this but foam has already exited-----*/

-----------------------------------------------------------------------------------------------------------------------

The following is additionally included at the top of the main solver:

Code:

#include "Python.h"
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include "/home/daep/a.del-ser/.local/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h"

-----------------------------------------------------------------------------------------------------------------------

The following python related things are included in Make/options:

Code:

EXE_INC = \
    -I/usr/include/python3.8/ \
    -I/home/daep/a.del-ser/.local/lib/python3.8/site-packages/numpy/core/include


EXE_LIBS = \
     -lpython3.8

-----------------------------------------------------------------------------------------------------------------------

The error stream looks like this (I have no idea what the question marks are indicating all over the place):

Code:

#0  Foam::error::printStack(Foam::Ostream&) at ??:?
#1  Foam::sigFpe::sigHandler(int) at ??:?
#2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
#3  ? at scalarmath.c:?
#4  PyNumber_Multiply in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#5  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#6  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#7  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#8  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#9  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#10  PyEval_EvalCodeEx in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#11  PyEval_EvalCode in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#12  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#13  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#14  PyVectorcall_Call in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#15  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#16  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#17  _PyFunction_Vectorcall in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#18  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#19  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#20  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#21  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#22  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#23  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#24  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#25  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#26  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#27  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#28  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#29  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#30  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#31  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#32  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#33  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#34  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#35  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#36  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#37  _PyFunction_Vectorcall in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#38  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#39  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#40  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#41  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#42  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#43  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#44  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#45  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#46  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#47  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#48  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#49  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#50  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#51  _PyFunction_Vectorcall in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#52  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#53  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#54  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#55  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#56  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#57  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#58  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#59  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#60  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#61  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#62  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#63  _PyObject_MakeTpCall in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#64  ? in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#65  PyObject_CallFunction in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#66  PyImport_Import in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#67  PyImport_ImportModule in "/lib/x86_64-linux-gnu/libpython3.8.so.1.0"
#68  ? in "/home/daep/a.del-ser/OpenFOAM/a.del-ser-9/platforms/linux64GccDPInt32Opt/bin/myPisoFoam"
#69  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
#70  ? in "/home/daep/a.del-ser/OpenFOAM/a.del-ser-9/platforms/linux64GccDPInt32Opt/bin/myPisoFoam"
Floating point exception (core dumped)

-----------------------------------------------------------------------------------------------------------------------

Additionally, I'd rather run this in a python virtual environment but haven't been able to get the local path working within Make/options. If anyone has any insight there too it would be great.

Thanks!

ukaya · January 12, 2023, 07:12

Have you managed to determine what was causing the issue?

mickbatti · January 29, 2023, 00:55

I also have the same identical error.
Any suggestions so far?

It seems an issue with python and numpy library paths, maybe related to multiple versions installed. But don’t know how to solve it.

s-rod · February 1, 2023, 10:17

Hi Adrian,

I'd assume this is not an issue any more since it's been here for a while, however, given the recent interaction from ukaya and mickbatti, I'd rather to chip in.

1. What you want to do, i. e. using the NumPy C API, has been covered by the pythonFoam project. (https://arxiv.org/abs/2103.09389)

2. If you do not mind using NumPy directly (instead of its C API) in OF, I suggest you have a look at the pythonPal4foam project. (https://journal.openfoam.com/index.p...rticle/view/79).

2.1. Regarding your question about how to run it on a Python virtual environment, note that the additional pythonPal4foam resources include video tutorials on how to do it. Check it out: https://www.youtube.com/watch?v=AU3X...Aar10uR1KE7YtG

3. For the sake of completeness, I think you can also run NumPy using swak4foam, however, I am not familiar with this so I can't tell you exactly how.

Hope it helps,

Simon

January 12, 2023, 07:12		#2
ukaya New Member Umut Kaya Join Date: Aug 2020 Posts: 5 Rep Power: 5	Have you managed to determine what was causing the issue?

January 29, 2023, 00:55		#3
mickbatti New Member Join Date: Jul 2012 Posts: 10 Rep Power: 13	I also have the same identical error. Any suggestions so far? It seems an issue with python and numpy library paths, maybe related to multiple versions installed. But don’t know how to solve it.

February 1, 2023, 10:17		#4
s-rod New Member Simon Rodriguez Join Date: Nov 2014 Posts: 13 Rep Power: 11	Hi Adrian, I'd assume this is not an issue any more since it's been here for a while, however, given the recent interaction from ukaya and mickbatti, I'd rather to chip in. 1. What you want to do, i. e. using the NumPy C API, has been covered by the pythonFoam project. (https://arxiv.org/abs/2103.09389) 2. If you do not mind using NumPy directly (instead of its C API) in OF, I suggest you have a look at the pythonPal4foam project. (https://journal.openfoam.com/index.p...rticle/view/79). 2.1. Regarding your question about how to run it on a Python virtual environment, note that the additional pythonPal4foam resources include video tutorials on how to do it. Check it out: https://www.youtube.com/watch?v=AU3X...Aar10uR1KE7YtG 3. For the sake of completeness, I think you can also run NumPy using swak4foam, however, I am not familiar with this so I can't tell you exactly how. Hope it helps, Simon