[fra76]

Yes, it does.
The mesh composed by:
4182212 cells
8661255 faces
876005 points

Any other suggestions, besides buying more ram?

[gschaider]

Yeah. Make smaller meshes.

(I know: answers like this are extremly non-sensitve because you'll have your reasons to use meshes of that size, but I could not resist).

Another thought (a bit less stupid): you want to calculate on this mesh. So I guess you have at least two machines for a parallel run (and in sum their RAM should be enough for the whole mesh): rip out the RAM from one machine, plug it into the other, do your thing with the mesh (including decompose), put RAM back, calculate. As I said: it's only a BIT less stupid.

The last (and really feasible) suggestion I have: could you give us a more concrete notion how splitMesh fails (error message etc). I guess (because you didn't post an error message) that it ends with a segmentation fault (or something equally non-descriptive). In that case compile OF with the debugging enabled, run splitMesh in gdb and post the backtrace (command bt in gdb). With that information it would be a lot easier to tell what the problem is.

[eugene]

If it is failing due to limited memory, add more swap space to your Opteron and run your utility over night or over the weekend if necessary.

[fra76]

Well,
I know the mesh is big, but I need it (for a comparison with Fluent, too).
The error message is a simple "Segmentation fault", but I'm really sure it depends on a lack of memory.
I'd like to add some memory from a computer to the Athlon64, which can handle more than 2Gb of ram, but, due to the limited number of memory slots, I can reach up to 3Gb.
I'll try to do this and use the swap space...
Thanks a lot
Francesco

[gschaider]

Hello Francesco!

Hope you didn't take my comment about the smaller meshes serious (people who ignore that kind of comment from me tend to live longer and happier lives).

If your extension of memory/swap doesn't work out compile a version of OpenFOAM with debug-symbols enabled. This would work like this (I'm only doing this from memory, so if anyone notices a mistake, please correct):

* copy the whole OpenFOAM-1.2-folder to a folder OpenFOAM-1.2-debug (for instance)
* in the bashrc-file in that folder change some lines:
export WM_PROJECT_VERSION=1.2
to
export WM_PROJECT_VERSION=1.2-debug
and
export WM_COMPILE_OPTION=Opt
# export WM_COMPILE_OPTION=Debug
to
#export WM_COMPILE_OPTION=Opt
export WM_COMPILE_OPTION=Debug

* source that bashrc
* recompile the new OpenFOAM-distribution

Now, instead of calling

splitMesh . theMeshThatFailed

call the debugger:

gdb splitMesh

in the command-line of the debugger say

run . theMeshThatFailed

When the program fails you get back to the command-line of the debugger. There you type bt (short for backtrace). The debugger now tells you at which statement the programm failed and from where this was called. If you copy that backtrace to this place it would make it much easier for everyone involved to tell what might be the problem.

[fra76]

Ok, i'll try to compile OpenFOAM in debug mode as soon as possible.

The problem I have now is compiling splitMeshWithSets with OpenFOAM 1.3.

Here follows the error I get:

--------------------------------------------------
[ splitMeshWithSets]$ wmake
Making dependency list for source file regionSide.C
Making dependency list for source file splitMeshWithSets.C
could not open file morphMesh.H for source file splitMeshWithSets.C
could not open file attachPolyMesh.H for source file splitMeshWithSets.C

SOURCE_DIR=.
SOURCE=regionSide.C ; g++ -m64 -DlinuxAMD64 -DDP -Wall -W -Wno-unused-parameter -Wold-style-cast -march=opteron -O3 -DNoRepository -ftemplate-depth-30 -I/home/francesco/OpenFOAM/OpenFOAM-1.3/src/dynamicMesh/lnInclude -I/home/francesco/OpenFOAM/OpenFOAM-1.3/src/meshTools/lnInclude -I/home/francesco/OpenFOAM/OpenFOAM-1.3/src/OpenFOAM/lnInclude -IlnInclude -I. -fPIC -c $SOURCE -o Make/linuxAMD64Gcc4DPOpt/regionSide.o
SOURCE_DIR=.
SOURCE=splitMeshWithSets.C ; g++ -m64 -DlinuxAMD64 -DDP -Wall -W -Wno-unused-parameter -Wold-style-cast -march=opteron -O3 -DNoRepository -ftemplate-depth-30 -I/home/francesco/OpenFOAM/OpenFOAM-1.3/src/dynamicMesh/lnInclude -I/home/francesco/OpenFOAM/OpenFOAM-1.3/src/meshTools/lnInclude -I/home/francesco/OpenFOAM/OpenFOAM-1.3/src/OpenFOAM/lnInclude -IlnInclude -I. -fPIC -c $SOURCE -o Make/linuxAMD64Gcc4DPOpt/splitMeshWithSets.o
splitMeshWithSets.C:50:23: error: morphMesh.H: No such file or directory
splitMeshWithSets.C:57:28: error: attachPolyMesh.H: No such file or directory
splitMeshWithSets.C: In function 'int main(int, char**)':
splitMeshWithSets.C:163: error: 'attachPolyMesh' was not declared in this scope
splitMeshWithSets.C:163: error: expected `;' before 'pMesh'
splitMeshWithSets.C:173: error: 'pMesh' was not declared in this scope
make: *** [Make/linuxAMD64Gcc4DPOpt/splitMeshWithSets.o] Error 1
--------------------------------------------------

Any idea?

[mattijs]

I think the splitMeshWithSets is a derivative of the splitMesh utility? You could a look there (mesh/manipulation) on what has changed.

[gschaider]

Hi Francesco!
I have done the changes and uploaded a version of splitMeshWithSets that compiles with 1.3 to the Wiki.
Be aware that there seems to be a change in the semantics for empty patches in the boundary file. While in OF 1.2

startFace ;
nFaces 0;

worked, in OF 1.3 startFace has to be set to the face after the last face.

Apart from that everything should work like it did for 1.2.

[fra76]

Thank you very much!
I've downloaded and compiled the new version of splitMeshWithSets with OpenFOAM 1.3!
I can use the single precision version of OF1.3 in order to reduce the memory request, but I still have problems in converting the mesh...
I guess I'll have to buy some more ram!

[fra76]

Well, I've tried to use SplitMeshWithSets on my mesh, in single precision in order to save ram, and it seemed to work.
But, after about 2 hours of computation (on an Athlon64 3500+), it returned this error:

{
type attachDetach;
faceZoneName membraneFaces;
masterPatchName carena-anteriore-prism_down;
slavePatchName carena-anteriore-prism_up;
triggerTimes 1(0);
manualTrigger off;
active on;
}
Foam::error::printStack(Foam:: Ostream&)
Foam::sigSegv::sigSegvHandler(int)
/lib64/libc.so.6 [0x364712f7c0]
Foam::primitiveMesh::calcCellEdges() const
Foam::primitiveMesh::cellEdges() const
Foam::faceZone::meshEdges() const
Foam::attachDetach::detachInterface(Foam::polyTopo Change&) const
Foam::attachDetach::setRefinement(Foam::polyTopoCh ange&) const
Foam::polyTopoChanger::topoChangeRequest() const
Foam::polyTopoChanger::changeMesh()
Foam::attachPolyTopoChanger::attach(bool)
splitMeshWithSets [0x40c28c]
__libc_start_main
__gxx_personality_v0
Segmentation fault


Any suggestion?

[gschaider]

[This is outside of the domain I'm qualified to speak about. Somebody please tell me to shut up]

To me the failing in libc suggests, that there is still a problem with memory.

In calcCellEdges there are two calls which allocate data structures whose size is the number of cells

My suggestion (not very helpful)
- put some Info<<-statements before and after these allocations to find out which one fails
- go with Eugene's suggestion of extending swap space (BTW: what's your current memory size?)

Other (random) thoughts:
- Maybe that valgrind-program everybody is talking about can help to pin down the problem (but I've never used that myself) Especially if the problem is due to a memory hole
- does the same thing happen if the stock splitMesh-utility (the call to Foam::attachPolyTopoChanger::attach happens in both just with a parameter differing)?

And then there's always the possibility of recompiling OF in Debug-mode and running it in the Debugger

[fra76]

Thank you for your suggestions.
I planned to put some output in the program, in order to find out where the problem is.

Concerning the memory/swap size, I have 2Gb of RAM and 8Gb of swap size. The virtual image of the program is about 2.5Gb, so I think the swap size should not be a problem.

I don't know the "valgrind-program". What is it?

I tried with the standard splitMesh utility, but the problem is exactly the same.

I have to recompile OF in debug mode and run the application again. Consider that it takes about 2 hours before giving the error, in optimize mode. I can't imagine how long could it take in debug mode...

I'll try and let you know!

Francesco

[gschaider]

@output in program: I'd try that before recompiling in debug mode (or parallel to recompilng). My bet is on the lines
List<dynamiclist<label,> > ce(nCells());
or
cePtr_ = new labelListList(ce.size());

@valgrind: that is a program that finds memory-leaks in your software by intercepting memory allocation calls. I just checked the docu and the problem for your case seems to be that the program needs to terminate correctly before it can say anything about memory leaks (but you can try valgrind+splitMesh(WithSets) for a smaller case and see whether it complains) But I have only used that program occasionally (so you might ask an expert)

@problem also splitMesh: that's bad, because then the problem is not in my code and then it's difficult for me to fix it. On the other hand: then it's not my fault

[fra76]

Hi Bernhard!
Finally I could compile OF1.3 in single precision debug mode, and I get some new errors!
Yuo say "There is a special version that compiles on Version 1.3. Please note when using this recipie that in OpenFOAM 1.3 empty patches without a starting face are treated differently: now the starting face must be one face after the last face. If it is left empty OpenFOAM assumes that it is zero."
Well, doing so, this is the error:
-------------------------------------------------
Create polyMesh for time = 0



--> FOAM FATAL ERROR : size 8661256 out of range 0 ... 8661255

From function UList<t>::checkSize(const label)
in file /home/francesco/OpenFOAM/OpenFOAM-1.3/src/OpenFOAM/lnInclude/UListI.H at line 88.

FOAM aborting
-------------------------------------------------

I've then tried to leave blank the startFace value (i.e. "startFace ;"), and this is the new error:

-------------------------------------------------
Create polyMesh for time = 0

Read 43634 faces to split



--> FOAM FATAL ERROR : Face 261804 in faceSet carena-anteriore-prism is not an internal face.

From function splitMesh
in file splitMesh.C at line 144.

FOAM exiting
-------------------------------------------------


Both with splitMesh and with splitMeshWithSets.
This seems to be a problem in the mesh.

The face 261804 is a part of this sets:
[francesco@donald sets]$ grep -w '261804' *
carena-anteriore-prism:261804
prism-cells-26:261804

It's the very first face in the set, and I really don't know how to check the mesh. I've checked that face, and it seems to be internal, looking at the coordinates of its vertexes.

Furthermore, it happens with each of the sets I have to split.

I'm wondering why in Opt mode I had a different behaviour...

Any suggestions to go on converting the mesh?
Thanks a lot

Francesco

[fra76]

Well, after trying a very simple case, I've notice that neither splitMeshWithSets nor splitMesh work with OpenFOAM 1.3!
In Debug mode, this is the error if I set "startFace ;" in constant/polyMesh/boundary:
----------------------------------------
Create polyMesh for time = 0

Read 14 faces to split

--> FOAM FATAL ERROR : Face 412 in faceSet foglio is not an internal face.

From function splitMesh
in file splitMesh.C at line 144.

FOAM exiting
----------------------------------------

or, If I follow Bernhard's suggestion and set it "one face after the last face", this is the error:
----------------------------------------
Create polyMesh for time = 0



--> FOAM FATAL ERROR : size 2571 out of range 0 ... 2570

From function UList<t>::checkSize(const label)
in file /home/francesco/OpenFOAM/OpenFOAM-1.3/src/OpenFOAM/lnInclude/UListI.H at line 88.

FOAM aborting

Foam::error::printStack(Foam:: Ostream&)
Foam::error::abort()
Foam:: Ostream& Foam::operator<<>(Foam:: Ostream&, Foam::errorManip<foam::error>)
Foam::UList<foam::face>::checkSize(int) const
Foam::SubList<foam::face>::SubList(Foam::UList<foa m::face> const&, int, int)
Foam::polyPatch::polyPatch(Foam::word const&, Foam::dictionary const&, int, Foam::polyBoundaryMesh const&)
Foam::wallPolyPatch::wallPolyPatch(Foam::word const&, Foam::dictionary const&, int, Foam::polyBoundaryMesh const&)
Foam::polyPatch::adddictionaryConstructorToTable<f oam::wallpolypatch>::New(Foam: :word const&, Foam::dictionary const&, int, Foam::polyBoundaryMesh const&)
Foam::polyPatch::New(Foam::word const&, Foam::dictionary const&, int, Foam::polyBoundaryMesh const&)
Foam::polyBoundaryMesh::polyBoundaryMesh(Foam::IOo bject const&, Foam::polyMesh const&)
Foam::polyMesh::polyMesh(Foam::IOobject const&)
splitMesh [0x40e2e2]
__libc_start_main
__gxx_personality_v0
----------------------------------------

With OF 1.3 without debugging, this is what I get:

----------------------------------------
Create polyMesh for time = 0

Read 14 faces to split

Adding point and face zones

Constructed topologyModifier:

Splitter
{
type attachDetach;
faceZoneName membraneFaces;
masterPatchName foglio_up;
slavePatchName foglio_down;
triggerTimes 1(0);
manualTrigger off;
active on;
}
Foam::error::printStack(Foam:: Ostream&)
Foam::sigSegv::sigSegvHandler(int)
/lib64/libc.so.6 [0x364712f7c0]
Foam::primitiveMesh::calcCellEdges() const
Foam::primitiveMesh::cellEdges() const
Foam::faceZone::meshEdges() const
Foam::attachDetach::detachInterface(Foam::polyTopo Change&) const
Foam::attachDetach::setRefinement(Foam::polyTopoCh ange&) const
Foam::polyTopoChanger::topoChangeRequest() const
Foam::polyTopoChanger::changeMesh()
Foam::attachPolyTopoChanger::attach(bool)
splitMesh [0x409eaf]
__libc_start_main
__gxx_personality_v0
Segmentation fault
----------------------------------------

With OpenFOAM 1.2 both splitMesh and splitMeshWithSets work perfectly on this simple mesh.

At this level, I guess this is a bug of OF 1.3
If you like, I can send the simple .msh file I've used for tests...
Francesco

[gschaider]

Hi Francesco!

"startFace ;" doesn't work in OF1.3 any more (that was, what I was aiming at in what you were qouting in the posting before). It is now treated like "startFace 0;" (not like "startFace 666;" like OF1.2 did when your mesh had 666 faces before splitting)
You now have to set it to a value (==nr of faces in the mesh).

But, yeah. You can send me the mesh and I'll give it a try.

About the big mesh: you are sure that gambit knows this as an interior face (one with cells on both sides). I'm not to big a Gambit expert (and I'm not too sure about the Gambit-nomenclature) , but some operations produce two exterior faces that lie at exactly the same place so it is easy to "ignore" one of them. In such a case checking the coordinates is not sufficient (but you won't need splitMesh too)

[fra76]

Hi Bernhard!
Well, in "constant/polyMesh/faces" I can see "2570" as the number of cells, so I've put this number in constant/polyMesh/boundary, adding the following two entries:
----------------------
foglio_up
{
type patch;
nFaces 0;
startFace 2570;
}

foglio_down
{
type patch;
nFaces 0;
startFace 2570;
}
----------------------

Then, I've ran splitMeshWithSets and I get a NEW error! Here it is (I added an output just before calling splitter.attach):
----------------------
Create time

Create polyMesh for time = 0

Read 14 faces to split

Adding set for faceSet internal.4
Adding point and face zones

Constructed topologyModifier:

Splitter
{
type attachDetach;
faceZoneName membraneFaces;
masterPatchName foglio_up;
slavePatchName foglio_down;
triggerTimes 1(0);
manualTrigger off;
active on;
}
fdc: BEFORE splitter.attach


--> FOAM FATAL ERROR : index -1 out of range 0 ... 628

From function UList<t>::checkIndex(const label)
in file /home/francesco/OpenFOAM/OpenFOAM-1.3/src/OpenFOAM/lnInclude/UListI.H at line 107.

FOAM aborting

Foam::error::printStack(Foam:: Ostream&)
Foam::error::abort()
Foam:: Ostream& Foam::operator<<>(Foam:: Ostream&, Foam::errorManip<foam::error>)
Foam::UList<foam::dynamiclist<int,> >::checkIndex(int) const
Foam::UList<foam::dynamiclist<int,> >::operator[](int)
Foam::primitiveMesh::calcCellEdges() const
Foam::primitiveMesh::cellEdges() const
Foam::faceZone::meshEdges() const
Foam::attachDetach::detachInterface(Foam::polyTopo Change&) const
Foam::attachDetach::setRefinement(Foam::polyTopoCh ange&) const
Foam::polyTopoChanger::topoChangeRequest() const
Foam::polyTopoChanger::changeMesh()
Foam::attachPolyTopoChanger::attach(bool)
splitMeshWithSets [0x410f09]
__libc_start_main
__gxx_personality_v0
Abortito
----------------------

Could you really use splitMesh with OpenFOAM 1.3? If so, could you send me a .msh example file?

Concerning Gambit (but the mesh was made using Tgrid and exported to .msh), I'm not an expert, but in the case of the big mesh the solid faces are internal, i.e. with cells on both sides. Even fluentMeshToFoam fails... So I really need splitMeshWithSets!

I'm trying to use OpenFOAM 1.2, but it can't be compiled in SP mode, so the 4Gb of RAM I have now on my PC are not enough to split that mesh!
That's why I would like to use version 1.3.

However, thank you very much!
Francesco

[gschaider]

Hello Francesco!

Yes I did already use it on 1.3. Could you send me the mesh, and I'll give it a try (EMail is in the profile information)

Bernhard

[fra76]

Hi!
as Bernhard knows, everything seems to work fine, now!
The problem was related to the order of the entries in the constant/polyMesh/boundary file. Infact boundary patchs have to be ordered by startFace in the boundary-file, but I wasn't aware of that.

Francesco

[fra76]

Here I am again.
Well, I've finally imported the big Mesh!
I've found a strange behaviour of splitMeshWithSets when two internal surfaces to be splitted are connected like a "T".
It can give an error or not, depending on the sequence of the conversions.
I've prepared a small example to show the problem. If you want, Bernhard, I can send it to you by mail.

In my case, I could join all "disturbing" surfaces, as a work around. But this could not always be possible.

Francesco