Mesh too big for memory. How to perform decomposition in parallel?
Hi,
I need to decompose a case to run it in many nodes because the mesh is too large for the memory of a single node. It is so large that running decomposePar in a single node is not possible. How can I decompose the case in parallel, using distributed memory? Running decomposePar in parallel, if possible, would be a solution. Best wishes, Thomas P.S.: A question like this was asked nine years ago. I am asking again because things may have changed since then. |
How did you generate the mesh? Can this be less memory consuming than the decomposition?
|
Dear Anton,
Indeed, your question makes sense. I have also had problems to generate the mesh using blockMesh. Since my geometry is sufficiently simple, I wrote a program which directly writes down the files points, faces, owner, neighbour and boundary without using OpenFOAM classes. An alternative would be to write another program that write those files in processor* directories, but this would require investigating how decomposePar does so. Anyway, this alternative would not solve the problem for the cases in which I have had access to a large-memory machine when creating the mesh but that machine is not available anymore for decomposing it. Best wishes, Thomas |
Quote:
I am surprised to learn that openFOAM does not do paritioning in parallel. In FVUS I do parallel partitioning using par metis but when i was porting the solver for windows par metis no longer reliably available and thus had to write partitioner myself. Since I am no expert the partitioner is serial only. I put it there very reluctantly because that was one step that handicapped the solver for large meshes and felt very bad about it. openfoam is out there for years so its surprising. |
Quote:
|
Quote:
No so far no-one complained. But all the people who are doing serious work using linux version that is parallel partitioning. Most people using it for VOF and multiphase so big sizes are avoided as much as possible. (There is someone going to try on more than 50 million cell cases but that would be in april). Even on windows the only step which is serial is the partitioning part so only that graph structure that is used for partitioning is serial (that is collected on root process). Rest of the solver is parallel. That means FVUS loads the mesh into partitions user is running solver on, then partitioning is performed (serial on windows and parallel on linux) and redistribution takes place. Solver then continues. Because only one graph structure is serial only the size of meshes user can run still be much higher than openFOAM. |
one solution is to create a coarse background grid (with blockMesh), then to decompose the domain on X CPUs and refine the background grid in parallel with refineMesh. Then you can use snappyHexMesh in parallel.
|
decomposing seperat file after the decomposition process
Quote:
As you are talking about the decomposition process I have a problem here, I'm preparing a new boundary condition but it refuse to be decomposed during the decompsePar so I have to redistribute it manually for all processors , is there any way to use specific command to distribute it directly or I have to paste it in each processor manually: Thanks in advance |
All times are GMT -4. The time now is 18:08. |