[Kaskade]

Hello,

the official support for metis has been dropped for license issues. Has someone been able to install and use metis on their own? My attempts so far have been fruitless.

Greetings
Kaskade

[AlmostSurelyRob]

Yes, you need to download metis from this site
http://people.sc.fsu.edu/~jburkardt/c_src/metis/metis.html

and GKLib. GKLib was a bit more tricky to find but I have found it for example here:
http://code.google.com/p/graphlabapi...0a9d384317c3bf
You can checkout the whole repository if you have mercurial installed.

Now it's the question of compiling these things. Set up your include paths correctly. For GKLib I had to add

DMALLOCINC = -I../libmetis/

in the supplied Makefile. libmetis was the position of libmetis.

Metis compiled seamlessly in my case using script although you must add -fPIC to the gcc command.

Finally go to

$FOAM_SRC/parallel/decompose/metisDecomp

change Make/option file to point to your metis library location and run

wmake libso

Let me know if you have any problems with that. I can send you the necessary files.


EDIT: ALSO! I removed the -I pointing to the dummy library. I guess that might be important. :-)

[Kaskade]

Thanks, I'll try it some time. My current case is pretty symmetric, so simple does the trick.

[pere]

Hi everybody,

I'm trying to compile metisDecomp, I'm getting this error while I'm doing wmake libso:

metisDecomp.C: In member function ‘Foam::label Foam::metisDecomp::decompose(const Foam::List<int>&, const Foam::List<int>&, const Foam::scalarField&, Foam::List<int>&)’:
metisDecomp.C:237: error: ‘METIS_WPartGraphRecursive’ was not declared in this scope
metisDecomp.C:254: error: cannot convert ‘int*’ to ‘real_t*’ for argument ‘9’ to ‘int METIS_PartGraphRecursive(idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, real_t*, real_t*, idx_t*, idx_t*, idx_t*)’
metisDecomp.C:275: error: ‘METIS_WPartGraphKway’ was not declared in this scope
metisDecomp.C:292: error: cannot convert ‘int*’ to ‘real_t*’ for argument ‘9’ to ‘int METIS_PartGraphKway(idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, idx_t*, real_t*, real_t*, idx_t*, idx_t*, idx_t*)’
make: *** [Make/linux64GccDPOpt/metisDecomp.o] Error 1

I've linked to libmetis.so.... perhaps any include that is not found?

[Kaskade]

I've compared scotch and metis for some of my old meshes. For some metis was a little faster (decomposing and/or simulation), for some scotch. You are probably wasting more time getting metis to run, than you would ever save by using it.

[pere]

Hi Kaskade,

I'm compiling OpenFoam from the source code so I need the same time to compile it with scotch or metis. Thanks for your reply.

[dkokron]

Mods to get 2.2.x to compile with METIS from parmetis-4.0.2.

Make sure you test the results!!

git diff src/parallel/decompose/Allwmake
diff --git a/src/parallel/decompose/Allwmake b/src/parallel/decompose/Allwmake
index 5cea1fb..581b877 100755
--- a/src/parallel/decompose/Allwmake
+++ b/src/parallel/decompose/Allwmake
@@ -62,6 +62,7 @@ else
     echo
 fi
 
+wmake $makeType metisDecomp
 wmake $makeType decompositionMethods
 
 wmake $makeType decompose

git diff src/parallel/decompose/metisDecomp/Make/options
diff --git a/src/parallel/decompose/metisDecomp/Make/options b/src/parallel/decompose/metisDecomp/Make/options
index a6a0f14..b046053 100644
--- a/src/parallel/decompose/metisDecomp/Make/options
+++ b/src/parallel/decompose/metisDecomp/Make/options
@@ -1,7 +1,6 @@
 EXE_INC = \
-    -I$(WM_THIRD_PARTY_DIR)/metis-5.0pre2/include \
-    -I../../dummyThirdParty/metisDecomp  /* location of dummy metis.h */ \
+    -I/home/dkokron/play/ParMETIS/parmetis-4.0.2/install/include \
     -I../decompositionMethods/lnInclude
 
 LIB_LIBS = \
-    -L$(FOAM_EXT_LIBBIN) -lmetis -lGKlib
+    -L$(FOAM_EXT_LIBBIN) -lmetis

git diff src/parallel/decompose/metisDecomp/metisDecomp.C
diff --git a/src/parallel/decompose/metisDecomp/metisDecomp.C b/src/parallel/decompose/metisDecomp/metisDecomp.C
index e1614a8..2530e61 100644
--- a/src/parallel/decompose/metisDecomp/metisDecomp.C
+++ b/src/parallel/decompose/metisDecomp/metisDecomp.C
@@ -202,6 +202,7 @@ Foam::label Foam::metisDecomp::decompose
 
     // Vertex weight info
     int wgtFlag = 0;
+    int ncon = 1;
     int* vwgtPtr = NULL;
     int* adjwgtPtr = NULL;
 
@@ -220,18 +221,19 @@ Foam::label Foam::metisDecomp::decompose
     {
         if (processorWeights.size())
         {
-            METIS_WPartGraphRecursive
+            METIS_PartGraphRecursive
             (
                 &numCells,         // num vertices in graph
+                &ncon,             // number of balancing constraints. It should be at least 1.
                 const_cast<List<int>&>(xadj).begin(),   // indexing into adjncy
                 const_cast<List<int>&>(adjncy).begin(), // neighbour info
                 vwgtPtr,           // vertexweights
+                NULL,
                 adjwgtPtr,         // no edgeweights
-                &wgtFlag,
-                &numFlag,
                 &nProcs,
-                processorWeights.begin(),
-                options.begin(),
+                NULL,              // NULL indicates graph should be equally divided among the partitions
+                NULL,              // load imbalance tolerance for each constraint; NULL -> 1.001
+                NULL,              // options array
                 &edgeCut,
                 finalDecomp.begin()
             );
@@ -241,14 +243,16 @@ Foam::label Foam::metisDecomp::decompose
             METIS_PartGraphRecursive
             (
                 &numCells,         // num vertices in graph
+                &ncon,             // number of balancing constraints. It should be at least 1.
                 const_cast<List<int>&>(xadj).begin(),   // indexing into adjncy
                 const_cast<List<int>&>(adjncy).begin(), // neighbour info
                 vwgtPtr,           // vertexweights
+                NULL,
                 adjwgtPtr,         // no edgeweights
-                &wgtFlag,
-                &numFlag,
                 &nProcs,
-                options.begin(),
+                NULL,              // NULL indicates graph should be equally divided among the partitions
+                NULL,              // load imbalance tolerance for each constraint; NULL -> 1.001
+                NULL,              // options array
                 &edgeCut,
                 finalDecomp.begin()
             );
@@ -258,18 +262,19 @@ Foam::label Foam::metisDecomp::decompose
     {
         if (processorWeights.size())
         {
-            METIS_WPartGraphKway
+            METIS_PartGraphKway
             (
                 &numCells,         // num vertices in graph
+                &ncon,             // number of balancing constraints. It should be at least 1.
                 const_cast<List<int>&>(xadj).begin(),   // indexing into adjncy
                 const_cast<List<int>&>(adjncy).begin(), // neighbour info
                 vwgtPtr,           // vertexweights
+                NULL,
                 adjwgtPtr,         // no edgeweights
-                &wgtFlag,
-                &numFlag,
                 &nProcs,
-                processorWeights.begin(),
-                options.begin(),
+                NULL,              // NULL indicates graph should be equally divided among the partitions
+                NULL,              // load imbalance tolerance for each constraint; NULL -> 1.001
+                NULL,              // options array
                 &edgeCut,
                 finalDecomp.begin()
             );
@@ -279,14 +284,16 @@ Foam::label Foam::metisDecomp::decompose
             METIS_PartGraphKway
             (
                 &numCells,         // num vertices in graph
+                &ncon,             // number of balancing constraints. It should be at least 1.
                 const_cast<List<int>&>(xadj).begin(),   // indexing into adjncy
                 const_cast<List<int>&>(adjncy).begin(), // neighbour info
                 vwgtPtr,           // vertexweights
+                NULL,
                 adjwgtPtr,         // no edgeweights
-                &wgtFlag,
-                &numFlag,
                 &nProcs,
-                options.begin(),
+                NULL,              // NULL indicates graph should be equally divided among the partitions
+                NULL,              // load imbalance tolerance for each constraint; NULL -> 1.001
+                NULL,              // options array
                 &edgeCut,
                 finalDecomp.begin()
             );
@@ -328,7 +335,16 @@ Foam::labelList Foam::metisDecomp::decompose
     }
 
     CompactListList<label> cellCells;
-    calcCellCells(mesh, identity(mesh.nCells()), mesh.nCells(), cellCells);
+    calcCellCells
+    (
+        mesh,
+        identity(mesh.nCells()),
+        mesh.nCells(),
+        true,
+        cellCells
+    );
+
+    // DSK replace after switching to 2.2.x calcCellCells(mesh, identity(mesh.nCells()), mesh.nCells(), cellCells);
 
     // Decompose using default weights
     labelList decomp;
@@ -362,7 +378,15 @@ Foam::labelList Foam::metisDecomp::decompose
     //   xadj(celli) : start of information in adjncy for celli
 
     CompactListList<label> cellCells;
-    calcCellCells(mesh, agglom, agglomPoints.size(), cellCells);
+    calcCellCells
+    (
+        mesh,
+        agglom,
+        agglomPoints.size(),
+        true,
+        cellCells
+    );
+    // DSK replace after switching to 2.2.x calcCellCells(mesh, agglom, agglomPoints.size(), cellCells);
 
     // Decompose using default weights
     labelList finalDecomp;

[pere]

Thanks I will try it!