[snappyHexMesh] motorBike snappyHexMesh -parallel freezes after few iterations

llirik · December 14, 2020, 05:49

Good day to all
need your help with motorBike OpenFOam benchmark
I have a prototype ARM CPU based cluster
125 separate nodes = 1000 cores in total
Each node hardware configuration:
8cores (Cortex-A53) + 2GB LPDDR3 ram + 16Gb emmc
nodes are interconnected 1Gb ethernet
OpenFoam-v2006
Ubuntu 20.04 aarch64
mpich-3.3.2
GlusterFS mount point for OpenFOAM installation

I checked first on single node: 8cores - everything is OK
but when i start to run on 4nodes/8nodes/16nodes - 32cores/64cores/128cores respectively
I have a problem at this stage:

mpirun -np 32 -hostfile ~/mpi_hosts4 snappyHexMesh -overwrite -parallel

after 2/3 iterations execution freezes, the processor load on the nodes drops from 100% to 70% and next iterations started after an hour or longer

I have no experience using OpenFOAM I am doing research and i need to show as many different tests as possible on my prototype
I will be grateful for any help or advice

me3840 · December 22, 2020, 18:29

According to the logfile your cells per refinement level are pretty low. How big is the mesh on 8 cores? Even using all the cells in the last complete level is about 80 cells/core. I don't think the code is going to scale well at that stage.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[solidMechanics] Support thread for "Solid Mechanics Solvers added to OpenFOAM Extend"	bigphil	OpenFOAM CC Toolkits for Fluid-Structure Interaction	686	December 22, 2022 09:10
pressure in incompressible solvers e.g. simpleFoam	chrizzl	OpenFOAM Running, Solving & CFD	13	March 28, 2017 05:49
simpleFoam error - "Floating point exception"	mbcx4jc2	OpenFOAM Running, Solving & CFD	12	August 4, 2015 02:20
Micro Scale Pore, icoFoam	gooya_kabir	OpenFOAM Running, Solving & CFD	2	November 2, 2013 13:58
pisoFoam with k-epsilon turb blows up - Some questions	Heroic	OpenFOAM Running, Solving & CFD	26	December 17, 2012 03:34

December 22, 2020, 18:29		#2
me3840 Senior Member Join Date: Nov 2010 Location: USA Posts: 1,232 Rep Power: 24	According to the logfile your cells per refinement level are pretty low. How big is the mesh on 8 cores? Even using all the cells in the last complete level is about 80 cells/core. I don't think the code is going to scale well at that stage.