RNG K-Epsilon parallel run crashing at writeInterval

b.simpson · July 6, 2021, 04:22

Hi all,

I have a simple natural ventilation model that I am running in OpenFOAM-8 using buoyantSimpleFoam and the RNG K-Epsilon turbulence model. I am running the model in parallel on 6 processors through Ubuntu 18.04.

Currently when I run my model it crashes when ever it reaches the writeInterval. I have tried setting the writeInterval in the controlDict to time steps between 100 and 1000. I have successfully ran the same model using the K-Omega SST turbulence model many times.

The error I get in the buoyantSimpleFoam log is as follows:

Code:

Time = 98

DILUPBiCGStab:  Solving for Ux, Initial residual = 0.084642688, Final residual = 0.00022274125, No Iterations 1
DILUPBiCGStab:  Solving for Uy, Initial residual = 0.13106511, Final residual = 0.00029400487, No Iterations 1
DILUPBiCGStab:  Solving for Uz, Initial residual = 0.13565445, Final residual = 0.00026879261, No Iterations 1
DILUPBiCGStab:  Solving for e, Initial residual = 0.99778473, Final residual = 0.061791952, No Iterations 1
DICPCG:  Solving for p_rgh, Initial residual = 0.041747707, Final residual = 0.00040698742, No Iterations 43
time step continuity errors : sum local = 3.8449625, global = -0.40380891, cumulative = -12.89106
DILUPBiCGStab:  Solving for epsilon, Initial residual = 0.017456754, Final residual = 0.00019965907, No Iterations 1
DILUPBiCGStab:  Solving for k, Initial residual = 0.01297698, Final residual = 0.00016623612, No Iterations 1
ExecutionTime = 204.45 s  ClockTime = 205 s

scalarTransport write:
DILUPBiCGStab:  Solving for CO2, Initial residual = 0.0035552031, Final residual = 9.6084704e-05, No Iterations 1

scalarTransport write:
DILUPBiCGStab:  Solving for AoA, Initial residual = 0.012096209, Final residual = 0.00026913087, No Iterations 1

Time = 99

DILUPBiCGStab:  Solving for Ux, Initial residual = 0.091478301, Final residual = 0.0002577593, No Iterations 1
DILUPBiCGStab:  Solving for Uy, Initial residual = 0.12497896, Final residual = 0.00031404754, No Iterations 1
DILUPBiCGStab:  Solving for Uz, Initial residual = 0.14731219, Final residual = 0.00033608703, No Iterations 1
DILUPBiCGStab:  Solving for e, Initial residual = 0.99791841, Final residual = 0.062730869, No Iterations 1
DICPCG:  Solving for p_rgh, Initial residual = 0.17235688, Final residual = 0.0015506243, No Iterations 42
time step continuity errors : sum local = 3.7518611, global = -0.23159506, cumulative = -13.122656
DILUPBiCGStab:  Solving for epsilon, Initial residual = 0.017323729, Final residual = 0.0002026168, No Iterations 1
DILUPBiCGStab:  Solving for k, Initial residual = 0.013025041, Final residual = 0.00017041142, No Iterations 1
ExecutionTime = 206.32 s  ClockTime = 206 s

scalarTransport write:
DILUPBiCGStab:  Solving for CO2, Initial residual = 0.0035338061, Final residual = 9.3818566e-05, No Iterations 1

scalarTransport write:
DILUPBiCGStab:  Solving for AoA, Initial residual = 0.012193801, Final residual = 0.00025705613, No Iterations 1

Time = 100

DILUPBiCGStab:  Solving for Ux, Initial residual = 0.08493483, Final residual = 0.00020312125, No Iterations 1
DILUPBiCGStab:  Solving for Uy, Initial residual = 0.1085, Final residual = 0.00032789983, No Iterations 1
DILUPBiCGStab:  Solving for Uz, Initial residual = 0.12894373, Final residual = 0.00031718948, No Iterations 1
DILUPBiCGStab:  Solving for e, Initial residual = 0.99787076, Final residual = 0.063485763, No Iterations 1
DICPCG:  Solving for p_rgh, Initial residual = 0.07760779, Final residual = 0.00073283949, No Iterations 31
time step continuity errors : sum local = 3.9056444, global = -0.43933573, cumulative = -13.561991
DILUPBiCGStab:  Solving for epsilon, Initial residual = 0.018083409, Final residual = 0.0002201838, No Iterations 1
DILUPBiCGStab:  Solving for k, Initial residual = 0.013392784, Final residual = 0.00018106851, No Iterations 1
ExecutionTime = 209.13 s  ClockTime = 209 s

scalarTransport write:
DILUPBiCGStab:  Solving for CO2, Initial residual = 0.0035812478, Final residual = 9.7399015e-05, No Iterations 1

-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[4935,1],3]
  Exit code:    145
--------------------------------------------------------------------------

I have tried adjusting my Epsilon BCs to several different values but the crash still occurs. My /0/Epsilon/ file is as follows:

Code:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  v1812                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       volScalarField;
    location    "0";
    object      epsilon;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

//epsilonInlet  0.03; // Cmu^0.75 * k^1.5 / L ; L =10

dimensions      [0 2 -3 0 0 0 0];

internalField   uniform 14.855;

boundaryField
{
   "xmin|xmax|ymin|ymax|zmax_north|zmax_south"
    {
        type            inletOutlet;
        inletValue      0.03;
		value           0.03;
    }
    "zmin|Ceiling|Floor|East_Wall|North_Wall|South_Wall|West_Wall|Computer|Person|zmax_ceiling|North_Window|South_Window"
    {
        type            epsilonWallFunction;
        value           0.03;
    }
	
	#includeEtc "caseDicts/setConstraintTypes"
}


// ************************************************************************* //

I have tried running the model on a single processor and the run crashes at the same writeInterval, just without any crash message unlike when running in parallel.

Does anyone have any ideas what could be causing this crash when using the RNG K-Epsilon model and/or any way of troubleshooting my model to better determine the cause of the crash?

Thanks in advance for any support.

Kind regards,
Ben

ControlDict:

Code:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  v1812                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  | Model:    VE_Practice                           |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application     buoyantSimpleFoam;

startFrom       latestTime;

startTime       0;

stopAt          endTime;

endTime         10000;

deltaT          1;

writeControl    timeStep;

writeInterval   100;

purgeWrite      3;

writeFormat     ascii;

writePrecision  8;

writeCompression off;

timeFormat      general;

timePrecision   6;

runTimeModifiable true;

/*libs (
      "libOpenFOAM.so"
      "libsimpleSwakFunctionObjects.so"
      "libswakFunctionObjects.so"
      "libgroovyBC.so"
     );*/

functions
{
	#include "surfaces"
	#include "surfaces2"
	  
	CO2
		{
			type    			scalarTransport;
			libs 				("libsolverFunctionObjects.so");
			writeControl       	timeStep;
			writeInterval      	1;
			active          	true;
			autoSchemes     	false;
			nCorr           	0;
			resetOnStartUp 		false;
			field				CO2;
			fvOptions       
			{
				generalSource
				{
					type            semiImplicitSource;
					active          true;
					selectionMode	cellZone;
					cellZone		spaceZone;
					volumeMode		absolute;
					sources
					{
						CO2		
						{
							explicit	1;
							implicit	0;
						}
					}
				}
			}
		}
		
	comfortAnalysis
		{
			type            comfort;
			libs            ("libfieldFunctionObjects.so");
			
			metabolicRate	1.2;
			clothing		0.7;
			extWork			0.0;

			executeControl  writeTime;
			writeControl    writeTime;

		}
		
	AoA
		{
			type    scalarTransport;
			libs 	("libutilityFunctionObjects.so");
			enabled true;
			writeControl writeTime;
			log yes;
			nCorr 1;

			field AoA;
			active          true;
			autoSchemes     false;
			resetOnStartUp false;
			DT              1e-5;
			nCorr 0;

			fvOptions
			{
				IncrementTime
				{
					type            semiImplicitSource;
					active          true;
					selectionMode all;
					volumeMode  specific;
					sources
					{
						AoA
						{
							explicit 1;
							implicit 0;
						}
					}
				}
			}
		}
	
	fieldMinMax
		{
			type                fieldMinMax;
			libs  				("libfieldFunctionObjects.so");
			log                 true;
			writeControl       	writeTime;
			//writeInterval      	1;
			//mode              magnitude;
			fields             	( U p T rho CO2 AoA);
		}
		
	breathingZoneAverages
		{
			type            volFieldValue;
			libs            ("libfieldFunctionObjects.so");
			log				true;
			writeControl    writeTime;
			writeFields		true;
			
			regionType		cellZone;
			name			breathingZone;
			
			operation		volAverage;

			fields
			(
				T
				CO2
				U
				PMV
				PPD
				AoA
			);
		}
		
	breathingZoneValuesAvg
		{
			type            volFieldValue;
			libs            ("libfieldFunctionObjects.so");
			log				true;
			writeControl    writeTime;
			writeFields		true;
			
			regionType		cellZone;
			name			breathingZone;
			
			operation		average;

			fields
			(
				T
				CO2
				U
				PMV
				PPD
				AoA
			);
		}
		
	breathingZoneValuesSum
		{
			type            volFieldValue;
			libs            ("libfieldFunctionObjects.so");
			log				true;
			writeControl    writeTime;
			writeFields		true;
			
			regionType		cellZone;
			name			breathingZone;
			
			operation		sum;

			fields
			(
				T
				CO2
				U
				PMV
				PPD
				AoA
			);
		}
	
	spaceVolAverage
		{
			type            volFieldValue;
			libs            ("libfieldFunctionObjects.so");
			log				true;
			writeControl    writeTime;
			writeFields		true;
			
			regionType		cellZone;
			name			spaceZone;
			
			operation		volAverage;

			fields
			(
				T
				CO2
				U
				AoA
			);
		}
		
	spaceAverage
		{
			type            volFieldValue;
			libs            ("libfieldFunctionObjects.so");
			log				true;
			writeControl    writeTime;
			writeFields		true;
			
			regionType		cellZone;
			name			spaceZone;
			
			operation		average;

			fields
			(
				T
				CO2
				U
				AoA
			);
		}
	
	residuals
    {
        type            residuals;
        functionObjectLibs ("libutilityFunctionObjects.so");
        enabled         true;
        writeControl   timeStep;
        writeInterval  1;

        fields
        (
            p_rgh
            U
            k
            epsilon
			CO2
			e
			AoA
        );
    }
	
	probes
    {
        type            probes;
        libs            ("libsampling.so");
        writeControl    timeStep;
        writeInterval   1;

        fields
        (
            p_rgh
			U
			T
			CO2
			k
			epsilon
			AoA
        );

        probeLocations
        (
            //(2.55 1.6 1.6)		//convergence point Occ1
			(1.6 3.1 2)		//convergence point Center
			//(2.55 4.6 1.6)		//convergence point Occ2
        );

    }
}

// ************************************************************************* //

fvSolution:

Code:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  v1812                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  | Model:    VE_Practice                           |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      fvSolution;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

solvers
{
    p_rgh
    {
        solver          PCG;
        preconditioner  DIC;
        tolerance       1e-06;
        relTol          0.01;
    }

    "(U|T|h|CO2|k|epsilon|e|AoA)"
    {
        solver          PBiCGStab;
        preconditioner  DILU;
        tolerance       1e-05;
        relTol          0.1;
    }

    G
    {
        $p_rgh;
        tolerance       1e-05;
        relTol          0.1;
    }
}

SIMPLE
{
    nNonOrthogonalCorrectors 0;
    pRefCell        0;
    pRefValue       0;

    residualControl
    {
        p_rgh           1e-2;
        U               1e-4;
		CO2             1e-4;
        T               1e-2;
		e               1e-2;
		h               1e-3;
        G               1e-3;

        // possibly check turbulence fields
        "(k|epsilon)" 1e-3;
    }
}

relaxationFactors
{
    fields
	{
		rho             1.0;
        p_rgh           0.9;
    }
    equations
    {
        U               0.1;
		CO2             0.3;
		h               0.9;
        T               0.5;
		e               0.5;
        "(k|R)" 		0.3;
		P				0.8;
		G               0.7;
		epsilon         0.3;
		AoA				0.4;
    }
}


// ************************************************************************* //

fvSchemes:

Code:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  v1812                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  | Model:    VE_Practice                           |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      fvSchemes;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

ddtSchemes
{
    default         steadyState;
}

gradSchemes
{
    default         Gauss linear;
}

divSchemes
{
    default         none;
	
    div(phi,epsilon) bounded Gauss upwind;
	div(phi,CO2) 	bounded Gauss upwind;

    div(phi,U)      bounded Gauss upwind;
    div(phi,e)      bounded Gauss upwind;

    div(phi,k)      bounded Gauss upwind;

    div(phi,Ekp)    bounded Gauss linear;

    div(((rho*nuEff)*dev2(T(grad(U))))) Gauss linear;
	
	div(phi,AoA) bounded Gauss upwind;
}

laplacianSchemes
{
    default         Gauss linear corrected;
}

interpolationSchemes
{
    default         linear;
}

snGradSchemes
{
    default         corrected;
}


// ************************************************************************* //

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Error running simpleFoam in parallel	Yuby	OpenFOAM Running, Solving & CFD	14	October 7, 2021 04:38
Some questions about a multi region case run in parallel	zfaraday	OpenFOAM Running, Solving & CFD	5	February 23, 2017 10:25
Can not run OpenFOAM in parallel in clusters, help!	ripperjack	OpenFOAM Running, Solving & CFD	5	May 6, 2014 15:25
simpleFoam in parallel issue	plucas	OpenFOAM Running, Solving & CFD	3	July 17, 2013 11:30
parallel Grief: BoundaryFields ok in single CPU but NOT in Parallel	JR22	OpenFOAM Running, Solving & CFD	2	April 19, 2013 16:49