CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > OpenFOAM Post-Processing

Reconstruction of the parallel case with dynamic mesh

Register Blogs Members List Search Today's Posts Mark Forums Read

Like Tree9Likes
  • 9 Post By makaveli_lcf

Reply
 
LinkBack Thread Tools Display Modes
Old   July 12, 2012, 10:10
Default Reconstruction of the parallel case with dynamic mesh
  #1
Senior Member
 
Dr. Alexander Vakhrushev
Join Date: Mar 2009
Posts: 214
Rep Power: 11
makaveli_lcf is on a distinguished road
Send a message via ICQ to makaveli_lcf
Hi all!

Today I faced a problem with the reconstruction of the parallel case with dynamic mesh refinement used in the solver.
First guess was that it is my solver's problem which could write not all the mesh data required for the reconstruction.
But first trial with the interDyMFoam in parallel (4 CPUs) indicated the same problem. Let's go into details.

So, simple run of the reconstructPar utility reports following symptoms:

Code:
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.1.1                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.1.1-221db2718bbb
Exec   : reconstructPar
Date   : Jul 12 2012
Time   : 15:08:53
Host   : "pc169"
PID    : 28123
Case   : /media/Sim/alexanderv/New_CD_Labor/Term_03/VOF/DyM_Reconstruct/OFv211/damBreakWithObstacle
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Time = 0.02



--> FOAM FATAL IO ERROR:
cannot find file

file:   /media/Sim/alexanderv/New_CD_Labor/Term_03/VOF/DyM_Reconstruct/OFv211/damBreakWithObstacle/processor0/0.02/polyMesh/pointProcAddressing  at line 0.

    From function regIOobject::readStream()
    in file db/regIOobject/regIOobjectRead.C at line 73.

FOAM exiting
Searching on the CFD-online gave tips and a way to solution:

1. Use reconstructParMesh for mesh reconstruction
2. Use reconstructPar for the data reconstruction

Step 1.
a) First problem is that reconstructParMesh uses reconstruction tolerance 1e-07 and suppose we used default values writing simulation data as well as mesh modifications in ASCII format with 1e-06 precision.
To override the defaults we use reconstructParMesh with -mergeTol 1e-06
b) Secondly reconstructParMesh during the run processes only one time step (correct?) either first or the lasr (wirh -latestTime flag). In any case I didn’t find automatic procession of all time dirs and -time <time> option works with the singe parameter only on the contrary to the reconstructPar -time <ranges>. So we will put it in a loop to reconstruct all time dirs.

Step 2.
Fails without clear reasons:
Code:
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.1.1                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.1.1-221db2718bbb
Exec   : reconstructPar
Date   : Jul 13 2012
Time   : 16:04:04
Host   : "pc169"
PID    : 19765
Case   : /media/Sim/alexanderv/New_CD_Labor/Term_03/VOF/DyM_Reconstruct/OFv211/damBreakWithObstacle_clean.run1
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Time = 0.02

#0  Foam::error::printStack(Foam::Ostream&) in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#1  Foam::sigSegv::sigHandler(int) in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#2   in "/lib/libc.so.6"
#3  Foam::objectRegistry::checkOut(Foam::regIOobject&) const in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#4  Foam::regIOobject::~regIOobject() in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#5  Foam::IOList<int>::~IOList() in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"
#6  Foam::processorMeshes::read() in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libreconstruct.so"
#7  Foam::processorMeshes::readUpdate() in "/opt/openfoam211/platforms/linux64GccDPOpt/lib/libreconstruct.so"
#8  
 in "/opt/openfoam211/platforms/linux64GccDPOpt/bin/reconstructPar"
#9  __libc_start_main in "/lib/libc.so.6"
#10  
 in "/opt/openfoam211/platforms/linux64GccDPOpt/bin/reconstructPar"
Segmentation fault


Solution is to set “startFrom startTime;” in the controlDict and removing processor*/0 folders.
In the attached script I backup them and then move back after data reconstruction.

So hereby I’ve attached a bash script for automatic reconstruction of e.g. interDyMFoam parallel case to avoid modified mesh isues.

Code:
#!/bin/bash


##  Merge tollerance (to override default value 1e-07 for ASCII output)
TOL=1e-06


##  Log file name
LOG='log.reconstructParMesh'


##  Start time folder name
START='0'


##  Check for empty time folders
TIME_DIRS=''
for proc_ in processor0/*; do
    dir_=`echo $proc_ | sed 's/processor0\///'`
    if [ ! -d $dir_ ]; then
        echo "Time $dir_ ist not reconstructed"
        ##  For empty list
        if [ ! -n "$TIME_DIRS" ]; then
            TIME_DIRS=$dir_        
        else
            TIME_DIRS="$TIME_DIRS $dir_"
        fi  
    fi
done


##  Exit if everything is reconstructed
if [ ! -n "$TIME_DIRS" ]; then
    echo -e "\nNothing to reconstruct! Exiting..."
    exit 0
else
    ##  Clean old log files
    rm -fv $LOG.*


    ##  Reconstruct mesh and data
    echo -e "Reconstructing mesh for:\n"

    ##  Mesh
    for dir_ in $TIME_DIRS; do
        echo -e "\tTime = $dir_ sec\n"
        reconstructParMesh -time $dir_ -mergeTol $TOL 2>&1 | tee $LOG.mesh
    done


    ##  Set startFrom to startTime
    sed -i "s/startFrom.*\;/startFrom    startTime\;/g" system/controlDict

    ##  Rename zero time folders in the processor directories
    for proc_ in processor*; do
        mv -fv $proc_/$START $proc_/$START.backup
    done

    ##  Data
    echo -e "Reconstructing data for times: $TIME_DIRS"
    reconstructPar -time `echo $TIME_DIRS | sed s/\ /,/g` 2>&1 | tee $LOG.data

    ##  Rename zero time folders backups in the processor directories
    for proc_ in processor*; do
        mv -fv $proc_/$START.backup $proc_/$START
    done
fi
It is possible to adjust the settings in the script for tolerance, start time and logfile name.

Here are some simulation and post-processing results:

https://docs.google.com/open?id=0BzC...1k5YmVTZDdOT1k
wyldckat, linch, aliqasemi and 6 others like this.
__________________
Best regards,

Dr. Alexander VAKHRUSHEV

Christian Doppler Laboratory for "Advanced Process Simulation of
Solidification and Melting"

Simulation and Modelling of Metallurgical Processes
Department of Metallurgy
University of Leoben

Franz-Josef-Str. 18
A - 8700 Leoben
Österreich / Austria
Tel.: +43 3842 - 402 - 3125
http://smmp.unileoben.ac.at

Last edited by makaveli_lcf; July 13, 2012 at 10:06. Reason: Fixed fail at the reconstructPar stage
makaveli_lcf is offline   Reply With Quote

Old   August 7, 2013, 10:18
Default
  #2
Senior Member
 
Illya Shevchuk
Join Date: Aug 2009
Location: Darmstadt, Germany
Posts: 176
Rep Power: 8
linch is on a distinguished road
Hi Alexander,

thanks for the script. Unfortunately, the data reconstruction doesn't work in my case (OF-2.1.x). I can't decrypt the error message:
Quote:
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Time = 0.0230874

#0 Foam::error:rintStack(Foam::Ostream&) in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#1 Foam::sigSegv::sigHandler(int) in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#2 in "/lib/x86_64-linux-gnu/libc.so.6"
#3 Foam:bjectRegistry::checkOut(Foam::regIOobject&) const in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#4 Foam::regIOobject::~regIOobject() in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
#5 Foam::IOList<int>::~IOList() in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libfiniteVolume.so"
#6 Foam:rocessorMeshes::read() in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libreconstruct.so"
#7 Foam:rocessorMeshes::readUpdate() in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/lib/libreconstruct.so"
#8
in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/bin/reconstructPar"
#9 __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
#10
in "/home/shevchuk/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64GccDPOpt/bin/reconstructPar"
Do you have any ideas?

Best regards,
Ilya
Attached Files
File Type: txt log.reconstructParMesh.txt (16.9 KB, 14 views)
linch is offline   Reply With Quote

Old   August 8, 2013, 04:27
Default
  #3
Senior Member
 
Illya Shevchuk
Join Date: Aug 2009
Location: Darmstadt, Germany
Posts: 176
Rep Power: 8
linch is on a distinguished road
Solved. the error was because of insufficient RAM.
linch is offline   Reply With Quote

Old   August 8, 2013, 04:56
Default
  #4
Senior Member
 
Dr. Alexander Vakhrushev
Join Date: Mar 2009
Posts: 214
Rep Power: 11
makaveli_lcf is on a distinguished road
Send a message via ICQ to makaveli_lcf
Effective)
__________________
Best regards,

Dr. Alexander VAKHRUSHEV

Christian Doppler Laboratory for "Advanced Process Simulation of
Solidification and Melting"

Simulation and Modelling of Metallurgical Processes
Department of Metallurgy
University of Leoben

Franz-Josef-Str. 18
A - 8700 Leoben
Österreich / Austria
Tel.: +43 3842 - 402 - 3125
http://smmp.unileoben.ac.at
makaveli_lcf is offline   Reply With Quote

Old   July 3, 2014, 05:51
Default
  #5
New Member
 
ttttoor
Join Date: Feb 2014
Posts: 10
Rep Power: 4
meng9019 is on a distinguished road
Hi Ilya,

How did you solve this problem?
I am running into the same problem now.
Thank you in advance.

Quote:
Originally Posted by linch View Post
Solved. the error was because of insufficient RAM.
meng9019 is offline   Reply With Quote

Reply

Tags
dynamic mesh, interdymfoam, pointprocaddressing, reconstructpar, reconstructparmesh

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Moving mesh Niklas Wikstrom (Wikstrom) OpenFOAM Running, Solving & CFD 122 June 15, 2014 06:20
how to parallel run in moving mesh case ELYOR CD-adapco 5 June 16, 2008 03:23
fluent add additional zones for the mesh file SSL FLUENT 2 January 26, 2008 12:55
Dynamic Mesh in an In-cylinder case venkitaraman FLUENT 0 March 26, 2007 17:15
Automatic Mesh Motion solver michele OpenFOAM Running, Solving & CFD 10 September 26, 2005 08:21


All times are GMT -4. The time now is 13:27.