Hi,
I would like to run a c
Hi,
I would like to run a case in parallel which has its root on a dynamically mounted partition '/scr/ceplx049/scr1/gcae504/OpenFOAM_Berechnung/ParallelTest' I decomposed the case in that directory and tried to run it, but somehow it looks for the information in a non-existing 'home'-directory... ceplx049/scr/ceplx049/scr1/gcae504/OpenFOAM_Berechnung/ParallelTest> mpirun --hostfile Klimakruemmer/machines -np 4 interFoam . damBreak -parallel > log & [1] 26003 ceplx049/scr/ceplx049/scr1/gcae504/OpenFOAM_Berechnung/ParallelTest> [2] [2] [2] --> FOAM FATAL IO ERROR : cannot open file [2] [2] file: /v/caenfs05/egb_user5/home/gcae504/damBreak/processor2/system/controlDict at line 0. [2] [2] From function regIOobject::readStream(const word&) [2] in file db/regIOobject/regIOobjectRead.C at line 66. [2] FOAM parallel run exiting [2] [ceplx050:20277] MPI_ABORT invoked on rank 2 in communicator MPI_COMM_WORLD with errorcode 1 [ceplx049][0,1,0][btl_tcp_frag.c:202:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed with errno=104 [3] [3] [3] --> FOAM FATAL IO ERROR : cannot open file [3] [3] file: /v/caenfs05/egb_user5/home/gcae504/damBreak/processor3/system/controlDict at line 0. [3] [3] From function regIOobject::readStream(const word&) [3] in file db/regIOobject/regIOobjectRead.C at line 66. [3] FOAM parallel run exiting [3] [ceplx050:20278] MPI_ABORT invoked on rank 3 in communicator MPI_COMM_WORLD with errorcode 1 mpirun noticed that job rank 0 with PID 26007 on node ceplx049 exited on signal 15 (Terminated). A parallel run with its root in my 'home'-directory works fine, but I limited space :-( Would be nice, if anybody has an idea!? Regards! Fabian |
Hmm,
You started from the d
Hmm,
You started from the directory '/scr/ceplx049/scr1/gcae504/OpenFOAM_Berechnung/ParallelTest' And MPI is reporting that it can't find the file Quote:
Check what 'mount -v' is showing and what the host is exporting (Linux: /usr/sbin/showmount -e HOST). Depending on the configuration, you might need some form of directory mapping. For some directories we use the GridEngine sge_aliases, which lets you specify stuff like this: #subm_dir subm_host exec_host path_replacement /tmp_mnt/ * * / |
Hi Mark,
yes, you are right
Hi Mark,
yes, you are right, I started from '/scr/ceplx049/scr1/gcae504/OpenFOAM_Berechnung/ParallelTest' and the case is located in that directory too, but I was wondering about the asking 'home' path, which obviously does not exists.... sorry, it works now when I run it with the complete path for the root and not just with '.'. Thanks! Fabian |
Hi,
as I mentioned before,
Hi,
as I mentioned before, it actually works now, but somehow I get the below error message after the first write to disk: Time = 50 DILUPBiCG: Solving for Ux, Initial residual = 0.0224561, Final residual = 0.000388077, No Iterations 1 DILUPBiCG: Solving for Uy, Initial residual = 0.0595427, Final residual = 0.00106835, No Iterations 1 DILUPBiCG: Solving for Uz, Initial residual = 0.0407827, Final residual = 0.000722178, No Iterations 1 DICPCG: Solving for p, Initial residual = 0.758773, Final residual = 0.00727437, No Iterations 269 time step continuity errors : sum local = 0.00111858, global = 9.43786e-05, cumulative = -0.00822085 DILUPBiCG: Solving for epsilon, Initial residual = 0.0116643, Final residual = 0.000240804, No Iterations 1 DILUPBiCG: Solving for k, Initial residual = 0.0600995, Final residual = 0.000742773, No Iterations 1 ExecutionTime = 4908.29 s ClockTime = 5530 s Time = 51 [2] --> FOAM Warning : [5] --> FOAM Warning : [5] From function Time::readModifiedObjects() [5] in file db/Time/TimeIO.C at line 222 [5] Delaying reading objects due to inconsistent file time-stamps between processors [6] --> FOAM Warning : [8] --> FOAM Warning : [9] --> FOAM Warning : [9] From function Time::readModifiedObjects() [9] in file db/Time/TimeIO.C at line 222 [9] Delaying reading objects due to inconsistent file time-stamps between processors [2] From function Time::readModifiedObjects() [2] in file db/Time/TimeIO.C at line 222 [2] Delaying reading objects due to inconsistent file time-stamps between processors [3] --> FOAM Warning : [3] From function Time::readModifiedObjects() [3] in file db/Time/TimeIO.C at line 222 [3] Delaying reading objects due to inconsistent file time-stamps between processors [4] --> FOAM Warning : [4] From function Time::readModifiedObjects() [4] in file db/Time/TimeIO.C at line 222 [4] Delaying reading objects due to inconsistent file time-stamps between processors [6] From function Time::readModifiedObjects() [6] in file db/Time/TimeIO.C at line 222 [6] Delaying reading objects due to inconsistent file time-stamps between processors [7] --> FOAM Warning : [7] From function Time::readModifiedObjects() [7] in file db/Time/TimeIO.C at line 222 [7] Delaying reading objects due to inconsistent file time-stamps between processors [8] From function Time::readModifiedObjects() [8] in file db/Time/TimeIO.C at line 222 [8] Delaying reading objects due to inconsistent file time-stamps between processors This Messages appears afterwards every time step, but the reconstruction and vtk-export works well at the end. Does anyone know, what kind of problem I face!? I run those calculations over ethernet... Regards! Fabian |
Yup, the time daemon is out of
Yup, the time daemon is out of sync on your machines. Either set up a timeslave to work properly or play around with:
~/.OpenFOAM-1.4.1-dev/controlDict OptimisationSwitches { fileModificationSkew 10; Enjoy, Hrv |
Hi Hrvoje,
thanks! I assume
Hi Hrvoje,
thanks! I assume the given switch accepts a sync problem of 10msec!? Fabian |
Hi Hrvoje,
I've encountere
Hi Hrvoje,
I've encountered the time-stamp problem as well, but it's a bit more mysterious. I'm running Xoodles on 8 cores of a single processor so it really can't be a time daemon problem. I get the time-stamps error when reading/writing files -- not all the time, but enough to make things unpleasant. Sometimes it shows up as an inability to read a file (and openFOAM crashes), other times it just doesn't write one of the files on one of the processors (and I get a 0 length file for whatever variable was writing). reconstructPar fails then. It's very inconsistent, and will not reproduce at the same point in the execution. Where exactly is the controlDict entry to do the fileModificationSkew, just in the controlDict in the system directory of my case, or elsewhere? Thanks in advance! Mike |
Look at:
~/.OpenFOAM-1.4.1-
Look at:
~/.OpenFOAM-1.4.1-dev/controlDict (the path may be adjusted for your version) and search for: OptimisationSwitches { fileModificationSkew 10; If you haven't got this, the equivalent bit in your OpenFOAM installation should be read instead (haven't checked): /home/hjasak/OpenFOAM/OpenFOAM-1.4.1-dev/.OpenFOAM-1.4.1-dev/controlDict Enjoy, Hrv |
Hi all!
I started to run a pa
Hi all!
I started to run a parallel OF 1.4.1 case on a small network (04Pcs). In the past versions, I used LAM/MPI without problelms. Now, when I decompose the case,I can't find the corresponds files on the others nodes and when I run mpirun (openmpi) it fails. what should I indicate in rhe decomposeParDict in the last lines, what is the problem, what is missing? N.B. the SSH works well in different nodes. Djemai |
Hi to all,
I have the same er
Hi to all,
I have the same error message of Fabian: [27] --> FOAM Warning : [27] From function Time::readModifiedObjects() [27] in file db/Time/TimeIO.C at line 222 [27] Delaying reading objects due to inconsistent file time-stamps between processors [36] --> FOAM Warning : [48] --> FOAM Warning : [48] From function Time::readModifiedObjects() [48] in file db/Time/TimeIO.C at line 222 [48] Delaying reading objects due to inconsistent file time-stamps between processors [29] --> FOAM Warning : [29] From function Time::readModifiedObjects() [29] in file db/Time/TimeIO.C at line 222 [29] Delaying reading objects due to inconsistent file time-stamps between processors [37] --> FOAM Warning : [37] From function Time::readModifiedObjects() [37] in file db/Time/TimeIO.C at line 222 [37] Delaying reading objects due to inconsistent file time-stamps between processors [49] --> FOAM Warning : [30] --> FOAM Warning : [38] --> FOAM Warning : [38] From function Time::readModifiedObjects() [38] in file db/Time/TimeIO.C at line 222 [38] Delaying reading objects due to inconsistent file time-stamps between processors I controlled the file ~/OpenFOAM/OpenFOAM-1.4.1/.OpenFOAM-1.4.1/controlDict and I found the section "OptimisationSwitches { fileModificationSkew 10; " that was set to 10 yet. So I tried to change this value into 20 but it didn't work, I have the same error message. The machine that I use is a cluster with 8 nodes with 2 INTEL XEON QUAD CORE for each node. Any suggestion? Thanks, Matteo. |
Hi to all,
sorry I've forgott
Hi to all,
sorry I've forgotten to tell you that I'm working on a mesh with 15000000 of cells. The same case with a coarser mesh (4000000 cells) don't give me any problem. Thanks, Matteo |
error while running in parallel on multi-cpus across nodes
Hi All,
While running a case in parallel I get the following error: PHP Code:
Thank you in advance, Srikara |
Hi,
I saw this quite frequently in the last month as well and assume, that it is a nfs error as Mark mentioned a long time ago. Unfortunately, I have no idea how to get rid of this... :-( Would be great, if you have an idea! Fabian |
Greetings to all!
It's not the first time I've seen reports about this issue with NFS, but I've never been able to reproduce that error with NFS to try and figure out the proper solution. But I do have an idea on how to fix that issue with NFS a while back, but didn't get a reply about it specifically: Quote:
So, if you guys can test these theories, perhaps we can get to the bottom of this problem! Best regards, Bruno |
Hello Bruno,
thanks for the advice! I will check our settings again... would be great if this works. As it occurs only occasionally it might take some days to give you a feedback. Thanks! Fabian Quote:
|
All times are GMT -4. The time now is 20:50. |