Loading timesteps comes to grinding halt
 

I've written a CFX POST power syntax (perl) script that loops through all the timesteps in a transient simulation. For each timestep forces acting on some of the geometry are stored in an array which is subsequently written to a text file once the loop has finished.

This works perfectly in the beginning; the first timesteps only take a few seconds to load. Then the whole s*** starts slowing down. After about 50 timesteps each new step takes about 5 minutes to load!

My first theory was that this has something to do with POST heaping up values in the memory or something, but the process memory remains constant. What on earth is happening?

Regards,
Einar


Script:
__________________________________________________ ____

COMMAND FILE:
CFX Post Version = 11.0
END
! my @theCube = ();
! my @theMatrix = ();
! my @theVector = ();
!$numSteps = 1050;
!$numLocations = 42;
!@locationNames = ("S_B3", "S_B2", "S_B1", "TEE_CM_MESH", "S_B13", "CHOKE", "S_B14", "S_B11", "S_B12",
!"S_B10", "S_B9", "S_B8", "S_B7", "S_B6", "S_B5", "S_B4", "TEE_MANIFOLD", "S_B15", "S_S4", "S_S3",
!"S_S2", "S_S1", "S_S5", "VCONE_MESH_PIPE", "S_S17", "S_S18", "S_S20", "S_S19", "ER_PROBE_MESH",
!"S_S16", "REDUCER1", "REDUCER2", "S_S13", "S_S12", "S_S11", "S_S10", "S_S9", "S_S8", "S_S7", "S_S6", "S_S14", "S_S15");
! $folder =">>E:/CFX/Output/";
! for($col = 0; $col < $numLocations; $col++) {
! push(@theVector,0);
! }
! for($row = 0; $row < $numSteps; $row++) {
! push(@theMatrix, [ @theVector ]);
! }
! for($var = 0; $var < 7; $var++) {
! push(@theCube, [[ @theMatrix]]);
! }

! for ($i=0; $i < $numLocations; $i++) {
! $location = $locationNames[$i];
! for ($j=0; $j < $numSteps; $j++) {
> load timestep=$j
! ($TstepVal, $Units) = evaluate( "Time" );
! ($ForceX, $Units) = evaluate( "force_x()\@$location" );
! ($ForceY, $Units) = evaluate( "force_y()\@$location" );
! ($ForceZ, $Units) = evaluate( "force_z()\@$location" );
! ($TorqueX, $Units) = evaluate( "torque_x()\@$location" );
! ($TorqueY, $Units) = evaluate( "torque_y()\@$location" );
! ($TorqueZ, $Units) = evaluate( "torque_z()\@$location" );

! $theCube[$j][$i][0] = $TstepVal;
! $theCube[$j][$i][1] = $ForceX;
! $theCube[$j][$i][2] = $ForceY;
! $theCube[$j][$i][3] = $ForceZ;
! $theCube[$j][$i][4] = $TorqueX;
! $theCube[$j][$i][5] = $TorqueY;
! $theCube[$j][$i][6] = $TorqueZ;
! }
!}

! for ($k=0; $k < $numLocations; $k++) {
! $fileS = $locationNames[$k];
! $fileName = "$folder $locationNames[$k].csv";
! open(fileS,$fileName);
! for ($l=0; $l < $numSteps; $l++) {
! print(fileS "$theCube[$l][$k][0] \t $theCube[$l][$k][1] \t $theCube[$l][$k][2] \t $theCube[$l][$k][3] \t $theCube[$l][$k][4] \t $theCube[$l][$k][5] \t $theCube[$l][$k][6] \n") or die ("FuckUp!!");
! }
! close(fileS);
! }

__________________________________________________ ______

Have a look at the task manager (if windows) or top (if linux) and check whether CFD-Post is gobbling up memory. If it is then you have found a memory leak bug and should report it.

I found a dirty big memory leak bug in CFX-Post V11 which I reported again in the beta of V12 but it did not get fixed. If you load a sequence of CGNS files it does not clear the memory of the old files correctly and it gobbles up memory until it grinds to a halt. To say I was not impressed that this failed to get fixed in the V12 beta is an understatement (what is the point of reporting major bugs if they don't get fixed?). I have now switched almost all my post-processing to Fieldview. End of rant.

Because timestep loop is inside location loop, your script opens and closes transient files 1050*42 times. If location loop is inside timestep loop, transient files will be opened and closed only 1050 times. Will this swap solve the problem? I don't know. But I believe later approach is more efficient. In this approach, each trn file only needs to be opened once and then data at 42 locations are saved. This process is repeated 1050 times. On the other hand, your current script opens each trn file 42 times and data at only one location are saved each time when a trn file is opened.

Correction
 

Thanks to both of you for good points.
I now see that I mistakenly reversed the order of the loops during debugging, longbow. When I tested the script the loops were in the correct order, however.
I just tested the whole thing without reading timesteps, ie just loading a simple timestep at the beginning and performing all operations on this single step. It took just over 2.5 minutes, so there seems to be nothing wrong here.
About memory usage, ghorrocks: The process memory for "PostEngine" goes from roughly 260MB to nearly 1000MB after a few hours of execution, but I have no idea if this explains the problem. The workstation in question has 32GB of memory, so there is plenty to spare.

No, the issue is apparent when the memory keeps climbing when it should not. And if you still have plenty of memory left then it seems unlikely memory problems are causing the issue.

I think longbow is onto the right track, it is a matter of optimising the loops.

Efficiency
 

I let the script run through 1040 timesteps, which took roughly 14 hours. This means that the average timestep loaded in about 48 seconds, and compared to the initial 3 seconds, this is certainly a dramatic increase. Using the 3D array does not affect the efficiency.
The conclusion is that something fishy is going on, but determining what it is could take forever. I have also noted that Perl sucks when it comes to reporting run-time errors. Errors like these are not reported once they occur, but rather after the script has finished and the computer has spent hours trying to execute rubbish. It is a nightmare to debug!

Hi Glen,

I am experiencing the same memry gobbling issue within CFX-Post 12.1, I can't build a 10 s animation as it is eating all of the memory and switches into virtual memory, however I started to think about FieldView, can you pls share your experience about FieldView's pros/cons in compare to CFX-Post?

Cheers,
Ali

Field view is very good at 3D stuff. Much faster and more memory efficient than CFD-Post. Its script language is a little weird but it works and is quite powerful. Fieldview's main shortcoming is it does not evaluate 3D volume integrals properly and you have to do it by a slow and approximate method. But that is the only short coming which comes to mind. It is approximately the same cost as CFD-Post.

But I rarely use CFD-Post now. I set all my scripts up in Fieldview when the memory leak bug (mentioned above) did not get fixed and now it is all set up I won't go back to CFD-Post even if the bug is fixed. Most of my simulations these days use hanging node meshes in Fluent and CFD-Post is not very good with these.