CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Post-Processing

Sampling turbulent fluctuation

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   May 6, 2020, 04:08
Default Sampling turbulent fluctuation
  #1
Senior Member
 
Gerry Kan's Avatar
 
Gerry Kan
Join Date: May 2016
Posts: 112
Rep Power: 6
Gerry Kan is on a distinguished road
Howdy Foamers:

I am wondering if there is a way to have OpenFOAM (line) samples *not* generate a dataset for each output interval.

For my validation case I would like to determine basic turbulent statistics (i.e., mean and fluctuation). For this purpose I take defined sampling line in the domain within controlDict. This will generate a dataset for each output interval, in my case each time step.

However, to match my LDA measurements, I need to sample my output at a very high frequency for a fairly long time (like 1 kHz for 1 hour). This is to ensure both the mean (duration) and fluctuations (frequency) are correctly represented in my openFOAM run.

The caveat is that, since with samples, openFOAM creates a directory for each sampling interval, I will end up having 3.6 million directories. I don't know if our Linux Cluster (on the OS side) would allow so many directories to be created, not to mention the time it would take to go through the files in each directory (a lesser problem, I would argue).

Is there another way to have openFOAM concatenate output sample data so that I don't end up having millions of directories, if this is at all possible in any OS?

Thanks in advance, Gerry.
Gerry Kan is offline   Reply With Quote

Old   May 6, 2020, 16:22
Default
  #2
HPE
Senior Member
 
Herpes Free Engineer
Join Date: Sep 2019
Location: The Home Under The Ground with the Lost Boys
Posts: 583
Rep Power: 5
HPE is on a distinguished road
I am not sure that sample creates a new directory and file at each write step in OpenFOAM; but, in any case, you can call a systemCall FO which can concatenate two files, and subsequently delete the former one at each time-step. Does it make sense?

FYI: I am sure that the cluster will not allow you more than 100k-500k files/dirs.
HPE is online now   Reply With Quote

Old   May 8, 2020, 03:23
Default
  #3
Senior Member
 
Gerry Kan's Avatar
 
Gerry Kan
Join Date: May 2016
Posts: 112
Rep Power: 6
Gerry Kan is on a distinguished road
Dear Herpes:

A system call in a cluster environment is probably not a very good idea. Lags in disk I/O, and the fact that computation might take place elsewhere and copied back to your working directory, would pose serious challenge in sychronizing your system call with the output. I know that not all clusters do that, some of the ones I use do.

Not to mention, doing anything that involves deleting output during a computation, you risk losing data. You will find yourself pulling out your hair very quickly, especially if your jobs take weeks to complete.

Having said that, I think it might be a better choice to write a script to monitor progress of the OpenFOAM output and move them to sub-folders accordingly. The combination and deletion of the data can take place afterwards, when I know everything is in a safe place.

Gerry.

P.S. - Judging from the links you attached in your signature on your posts, I assume you are commercially affiliated with ESI, and you answering user questions on CFD-online forms part of your regular support activity? If that is the case, I would strongly suggest you revise the way you respond to them. The helpfulness of your comments notwithstanding, they do come across as being rather condescending.

Last edited by Gerry Kan; May 8, 2020 at 04:31.
Gerry Kan is offline   Reply With Quote

Old   May 8, 2020, 05:40
Default
  #4
HPE
Senior Member
 
Herpes Free Engineer
Join Date: Sep 2019
Location: The Home Under The Ground with the Lost Boys
Posts: 583
Rep Power: 5
HPE is on a distinguished road
Edited:

Nobody is paying me to be here (all free work - free time to try to help others as much as I can), and I must refuse your perception/opinion for the 95% of my posts, I'm afraid. Most of the time, it is just international English: see an example potentialFoam, erasing the U and p fields.

Last edited by HPE; May 8, 2020 at 16:50.
HPE is online now   Reply With Quote

Old   May 11, 2020, 07:59
Default
  #5
New Member
 
Join Date: Apr 2020
Location: UK
Posts: 14
Rep Power: 2
Tobermory is on a distinguished road
Perhaps consider taking a different tack. Presumably this is an LES or DNS simulation? How about using the fieldAverage functionObject to calculate mean and mean square statistics, and then when you have averaged for long enough, run the sample as a postProcessing exercise on the mean and rms fields.

Good luck!
Tobermory is offline   Reply With Quote

Old   May 12, 2020, 04:07
Default
  #6
Senior Member
 
Gerry Kan's Avatar
 
Gerry Kan
Join Date: May 2016
Posts: 112
Rep Power: 6
Gerry Kan is on a distinguished road
Dear Tobermory:

Thanks. Yes, it is an LES simulation. The fieldAverage function will come in handy, but at the moment I need to extract point measurements for mean and fluctuations along different line stations. I am still in the process of figuring out what to do.

Thanks, Gerry.
Gerry Kan is offline   Reply With Quote

Old   May 12, 2020, 04:14
Default
  #7
New Member
 
Join Date: Apr 2020
Location: UK
Posts: 14
Rep Power: 2
Tobermory is on a distinguished road
Understood. Here's the thing though (and I only realised this recently) - you can always run your sampling as a post processing activity, choosing what time step(s) you want to run it on. You don't have to do your sampling during the run (unless you want to save the time history of the samples). For example, with pisoFoam, you can run pisoFoam -postProcess. When you do that, instead of continuing the run, it just runs any postprocessing function objects.

So, if you have run your LES simulation for a while and have developed a time-averaged field at say time t1, you can then play around with as many different samples as you wish, all on the t1 data fields, as a postprocessing activity.

Just a suggestion. May not work for you, but have a think.
Tobermory is offline   Reply With Quote

Old   May 13, 2020, 04:30
Default
  #8
Senior Member
 
Gerry Kan's Avatar
 
Gerry Kan
Join Date: May 2016
Posts: 112
Rep Power: 6
Gerry Kan is on a distinguished road
Quote:
Originally Posted by Tobermory View Post
Understood. Here's the thing though (and I only realised this recently) - you can always run your sampling as a post processing activity, choosing what time step(s) you want to run it on.
Hi Tobermory:

I have used this feature before, and this requires you to have the solution already computed and saved, so that it is available for sampling / probing for the time requested. If you are talking about 1 ms for an hour, it's going to be a lot of data. In fact, much worse than having just 3.6 milliion line sample data.

In the meantime I talked to my cluster manager. He believes that the OS *should* be able to handle 3.6 million directories, but the only issue is how long it takes to list all of them. Previously I had a case with only 360k directories (I think, I was sampling at 10 ms) and it took "a long time" (again I didn't take a stop watch, but it was very long) for an ls operation in my script to retrieve the listing.

The compromise seems to be to have another process monitor the postProcessing output and move them to a directory tree structure, each with a more manageable number of subdirectories containing the sampled data.

Thanks for your suggestions, though, Gerry.
Gerry Kan is offline   Reply With Quote

Old   May 13, 2020, 04:33
Default
  #9
New Member
 
Join Date: Apr 2020
Location: UK
Posts: 14
Rep Power: 2
Tobermory is on a distinguished road
Understood - sounds like you have a good plan. Good luck!
Tobermory is offline   Reply With Quote

Reply

Tags
sampling frequency

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Oscillating Flat plate in turbulent region Shiv1510 Main CFD Forum 7 May 5, 2020 03:36
Problem with divergence TDK FLUENT 13 December 14, 2018 06:00
3D velocity fluctuation profiles from a file in initialization section using LES. Aramisss CFX 0 July 8, 2015 16:24
Turbulent Boundary Layer on a Flat Plate Hoshang Garda FLUENT 1 November 27, 2013 10:24
low turbulent reynold number C-CARE CFX 4 March 12, 2004 06:36


All times are GMT -4. The time now is 08:44.