Hardware for OpenFOAM calculation

Eko · February 5, 2018, 03:59

Hey guys,

I need your help. I have to simulate a case using the chtMultiRegionFoam solver and a mesh size of about 2.2 million. The idea was to simulate one whole day, so 86400 seconds. I wanted to ask if anyone did something similar or has some suggestions on which hardware I should use.

Currently I have access to a server with around 100gb RAM and 64 cores (4x16cores, up to 2.7 GHz). This should be enough I guess?

Another question: Does anyone have an idea how long simulations like this usually take? It's my first simulation of that size and I fear that it will take a lot of time till it's finished.

eric · February 5, 2018, 05:12

Simulating a whole day will take a long time. Have you tried running the tutorial for conjugate heat transfer? You will see that the time step in that simulation is around 0.03 s. So a full day is almost 3 million time steps. Of course this can be different in your case depending on the problem so I would run some tests on a smaller grid or fewer time steps and extrapolate from there.

A common way to speed up heat transfer problems is to use the frozen flow assumption, this can be used if the flow is relatively steady so that we can assume that the flow field does not change, only the temperature.

Eko · February 5, 2018, 05:34

Quote:

Originally Posted by eric

Simulating a whole day will take a long time. Have you tried running the tutorial for conjugate heat transfer? You will see that the time step in that simulation is around 0.03 s. So a full day is almost 3 million time steps. Of course this can be different in your case depending on the problem so I would run some tests on a smaller grid or fewer time steps and extrapolate from there.

A common way to speed up heat transfer problems is to use the frozen flow assumption, this can be used if the flow is relatively steady so that we can assume that the flow field does not change, only the temperature.

Well I ran some tests and it takes too much time. I have time and could wait some weeks but it seems that it would take much longer than that.
Currently I am trying to make my mesh more corse but without any success till now.
I did also think about setting a higher Courant Number but I don't know...

I can't use the frozen flow assumption since I have to look at the flow field and the temperature :/

trampoCFD · February 5, 2018, 20:23

Hi Eko, you can use frozen flow: here is how.
1/Run your simulation for 10 min at a time of the day where your boundary conditions are changing the most.
2/run the same simulation but freeze the flow for 1sec, 10 sec, 30sec 1 min, 2min etc... Work out how long you need the flow on and how often you need it on to get exactly the same results as your detailed 10 min simulation.
3/Further reduce your simulation wall time by using different freeze/unfreeze frequency for different rate of change at different time of the day.

Doing the above in STAR-CCM+ is super easy because you can use co-simulation to have a much larger time step in the solid than in the fluid, and can automate the freeze/unfreeze of the flow solver with a very simple macro. You can get many orders of magnitude speed ups with zero loss of accuracy.

Eko · February 6, 2018, 06:54

I will have to look at that first. I am really new to OpenFOAM and it's the first time I read and hear about frozen flow assumption.
Before I can look at that I have to solve my current problem.

Code:

mpirun noticed that process rank 7 with PID 43266 on node cfdserver exited on signal 8 (Floating point exception).

This appears seconds after starting the solver. I am trying to solve this but don't really know where to start or look. I guess it has to do something with the boundary conditions of my fluid regions or the Courant Number. (The fluid temperature decreases which it shouldn't)

trampoCFD · February 6, 2018, 18:18

Hi Eko,
Please submit your openfoam specific questions in the openfoam forum. You'll have a much better chance to get an answer :-)

Eko · February 7, 2018, 05:24

Quote:

Originally Posted by trampoCFD

Hi Eko,
Please submit your openfoam specific questions in the openfoam forum. You'll have a much better chance to get an answer :-)

I know but it doesn't seem necessary anymore. Looks like it's running now. But I will do some more tests and changes to be sure

February 5, 2018, 03:59	Hardware for OpenFOAM calculation	#1
Eko Member Join Date: Dec 2017 Location: Germany Posts: 48 Rep Power: 8	Hey guys, I need your help. I have to simulate a case using the chtMultiRegionFoam solver and a mesh size of about 2.2 million. The idea was to simulate one whole day, so 86400 seconds. I wanted to ask if anyone did something similar or has some suggestions on which hardware I should use. Currently I have access to a server with around 100gb RAM and 64 cores (4x16cores, up to 2.7 GHz). This should be enough I guess? Another question: Does anyone have an idea how long simulations like this usually take? It's my first simulation of that size and I fear that it will take a lot of time till it's finished.

February 6, 2018, 06:54		#5
Eko Member Join Date: Dec 2017 Location: Germany Posts: 48 Rep Power: 8	I will have to look at that first. I am really new to OpenFOAM and it's the first time I read and hear about frozen flow assumption. Before I can look at that I have to solve my current problem. Code: mpirun noticed that process rank 7 with PID 43266 on node cfdserver exited on signal 8 (Floating point exception). This appears seconds after starting the solver. I am trying to solve this but don't really know where to start or look. I guess it has to do something with the boundary conditions of my fluid regions or the Courant Number. (The fluid temperature decreases which it shouldn't) Last edited by Eko; February 6, 2018 at 09:40.

February 6, 2018, 18:18		#6
trampoCFD Member Guillaume Jolly Join Date: Dec 2015 Posts: 64 Rep Power: 10	Hi Eko, Please submit your openfoam specific questions in the openfoam forum. You'll have a much better chance to get an answer :-) flotus1 likes this.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Dynamic mesh adaption in parallel calculation	xh110120	FLUENT	1	March 12, 2016 08:05
Determining the calculation sequence of the regions in multe regions calculation	peterhess	OpenFOAM Running, Solving & CFD	4	March 9, 2016 03:07
Increase in CPU number during the FIRE calculation	amin_u50	AVL FIRE	0	August 6, 2014 08:34
MRF and Heat transfer calculation	Susan YU	FLUENT	0	June 2, 2010 08:46
Warning 097-	AB	Siemens	6	November 15, 2004 04:41

February 5, 2018, 05:12		#2
eric Member Knut Erik T. Giljarhus Join Date: Mar 2009 Location: Norway Posts: 35 Rep Power: 22	Simulating a whole day will take a long time. Have you tried running the tutorial for conjugate heat transfer? You will see that the time step in that simulation is around 0.03 s. So a full day is almost 3 million time steps. Of course this can be different in your case depending on the problem so I would run some tests on a smaller grid or fewer time steps and extrapolate from there. A common way to speed up heat transfer problems is to use the frozen flow assumption, this can be used if the flow is relatively steady so that we can assume that the flow field does not change, only the temperature.

February 5, 2018, 20:23		#4
trampoCFD Member Guillaume Jolly Join Date: Dec 2015 Posts: 64 Rep Power: 10	Hi Eko, you can use frozen flow: here is how. 1/Run your simulation for 10 min at a time of the day where your boundary conditions are changing the most. 2/run the same simulation but freeze the flow for 1sec, 10 sec, 30sec 1 min, 2min etc... Work out how long you need the flow on and how often you need it on to get exactly the same results as your detailed 10 min simulation. 3/Further reduce your simulation wall time by using different freeze/unfreeze frequency for different rate of change at different time of the day. Doing the above in STAR-CCM+ is super easy because you can use co-simulation to have a much larger time step in the solid than in the fluid, and can automate the freeze/unfreeze of the flow solver with a very simple macro. You can get many orders of magnitude speed ups with zero loss of accuracy.