Full-3D far too slow but not Quasi-3D
 

Hello CFD community,

I am trying to simulate a uniform laminar flow normal to a thin flat plate. It's a lot like the classic circular cylinder case with which we are all familiar, except the geometry of the bluff body is different. When I run quasi-3D simulations, they run very fast, but full-3D simulations on the same mesh extruded take more than 10 times as long - too long to be of any use. I am aware that quasi-3D should be faster, but not by this much. I have tried several different mesh designs, different fluid solvers which use different mathematical techniques, different numbers of cores and different CPU models, but nothing has worked yet. I even noticed the same thing with the circular cylinder. I must be setting up my model wrong because I know much more intense models are possible on very few cores. I just haven't been able to find the bad actor. Also, I need full control over all of my boundary conditions so I require the use of full-3D simulations.

I would be very grateful for any help or insight you can provide.

Best regards

Some questions:


- are you prescribing periodic BCs in the third direction?
- do you see a vortex shedding in the 3D problem but not in the 2d?
- the flow become fully 3D and unsteady?


Note that:
- the stability constraint could require a smaller time step in the fully 3D solver.
- the development of the unsteady flow can be a further reason.


I am not suprised by the fact that the 3D case is slower. However, you should provide all the details of yours simulation.

Thank you for your response. To your questions:
1. Yes. I am prescribing periodic BCs in the third direction for now, but I will change to different BCs in the third direction once I have validated the velocity field statistics.
2. I see vortex shedding in both the 2D and 3D problems.
3. Yes and yes. The flow is fully 3D and unsteady at this Reynolds number (400).

I have some questions about your notes:
1. How much smaller should the time step be in full-3D than in quasi-3D? Also, could you be more specific about what you mean by the "stability constraint"?
2. Why would the unsteadiness in the flow not slow down the quasi-3D simulations to the same degree?

I agree it is not surprising that full-3D is slower than quasi-3D, but not to the extent I've been seeing. I am not sure what other info you would like me to provide about my simulations, so I will give some general info that I feel is important:
- I am currently using a spectral/hp element method solver (Nektar++), doing a p-refinement study at p=5,7,10. At p=10 the full-3D simulation runs out of memory on a cluster I use, regardless of how many nodes I run on. p=10 works fine in quasi-3D.
- For full-3D, p=5 has 3,072,528 degrees of freedom, p=7 has 7,179,008 and p=10 has 18,441,368
- My mesh was created using GMSH
- I selected my time step size based on what the CFL number was converging to. I like to keep it below 1.
- I am solving the incompressible Navier-Stokes equations using direct numerical simulation.

Let me know if there is any other info I can provide. Once again, thank you for your help.

Spectral elements methods are computationally expensive. But your dt depends on the time-integration method, too. At Re=400, the stability limit can be due to the diffusive terms (if explicit) rather then the CFL<=1.
The periodic conditions can produce some communication issue in your cluster, cache miss and so on.

By a physical point of view, I suppose that the shedding you have in the 3D case is no longer two-dimensional.
In conclusion, there are several reasons for a slow run.

I am not sure what you are looking for here. You are doing a study and you are finding the findings.

It is true that higher the order computationally expensive it becomes. It becomes unstable and it uses more and more memory. This is your finding too.

One can not even advise you to use another software here because you are studying p refinement and you are using software that allows it.