[juxw28]

Hi,

I would like to set up a closed-loop interaction in OpenFOAM to perform active flow control. The idea is to reproduce some recent results of active flow control using deep reinforcement learning:

https://www.researchgate.net/publica...e_flow_control

but using OpenFOAM instead of Fenics as a CFD solver. Changing to OpenFOAM as a solver should also give much better performance and allow to study more complex cases.

This means that one needs to implement the classical DRL framework:

- 1 prepare for the training, i.e. start the training agent, and the CFD framework
- 2 for each episode (i.e. individual simulation used for training):
- 2.1 perform a flow reset (i.e., read from pre-defined initial conditions)
- 2.2 for each agent action to be used for control:
- 2.2.1 provide a state estimate, i.e. pressure or velocity at a number of points, to the controlling agent
- 2.2.2 receive the action set by the controlling agent
- 2.2.3 for the number of numerical timesteps within one agent action:
- 2.2.3.1 get the instantaneous control value from an interpolation law and the current and previous action values
- 2.2.3.2 advance the simulation by 1 timestep, applying the control
- 2.2.3 provide to the controlling agent the value of the reward


For an explanation of why there should be an inner loop for converting actions to control, see the discussion here:

https://www.researchgate.net/publica...E_OPTIMIZATION

Are there any examples of an interaction in this kind that is already available / implemented, or anything similar? If not, is there a general way to implement this?

Also, wondering if this could be somehow interfaced with sockets, as this would allow easy interaction with the python code and naturally bridge between the controlling agent (usually written in Python) and the CFD, a bit like what is done there:

https://github.com/jerabaul29/Cylind...rolDRLParallel

[sy_Shane]

Hi,juxw28,

Maybe we are working on the same trade.

I am in trouble for a long time.

I want to find a way to interactive simulation about RL(by python) and OpenFoam and I don't know how to write a sockets, as this would allow easy interaction with the python code and naturally bridge between the controlling agent (usually written in Python) and the CFD.

How about your progress now? If possible, could I beg you to share your idea?

Many thanks and best regards.

[crubio.abujas]

I've seen that in OF7 exists a PatchType called externalCoupledMixed where you can bind OpenFoam a boundary condition with an external software. I have never used this feature, but I think it may help both of you to handle this kind of software collaboration.

I think that the binding is made by file checking. When OpenFoam solves the patch it writes a patch_name.out (in a folder you shall specify or the folder "comms" by default) and waits you a file patch_name.in to be generated. So the idea is that the external code will be periodically checking the comms file and when any new patch_name.out is created, read it, process it, create a patch_name.in and keep waiting until the patch_name.out is updated again. There are some options to define the checkign periodicity, the waiting time and so on, so please check the source to be sure on how to setting this up.

This can be useful for any model that is solelly controlled by boundary conditions. If you have some sources in fvOptions I didin't found any way of doing the same, but for sure the same methodology can be applied.

I think there is an example in "heatTransfer/buoyantSimpleFoam/externalCoupledCavity" where this approach is used, so you can have a better understanting on how to use it.

I hope that help you!