[janGaertner]

Hello OpenFOAM Community,


I am excited to announce the release of a new load balancing library designed to enhance the performance of OpenFOAM's reacting solvers in parallel environments. This development is particularly significant for those working with reacting flows, where the computational cost of calculating chemical reaction rates can greatly surpass the costs associated with integrating convective and diffusive transport terms.


Key Features and Benefits:

• Seamless Integration: The library integrates smoothly with any OpenFOAM reacting solver that incorporates finite-rate chemistry, ensuring ease of use and wide applicability.
• Support for Standard and TDAC Models: It is compatible with both the standard and dynamic adaptive chemistry model (TDAC) in OpenFOAM.
• Enhanced Load Balancing: By exchanging information between processes via MPI calls and tracking ODE solution times on a cell level, the library significantly addresses load imbalances.

Performance Validation:
The library has been rigorously validated on various test cases, including extensive simulations on the HLRS Hawk super-computer using up to 8000 cores. Results show that the load-balanced models produce identical results to the original unbalanced models, with substantial speed-up factors:

• Up to 6x speed-up for the standard model.
• Up to 5x speed-up for the TDAC model.

Despite observing non-linear scaling at lower cell counts per processor, the load-balanced models consistently outperform their unbalanced counterparts, making them the preferred choice for reacting flow simulations in OpenFOAM.

Conclusion:
This novel load balancing library is a robust solution for improving the efficiency of reacting flow simulations in OpenFOAM, especially in massively parallel computing environments. I encourage the community to explore and integrate this library into their simulations to take advantage of the significant performance enhancements it offers.
Feel free to reach out for more information or if you have any questions. Your feedback and experiences with the library would be greatly appreciated!


Best regards,


Jan Gärtner


GitHub Link: https://github.com/ITV-Stuttgart/loa...ChemistryModel
Article Link: https://www.sciencedirect.com/scienc...10465524002455

[Pablok]

Dear Jan,
Thanks for your contribution. I tested the library on a simple test case and it worked well.
I have my own custom chemistry model and would like to add it to the list of models available for the load balancing library. Do you know how much of an effort it would be for me to modify the source file for this task?

Best regards,

Pablo

[janGaertner]

Hello Pablo,


The chemistry model is the main part of the library. If you add additional functionalities on top of the StandardModel, hence you can derive from it, it should be fairly straight forward. Otherwise it will require substantial changes to your existing model to use these functions. On the other hand, if you modify the ODE solver it is plug-and-play.



Best to see how it is done for the TDAC model, which builds on top of the StandardModel.



Best,


Jan

[Pablok]

Thanks for your quick answer.

My chemistry model is a copy of the StandardChemistryModel with some modification (it calls Cantera to calculate the reaction rates).
So I was thinking that I could create a class like loadBalancedMyChemistryModel by copying the loadBalancedChemistryModel class, change the name in the files, modify the macro in makeLoadBalancedChemistrySolverTypes.H and try to compile the library.

I'll give it a try and see if it works.

Best,

Pablo

[janGaertner]

Hello Pablo,


it seems you replace OpenFOAMs ODE integration by calling the Cantera function to solve the concentration. This should be then fairly easy to integrate. Copy the loadBalancedChemistryModel, call it e.g., loadBalancedCanteraChemistryModel and then replace in the solveCell function the solve command in Line 392-398 in LoadBalancedChemistryModel.C.


Best of luck,


Jan

[Pablok]

Hi Jan,
I just compiled the load balancing library with my chemistry model and it seems to work. I just tested it on 4 cores for a 1D flame simulation, I guess I have to try it with 100+ cores if I want to observe a speed up of the computational time.
Thanks for your help and feedback!

Pablo

[arijitb]

Hi Jan,
I am trying to implement your chemistry load balancing model for a turbulent combustor. When I used PaSR model with turbulent reaction the load-balanced simulation provides the exact same result as that of the load-unbalanced simulation. The time saved is significant - load balanced approach seemed to be approx. 7/8 times faster. However, when I am trying the implement the EDC model with load balancing, the flame is not established. Can you tell why this issue is occurring and how to solve it?

[janGaertner]

Dear Arijit,

could you provide a simple test case consisting of a system/, constant/, and 0/ directory, which I can use to check what the issue is?
Please also use the GitHub issue tool to document and raise these issues.

Best,

Jan

[arijitb]

Hi Jan,
Thanks for the quick reply. Turns out it simply required a bigger ignition patch to start the flame. It now seems obvious - establishing flame in EDC is much difficult than in PaSR! Thanks again for building this tremendously useful model.