CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > ANSYS > FLUENT

Fluent error every time opening a large data file (>10 GB) with Discrete Ordinate

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   December 2, 2022, 06:22
Default Fluent error every time opening a large data file (>10 GB) with Discrete Ordinate
  #1
New Member
 
Join Date: Jan 2021
Posts: 4
Rep Power: 5
fluidizedbed is on a distinguished road
Dear all,


I have been working with a Discrete Ordinate radiation model (DOM) on top of using species and volumetric reaction model using a UDF. Due to the large mesh size (cell counts) as well as high number of directional discretization used in the DOM, my case files can have a size of 100ish MB while the data files range from 10 GB to 60 GB.


In the HPC cluster, normally I use 128 cores to do calculations in Fluent. The cluster is AMD EPYC 7H12 2x64 cores with 512 GB RAM running in Linux. Typically I often do calculations and save the cas and dat file and then reopen to continue calculations later. Also this is used to do step-by-step calculation iterations (flow->radiation and energy->species & flow), some sensitivity testings, or testing the solver settings.


The problem now is I can save the cas and dat file after doing some calculations but I cannot open the dat file when I want to open them in any future sessions.

There is always an error that causes Fluent process to stop. The error message in the TUI says:



Code:
Reading from nodeXXXX.XXXX.os:".../datafile.dat.h5" in NODE0 mode ...


  Reading results.
(cx-use-window-id 51)

==============================================================================

Node 0: Process 3281202: Received signal SIGSEGV.

==============================================================================
 The fluent process could not be started.
I have tried several things with my findings below:
  • I did some trials to open the data file using various no of CPU. Nothing works except when opening the cas and dat file with 1 CPU. The problem with opening the dat file using 1 CPU is that any calculation that is made after that will be extremely slow (1 CPU vs 128 CPUs)
  • I tried changing all combination of Interconnects and MPI types setting in the Fluent launcher for the Parallel setting. The default settings seem to be the best one (default for Interconnects and MPI types), also when tried to open with 1 CPU.
  • I tried to add a compiler directive to parallelize the UDF, compile and load the parallelized UDF, redo calculations, and save as a new cas and dat file. I still cannot open the data file using 128 CPUs.
  • I tested single precision setting (normally double precision) and also not use UDMI (normally I need UDMI) but none of them were the problems.


I am a bit running out of ideas and could find out how to solve this.
Anyone here might be familiar with such issues?

Is there any other step(s) that I can do also to help me understand what happens and report this error properly to the System Administrators?
fluidizedbed is offline   Reply With Quote

Old   December 2, 2022, 19:19
Default
  #2
Senior Member
 
CFDKareem's Avatar
 
Kareem
Join Date: Nov 2022
Location: New York
Posts: 123
Rep Power: 3
CFDKareem is on a distinguished road
Hello, I have received a similar SIGSEGV working with very large meshes. I solved my problem by unselecting my GPU in the Fluent launcher. For my problem I believe it was exceeding the GPU VRAM.

I know you didn't mention you were using a GPU for computing, but hope it may help!

-Kareem
CFDKareem is offline   Reply With Quote

Old   December 3, 2022, 12:10
Default
  #3
New Member
 
Join Date: Jan 2021
Posts: 4
Rep Power: 5
fluidizedbed is on a distinguished road
Quote:
Originally Posted by CFDKareem View Post
Hello, I have received a similar SIGSEGV working with very large meshes. I solved my problem by unselecting my GPU in the Fluent launcher. For my problem I believe it was exceeding the GPU VRAM.

I know you didn't mention you were using a GPU for computing, but hope it may help!

-Kareem
Hello Kareem, thanks for the reply!

My setting has been always with 0 GPU. So, this mean it is already done without GPU I suppose?
fluidizedbed is offline   Reply With Quote

Old   December 3, 2022, 19:19
Default
  #4
Senior Member
 
CFDKareem's Avatar
 
Kareem
Join Date: Nov 2022
Location: New York
Posts: 123
Rep Power: 3
CFDKareem is on a distinguished road
Quote:
Originally Posted by fluidizedbed View Post
Hello Kareem, thanks for the reply!

My setting has been always with 0 GPU. So, this mean it is already done without GPU I suppose?
Yes, what solved it for me was switching GPU from 1 to 0. So unfortunately my fix isn't going to work for you!

This is now all speculation, but maybe try turning GPGPU computing on if you have a capable card. The DO models is one of the few models that can be accelerated with a GPU. See this link for turning on the acceleration and using NVIDIA MPS: https://cfdresearch.com/ansys-fluent-cfd-post-scripts/

Other than that I am not sure where to go. I'll try to do some more research on your setup and report back if I find anything. I've always struggled to "root cause" SIGSEGV errors.

Good Luck!

-Kareem
CFDKareem is offline   Reply With Quote

Reply

Tags
ansys fluent, data file, discrete ordinate model, error, received signal sigsegv

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
[foam-extend.org] Problems installing foam-extend-4.0 on openSUSE 42.2 and Ubuntu 16.04 ordinary OpenFOAM Installation 19 September 3, 2019 18:13
polynomial BC srv537 OpenFOAM Pre-Processing 4 December 3, 2016 09:07
[OpenFOAM.org] Error creating ParaView-4.1.0 OpenFOAM 2.3.0 tlcoons OpenFOAM Installation 13 April 20, 2016 17:34
[foam-extend.org] problem when installing foam-extend-1.6 Thomas pan OpenFOAM Installation 7 September 9, 2015 21:53
DecomposePar links against liblamso0 with OpenMPI jens_klostermann OpenFOAM Bugs 11 June 28, 2007 17:51


All times are GMT -4. The time now is 06:09.