CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > SU2

SU2 and ParMETIS

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   February 4, 2014, 12:34
Default SU2 and ParMETIS
  #1
New Member
 
Ben N.
Join Date: Dec 2012
Location: San Jose, CA
Posts: 1
Rep Power: 0
bensky is on a distinguished road
Greetings,

I am running into a memory limit when running SU2_DDC on a ~40M cell grid (SU2 version 3.0). If I understand correctly, because SU2_DDC/METIS only runs serially, there is an upper bound for the allocated RAM per CPU and thus size of grid to be decomposed. Here is the error I get when SU2_DDC is called:

------------------------ Divide the numerical grid ----------------------
[METIS Fatal Error] ***Memory allocation failed for METIS_MESHPARTNODAL: adjncy. Requested size: -1385723056 bytes

A 12M cell grid partitions and solves without issue. I have heard that it may be possible to utilize ParMETIS instead of METIS to parallelize the partitioning process in SU2_DDC? If so, is there any instructions/documentation on how to build SU2 with ParMETIS?

Thanks in advance and best regards,

Ben
bensky is offline   Reply With Quote

Old   February 4, 2014, 21:58
Default
  #2
Super Moderator
 
Francisco Palacios
Join Date: Jan 2013
Location: Stanford, CA
Posts: 301
Rep Power: 5
fpalacios is on a distinguished road
Hi Ben,

You are right. It seems that you don't have enough memory in your master node to perform the partition (we have done larger simulations but with more memory in the master node).

ParMETIS is a future plan for SU2, but we are not still there. So, at this point the quickest thing that you can do is to use a machine with more RAM memory. Another possibility, if the grid is easy to divide, is to bypass METIS and implement a geometry based grid partitioner, or run METIS several times in different parts of the domain.

The call to METIS is very simple (void CPhysicalGeometry::SetColorGrid(CConfig *config)
in geometry_structure.cpp)… you will see that METIS is nothing more than a method to color the grid, so you can modify the subroutine to reduce the size the grid that METIS is using.

Thanks for using Su2,
Best,
Francisco
fpalacios is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 03:34.