CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > CFX

CFX parallel multi-node jobs fail w/ SLURM on Ubuntu 10.04

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   February 17, 2012, 07:20
Default CFX parallel multi-node jobs fail w/ SLURM on Ubuntu 10.04
  #1
New Member
 
Daniel Petersen
Join Date: Feb 2012
Posts: 6
Rep Power: 5
danieru is on a distinguished road
Hi,

I help maintain a cluster which we've just installed ANSYS 14.0 on, for one of our users. We're running Ubuntu 10.04 LTS with SLURM for job control.

The installation had a couple of 'unexpected operators' errors. Focusing on CFX first, despite the install errors we've successfully run interactive pre, solver and post, as well as single node solver parallel jobs via the job scheduler SLURM, so we're mostly there.

The problem is trying to run parallel jobs on multiple nodes via SLURM. When we submit a multiple node job to SLURM, the job fails, complaining that it cannot connect via RSH.

This isn't surprising since we don't use RSH, it's too insecure, so the question is: Does anyone know how best to setup/allow CFX parallel to play nice with SSH/SLURM?

As a side note, CFX seems to be scaling terribly when running in parallel, in the context of CPU utilization at least. As an example, our nodes each have 4 CPUs, 12 cores each: when running CFX with 4 processes on the node, each process is only utilizing 25% of the core it's running on. At 8 processes, it's only 8% per core, at 48 processes, it's reported down at 1 or 2% per process! This is with no other processes competing for CPU cycles. Can anyone comment on their experience with how well the CFX parallel solver scales? Perhaps it's a configuration issue on our side? If this is the best CFX can do, then there's no point in worrying about trying to get multi-node parallel computing to work...
danieru is offline   Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to Run Parallel Jobs in Rocks Cluster asaha OpenFOAM Running, Solving & CFD 12 July 4, 2012 22:51
CFX 13 Linux (Ubuntu 11.10) local parallel setup TSokl ANSYS 0 January 16, 2012 08:35
[ICEM] Node Number vs. Node_ID & ICEM vs. CFX Araz ANSYS Meshing & Geometry 1 April 25, 2011 11:03
CFX parallel jobs on LSF Dominik CFX 3 August 7, 2007 07:19
CFX Parallel jobs with LSF Terry Suckling CFX 0 December 9, 2005 06:37


All times are GMT -4. The time now is 04:27.