CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Main CFD Forum

Maximum number of nodes in cluster

Register Blogs Members List Search Today's Posts Mark Forums Read

Reply
 
LinkBack Thread Tools Display Modes
Old   April 3, 2001, 10:27
Default Maximum number of nodes in cluster
  #1
Marat Hoshim
Guest
 
Posts: n/a
Hi,

I'd like to know what maximum number of nodes do you use in your clusters for your cfd-jobs.

Does someone has run jobs on 32 nodes ?

Bye,

Marat
  Reply With Quote

Old   April 3, 2001, 11:02
Default Re: Maximum number of nodes in cluster
  #2
Jonas Larsson
Guest
 
Posts: n/a
We're just now installing a new cluster with 100 nodes. To get good scaling on all 100 nodes you need a very large case though.
  Reply With Quote

Old   April 3, 2001, 11:24
Default Re: Maximum number of nodes in cluster
  #3
Marat Hoshim
Guest
 
Posts: n/a
Hi Jonas,

what network technology is required for 100 nodes ? I guess that 100 Mbit/s might be to slow if all 100 nodes do communicate with eachother ?

What's the reason, why small jobs do scale that well on many nodes ? Is it because the ratio of volume to surface is smaller ?

Thanks,

Marat
  Reply With Quote

Old   April 3, 2001, 11:34
Default Re: Maximum number of nodes in cluster
  #4
Jonas Larsson
Guest
 
Posts: n/a
We use std. 100 mbit Fast Ethernet. We considered using a faster network with lower latency (myrinet or equivalent), but the extra cost for this couldn't be justified. However, if you want to be able to use all 100 nodes for medium-size jobs (a million cells or so) then you will have to use something faster than fast ethernet. This isn't very critical for us though since we will only need to use all nodes for extremely large case. Most of the time we will use around 10 to 20 nodes for one case I guess.

The poor scaling for small jobs is because the sub-domains placed on each node becomes too small (high surface/volume ratio) and this gives a lot of overhead communication. For some parellelisation models it might also make the numerics more problematic.

  Reply With Quote

Old   April 4, 2001, 03:46
Default Re: Maximum number of nodes in cluster
  #5
Marat Hoshim
Guest
 
Posts: n/a
Hi Jonas,

will you purchase AMD or Pentium processors ? Have you compared the two processors with a real cfd job ? I'm not quite sure if the SPEC benchmarks are representative for real cfd jobs! (If that would be the case, AMD should be much faster !)

Regards,

Marat
  Reply With Quote

Old   April 4, 2001, 09:00
Default Re: Maximum number of nodes in cluster
  #6
Jonas Larsson
Guest
 
Posts: n/a
We bought PIII 1GHz nodes this time. We got a very good deal on these though - if we would buy for "street prices" the AMD is probably more attractive, especially if you can get one of the new boards with DDR memory. We haven't benchmarked any AMD yet.

Some of the spec tests are quite represantative for CFD I think. But as always you have to be very critical of these numbers - you will, for example, never come close to the performance that spec-numbers given for the P4 since these numbers were produced by intel with special in-house compilers which optimize code for the P4.

There has been a lost of discussion about AMD/P3/P4 on these forums over the last year - if you use the main search tool (top right corner) and search for "AMD" you should get a lot of interesting opinions.
  Reply With Quote

Old   April 5, 2001, 13:17
Default Re: Maximum number of nodes in cluster
  #7
George Bergantz
Guest
 
Posts: n/a
The mobo's with the AMD 761 (or 760) chp set that support DDR ram are not widely available (if at all) and my reading of the Linux forums is that there is no stable version yet for this hardware configuration. The speed-up with DDR ram is, under *optimal* circumstances maybe 10% (see discussions on Tom's Hardware page). While a terrific hardware package it is rather new and one could risk spending a lot of time screwing around trying to get the thing to be stable.

I suggest going with the mature PIII systems as Jonas describes above. He is right that the SPEC-mark's are only guidelines, but the are useful is showing relative performance between platforms, not absolute expected fpu through-put.

Heck the real issues is teh degree to which you can write parallel code, compiler, and bandwidth/latency issues. Those are usually the big hang-up's.
  Reply With Quote

Old   April 9, 2001, 17:11
Default Re: Maximum number of nodes in cluster
  #8
Charles Crosby
Guest
 
Posts: n/a
Looking at some of the floating point benchmarks that have been run on DDR-equipped systems, it appears that the advantage to be gained from DDR is relatively small. However, the parameter that seems to be very significant to CFD work (or any floating point work on largish data sets) is the bus speed. I have experimented with the Linpack benchmark program, and the results give you more or less the same picture as real CFD analysis results (although we have encountered a Windows / Linux performance anomaly, most likely attributable to incorrect compiler settings when the Windows executable was compiled ...)

Real-world sized data sets are much larger than the AMD or Pentium cache memories, and as a result, performance is largely determined by how fast data can be fed to the CPU, through the frontside bus. 133 MHz is much better than 100, which is much better than 66. CPU clock-speed becomes less important, e.g. an 800 MHz CPU does CFD work only about 25% faster than a 400 MHz CPU, if both operate at the same FSB frequency. The AMD's have an advantage here, with their double rate FSB (100 MHz effectively gives you almost 200 MHz), even though the memory bus may only be running at 100 MHz. The VIA KT133A chip set has proved to be astonishingly effective with the AMD processor, because it supports effectively 266 MHz data transfer to the CPU, even when using only ordinary PC 133 SDRAM (see www.anandtech.com). FWIW, the much-maligned Pentium 4 is a very good option to consider, the (effectively) 400 MHz FSB giving it outstanding performance when doing floating point work with big data sets. It does come at a price though ... P4 aside, the other good option must be the AMD Athlon C processors, which use the (effectively) 266 MHz bus speed. (Actual FSB is still only 133 MHz, but data are transferred on the rise and fall of the clock)
  Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Upgraded from Karmic Koala 9.10 to Lucid Lynx10.04.3 bookie56 OpenFOAM Installation 8 August 13, 2011 04:03
snappyHexMesh won't work - zeros everywhere! sc298 OpenFOAM Native Meshers: snappyHexMesh and Others 2 March 27, 2011 21:11
external flow with snappyHexMesh chelvistero OpenFOAM 11 January 15, 2010 20:43
RPM in Wind Turbine Pankaj CFX 9 November 23, 2009 05:05
CFX4.3 -build analysis form Chie Min CFX 5 July 12, 2001 23:19


All times are GMT -4. The time now is 00:21.