CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

CFD workstation - memory bandwidth

Register Blogs Community New Posts Updated Threads Search

Like Tree6Likes
  • 1 Post By flotus1
  • 1 Post By vbchris
  • 1 Post By flotus1
  • 1 Post By flotus1
  • 1 Post By fusij
  • 1 Post By fusij

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   December 3, 2016, 12:50
Question CFD workstation - memory bandwidth
  #1
Member
 
Join Date: Nov 2009
Posts: 56
Rep Power: 16
fusij is on a distinguished road
Hello all,

I am buying a workstation machine and I am wondering about the RAM setup.

I intend to buy:
processor: I7-6900K
motherboard: ASUS X99-II
RAM: 32 (4x8) GB DDR4 PC-24000 PATRIOT VIPER 4 @ 3000 MHz

Now I have 8 cores and 3000 MHz * 64 (bits) * 4 (quad channel) = 96 GB/s theoretical memory bandwidth. Dividing this bandwidth on each core under full load gives me 12 GB/s per core.

Total amount of memory is 32 GB and then I have 4 GB per core.

Is this enough memory? Which one is the bottleneck, memory or CPU?
fusij is offline   Reply With Quote

Old   December 4, 2016, 10:18
Default
  #2
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,399
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
An I7-6900k with fast quad-channel memory should provide a fairly reasonable balance between memory bandwidth and raw computing power for a CFD workstation. I don't see an obvious bottleneck here.
Concerning the amount of memory: you are the only person who can answer this question. How much memory you need depends entirely on the software you use and the size of the models you run. However, 32GB are the absolute minimum I would recommend for this CPU. If you want to use 64GB use a 4*16GB setup instead of 8*8GB.
fusij likes this.
flotus1 is offline   Reply With Quote

Old   December 6, 2016, 19:13
Default
  #3
Member
 
Join Date: Nov 2009
Posts: 56
Rep Power: 16
fusij is on a distinguished road
Thank you for your answer flotus1.

What CPU cooler do you recommend? I have been thinking about Noctua NH-D15 but is maybe Noctua NH-U14S sufficient? Are liquid cooling solutions an option here? I see alot of prebuilt systems with the I7-6900K using liquid cooling but I am little afraid about the noise when compared to the Noctua air coolers.

And another thing; If I will be running GF-1060GTX, 1x250GB SSD and 1x2TB HDD, how big should my PSU be? Can I get away with 650 W or should I go for 800 W?
fusij is offline   Reply With Quote

Old   December 6, 2016, 19:58
Default
  #4
Member
 
Chris L
Join Date: Sep 2012
Posts: 53
Rep Power: 13
vbchris is on a distinguished road
I'd recommend this:

https://ca.pcpartpicker.com/product/...r-rr212e20pkr2

In the old Sandybridges I notices throttling when running CFD simulations and moved to water cooling. But those CPU's run really hot.

The new generation of intel are much better and air cooling has worked fine for me.

On the note of PSU's.

Use PC Partpicker to get a wattage estimate then I would look for a PSU with twice the capacity.
I don't normally skimp on PSU, get something gold rated from a reputable brand.
fusij likes this.
vbchris is offline   Reply With Quote

Old   December 7, 2016, 03:54
Default
  #5
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,399
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Originally Posted by fusij View Post
What CPU cooler do you recommend? I have been thinking about Noctua NH-D15 but is maybe Noctua NH-U14S sufficient? Are liquid cooling solutions an option here? I see alot of prebuilt systems with the I7-6900K using liquid cooling but I am little afraid about the noise when compared to the Noctua air coolers.

And another thing; If I will be running GF-1060GTX, 1x250GB SSD and 1x2TB HDD, how big should my PSU be? Can I get away with 650 W or should I go for 800 W?
Noctua Coolers are always a good choice. For moderate or no overclocking at all the U14S will do. Liquid cooling is not necessary unless you are planning on overclocking the CPU to a power consumption of more than ~200W.
Any decent quality power supply with ~500W will do. My preferred choice would be a BeQuiet P11 550W. In an unrealistic maximum load scenario you have 200W for the CPU (overclocked), 120W for the GPU, <40W for memory (8 DIMMs) and <50W for the rest of the system. More than 550W is really not necessary.
fusij likes this.
flotus1 is offline   Reply With Quote

Old   December 15, 2016, 13:13
Default
  #6
Senior Member
 
M. C.
Join Date: May 2013
Location: Italy
Posts: 286
Blog Entries: 6
Rep Power: 16
student666 is on a distinguished road
Hi Fusij,

Quote:
I intend to buy:
processor: I7-6900K
motherboard: ASUS X99-II
RAM: 32 (4x8) GB DDR4 PC-24000 PATRIOT VIPER 4 @ 3000 MHz
Why did you buy RAM: 32 (4x8) GB DDR4 PC-24000 PATRIOT VIPER 4 @ 3000 MHz if i7-6900K only support 2400MHz maximum?
http://ark.intel.com/it/products/941...up-to-3_70-GHz

Quote:
Now I have 8 cores and 3000 MHz * 64 (bits) * 4 (quad channel) = 96 GB/s theoretical memory bandwidth. Dividing this bandwidth on each core under full load gives me 12 GB/s per core.

Total amount of memory is 32 GB and then I have 4 GB per core.
Are your calcs correct?
I mean: 3000*64*4=768000/8=96Gb/s why you should divide by 8 to consider each core (12Gb/s)?

Thanks Michele
student666 is offline   Reply With Quote

Old   December 15, 2016, 17:57
Default
  #7
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,399
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Originally Posted by student666 View Post
if i7-6900K only support 2400MHz maximum?
http://ark.intel.com/it/products/941...up-to-3_70-GHz
DDR4-2400 is the lower limit that is guaranteed by Intel. Faster RAM is usually no problem with the I7 processors.
flotus1 is offline   Reply With Quote

Old   December 15, 2016, 19:18
Default
  #8
Senior Member
 
M. C.
Join Date: May 2013
Location: Italy
Posts: 286
Blog Entries: 6
Rep Power: 16
student666 is on a distinguished road
Quote:
DDR4-2400 is the lower limit that is guaranteed by Intel. Faster RAM is usually no problem with the I7 processors.
Thanks for answering.

Checking this:
http://www.intel.com/content/www/us/...eet-vol-1.html

you may see at section 1.3.1 that intel only support up to 2400 MT/s (why MT/s and not MHz? is the same?)

and on:

http://www.intel.com/content/www/us/...eet-vol-2.html

you may see on section 4.4.3 that higher transfer rate are "reserved".

So, does it come from your experience? have you tested higher transfer rate?

Thank you
student666 is offline   Reply With Quote

Old   December 15, 2016, 19:25
Default
  #9
Senior Member
 
M. C.
Join Date: May 2013
Location: Italy
Posts: 286
Blog Entries: 6
Rep Power: 16
student666 is on a distinguished road
Just another question,

ASUS X99-II tech specification said that it supports only windows OS, so if I have to use linux what is the alternative solution? is there a comparable motherboard that can support both OS with similar tech specification?

https://www.asus.com/it/Motherboards...pecifications/

Thanks

Michele
student666 is offline   Reply With Quote

Old   December 17, 2016, 10:50
Default
  #10
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,399
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Originally Posted by student666 View Post
Thanks for answering.

you may see at section 1.3.1 that intel only support up to 2400 MT/s (why MT/s and not MHz? is the same?)

So, does it come from your experience? have you tested higher transfer rate?
DDR memory effectively performs two transfers per cycle. Hence the name. This means that DDR4-2400 (see how I try to avoid a MHz suffix here) actually runs at 1200MHz. Nevertheless it is able to perform 2400MT/s as if it was running at 2400MHz SDR.
I have no first hand experience with Broadwell-E processors. I will let you decide if this invalidates the advice I give.

Quote:
Originally Posted by student666 View Post
Just another question,

ASUS X99-II tech specification said that it supports only windows OS, so if I have to use linux what is the alternative solution? is there a comparable motherboard that can support both OS with similar tech specification?
You should not worry too much about missing official Linux support. The X99 chipset has been around for long enough that most common Linux distributions will just run out of the box with every feature you need. I am not a Linux expert to say the least but I still got it running on every single piece of hardware I tried so far, no matter if it was officially supported or not.
Missing official Linux support for consumer hardware is just a question of numbers. Less than 2% of all PCs run on Linux, so for most hardware manufacturers supporting it officially would be a waste of effort and money.
student666 likes this.
flotus1 is offline   Reply With Quote

Old   December 18, 2016, 05:45
Default
  #11
Member
 
Join Date: Nov 2009
Posts: 56
Rep Power: 16
fusij is on a distinguished road
Just bought the system.

Setting up Ubuntu was not completely straightforward since you have to disable secure boot in BIOS (delete PK key) before using Nvidia drivers for the GTX1060.

I enabled the XMP onboard switch on the ASUS X99-A II and now I have 32GB running @ 3200 MHz.

There is one thing that is bothering me. It takes the computer ca. 10 seconds to load up the boot screen and then it loads up really fast. I saw somewhere that it was memory related, takes longer for the motherboard to prepare the memory for booting when you have large bulk of memory. You know something about that Flotus1?
flotus1 likes this.
fusij is offline   Reply With Quote

Old   December 18, 2016, 05:59
Default
  #12
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,399
Rep Power: 46
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
X99 motherboards take longer to post than most smaller chipsets like Z170, especially with larger amounts of memory. C612 takes even longer. I can hardly imagine that these 10 extra seconds really mess up your productivity. There is not much you can do about it anyway.
flotus1 is offline   Reply With Quote

Old   December 18, 2016, 06:06
Default
  #13
Member
 
Join Date: Nov 2009
Posts: 56
Rep Power: 16
fusij is on a distinguished road
You are right, it is just that I am not used to it. Extra 10 seconds are nothing compared to the total usage time

Thank you for your help Flotus1, I am really pleased with my new system.
fusij is offline   Reply With Quote

Old   December 20, 2016, 06:35
Default
  #14
Senior Member
 
M. C.
Join Date: May 2013
Location: Italy
Posts: 286
Blog Entries: 6
Rep Power: 16
student666 is on a distinguished road
Hi Fusij,
what is the software CFD your're planning to use with this hardware configuration?

Thanks

Regards

Michele
student666 is offline   Reply With Quote

Old   December 20, 2016, 06:44
Default
  #15
Member
 
Join Date: Nov 2009
Posts: 56
Rep Power: 16
fusij is on a distinguished road
Quote:
Originally Posted by student666 View Post
Hi Fusij,
what is the software CFD your're planning to use with this hardware configuration?

Thanks

Regards

Michele
I am using OpenFOAM
student666 likes this.
fusij is offline   Reply With Quote

Old   December 20, 2016, 13:28
Default
  #16
Senior Member
 
M. C.
Join Date: May 2013
Location: Italy
Posts: 286
Blog Entries: 6
Rep Power: 16
student666 is on a distinguished road
Can someone suggests me where should I look for some motherboard with linux - intel compability?

I would like to set up a workstation with similar hw to fusij:

processor: I7-6900K
motherboard: ???
RAM: 64 (4x16) GB DDR4 > 2400 MHz

but I would like don't have problem with linux.

thanks

Michele
student666 is offline   Reply With Quote

Old   December 20, 2016, 14:40
Default
  #17
Member
 
Join Date: Nov 2009
Posts: 56
Rep Power: 16
fusij is on a distinguished road
Quote:
Originally Posted by student666 View Post
Can someone suggests me where should I look for some motherboard with linux - intel compability?

I would like to set up a workstation with similar hw to fusij:

processor: I7-6900K
motherboard: ???
RAM: 64 (4x16) GB DDR4 > 2400 MHz

but I would like don't have problem with linux.

thanks

Michele
I would recommend asus x99-A II, the board that I bought.

Sent from my SM-G920F using CFD Online Forum mobile app
fusij is offline   Reply With Quote

Old   December 20, 2016, 15:10
Default
  #18
Senior Member
 
M. C.
Join Date: May 2013
Location: Italy
Posts: 286
Blog Entries: 6
Rep Power: 16
student666 is on a distinguished road
ok, thanks, i'll have a closer look.

thx

Regards

Michele
student666 is offline   Reply With Quote

Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
CPU for new CFD workstation Andrew Norfolk Hardware 3 February 18, 2016 17:36
CFD Design...The CFD Future John C. Chien Main CFD Forum 20 November 19, 2015 23:40
Workstation for HPC CFD tubois Hardware 5 November 13, 2015 14:36
Future CFD Research Jas Main CFD Forum 10 March 30, 2013 12:26
PC vs. Workstation Tim Franke Main CFD Forum 5 September 29, 1999 15:01


All times are GMT -4. The time now is 10:23.