[Deedledot]

Hello,


Thanks to the help of this forum, I was able to successfully build and setup a dual EPYC workstation!


I am getting some weird fan behavior that I want to make sure I understand before running anything particularly demanding. After booting, the fan RPM seems to smoothly ramp up and down in a cycle with a period of about 10 seconds.


The H12DSi-N6 does not seem to have fan options in the BIOS. I have the CPU2 cooler plugged into FAN2, CPU1 cooler plugged into FAN6, and the Enthoo Pro case fan controller plugged into FANB. I am not certain if all fans are oscillating or just the case fans.


How can I get the fans to operate at a more consistent RPM?
What is the best way for me to monitor sensors and modify fan settings while running this system with Fedora?


I apologize if these are naive questions.

[flotus1]

It's a very common issue with Supermicro boards. The solution is to lower the fan thresholds via IPMI

https://calvin.me/quick-how-to-decre...-fan-threshold
or
https://www.informaticar.net/supermi...ard-loud-fans/

[Deedledot]

Originally Posted by flotus1

It's a very common issue with Supermicro boards. The solution is to lower the fan thresholds via IPMI

https://calvin.me/quick-how-to-decre...-fan-threshold
or
https://www.informaticar.net/supermi...ard-loud-fans/

I tried that first link and got some weird results. According to the ipmi web interface, the thresholds are now lower, but not what I set them to, and the RPM is now stuck on 1400. It also says one of the fans is critical for some reason, even though it’s spinning and clearly within thresholds. I will try again after I get off work in a few hours. Let me know if you have any ideas about what could be going wrong.

[flotus1]

I'm fairly certain this is what's going on here. I had the same issue with my Supermicro H11DSI and solved it with that. And there are plenty of people on the internet with the same problem.

What happens is this: the SM software detects a fan spinning lower than the critical limit. As a result, all fans are set to 100% for a while to unstuck them and prevent damage. Fan speed then returns to normal, and the check is repeated every 10s. Which leads to the cycling behavior when slow-spinning fans are connected.

Maybe you just had a typo in your commands. Here are some commands to read the current settings, that should help you figure out what went wrong:
https://blog.pcfe.net/hugo/posts/201...pyc-ipmi-fans/

[Deedledot]

Ok I did more testing. I set the lower thresholds to 0 100 200. IPMI then says Low NR and Low CT are 0 and 140 respectively. It doesn't seem to let me raise or lower these thresholds after this. I get a confirmation that it changes in terminal but then when I run ipmitool sensor it still shows 0 and 140. This persists after a reboot as well.


I reset my BMC, which made the fans start surging up and down again for the reasons mentioned. I then set the thresholds again. Low NR and Low CT are stuck at 0 and 140, but the RPMs are both reasonable this time and I do not have any "critical" fans anymore.


So my fans seem to be working properly now, but I can't change the threshold anymore and I am not sure why.



FanThresh.png

[flotus1]

No idea about that, sorry. I just set these limits once, and left it there when it did what I wanted.
Maybe setting one of them to 0 is a problem. or maybe SM changed the interface slightly this time around

[Deedledot]

Originally Posted by flotus1

No idea about that, sorry. I just set these limits once, and left it there when it did what I wanted.

Maybe setting one of them to 0 is a problem. or maybe SM changed the interface slightly this time around

I might try other IPMI tools eventually out of curiosity, but I don’t think I really have any need to change it again.

Thanks for all the help! I really appreciate it. Without your advice on this forum, I would have never been able to put together such a fantastic system.

[wylie_uk]

i had the same issues and i resolved... here's how.


the IPMI path is a no good unless your running some very old firmware (not recommended) in the latest firmware the critical low fan speed is hard coded, im guessing supermicro dont want people burning out stuff and blaming them


The issue for me was specifically the Noctua 120mm CPU fans, they have a very low min rpm and were triggering the critical low alert. - but it could be any fan doing it.



It wasn’t obvious which fans were causing the pulsing, as all the fans as a whole pulsed up and down when only one was critical, (BMC health logs help me track it down).


i used Arctic P14 Max chassis fans and Arctic P12 Max for CPU fans, these worked/scaled correctly without triggering a low critical alert and are quiet when low and good airflow when maxed.


Supermicro support informed me that the fan minimum speed cannot be less then 700RPM. they suggested (unoffically https://gelidsolutions.com/product/silent-14-pwm-black/ - which does work but not very good airflow)


as a hack you could isolate the offending fan, and remove the 4th PWM wire. then that fan will run flat out all the time whilst the other fans will scale and mobo wont bitch.

[flotus1]

Thanks for the info, I had no idea.
A bit of a dick move by supermicro. No wonder their attempt at breaking into the market for consumer motherboards was short-lived. There were probably more reasons, but being overly restrictive just doesn't fly there.
It's a shame there aren't more options for workstation type boards with AMD epyc. Asrock delivers a slightly better user experience here, at least they have somewhat better fan control.

[wkernkamp]

On my supermicro boards the FANA and FANB are CPU fans and the others case fans. You don't have FAN A in use and maybe the FANB does not actually connect to a fan for CPU2? I could see how an odd control loop behavior could develop when "CPU" fans do not influence CPU temp.

[JBeilke]

I also had this behaviour after changing all fans to Noctua.
The solution was a fan hub (Noctua NA-FH1) together with a fan controller (Noctua NA-FC1). Now, for the first time, the machine runs almost silently even under full load. The whole thing, including the fans, cost maybe 200 euros, but it was probably my best investment in years :-)


https://t.me/wildkatze_cfd/44