I replaced all 4 of them with Noctua NF-A4x20s, wired to run at full speed all the time. They still report their speed so the IPMI management interface doesn't consider the power supply fan to have failed, but the PSU can no longer control the fan's speed.
The PSUs don't run any hotter and I can't hear them now.
I have a used Eaton PW9130 UPS in the bottom of the rack. The 80mm (exhaust) and 60mm (inverter heatsink) fans were likewise louder than I'd like. I replaced them with Noctuas too, again wired to run at full speed all the time, and the UPS' Web/SNMP card confirms it's still no hotter than 30'C internally. I can't hear that now either.
Hilariously, the most critical fan, the original inverter heatsink fan, is a 2-pin fan, so it probably can't even detect when it has failed (unless it's detecting fan failure by monitoring current consumption). The original rear exhaust fan uses a locked rotor sensor rather than a tachometer, which required a bit of bodging to convince the UPS that it has not failed. Oh well.