RMS thinks that free hardware is a different matter than free software, on the basis that software is easily copyable but hardware is not.
If a piece of hardware contains unmodifiable software, RMS considers it an "integral part of the hardware". It is no different to him than a cog or any other physical piece in that thing.
In contrast, if said software (firmware) is replaceable, it means that it can be copied at no cost, modified, used on other hardware, etc.. It is a fine line, but I can see some difference.