For the average case it probably doesn't matter, and you can optimize it, but I think it is totally understandable that the average novice could end up with bad https performance if only because the defaults are bad or they made a mistake. If hardware assist for the handshake and/or transfer crypto is shut off (or unavailable, on lower-spec CPUs) your perf is going to tank real hard.
I ended up using ssh configured to use the weakest (fastest) crypto possible, because disabling crypto entirely was no longer an option. I controlled the entire network end to end so no real risk there - but obviously a dangerous tool to provide for insecure links.
Also worth keeping in mind that there are production scenarios now where people are pushing 1gb+ of data to all their servers on every deploy - iirc among others when Facebook compiles their entire site the executable is something like a gigabyte that needs to be pushed to thousands of frontends. If you're doing that over encrypted ssh you're wasting cycles which means wasting power and you're wasting that power on thousands of machines at once. Same would apply if the nodes pull the new executable down over HTTPS.