Here the English language obscures the physics. Sure, the black line on the track is at a constant elevation. But the tire's point of contact is different from the system's center of mass (CoM). CoM is key here. When a rider tilts in the turns, the CoM lowers. In the straights, it raises. So, you _are_ going up and down during the hour record.
The question now becomes: how much effect does this elevation change have?
It is one thing to be aware of the effect, run the calculations, and find the result is negligible. Has anyone done this? That would be an interesting analysis, and I'd like to see it.
With this in mind, I will make another claim: for a particular rider, there is an ideal line around a velodrome that would minimize center-of-mass elevation change. This line would be faster than the current black line. How much faster? This would be a fun simulation problem.
Another interesting connection: center of mass and bicycling explains why pumping works on a BMX track, a pump track, a trail, and so on. (There are other mainstream explanations, but I think the CoM explanation is the most elegant.)