On a nice track, assuming a perfectly smooth surface and zero elevation change, I'm willing to accept the effect may not matter enough to care. But introduce even just a little bumpiness or some elevation change (perhaps in the track curves), and it might matter for someone pursuing the hour record.
However, cycling tracks are designed to be very smooth which is why high pressure tyres are still used there.
Here the English language obscures the physics. Sure, the black line on the track is at a constant elevation. But the tire's point of contact is different from the system's center of mass (CoM). CoM is key here. When a rider tilts in the turns, the CoM lowers. In the straights, it raises. So, you _are_ going up and down during the hour record.
The question now becomes: how much effect does this elevation change have?
It is one thing to be aware of the effect, run the calculations, and find the result is negligible. Has anyone done this? That would be an interesting analysis, and I'd like to see it.
With this in mind, I will make another claim: for a particular rider, there is an ideal line around a velodrome that would minimize center-of-mass elevation change. This line would be faster than the current black line. How much faster? This would be a fun simulation problem.
Another interesting connection: center of mass and bicycling explains why pumping works on a BMX track, a pump track, a trail, and so on. (There are other mainstream explanations, but I think the CoM explanation is the most elegant.)