> But you'd have to load N iframes for N vote buttons on page load.
No you don't. Use a FORM with a target of the iframe, make each arrow an <input type=image name="id#">
One form can cover all the arrows at once. Or you can have a form per set of arrows. Either way you only need one iframe.
Making it hide is a bit more complicated:
One way is have it hide on focus using visiblity, then set a very long transition in the CSS rules to unhide.