You've just described a ballocket. You launch a rocket from a very high altitude balloon.
You gain nothing from the height, because you still need to spend ~10km/s getting up to orbital velocity, but you don't have to push your way through the thick lower atmosphere any more. Rockets don't work very well in atmosphere, so you do get fuel savings due to more efficient engines.
I don't know if this has ever been tried for real.