Strangely, I wasn't suggesting the Montgolfier brothers were going to take Gibraltar single-handedly.
There is a huge difference between climbing a wall with seige ladders where once you get a few men across everyone else can follow, and 2-3 men floating in a wooden basket and trying to land (!) in a fortress.
Granted reconnaissance may have been useful, but if after 4 years of seige you haven't worked out your enemy through espoinage and being shot at, a balloon is not going to help.