Talk about twisting the argument. The Manhattan project was in competition with the Soviets and the Nazis, it was less a result of some unified front of all the smartest scientists in the world, and more of a result of "if we don't succeed first, someone much worse probably will".
Your logic of "if there were 10 of them simultaneously, it wouldn't have worked out" is flawed and self serving in that if you ignore all competition and just divide a single competitor into any number of smaller entities, of course at some point they'll be too small to be viable.
