This is a fundamentally different situation than a backdoor in a parameterized encryption standard, such as ECDSA (which is often referenced in these discussions): there, only the people who built the backdoor can use the backdoor. Here, the backdoor exists in a shared resource, waiting for others--including your enemies--to take advantage of; that's quite a risk, and unless you've been seeing some weird behavior--such as the NSA distributing heartbeat-disabled builds of OpenSSL for any potential government usage--I think it is a horrible stretch to believe that they've been sitting on this bug (or even having themselves planted the bug), using it as the long-term surveillance means that some people seem to be want to believe.
Frankly, the fact that they've been logging SSL traffic is enough: for systems without perfect forward security, if they don't already have the keys through other means, they just wait for an opportunity like yesterday and then attempt to quickly get the keys they want. I would almost go so far as to claim the NSA was being negligent in their strategy (not that I like this strategy, mind you) if they didn't follow through to that point. But I just don't see it as being rational to believe the NSA is willing to make our own country's secrets less secure if they are seeing benefits using the bug against others; if anything, I could see them trying to secretly (so as not to tip their hand as having had any advanced notice) fix the bug (after using it for a short time period to pull a bunch of keys, of course ;P).
Its the same methods as in 1940, that is, classic intel methods.
Security does not just mean strong crypto algorithms.
If you find your enemy has found a flaw in openssl or some other methods which you are using to communicate - the best way forward is to continue using that - keep the enemy thinking its all good information when its in fact worthless, and move to another method for the real secure stuff, such as steganography or pidgeons.
Anyway, there probably isnt much "really highly, this kills the cat"-type of information goin on the internets, I guess one point of NSA would also be to keep highly classified information to a minimum. Think thats one reason why Navy and others have their own networks parallel to the internet. Where much secrets flow - isolate.
Say, for example, china wants to spy on a military contractor. Unless the NSA is sharing its secure pigeon network with every US defense contractor (and many of them, large and small) some pretty important US national security assets might be in play. So, perhaps not "state secrets" but things like technology inside of some tactical weapons guidance systems, or similar. The downside for the NSA of sharing any secret-pigeon networks would op-sec goes down as info dispersal goes up.
* Also for companies like a tesla or a space-x who may have purely industrial know how.
Now, would a large state actor involved in offensive black hat hacking have known of heartbleed? I think the answer is likely yes.
Any decently funded team with a dozen good auditors to commit to the project would be watching popular open source projects like openssl, linux, chromium, firefox, apache, nginx, gnupg, openssh, boost, gmp, berkeley db, qt, gtk, etc.
For this part of the project, you only have to grep for low hanging fruit in each new patch that is released for each project, that is usage of: gets, scanf, strncpy, strncat, memcpy etc (or the equivalents for each project that has wrappers or handling functions).
Any large state actor with any decent team running such a project would have discovered heartbleed within days of it being committed. They also would have discovered a lot of other bugs that we either don't know about yet or have fixed.
With heartbleed the state actors are kicking themselves either way: either because they didn't know about the bug and missed it, or they did know about the bug and now can no longer use it as effectively.
"They" (and you can include black hat groups that don't disclose in this as well) combined likely have more resources dedicated to uncovering these bugs than what the open community does, and it might be an order of magnitude larger.
When you think about this further, you realize that the state actors having discovered heartbleed or not doesn't matter - what does matter is that they do have a lot of exploits that we don't know about and it has been confirmed that they are not only looking for these bugs and have a lot of people working on it, but are actively discovering them, using them and purchasing them on the market.
The response to this shouldn't be heartbleed specific - it should be what do "we" do to stop "them" from discovering and using exploits from open source and projects. There needs to be a heck of a lot more effort or a whole new approach to defeat the level resources that are out there dedicated to uncovering and not disclosing these exploits.
The best thing that could have happen did happen: heartbleed was discovered and it was disclosed, and a hell of a lot of people are now more aware of just how frail some of this infrastructure is and what the risks are.
If the NSA has not, they're incompetent.
Has the NSA ever used a 0day to access a machine they were interested in?
Are the people that work for the NSA likely to be smart enough to realise the NSA's upside in finding security flaws and not telling people about them?
Will the NSA have ever done a security review of popular opensource libraries?
I'm not begging the question. What we know is incomplete. However, answer those questions yourself and then imagine how you might answer those questions if you were rich, liked playing dirty, full of smart people, and in a position of power. That's as good a bet as any on what might have been going on.
My two cents: all that is needed is a small crack in security. I wouldn't bet on the internet being secure, because men and women are fallible and security is complex.
Yeah, having to collect and process all that random data... I'm sure they gave up after a couple weeks.
Can you imagine those guys doing anything remotely evil, like extracting metadata and data of anything they could get their hands on?
Then when their honeypots attract alot of bees, is time to tell Google to seal the hole, to protect all the medium sensitivity networks and info.
Disclaimer: All characters and events appearing in my comment are fictitious. Any resemblance to real events or persons, living or dead, is purely coincidental.