More People Were Paid To Exploit Heartbleed For The NSA Than To Fix It

Unsurprisingly, it turns out that the NSA knew about the Heartbleed bug since shortly after it was added to OpenSSL. While thousands of salaried NSA personnel search for bugs like these to exploit, OpenSSL has only four part-time volunteers maintaining it. Of course this was going to happen.

The idea behind open source software is that “given enough eyeballs, all bugs are shallow.” This only works if there actually are enough eyeballs. Code audits can only happen if there are people with the will, expertise, and time to do so. Rusty Foster pointed out the problem with OpenSSL:

The project’s code is more than fifteen years old, and it has a reputation for being dense, as well as difficult to maintain and to improve. Since the bug was revealed, other programmers have had harsh criticisms for what they regard as a mistake that could easily have been avoided.…

Unlike a rusting highway bridge, digital infrastructure does not betray the effects of age. And, unlike roads and bridges, large portions of the software infrastructure of the Internet are built and maintained by volunteers, who get little reward when their code works well but are blamed, and sometimes savagely derided, when it fails. To some degree, this is beginning to change: venture-capital firms have made substantial investments in code-infrastructure projects, like GitHub and the Node Package Manager. But money and support still tend to flow to the newest and sexiest projects, while boring but essential elements like OpenSSL limp along as volunteer efforts.

This point is only compounded by the NSA news. As it turns out, a great deal of funding was going towards meticulously auditing OpenSSL. The problem is that the NSA keeps the results of these audits to themselves. No bugs are fixed. No patches are committed. Critical flaws are kept under wraps so that they can be used to siphon more data and break into more computers.

Never mind the fact that the NSA’s priority is supposed to be the defense of the United States, when critical infrastructure in the US was potentially affected by this bug. If they wanted to call this defense, then the NSA must have been really confident that the classic go-to bogeymen of China, Russia, Iran, or Al Qaeda hadn’t also discovered Heartbleed. Which, of course, they couldn’t be, because Neel Mehta at Google eventually reported it, so it’s not like it was impossible to find without NSA super-wizardry.

But back to the issue at hand: the NSA has, potentially, a small army of security researchers doing all of the code audits that tech companies and the open source community should be doing, and hoarding the benefits for themselves. The Is TrueCrypt Audited Yet? project might as well change its website header from “Not Yet” to “Who Knows?” This is awful. Economically, it’s also unsurprising.

The NSA has an entire budget devoted to doing just this: “$1.6 billion a year on data processing and exploitation, more than a thousand times the annual budget of the OpenSSL project” reports The Verge. Their prime directive is to find bugs, keep them quiet, and exploit them for their own gain (sorry, “national security”). OpenSSL’s volunteers, on the other hand, need jobs to feed their families. As much as they might want to, they don’t have the time to devote the effort needed to make sure their code is rock-solid. And apparently, neither do its users. It took a Google employee two years to discover Heartbleed, despite the fact that they’re a multi-billion dollar corporation that depends on the integrity of things like OpenSSL. Evidently, though, it’s still not cost-effective to have dedicated teams keeping an eye on the code.

My instinct is to just say that this is another infopolicy case for a universal basic income, to free up volunteers who are willing and able to perform these audits from the pressure of having to work another job. While that would certainly help, I admit it’s a bit reductionist. Code audits can be boring, tedious work, and while with 7 billion people in the world I’m sure some of them would be jumping out of the woodwork to proofread thousands of lines of code, I can’t say how many. But the NSA has apparently figured out how to efficiently spot glaring security flaws, so it’s high time the white hats did too.

Discussion

  1. Anonymous

    the only thing i am surprised at over this is that the NSA weren’t the instigators (or it hasn’t come to light yet that they were!)!

    1. Scary Devil Monastery

      That would be too risky by far – imagine trying to tell the open-source community to insert a deliberate bug and then keep their trap shut about it. Guaranteed to eventually leak, if for no other reason that the pertinent area of vulnerability would be passed around like a big, red, glowing, ticking bomb among the various coding teams.

      Makes more sense to realize the NSA was simply betraying the security of their nation out of a malplaced hope they’d be the only ones sitting on the discovered security hole.

  2. Aelius Blythe

    “the NSA’s priority is supposed to be the defense of the United States…”

    Well, there’s the final nail in the coffin of THAT defense of the NSA. It’s hardly surprising, but it is appalling. Putting so many people at risk for the (supposed) sake of national security is beyond moronic – even for the NSA.

  3. Zirgs

    OpenSSL has a shitty code-review process.
    Only two guys are responsible for the Heartbleed bug.
    A code author and a reviewer – and that’s it.
    That’s just insane – they really should take thing a lot more seriously.
    Even if they’re underpaid – that’s not an excuse.
    More reviewers/approvers are definitely needed.

    And it’s not just the NSA. Many hackers try to find bugs in open source projects, but choose not to report them.

    1. Caleb Lanik

      What exactly would you suggest they do with no money and no volunteers? The whole thing is maintained by four part time people. Ultimately, this is our fault, everyone uses OpenSSL, but nobody helped maintain or audit it.

      1. Me

        Let computers do the testing, that is why they are made, to do boring tedius repetitive jobs.

        So unit testing is probably the answer.

      2. Zirgs

        Their code is pure shit. Hard to read and maintain.
        At least we learned a lesson – never take a random piece of code from the internet without auditing and reviewing it yourself.

        1. Caleb Lanik

          I would also hope that we learned that if we’re going to make a given piece of software the backbone of the internet, we should probably give them some money and help contribute to the codebase instead of just getting mad that four people who work part time didn’t do it right. FOSS, sometimes you get what you pay for.

        2. Zirgs

          We should get mad – as I said – lack of time and money are not an excuse to release crappy code.
          They should have postponed the release and reviewed the code more carefully.
          No one needs rushed and half assed releases.
          And if someone offers you a crappy piece of code for free – simply reject it.

        3. gurrfield

          Zirgs:

          Seriously, any free open source code is “as is” and if there’s just 2 coders on such a big important project, well.. then it’s the communitys fault for not helping out on the project – or coming up with alternative projects.

    2. A nonymous

      So in hindsight what did _YOU_ do to help avoid this! That is a question I would really like you to answer!

      1. Zirgs

        I’m not a server owner and I don’t use OpenSSL.

        And at the company that I work for we take things a lot more seriously than the OpenSSL developers.

        1. gurrfield

          It’s a bit easier to “take something seriously” if you get any pay for it…

        2. Zirgs

          That’s not an excuse gurrfield.

          Why did he merge that commit on 31/12/2011 when normal people are celebrating the New Year’s Eve?
          What kept him from postponing the release for 1 or 2 months to review it more thoroughly?
          That was not an urgent update or bugfix.

          No matter if you’re paid or not – DO NOT release half assed crap.

        3. gurrfield

          The project developers are free to do whatever they please – they have no obligation to make the code fit your needs – unless you pay them for it.

          What really has no excuse is to build any entire security infrastructure on pieces of unreviewed software. If the project is too badly reviewed or is too poorly managed and/or has too few developers, then you should have replaced it by something else which is more thoroughly reviewed by the community.

    3. Scary Devil Monastery

      “Many hackers try to find bugs in open source projects, but choose not to report them.”

      Many hackers, being either black hat or gray hat, certainly choose not to report bugs which allow them to perpetrate criminal acts.

      For the National Security Agency to find and allow a security flaw to stand which impacts their entire nation the hardest of all, simply because they vainly hope no one else has found the hole…that’s outright betraying their charter.

      Or, assuming they’re under a military legal umbrella, outright treason.

  4. Renee Marie Jones

    The situation is even worse than the article states. There are a lot of people that are capable of auditing code like openssl for bugs, bug there is no point. I once found a ghastly bug in a low-level linux driver. I tried to report it, but there was no point. If you are not one of the “in” people on an open source project, nothing you report will be taken seriously. There may only be a handful of people responsible for maintaining openssl, but that is their own fault. I guarantee they would not listen to a word I said.

    1. Aaron Wolf

      Hi Renee,

      I have no programming experience and a couple years ago when I pointed out some bugs in some Open Source software, I was thanked and encouraged. I ended up being welcomed to help out further with testing and support, despite nobody knowing me and my having no background in this.

      It’s unfortunate that you had a bad experience, but you have no business assuming that your experience is representative of all projects. There’s even an entire initiative aimed around helping projects be as welcoming as possible to newcomers. Check out OpenHatch.org

      1. gurrfield

        There’s always a bunch of people in here (or any place on the internet, really) miscrediting open source as fast as they get a chance to do so. Some of them may have substance, but let’s not forget that quite some companies by now have money to lose on if Open Source is competitive with their own closed source solutions so there’s money in trying to make it look that more are furious with open source projects than there really are.

  5. Perché l’NSA poteva sapere di Heartbleed | Notizya.it

    […] (Immagine via) […]

  6. G

    Things would be so much nicer if people believe things based on their merit instead of fitting with the rest of they want to believe. Bloomberg has exactly 0 reliable sources for their rumor, yet funnily it turns into “facts” when reposted.

  7. Zirgs

    BTW Rick, why have you stopped writing about scamcoin?
    Is that because the price is on a constant downhill path for more than 4 months now?
    Thousands of people have lost their money.
    Miners have already started to abandon this sinking ship, because it’s not profitable for them anymore.
    I think that you should better sell now, because the price will tank to 2011 levels and it’s not coming back.

    1. gurrfield

      You don’t call dollars a “sinking ship” because one single company or bank trading in dollars have commited fraud or stolen money – that’s what MtGox was.

      There was no failure with the currency as such, but an “inside” or “outside” job of economic / financial crime where people’s deposited assets “disappeared”.

      Could have happened to people trusting any company to safekeep their dollars or euros or beer bottle-caps or rare post-stamps or whatever.

      1. Zirgs

        I’m not talking about the Gox – which wast just a hilarious and epic failure..

        I’m talking about the bitcoin itself.
        Which is losing value fast.
        Current infrastructure simply DEPENDS on increasing prices, but that’s not happening any longer.
        In fact – bitcoin is inflating a lot quicker than the dreaded dollar.
        The price is ~2.5x lower than in December and mining difficulty is way higher – you do the math.

        That’s why many miners are abandoning bitcoin.
        Ebay is full of used mining rigs that are no longer profitable.

        Also ordinary users are seeing their investments evaporating.
        And that’s also accelerating the price drop – sell while BTC is worth anything.
        The future looks bad for BTC.

        1. gurrfield

          Yes there’s been some big bumps in what bitcoins have been Traded for.

          Well it’s “value” is still mostly just speculation and no merit or connection to any real world trade. If people start using it for real world goods and services we may see a big change.

          For as long as the value is based purely on speculation – the “price” will keep being very fluctuating. If people start wanting it for services they offer or stuff they sell, that’s when bitcoin will get it’s real value. And that is not anything like what we’ve seen so far.

  8. frank87

    There are lot’s of young people in schools and universities studying software. Why don’t they do open source software?
    Final exam: find a bug in Linux…

    1. gurrfield

      That’s just what they do! Linux was concieved by such a student and many other open source projects are started by university students as well.

  9. Week 15, 2014 | Karl Andersson

    […] More people were paid to exploit Heartbleed for the NSA than to fix it […]

  10. […] Image via falkvinge.net […]

  11. Renegade

    That was a **great** article up until you went full commie. “…universal basic income…” Nice idea, but who are you going to steal the money from to pay people?

    1. gurrfield

      Well people out of work get “social security” today already. That money is also “stolen” if you consider taxes to be stealing.

      1. abc

        That doesn’t make it right, of course.

    2. Autolykos

      I know some people consider human rights to be optional (and I can understand where you’re coming from), but I still prefer to live in a society that respects and protects them.
      And once you’ve accepted that, you can’t just let people starve, no matter what they do. So the only difference with UBI is that we could do away with a bureaucracy that has little else to do (and in fact can do little else) than harass the unemployed. And I also firmly believe that anyone who needs to be bullied into work will do more harm than good anyway and should be replaced with machines ASAP (very probably, this has already happened and is the reason this guy won’t get a job).
      I don’t think UBI should be any higher than the bare minimum you need to survive, but it also shouldn’t be lower. And once we have it, pretty much all other benefits we’re paying right now (as well as most regulations on labor and wages) would be completely obsolete.
      Basically, UBI does not contradict libertarianism (except for the most extreme forms), it is a necessity for libertarianism to work properly with humans.
      And if you consider this a violation of your free choice, I’d have nothing against allowing you to opt out of it, once and for all. You won’t pay for it, and you won’t get any of it (or any other benefits) you might need later. I don’t expect anyone but a few super-rich to actually consider this to be sensible option, though.

      1. gurrfield

        Yes it is inhumane to let people starve.

        But also from a strictly economic and consequential point of view it pays off to have social security, because without those it would increase crimes such as stealing food and breaking into places to get roof over head to sleep.

        Frankly you can view the taxes you pay for social security as an “insurance” you won’t get robbed by someone desperate to get food for the day.

        So either we pay for some level of social security or we end up having to pay when the poor people hit the prisons and/or hospitals anyway.

  12. Perpetualrabbit

    We should start a crowdfunding campaign, possibly kickstarter, for a mega-audit campaign. Hell, we could ask Kim Dotcom to help funding it and set it up.
    It should have the following goals:

    1) Identify all libre/open source software projects that make up the security infrastructure of the internet. Such as openssl, gnutls, openssh, apache, nginx, postfix, dovecot, `openstack´ and many others
    2) Estimate how many man-years it will take to audit that software
    3) Attract software developers to audit and fix the code. This includes cleaning up and adding comments and documentation where there is none now. Probably a lot of developers already working in the target projects who are currently either not paid or working part time could be hired. And new people, with a fresh eye on the code.
    4) Set up a number of example servers for hackers to try and hack, which are continually kept up to date with the latest output of this mega-audit campaign. The target for the hackers is to obtain a secret token (like capture the flag) with a successful hack, and earn a price in exchange for an exact report on how the hack was done. So this way the faulty software can be fixed.

    1. abc

      +1

      That’s an awesome use for crowdfunding!

  13. NSA has 'piggybacked' on corporate surveillance efforts

    […] according to two sources “familiar with the matter”, reports Bloomberg. Infopolicy site Falkvinge even cites that “a great deal of funding*was*going towards meticulously auditing […]

  14. […] unite al recente scandalo Datagate di Edward Snowden, rendono palese all’opinione pubblica il modus operandi di questa famigerata agenzia: scovare errori, sfruttarli e non dire niente a […]

  15. […] Speculation on my part: the above is suggestive of a Heartbleed attack in which the login details of administrators were captured and used to access servers to covertly install the malware. In other words, it’s hard to blame SingleHop for falling to an exploit which the NSA were more interested in exploiting than reporting or fixing. […]

  16. […] Speculation on my part: the above is suggestive of a Heartbleed attack in which the login details of administrators were captured and used to access servers to covertly install the malware. In other words, it’s hard to blame SingleHop for falling to an exploit which the NSA were more interested in exploiting than reporting or fixing. […]

Comments are closed.

arrow