Earlier this week, engineer Antti Karjalainen was simply doing his job when he stumbled upon one of the biggest security holes the Internet has ever seen — the Heartbleed bug.
Karjalainen works for security firm Codenomicon and is one of three engineers from the company credited with uncovering Heartbleed.
The Heartbleed bug affects a widely used encryption protocol known as OpenSSL. In short, the vulnerability tricks servers into spilling (or "bleeding") data from its memory, potentially giving hackers aware of the flaw access to sensitive information such as credit-card numbers and passwords.
Karjalainen — along with Codenomicon colleagues Riku Hietamäki and Matti Kamunen and Google security's Neel Mehta (who worked independently of Codenomicon) — is credited with being the first to discover the flaw. Karajalainen said he was working with Hietamäki on updating new features for Codenomicon's protocol test suite when they discovered the bug.
The new feature, called Safeguard, is designed to recognize new kinds of software vulnerabilities, according to Karjalainen. He said he initially noticed something was wrong when he added support for OpenSSL's Heartbeat feature, a mechanism used to test whether a connection is secure.
Heartbeat allows one server to send arbitrary data to another. The recipient then sends back an exact replica of the data to the original sender to prove that the connection is secure and nothing has been compromised.
That's not what happened in this case.
The crown jewels of the secure Internet
Codenomicon began to notice new test cases that would trigger vulnerabilities within the Heartbeat protocol.
"We were in the right place with the right tool," Karjalainen said.
After noticing that one response to a test case had turned out unusually large, Karjalainen and Hietamäki tried a different test. This new test would confirm that a flaw in the Hearbeat feature was stealthily leaking extraneous data.
"It worked just as we feared," he said. "The server dumped the requested amount of its memory. When we investigated the response closer ... we soon understood that this is potentially a very, very bad vulnerability."
The next day, Codenomicon's security specialist Marko Laakso discovered that the version of OpenSSL being used to power the company's test protocol suite was leaking private keys from the server.
"The private keys are the crown jewels of the secure Internet," Karjalainen said. "They are used for proving that you are who you really say you are. So this was potentially the worst vulnerability in the history of the Internet."
Heartbleed was born
Codenomicon immediately began to patch its servers and then brainstormed a way to help inform others. The company had been referring to the issue as Heartbleed, so it purchased the domain name heartbleed.com, created a logo to visually represent the bug, and began writing a lengthy, detailed FAQ about the problem.
Heartbleed is particularly dangerous for a few reasons. It affects OpenSSL, which is used across a majority of the Internet, meaning the bug likely affects every Internet user in one way or another. Heartbleed is difficult to detect because it allows intruders to attack at such an early phase of the communication process when data is being transferred. Ari Takanen, Codenomicon's co-founder and chief research officer, compared it to robbing a building before you even compromise the lock on its door.
Some larger Web services such as Twitter and Google have already said that they've applied the necessary patch to address the issue. However, there are several other types of devices, including cable boxes and traffic lights, that may not ever get fixed because they're systems are updated so infrequently, as MIT Technology Review reported.
"Since then I haven't been able to think about anything else than the vulnerability and its consequences," Karjalainen said. "It would be really nice to have something else to think about for a while."