In one of the biggest security goof ups of recent times, user data of as many as 3,400 websites have been leaked online after content delivery network Cloudflare fell victim to a rather nasty bug. Worse still, the leaked user data has been already cached by search engines.
Among the affected websites are popular services such as Uber, Fitbit, and dating website OKCupid — each with millions of users worldwide. 1Password is another website that relies on Cloudflare’s platform, although they are unlikely to be affected nearly as badly because of their native end-to-end encryption.
Cloudflare’s services include optimization of data security and overall performance of its more than 5.5 million client websites. The company warned its customers earlier today that a recently fixed software bug exposed an array of sensitive personal information which may include passwords, as well as cookies of end users.
Adding to the worries of all those affected, it seems the leakage may have occurred last year on September 22. If that’s the case, then all the compromised data had been out there to be misused by cyber criminals for at least five months before the issue was finally discovered.
However, if you thought things could not go any worse from there, there’s more to the breach. Apparently, Google and other search engines had already cached some of the highly sensitive personal data before efforts were made to control the damage.
“The bug was serious because the leaked memory could contain private information and because it had been cached by search engines,” Cloudflare CTO John Graham-Cumming wrote in a blog post published Thursday. “We are disclosing this problem now as we are satisfied that search engine caches have now been cleared of sensitive information. We have also not discovered any evidence of malicious exploits of the bug or other reports of its existence.”
The bug was in an HTML parser chain
The leakage reportedly resulted from a bug in an HTML parser chain used by Cloudflare to optimize webpages as they pass through the servers at the service’s end. The parser is important as it oversees a number of tasks including, but not limited to, converting HTTP URLs to secure HTTPS variants, adding Google Analytics tags, obfuscating email addresses, and such.
Security experts who analyzed the goof up are saying that when the parser was deployed alongside three other features — Automatic HTTPS Rewrites, e-mail obfuscation, and server-side excludes — it led to the servers at Clourflare’s end to start leaking pseudo random memory contents into various HTTPS responses.
The bug was brought to Cloudflare’s attention only on Saturday evening by Tavis Ormandy, a security researcher at Google.
“It became clear after a while we were looking at chunks of uninitialized memory interspersed with valid data,” Ormandy explained in a blog post
The company says its engineers were quick to react and all measures had been taken to disable e-mail obfuscation within an hour. The move fixed most of the loophole, but as we now know, the damage was already done. The Cloudflare team took six more hours to figure the underlying reason behind the leak and fix it.