Pwn2own considered (somewhat) harmful

I think that hacking challenges and bug bounty programs can be extremely valuable. This is true when they involve transparent, sustained efforts to evaluate the security of a particular platform. For example, I believe that there is a substantial value in Mozilla bug bounties, or in the Chrome reward program. These programs greatly improve the security of the browsers in question, occasionally advance our understanding of web security, and provide tons of statistical data about vendor response processes and attitudes toward security flaws. That last part is arguably the most important metric when dealing with code so complex that for better or worse, it is unlikely to ever be perfectly safe.

I also think that Pwn2own, an annual browser hacking contest run by TippingPoint, does not deliver the same value. The formula of the contest boils down to this: once a year, a single, secretly developed exploit is exchanged for a substantial amount of money. No information about the flaw or its back story is revealed in the process, and given that this trade is negligible in comparison to the annual volume of browser vulnerabilities, there is absolutely no intrinsic value in observing it.

That, alone, is not a compelling criticism; at best, it's a reason not to watch. But then, there are some negative consequences, too: it is in the interest of the conference and contest organizers, and the participating researchers, to get publicity for their findings - and journalists, who do not necessarily have a holistic view of the day-to-day browser security research, embrace such high-profile developments with disproportionate enthusiasm. The resulting ecstatic press coverage ultimately undermines any attempt to have a meaningful and reasonable discussion about the state of browser security.

Take this quote, which likely will be repeated in every Safari-related story for the next twelve months:

"A team was able to exploit Safari to exploit a MacBook Air in five seconds. Yes, five seconds - less time than it takes most people just to type 'Safari got hacked in less than five seconds'."

That's remarkable, but also completely wrong. It takes days or weeks to find and exploit a vulnerability, and Pwn2own is no exception: the actual exploits are prepared months or weeks in advance, and simply executed on the day the contest takes place. I do not think there is a single person in the information security industry who would say that the discovery of a normal browser vulnerability is a notable event: several hundred such flaws are discovered and resolved every year in every browser, as evidenced by release notes maintained by the vendors with varying degrees of accuracy. Neither the fact that somebody discovered a vulnerability before Pwn2own, nor that this person needed needed five seconds to execute that pre-made code, is a useful measure of anything.

Similarly, the survival of Firefox and Chrome intuitively makes me happy, because I know that these browsers give a lot of thought to security - but I do not think that Pwn2own is a meaningful testament to this. Perhaps these two vendors merely patched up the vulnerability somebody wanted to use, and there was not enough time to find a new one. Or perhaps nobody attending the event (which brings together only a tiny fraction of the infosec community) had the expertise and the inclination to target this particular browser.

Yes, there are vendors who lag behind the rest when it comes to vulnerability response and proactive security work; and there are some hard problems we still have to solve to make the web a safer environment. But the headlines inspired by Pwn2own (and probably encouraged by the organizers) are very unfair, and unnecessarily alienate the parties who should be paying attention to their security posture. Investigating real data, and asking some hard-hitting questions, can make more of a difference... and if done right, it can be more fun.