Bug bounty programs work for the giants but pose problems for others
SAN FRANCISCO -- Ask Google and Facebook if their bug bounty programs are a success, and they offer a resounding "yes," in part because such programs prove to be a cost effective way to find security flaws in their products.
That doesn't mean that such programs are for all companies. If you're tempted to offer to pay for anyone to look for bugs in your web site, app or service, there's a lot to think about to prevent the program backfiring on you.
For instance, before launching a bug bounty program, it's crucial to prepare for the response. "If you aren't prepared to handle the volume, it can make it difficult for your response team to get through the noise to pull the signal out," said Nate Jones, technical program manager for Facebook, speaking during a panel discussion at RSA this week. He said sometimes Facebook gets a high volume of bug reports that aren't really important or that aren't actually bugs. It takes work to sift through those reports to find the really important ones.
It's also a good idea to offer as much information as possible about payment to researchers in advance to avoid any misunderstandings, Chris Evans, who works on Chrome security at Google, said.
Trying to manage the volume of bug submissions seemed to be a problem for one audience member who asked how to handle tweaking a program after getting too many submissions. Taking down the program is one option, although it comes with the risk of angering researchers.
However, the flip side is that a high volume of submissions may indicate a high number of problems that a company likely should want to know about, Evans said. Businesses can try hiring a standard pen tester before launching a bug bounty program to try to get an idea how buggy their product is, he said.
Pricing can help control the volume of bugs that researchers are likely to submit. Google will sometimes set a low price for finding certain kinds of bugs as a way to keep response low, Evans said. If researchers end up finding good bugs, the company will increase the bounty slowly to start attracting more submissions, he said.
That's a wise approach also because it can help control spending. "The caveat is to make sure you don't run out of money," Casey Ellis, CEO of Bugcrowd said. That's hard without a good gauge for how buggy your product is.
The dark side of bug bounty programs
Still, generally speaking, the economics of bug bounty programs are complicated. While the big companies were very positive about their bug bounty programs, it was clear that the current set up is tenuous.
The first reason Jones gave for why Facebook's program works is because it's cheap. "It's a good way to pay for legitimate results rather than to pay people for their time. It turns out that has been cost effective for us," he said. While Facebook does do penetration testing, the bug bounty program is "much less expensive," he said.
Not all researchers are happy with that arrangement. One audience member said that the model doesn't work for so-called "bounty hunters" who might want to make a living at it, even for researchers who live outside of expensive countries like the U.S. He suggested that most bug submissions that companies get are from researchers doing other work who stumble on bugs.
Ellis agreed that the current model has problems. "The hunters take on all the risk," he said. They may spend time and never find a bug, meaning they don't get paid for that work. He said he knows of some researchers in India and the Philippines who are making a living off of bug bounties and he said that trends indicate at some point bug hunting could be a reliable income source for people in the West.
There's also the reality that researchers can often make more money selling or using bugs they find for nefarious reasons. However, most panelists agreed that money doesn't typically convince people to use bugs for bad reasons. Ellis quoted researcher Dan Kaminsky who noted that plenty of people aren't swayed by a bigger payout for doing black hat work just like "not everyone wants to be a drug dealer."
There are yet other potential unintended consequences to bug bounty programs. Jake Kouns, CISO of Risk Based Security, suggested that it's possible that some people might feel an incentive to insert bugs into software that's developed by the community in order to cash in later through bug bounty programs.
That's possible but Casey noted that there are so many processes in place that track code submissions that it'd be relatively easy to find the person who inserted the bad code in the first place.