Hey Crowd, Who's Feeling Clever?

Let's say you run a research lab, and you've got a really tough problem. I mean a real doozy - your best scientists have been working on it for years and they haven't been able to figure anything out. But you still need an answer, so how are you going to get one? Enter a new phenomenon: Open Innovation. You've probably heard of companies relying on the "wisdom of the crowds" for things like the latest Doritos ad campaigns, Threadless T-shirt designs, and the world's largest encyclopedia, but you may not know that firms are also turning to crowds with their really high-tech conundrums. A number of platforms have sprung up to connect problem seekers with problem solvers, of which Innocentive (founded 2001) is probably the most famous.

I'll go more into Open Innovation (and it's cousin, Crowdsourcing) in a later post, but the basic idea behind Innocentive is that companies can post problems and associated rewards on a website where individual scientists can sign up to view them. If someone thinks they might be able to answer a problem, they connect with the company and send in a solution. The company then determines the best solutions and gives (big) rewards to the solvers - up to the tune of $100,000. Keep in mind, though, these are complex research problems. Examples include designing an injectable suspension placebo with no pharmacological or biological activity, or synthesizing a food grade polymer delivery system. So then the big question: does it actually work? 

Short answer: yes. A recent paper by Lars Bo Jeppesen (Copenhagen Business School) and Karim Lakhani (Harvard Business School) explains just how well: for the 166 problems posted on Innocentive during the study period, 933 potential solutions were received and 43 of the problems (26%) were solved. Given the difficulty and specialization of the problems involved, that's pretty impressive. In fact, winning solvers devoted on average more than 100 hours to develop their solutions - without any guaranteed compensation.

What's even more interesting is who is actually winning the contests. Certainly expertise is required to solve complex scientific problems, but Jeppesen and Lakhani find that the problems are disproportionately solved by people whose expertise lies outside of the domain of the problem. The food grade polymer problem above, for instance, is a biochemistry question that was solved independently by an aerospace physicist and a small agribusiness owner. Successful responses also come disproportionately from women, who tend to have more marginal positions in science (nobody likes it, but that's what the research suggests). So what's going on?

The underlying intuition is that people come up with strategies to solve tricky problems, and that those strategies are a function of our expertise, backgrounds, and personalities. As a result, different people tend to approach similar problems from different angles. Unfortunately, we often don't know what approach will be effective ahead of time, so even the most brilliant person (or company) might look at a problem a hundred different ways and still not be able to solve it.

Open innovation is effective because it allows a very diverse set of people to try their hand in solving a problem. By broadcasting out to a broad (and diverse) group, Innocentive is able to bring in a broad (and diverse) set of problem-solving strategies. This diversity makes it more likely someone in the group will have an effective solution, and the trickiness of the problem makes it more likely that that person will be someone with an unusual background (i.e., a unique perspective). That might be a scientist from outside the field, or a woman in a field dominated largely by male scientists. Jeppesen and Lakhani also dig into some old sociology studies and find that breakthrough inventions are actually frequently developed by outsiders and eccentrics; those who are "not engaged in the occupation…and are not bound by [its] professional customs and traditions."

Pretty neat, right? But before you go and sign up to start applying your liberal arts degree to an X-Ray crystallography problem, remember what we've learned about creativity. Although a fresh perspective is critical, it isn't enough to be different. Innovation is driven by ideas that are both novel and relevant.  The physicist was able to solve the protein problem because, once she read the problem statement, she realized that her unique expertise was probably applicable. And she was successful because even though she was an expert scientist, she didn't think like a traditional biochemist. And that's what creativity's all about.

[Comic: courtesy of XKCD]