Disruptive Innovation Could Up-End an Innovative Industry

I was amused and somewhat unsettled when I heard of Kaggle, a company with a novel approach to data analysis. As I understand it, Kaggle is a middleman between companies that have large amounts of data and are looking for certain kinds of analysis and the people or companies that can provide that analysis. But with a twist.

The companies with the data can be any kind of company. And the analysts? They could be anybody, too, and some of them might even be nobodies. Here's a quotation from the website, (http://www.kaggle.com/): “Kaggle is a platform for data prediction competitions that allows organizations to post their data and have it scrutinized by the world's best data scientists. In exchange for a prize, winning competitors provide the algorithms that beat all other methods of solving a data crunching problem. Most data problems can be framed as a competition.” You don't have to be a big-time scientist to engage in the competition. You just need a big idea.

One competition, sponsored by California’s Heritage Provider Network, is to “identify patients who will be admitted to a hospital within the next year, using historical claims data.” Practically the holy grail of managed care. There are over 1,000 contestants, and Heritage will hand out $3 million to the winner.

This is a pretty ingenious way to get people to work on a project that you might otherwise spend $3 million contracting with a single company and not get what you had hoped to get. Using a Kaggle competition, you could get exceptional value for your money. But if lots of jobs were done this way, there would be an awful lot of work done without being paid for, which is bad in itself, and which might shred the business of many kinds of consultants.

In managed care, I am thinking of the predictive modeling (PM) consultants who emerged 10 or 15 years ago and have been more and more accepted by health plans. Will operations like Kaggle mean the end of these vendors? Will competitions make PM consulting less lucrative and reduce the number of players? Or will it result in improved modeling with no downside for the sponsors of the competition?

This sounds like a disruptive innovation that bears watching.