Tuesday, June 24, 2008

What do you have against science, Wired?

I thought we spoke about this before, but you persist with the silliness: "The End of Theory: The Data Deluge Makes the Scientific Method Obsolete"
This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear. Out with every theory of human behavior, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.

The big target here isn't advertising, though. It's science. The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists. The models are then tested, and experiments confirm or falsify theoretical models of how the world works. This is the way science has worked for hundreds of years.

Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.

But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete.


There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.


No. No, it's not. Data without a model *is* just noise. The value of a model is that you can make predictions with it. You can't do that with just data points, you have to connect them in some way. Even if you're only making inferrences from correlation, you're still creating a model. The hypothesis is the model. From it you make predictions, which you test. This is often how things start in a proper scientific investigation. Someone notices an interesting correlation and studies it further. You're just skipping the testing phase, relying on lots of correlation being sufficient instead.

Why do you refuse to learn how science works, Wired magazine? Is it because you fear that too much critical thinking will expose the singularity as a pipe-dream, that you won't get nerd-raptured away?

Come on. We can work through this together, if you'd only tell me why.

1 comment:

Andreas Öjerfors said...

The problem with Wired is that it still thinks that technology is cool. That its something special, alien to people, something that Wired and Wired alone have extraordinary insight into. Their edge is presenting technology as something that will change everything.

In reality, the tech have become everyday tools that even my grandmother uses, and it is just that - tools.

With their view of technology, they will continue to misinterpret its value for humanity.