Since the beginning of this blog, we’ve been talking about ways to re-use and mash up data that already exists online. This is the core of what the programmable web is about, and there are many potential data sources to use. Figuring out ways to use them that advances skepticism and critical thinking is the key.
Among the others who noticed the utility of re-using existing data this way were journalists. This is because at the same time these fantastic web APIs and tools have become available, governments and other public institutions have moved to open up many of their massive public-domain databases for use by the public. When these datasets contain information that might bear on policy issues and decisions, they are potential gold mines for journalists.
This has kicked off a trend called data-driven journalism. Simply put, it is journalists using data mining and other data analysis techniques in order to find the basis for stories. I think skeptics could learn from the techniques of data-driven journalism, and use them for our purposes too. Indeed, I’ve done some very small experiments in that direction in my metrics articles.
Beware: it’s not the easiest thing in the world to get right. There are definitely many ways you can be tripped up if you aren’t careful. But I think if you are careful there are some interesting techniques here that will be helpful to skeptics.
So let’s explore what it would mean to do data-driven skepticism.
You may have noticed a new badge appeared recently on the right side of this blog that says Keep Libel Laws out of Science. It has to do with an ongoing legal case in England where a major chiropractic association is suing science writer Simon Singh for libel over an article in which he referred to certain chiropractic procedures as “bogus.” I encourage you to click the badge and sign the petition.
Today there was a fascinating development in this case that relates directly to skeptical software tools. Because the case hinges on whether or not chiros promote procedures they know to be “bogus”, skeptics have been scouring chiropractic websites in the UK looking for evidence of this. In response, another chiropractic association has advised its members to take down their websites entirely! This is stunning.
Internet Archive Wayback Machine
As skeptics one of the key things we do is hold woo-woos feet to the fire when they make ridiculous claims. Perhaps the most public place of all to make a claim is on a website, because it is instantaneously visible to everyone on earth who chooses to look. Tracking claims made on websites is thus an important skeptical technique.
But this move by the chiropractors reminds us that the web is mutable thing. Any content anywhere on the web can be changed at any time. Paranormalists and pseudoscientists can edit their web sites constantly to present a moving target or to remove evidence of their missteps. In order to do our jobs as skeptics, we need to be constantly aware of this and use tools to compensate. Fortunately such archival tools exist. One is the well-known Internet Wayback Machine, but several others (including commercial products) exist.
After the jump, I’ll talk more about some of the uses of these tools and show you how to use them as a skeptic.