Data-Driven Skepticism

Since the beginning of this blog, we’ve been talking about ways to re-use and mash up data that already exists online. This is the core of what the programmable web is about, and there are many potential data sources to use. Figuring out ways to use them that advances skepticism and critical thinking is the key.

Among the others who noticed the utility of re-using existing data this way were journalists. This is because at the same time these fantastic web APIs and tools have become available, governments and other public institutions have moved to open up many of their massive public-domain databases for use by the public. When these datasets contain information that might bear on policy issues and decisions, they are potential gold mines for journalists.

This has kicked off a trend called data-driven journalism. Simply put, it is journalists using data mining and other data analysis techniques in order to find the basis for stories. I think skeptics could learn from the techniques of data-driven journalism, and use them for our purposes too. Indeed, I’ve done some very small experiments in that direction in my metrics articles.

Beware: it’s not the easiest thing in the world to get right. There are definitely many ways you can be tripped up if you aren’t careful. But I think if you are careful there are some interesting techniques here that will be helpful to skeptics.

So let’s explore what it would mean to do data-driven skepticism.

Finding targets for skeptical analysis via RBUTR

One of the interesting side-effects of the anti-misinformation tools I wrote about on Sunday may be better availability of metrics about what misinformation is actually making the rounds.  That could be very useful for skeptics.

I often wonder whether skeptics are staying focused on the right topics. Skeptics are reactive. We often find ourselves responding to news articles, social media trends and other ephemera needing critical analysis. While this is necessary, there is always the danger that we might be distracted from other topics needing us. Those neglected topics could affect equally as many people but are not getting media attention. This is why I often talk about the long tail and focusing on a niche, because the more skeptics who do that, the better overall topic coverage we can get.

I was reminded of this while listening to this week’s Skeptics’ Guide to the Universe podcast, in which host Steven Novella pointed out that although the pseudoscience of neuromuscular dentistry has existed for half a century, there is “very little written about it, skeptically.” I’ve also seen evidence of this when responding to earnest requests for information on the James Randi Educational Foundation’s forum.  Requests occasionally arrive there for a skeptical analysis on some product that has been around for quite some time, and yet nothing appropriately critical about it can be found online.

Let me give you a quick example of how information generated by one of those new tools might help us see whether the focus problem exists and solve it at the same time.

How many skeptic podcasts are there? Please help me find them all

Last May I attempted to measure the amount of skeptic podcasting being produced. We learned a number of interesting things in that post, one of which was that there are over two hours of new skeptic podcast material being produced every day.

It is almost a year later, and we are coming up on the end of the seventh year of skeptical podcasting. It seems appropriate to revisit the census and see where the current numbers are. Unfortunately, there’s no one place to go to find all the skeptical podcasts. Last year I had to create my list by hand, and I definitely missed several.

This year I have that list to start with, plus some known announcements of new podcasts. But I’m sure I must have missed a few. Check out the list in the rest of this post and let me know which ones I’m missing.

My Top Posts of 2011: A Lesson Learned Again

This is the time of year you see a many “year in review” posts. It’s good to take a moment and look back at what you’ve done.

In the spirit of this blog, I’ve got a year in review post coming later that will be a how-to about measuring your own skeptical contribution for the year. But for this one I just thought I’d look at what my top posts in each venue (blog, social media and so on) were and how much traffic or attention they got.

As I’ve pointed out before, raw traffic levels are often a misused indicator. Traffic can surge for a variety of reasons that have nothing to do with the quality or importance of the post itself, so it can be dangerous to let yourself be guided entirely by “ratings” as it were.

That having been said, the traffic levels may have some a lesson to teach us (albeit one that we have seen before on this blog). Read on.

How is The WOT Project doing after six weeks?

Back in March I blogged about Web of Trust and how it could be a powerful tool for skepticism. Web of Trust is a crowdsourced web site safety rating system, that can warn unsuspecting internet users (and now, everyone on Facebook) when they are about to visit a site that contains scams, malware or other potential danger. I suggested that by rating sites selling products based on superstition or pseudoscience, skeptics could turn WOT into a tool for skeptical outreach. Indeed, many skeptic targets such as PowerBalance already sport negative WOT ratings.

In June Canadian skeptic Erik Davis launched a site called The WOT Project. His focus is the opposite side of the equation: protecting the WOT ratings of skeptic sites. Each week the WOT Project posts a list of skeptic sites and encourages participating skeptics to give them a good rating in WOT. The sixth such set was published on Monday.

Since WOT has an API, the ratings can be measured over time. Since my two most recent blog posts were about measuring skeptic outreach on Wikipedia, I thought it would be appropriate to do this for WOT as well. So let’s see how well has WOT Project done in their efforts to protect skeptic web sites on WOT.

How much traffic does a Wikipedia “Did You Know” attract?

If you follow me on social media (Twitter, Facebook and so on) you may have seen me congratulate Dr. Karen Stollznow over the weekend. It was because her new Wikipedia biography (launched just before TAM) appeared on the main page of Wikipedia as part of the “Did You Know?” feature. This is a box on the left side of the page that pulls interesting trivia from articles recently added to Wikipedia.

The main page of the English Wikipedia is apparently used by many as an entry point. It currently receives between 4 and 5 million page views every day. That’s a tremendous amount of traffic, and it guarantees that anything linked from that page is going attract alot of readers.

Like the rest of Wikipedia, the Did You Know? feature is collaboratively edited. Anyone can nominate a page to appear there, as long as they follow certain rules. When I create a new article that is relevant to skepticism, I nominate it in the hopes that it will be displayed here one day. The goal is to get the attention of those 4 million people, and expose them to skepticism.

But how effective is this? Fortunately, Wikipedia’s transparency allows us to examine the traffic numbers and answer that question. In this article I hope to show it is a very good way to get new people exposed to skeptic concepts.

Why should skeptics edit Wikipedia? Traffic, traffic, traffic!

A recurring topic on this blog and in my public talks is getting skeptics involved in editing Wikipedia. I’ve been writing about it here and talking about it at events like Skepticamp and Skeptics in the Pub for almost 3 years. I believe it to be very important.

Wikipedia - The Free Encyclopedia

But it is sometimes difficult to impress people with the importance. I often approach this by talking about things like SEO and SERP placement. But to many non-webmasters these are confusing concepts. It can be hard to visualize how they translate into readers.

Last weekend at The Amazing Meeting (TAM9) Susan Gerbic-Forsyth gave a Sunday paper presentation on the topic of Wikipedia. Susan has taken up the gauntlet I threw down in a big way, for which I thank her. She’s contributed a ton of photos she’s taken at skeptic events, which are a fantastic addition to any article. She’s also started her own blog where she shares what articles she’s been working on, tips on formatting and so on. It’s good stuff, check it out.

During TAM9, Susan and I were talking about how it is sometimes difficult to convince skeptics that Wikipedia is worth the effort. Skeptics, due to their nature, are painfully aware of the many limitations of Wikipedia. Some express doubts that these can be overcome, and have written off the site. Others who have actually tried to edit express frustration in dealing with other editors and the culture of Wikipedia (a topic for another time).

But as we were talking, it occurred to me there is a very simple way to show exactly how important Wikipedia is, without using any obscure terminology. And that’s what this post is about. By the end of it, it should be crystal clear why skeptics need to edit Wikipedia.

How much skeptical podcasting is being produced?

Update May 20: Updated the podcast list at the bottom of the post.
Update May 29: Updated the podcast list again. Tell me if I’m missing any.

This is the first of what I hope will be a series of posts on measuring skepticism’s digital footprint. Skeptics often talk about our outreach and effect, but how can we know what that is unless we are measuring our efforts? Fortunately, measuring things online is pretty straightforward. So I set out to generate some basic measurements of the content skeptics are generating online. In this post I will tackle podcasts.