Web of Trust is a useful tool for skeptics

It’s frustrating being a skeptic. We do our best to explain why homeopathy or Power Balance wristbands or MMS are (at best) a waste of money or (at worst) a dangerous fraud. And yet people still buy these products every day. You just want to slap them sometimes.

What we need is a tireless skeptical robot, that would catch people right as they were about to buy one of these products and give them a good solid “dope slap” to the back of the head. That would be awesome.

Of course, the robot idea has the same flaw as our outreach efforts: how do you get a robot to everyone who needs it? The marketers of these products are everywhere and have lots of money to spend. We don’t have a budget, and robots are kind of expensive.

But suppose these imaginary skeptical robots had other purposes too? Then folks might seek them out on their own, and we wouldn’t have to pay for their development.

Well, at least for when people buy products via a web site, the appropriate robot already exists. It is called Web of Trust, and (like The Mad Skeptic, who scooped me on this) I think skeptics should be promoting it and helping create its crowdsourced ratings.

In this post I’ll show you how we can use it and I’ll give you a look at what WOT’s ratings of skeptic and believer web sites look like already.

What is WOT? What is it not?

Web of Trust (WOT) is a service that compiles ratings on millions of websites. These ratings help you know if you can safely use a website you’ve never encountered before, or trust it with your personal information. With virus, malware and phishing attacks rampant on the internet, there is a real need for this type of service.

The ratings in the service are based on a number of sources of information, including whether the site is getting social links and whether it has been blacklisted by anti-virus companies. But the biggest component of the ratings, according to the service, are votes by users of the service. It is a classic use of crowdsourcing – a vast unpaid team of volunteers (the users of the service) help make it better through their contributions. To avoid potential for abuse, the software uses a Bayesian algorithm to factor in user reputations and other factors.

For convenience, the WOT ratings are available through a web browser add-on, available for Firefox, Chrome, Internet Explorer, Safari, Opera and other browsers. It is compatible with Windows, Mac OSX and Linux operating systems.

Once it is installed in your browser, the add-on flags hyperlinks in many web pages with little round icons. Green for good, yellow for unsatisfactory and red for poor. So even before you click, you can see that where there might be a problem. This works on most normal web pages including Google results pages, Wikipedia pages and so on. It has recently been updated to support Twitter and Facebook (with their common use of shortened URLs) as well.

But what if you click anyway? Then the real fun begins! The robot wakes up, in the form of a screen similar to what you see to the right. (Click for larger version). This is the actual warning I get when I click on the infamous Jim Humble’s web site in my browser.

If that isn’t a dope slap, I don’t know what is.

If you want more information, you can click the button indicated and see (in most cases) the exact reason the website has been given this poor rating.

You’ll note on that scoreboard page that WOT rates sites on four different criteria:

• Trustworthiness – Do you trust the content of the website?
• Vendor reliability – Is the site safe for business transactions?
• Privacy – Does the site have an appropriate privacy policy?
• Child Safety – Does the site contain age-inappropriate material?

The purpose of these four ratings are explained in much more detail on the WOT website. Suffice it to say that skeptics are primarily going to be interested in the “Trustworthiness” rating.

It is important to note that WOT is not really intended to point out misinformation. It’s intended to point out safety and privacy risks. As annoying as misinformation is, it is not always a threat to safety. But there is some dispute in the WOT community. As skeptic Jesse Brydle (already a user of WOT) told me:

The community is split somewhat when it comes to downright lies and misinformation. Some say the service is just for standard Internet-based overt spam, scams, malware, etc. and that anything more esoteric or content based is an issue of free speech and buyer beware – not their domain. They understandably don’t want to get involved in essentially doing meta-analyses of scientific journals before giving out ratings. Others see conventional scams like homeopathy or power bands as well within their domain.

But WOT definitely is intended to help you avoid scams and other online financial risks. Where misinformation becomes a product that is sold for money, that is squarely where WOT’s goals and skeptic’s goals align. And that’s where we can focus our efforts.

How can skeptics use WOT?

We should be promoting WOT as a useful tool to avoid bad things on the internet, whenever we can. The more people using WOT, the more effect it will have.

But skeptics should also become users of the service ourselves, so we can help give appropriately negative scores to the sites that are selling products based on lies and misinformation. Be sure to create a login on the site to facilitate this.

Then, you can simply rate sites that you encounter that are skeptic relevant. Click the WOT toolbar icon to get the scoreboard for the site, and then choose Edit my rating. You can then click inside the red/yellow/green scales to assign your score on one or more of the four criteria.

Another feature of WOT is the ability to leave comments with the ratings. These comments should explain (clearly, simply, and without snark) why the product or website is fraudulent. Here’s an example on WOT’s rating page for the Power Balance website. It simply says the site is making untrue claims and links to a supporting BBC News story.

As you can see from the PowerBalance scorecard, WOT users have been all over this high-profile product. As another example, in the WOT Forums you can see one user sought out many of the different sites selling Jim Humble’s MMS and gave them negative ratings.

Now, there are other services that allow you to leave critical comments that are tied directly to dangerous websites. Google’s Sidewiki experiment allows users to edit a small wiki page that is tied to other websites. Some skeptics like Travis Roy of Granite State Skeptics have used that to tie FTC Warnings to the sites that they warn against. This is useful, but I don’t see Sidewiki getting as much traction as WOT has already.

Just as I advised in my post on skeptics editing Wikipedia, always be aware of the rules of the service. You should only rate sites that you have actually visited yourself. You should only rate on criteria that actually apply to that site. For instance, don’t give a negative child safety rating to a site unless it actually has age-inappropriate material. As Jesse put it to me:

They don’t like it if you rate a site poorly across the board just because you don’t like their message.

Skeptics should not open ourselves up to accusations of abuse. Play by the rules!

How are we doing?

That potential for abuse got me wondering – are there skeptic websites that have received negative ratings from disgruntled believers? And aside from the anecdotal examples cited above, how are the “believer” websites doing in WOT ratings? I decided to find out.

It turns out that WOT has a very simple API that allows the current ratings for sites to be obtained automatically. I happen to have a database of about 900 skeptic and 2,400 believer website addresses that I’ve used in the past for other research. It’s not exhaustive by any means, but it certainly has all of the most popular web sites in it and should be a decent enough sampling.

One limitation of WOT is that it rates based on domain name, not by URL. This means that sites which share a domain will share ratings too. (For example, the blogs Pharygula and Respectful Insolence are both located on scienceblogs.com, so they will both have the same WOT rating numbers).

I started with 3,410 URLs. I wrote a script that would boil my list of URLs down to domain names, request the ratings from WOT, and dump the results into a spreadsheet for analysis. Once duplicates and some other uninteresting sites were removed, I ended up with 894 skeptic sites and 2,421 believer sites. I discounted the 185 skeptic sites (21%) and 694 believer sites (29%) that had no Trustworthiness rating at all, leaving 709 skeptic sites and 1727 believer sites.

Naturally I was hoping that skeptic sites would all have very high Trustworthiness, and believer sites would rate low. I was disappointed to find the spread, while present, is not especially large. As you can see from the pie charts, skeptics do have a higher percentage of green icons, 97% versus 84%. But the difference is not what we might like to see. And some notorious believer sites have quite high Trustworthiness ratings in WOT.

In these charts I chose not to show the five different colors that WOT uses, but just grouped the ratings by the color of icon that WOT would use for these URLs (green for excellent or good, yellow for unsatisfactory, and red for poor or very poor).

WOT ratings are accompanied by a confidence value (shown by an icon of 5 cartoon people on the site) which gives you a rough idea of how “solid” that rating is. A site that has only been rated by one person will have a very low confidence value, and its rating might change easily if others chime in. A site that has been rated by thousands of people will have a high confidence value, and the rating is unlikely to change very rapidly.

Confidence values are interesting because they give us an idea of how much of an effect skeptics are likely to have on WOT ratings. The news here is better.

As you can see, the majority of both skeptic and believer sites have zero or very low confidence values on their Trustworthiness ratings. Usually this means only a handful of people have rated these sites. This means that if skeptics were to chime in here, we might have a profound effect on ratings.

You might be wondering at this point: why should skeptics commit time with this particular service? There are other services like HONcode or even the various anti-malware products that do similar things. This one might go out of business tomorrow. A few reasons.

First, WOT is the only one of which I am aware that has crowdsourced ratings. That means we can directly affect the results.

Second, if you are concerned about whether WOT will stick around, be aware that the service is at least 3 years old.

Third, WOT ratings are increasingly being used by other tools and services that are visible to end users. For instance, the LinkExtend browser toolbar, among its many other features, will show WOT ratings. The reputation/safety checking service URLvoid uses WOT ratings as one of its signals, as does Global Sign’s Website Passport service. Because WOT makes its ratings available via the aforementioned API, there are many opportunities to build services on top of it.

Remember the Home Front

Once you start using WOT, don’t forget to rate good websites too! We need to make sure that links to skeptic articles will show nice clean green icons next to them for WOT users, so people will feel safe to click on them. Be sure to give appropriate ratings to the skeptical websites that you visit regularly.

If you are a skeptic webmaster, be sure to check the ratings on your own site(s) regularly. Bayesian magic aside, the opportunity for abuse does exist. I’ve found a few skeptic sites (like Amy Tuteur‘s old home birth debate blog) which have low ratings that clearly seem to be the work of disgruntled believers. So it can happen.

Don’t forget to check alias domains if they have been used by others to link to you in the past – remember that WOT puts the icon on the link based on the domain in the link, it doesn’t know that one domain has been forwarded to another.

If you use an RSS reader you can subscribe to a feed from your site’s WOT scoreboard page. Just click the orange logo in the lower right. This feed will give you a notification whenever someone leaves a WOT comment about your site.

NOTE: I followed up this post with another on WOT in June. Be sure to read that too.