Reverse image search as a skeptic tool – with a twist

The Beast of Trowbridge - in duplicate

The Beast of Trowbridge?

I must confess I’ve been remiss in not blogging about this particular topic earlier.  Investigating anomalous photos has always been a skeptic mainstay – for over a century and a half in fact. Ghost photos of one type or another have existed practically since the invention of photography. Those have been subsequently joined by photos of cryptids, UFOs and other alleged anomalous phenomena.

Now that practically everyone has a camera in their pocket all the time (in the form of a mobile phone) photos of this type pop up constantly – along with opportunities to investigate them. And so we need as many skeptics as possible to have some skills in investigating the latest local ghost or UFO photo.  There are just too many of them to send them all to Joe Nickell or Richard Wiseman.

Because photos are ubiquitous, and doctoring photos using software is so incredibly easy, tools for detecting photo manipulation (like FourMatch and Tungstene) are becoming more common.  But software like that can be quite expensive – out of the range of the average skeptic.

But today I’m writing about something far, far simpler.  In fact, it’s an incredibly easy way to detect obvious hoaxes based on stolen or misrepresented photos.  It should be in every skeptic’s toolkit, along with a trick for using it that I’ve never seen suggested before.

The Psychic Wayne Case

Although I haven’t blogged specifically about it, I’ve spoken about this technique before. In both my TAM 2012 and SkepTech talks I related the story of Irish skeptic Alan Rice‘s skeptical win over a cheesy psychic program running on late-night TV.

Alan noticed that during “Psychic Wayne’s” program, a seemingly endless series of other psychic readers were promoted via photos and names in the the lower third graphics as Wayne spoke.  He reasoned that it was unlikely that the company producing the program actually had that many psychics on call in the wee hours of the morning.  So he took screen captures and clipped out just the portion that showed a head shot of one of the other psychics and ran it through a reverse image search engine. He found something interesting.

Most people are familiar with image search – you type a word or phrase and the search engine shows you images that relate to it. You’ve been able to find photos and logos and diagrams this way for years. But a more recent development is reverse image search. Instead of typing a phrase, you upload or provide the URL to an image, and the search engine finds other images which are similar or are exact matches. They can be used to identify people, find the source of a piece of art, and so on – and can even find inexact matches, i.e. altered versions of the original image. Two well-known reverse image search engines are Google Images and TinEye.

In the Psychic Wayne case, Rice was able to crowdsource the task of searching a large number of the other pictures. He and other skeptics were able to use reverse image search to identify that at least 25 of the alleged psychics on call were actually simply stock photos, available for anyone to purchase. Do follow that last link and read Alan’s post about it, as its a great case study of straightforward skeptical activism using online tools.

And best of all, it had an effect. Between the embarrassment of this incident, and a number of complaints to the television station about the content of the program, pressure was brought on the television channel.  Before the end of 2012 the show had been cancelled.

The Beast of Trowbridge

Last week a local paper in England reported that a local legendary creature (“The Beast of Trowbridge” – a big cat of some kind) had been photographed. The article displayed the photo and attributed it to an ostensible local couple.

As Hayley Stevens documented on her blog, the photo was quickly found to be a fake. Not only was it not a recent photo (dating from 2007) but it hadn’t even been taken in England. It was a photo of an actual wild cougar (not a cryptid) in Michigan, in the United States.

I was interested in how this hoax was discovered, because Hayley doesn’t specifically say in her post, although she does credit the person who uncovered it. Obviously, reverse image search might have done the trick here.  But a quick try using Hayley’s copy of the image doesn’t turn up any useful results.  Hmmm.

If you’ve looked carefully at the photos in her blog, you might guess why.  The Trowbridge hoax image is left/right reversed from the original from Michigan.  It also has a time/date stamp on the lower part of the image (obviously also fake, we now know) that might be interfering with the match.  But this does suggest an experiment.

Manipulate Before Matching

Knowing Alan Rice and his cohorts had to crop the head shots to find them on stock photo sites, I figured some manipulation of the Trowbridge photo might help Google match the image. As an experiment, I pulled the original image into a program and left-right reversed it, prior to uploading.

This is easy to do, almost any graphics editing program will have a way to flip or rotate an image.  Even if you never edit graphics, you probably already have a program on your computer that will do this. Windows computers come with a program called Paint that can be used, just click the Rotate button on the toolbar, pick an option and save. On a Macintosh the Preview program that comes with OSX can also do this for graphic images – use Tools | Flip from the menu, then re-save the file.

I tried cropping the image to eliminate the time stamp, and also flipping the image. It turns out cropping is not necessary in this case, merely left/right flipping it will return this result from Google Images search:

Black cougar in michigan

As you can see, in big bold letters Google states this is a black cougar in Michigan.  The upper left of the six visually similar images is the one we’re after – clicking it eventually leads us to this page which is the actual 2007 source of the image. Voila.

Although we worked back from the result, it is clear that a left-right flip is a good choice here, because this photo has an obvious “up” direction.  Any other rotation or flip would result in an upside down or sideways image – not likely to match this photo.  (Although – you do occasionally see smartphone photos posted in the wrong rotation online).

But it is easy to imagine other cases such as UFOs or ghost photos where there is no obvious up or down, that various rotations or up/down flips might be worth trying.  With various combinations of flips and 90 degree rotations, there are eight possible orientations for any given image, as shown in this diagram:

Eight photo transformations

So we might have to create and upload eight files to try all the easy possibilities for a UFO or blobsquatch.  (If there is visible text in the image, you could limit yourself to four versions, skipping the ones that would reverse the text).  Rotations other than multiples of 90 degrees are possible, of course, but the combinations quickly get out of hand.

We can put all this together to create a best practice recommendation for skeptics.

Best Practice

Skeptics should adopt a best practice of using reverse image search to eliminate the possibility of an obvious hoax for any anomalous photo.  My recommendations include:

  • Become familiar with both Google Images and TinEye search.  (On Google, click the camera icon or drag-and-drop your photo onto the search box to perform a reverse search).
  • Both services offer browser plug-ins or bookmarklets to search images with just a click or two – install them. (Google supports Chrome and Firefox, TinEye supports the five major desktop browsers).
  • (UPDATE) Chrome users can use this extension instead, which supports multiple search engines. There is also a Firefox version of the addon.  (h/t Hayley Stevens)
  • Crop the image before uploading to search for only one particular section of the image, as appropriate (Psychic Wayne)
  • Images with an obvious “up” direction should be searched as-is and also left-right reversed (Trowbridge)
  • Images without an obvious “up” can be searched rotated and flipped, you may wish to try up to 8 versions to get all the possible permutations of simple flips and 90 degree rotations.

These simple steps don’t take much time, and can be done by anyone.  They won’t always turn something up, but occasionally they should result in some easy skeptic wins regarding hoaxed or misappropriated photos.

Further Resources

There is certainly a whole lot more to investigating anomalous photos – many books have been written about this subject.

If you are interested in other new techniques, I highly recommend the FourAndSix blog where Dartmouth computer science professor Hany Farid writes fantastic tutorials on digital image analysis techniques.

I also suggest you read Andrew Hansford’s investigation of a UFO photo (just posted yesterday), which he also presented at TAM 2013.  It’s a fine example of using online tools and knowledge of digital camera technology to debunk an alleged UFO photo.

(UPDATE) And of course Stuart Robbins presented an entire workshop at TAM 2013 on photo analysis, you can see various posts from him on this photo analysis here.  (h/t Reed Esau)

About Tim Farley
Focused on online misinformation, Tim Farley is a software engineer, computer security expert and scientific skeptic who created the site What's The Harm. He is a Past Fellow of the James Randi Educational Foundation.

15 Responses to Reverse image search as a skeptic tool – with a twist

  1. idoubtit says:

    I plan to use these tools in the future for sure. I think we can make some pretty quick debunking with the.

    One point of clarification on the particulars of the beast – I don’t see that the Michigan photo was authenticated as a cougar, let alone a black one. They have never been documented to exist in a melanistic state. The photo is CLAIMED to be a black cougar in Michigan but that’s highly dubious. That it was taken in Trowbridge is certainly debunked.

  2. John Nuttall says:

    Hi Tim,

    I’m the person referenced in Hayley’s blog. The method you describe to look for flipped images is exactly the one I used. Here is an extract from my post in the LAPIS Facebook group from 17th August…

    “To find the original source for the image:

    I downloaded the image from the Wiltshire Times’ article http://www.wiltshiretimes.co.uk/news/10610757.Beast_of_Trowbridge_spotted_in_Winsley/ , performed a horizontal flip in Photoshop, and searched for the image using a little used Google technique.

    The result was….”Best guess for this image: black cougar in michigan”…with a link to the site hosting the original image. Full details of this technique will appear in the forthcoming LAPIS Investigation Guide!”

    Flipping an image is a well known technique amongst the internet marketing community for preventing the discovery of “borrowed” copyrighted images. Hence, I always look for flipped versions of an image, should Google Images initially return a null result, because I know other variants sometimes exist.

    I’m very pleased to see the technique shared with the wider skeptical community via your well researched article, and I’ve bookmarked your blog for future reference.

    • Tim Farley says:

      Thanks! Good to have confirmation that was your method. Didn’t know internet marketers used this technique, all the more reason to have it one’s toolkit.

      I’ve suggested to one of the authors of the browser plug-ins for these searches that incorporating the flip-and-search into the browser would be a cool feature to have. We’ll see if he agrees.

  3. Reed Esau says:

    I’m surprised that Google Image search can’t yet compensate for the horizontal flip.

    Another resource is TAM2013 Workshop Presenter Stuart Robbins’ series on image analysis.

    https://pseudoastro.wordpress.com/category/image-analysis/

  4. Tim Farley says:

    Lo and behold I plugged Hany Farid’s excellent blog on Monday and on Tuesday there’s a brand new (and very skeptic-friendly) post from him on analyzing shadows in Apollo moon landing photos. Enjoy!

  5. Tim Farley says:

    Courtesy of John Ping, here’s an example from this very week where a photo contest was won with a stolen photo. Sure enough, if the contest judges had just used TinEye or Google (with a crop & flip) they could have avoided this controversy with just seconds of work.

  6. Pingback: Why Google’s Chrome is currently the skeptic browser of choice | Skeptical Software Tools

  7. Pingback: Content Roundup for August 2013 | Skeptical Software Tools

  8. Pingback: Virtual Skeptics #54 – 8/28/2013 | The Virtual Skeptics

  9. Pingback: Granite State Skeptics, skepticamp- The Third Year | Yankee Skeptic

  10. Pingback: Πρακτικά εργαλεία για σκεπτικιστές στο Ίντερνετ | On the way to Ithaca

  11. Pingback: My Skeptical 2013 in Review | Skeptical Software Tools

  12. Pingback: See how to debunk viral photos in seconds using image search | Skeptical Software Tools

Follow

Get every new post delivered to your Inbox.

Join 13,179 other followers