Animals

canine confirmation confound – lessons from poorly performing drug detection dogs

Intuitively, the use of police dogs as drug detectors makes sense. Dogs are known to have a better sense of smell than their human handlers. Furthermore, they cooperate easily. Still, compared to the generally good picture sniffer dogs have in the public eye, their performance as drug detectors in real life is terrible. The reason why scent dogs get used anyway holds important lessons for behavioural researchers working with animals or humans.

Survey data coming out of Australia paints an appalling picture of sniffer dog abilities. Their noses hardly ever detect drugs that they are trained on. For example, only about 6% of regular ecstasy users in possession of drugs reported that they were found out by a sniffer dog they saw (Hickey et al., 2012). But once they bark, you can be pretty sure that a drug was found, right? No, you can’t be sure at all. An Australian review by the ombudsman for New South Wales found that nearly three quarters of dog alerts did not result in any drugs being found. It’s clear: using sniffer dogs to detect drugs just does not work very well.
drug detection, military, dog

Both looking in the same direction. Who is following whom?

This raises the question why scent dogs are actually used at all. My guess is that they perform a lot better in ability demonstrations compared to real life. This is because in demonstration scenarios their handlers know the right answer. This answer can then be read off unconscious behavioural cues and thus guide the dog. This is exactly what a Californian research team led by Lit et al. (2011) found. When an area was marked so as to make the handler believe that it was containing an illicit substance, more than 80% of the time the handler reported that his/her dog had found the substance. However, the researchers in this study misled the dog handlers and in fact never hid any illicit substances, i.e. every alarm was a false alarm. Interestingly, when an area was not marked, significantly fewer dog alerts were reported. This suggests that the dog owners control to a large extent when their own dog responds. Apparently, sniffer dogs game the system by trusting not just their nose but also their handler when it comes to looking for drugs. This trick won’t work, though, if the handler himself doesn’t have a clue either, as in real life scenarios.
The deeper issue is that good test design has to exclude the possibility that the participant can game it. The most famous case where this went wrong was a horse called Clever Hans. Early last century this horse made waves because it could allegedly count and do all sorts of computations. Hans, however, was clever in a different way than people realised. He only knew the answer if the person asking the question and recording the response also knew the answer. Clearly, Hans gamed the system by reading off the right answers from behavioural cues sent out by the experimenter.
Whether reading research papers or designing studies, remember Hans! Remember that the person handling the participant during a test should never know the right answer. If s/he does, the research is more likely to produce the intended result for unintended reasons. This can happen with scent dogs (Lit et al., 2011), with horses but also with adult humans (see the Bargh controversy elicited by Doyen et al., 2012). Unfortunately, after 100 years of living with this knowledge, reviewers start noticing that the lesson has been forgotten (see Beran, 2012). Drug detection dogs show where this loss leads us.

——————————————————————————————————–

Beran, M.J. (2012). Did you ever hear the one about the horse that could count? Front. Psychology, 3 DOI: 10.3389/fpsyg.2012.00357

Doyen S, Klein O, Pichon CL, & Cleeremans A (2012). Behavioral priming: it’s all in the mind, but whose mind? PloS one, 7 (1) PMID: 22279526

Hickey S, McIlwraith F, Bruno R, Matthews A, & Alati R (2012). Drug detection dogs in Australia: More bark than bite? Drug and alcohol review, 31 (6), 778-83 PMID: 22404555

Lit L, Schweitzer JB, & Oberbauer AM (2011). Handler beliefs affect scent detection dog outcomes. Animal cognition, 14 (3), 387-94 PMID: 21225441

NSW Ombudsman (2006). Review of the Police Powers (Drug Detection Dogs) Act 2001 Sydney: Office of the New SouthWales Ombudsman
—————————————————————————————————————————-

ResearchBlogging.orgIf you liked this post you may also like:
Correcting for Human Researchers – the Rediscovery of Replication

images:

1) By U.S. Navy photo by Photographer’s Mate 3rd Class Douglas G. Morrison [Public domain], via Wikimedia Commons

—————————————————————————————————————————-

If you were not entirely indifferent to this post, please leave a comment.

Advertisements