Behavior and Training · dog-human relationship · Science

The Nose Knows Bias

Despite the presumed superiority of our brains, humans are susceptible to a wide range of mental mistakes. These are collectively called “cognitive errors” and they impinge upon our judgement and can lead our decision-making astray. I have written about several of these previously in The Science Dog (attribution error, the availability heuristic and negativity bias), and also explore their impact upon our feeding decisions in “Dog Food Logic“. These are important to consider because cognitive biases (and their ne’er-do-well cousins, logical fallacies) actively interfere with our critical thinking skills and can cause us to come to conclusions that are not always logical or well-supported. A stunning example of this, and one that could have serious implications, was presented in a recent research paper about detection dogs.

We are influenced by confirmation bias not only when we seek out supporting evidence but also when we discount or ignore disconfirming evidence for strongly held beliefs. In this study, human perceptions of drug detection dog reliability (and validity) were studied. Reliability refers to a dog’s ability to be consistent across numerous search trials while validity refers to the dog’s accuracy at finding (and not finding) the substance for which he or she is trained.

The Study: Researchers collaborating at the University of Davis (California), Hobart and William Smith Colleges (New York), and University College London (UK), examined how beliefs about detection dog reliability influenced potential jurors’ decisions regarding a drug-related court case. A group of 554 jury-eligible US residents read a description of a criminal trial case study. In the scenario, a drug detection dog’s handler reported, on two occasions, first in a car search and second in a home search, that his dog had alerted to the presence of drugs. However, no drugs were found in either location. A large sum of hidden money was found in the suspect’s home where the dog alerted, and the suspect was arrested on drug-trafficking charges. After reading the case, participants were asked to do three things: (1) Issue a verdict (guilty or not guilty); (2) Rate their level of confidence in their verdict (rated between 0 and 100); and (3) Rate their level of confidence in detection dog alerting.

Following this phase of the experiment, participants completed surveys that examined their beliefs about science, their motivation to think deeply about new information (called the “need for cognition” scale), and their beliefs about the abilities and degree of infallibility of drug detection dogs. They were also asked a series of questions regarding their perceptons about the regulation and certification of detection dogs.

Results: After reading the case, approximately 2/3 (66.5 %) of participants rendered a not guilty verdict and 1/3 (33.5 %) rendered a guilty verdict. The researchers also reported that:

  • Participants who assigned a guilty verdict had statistically higher belief scores in the reliability of detection dogs compared with participants who assigned a not guilty verdict. They also rated their personal confidence in the verdict significantly higher.
  • Other than their level of confidence in detection dog infallibility, no other factors (need for cognition score, belief in science, prior experience with dogs, or participant demographics) influenced participant decision-making. (Emphasis my own; this result just blew me away).
  • There was extraordinary agreement among all of the participants about detection dog reliability, scientific basis, and certification:
      • 96.2 % believed that detection dogs provide a reliable method for finding hidden items.
      • 71.2 % believed that detection dogs will only alert when they find their trained odor.
      • 80.0 % believed that a drug detection dog who alerts in the absence of drugs must be alerting to residual odor.
      • 87.2 % believed that both the training and the certification of detection dogs is based upon scientific research.
      • 82.5 % believed that detection dogs must be certified through a rigorous set of state and/or federal criteria.

Conclusions: The participants of this study believed almost universally that detection dog evidence is reliable and that their training/certification is based upon scientific research. Strong convictions about detection dog infallibility were associated with finding a defendant to be guilty of drug trafficking, even when participants were told that no drug evidence had been found.

CONFIRMATION BIAS IN THE COURT ROOM

Are Detection Dogs Reliable? Are they Certified? These are important issues when the evidence provided by these dogs may be used in a court of law. A 90 percent reliability rate would mean that these dogs are correct 90 percent of the time (and incorrect 10 percent of the time). Legal experts state that a 10 percent error rate would still be of concern because one must consider the 10 percent of defendants who could be adversely affected by these errors. Unfortunately, the evidence that is available in the scientific literature (and there is not very much of it), reports success rates of trained detection dogs that are significantly below 90 percent. Numerous factors influence and limit this reliability. These include the subjective nature of interpreting a detection dog’s behavior, the range of training methods used, and variability in proficiency among dogs. Second, despite public perception, there are no third-party or nationally mandated standards for certification. While there is a growing body of information that provides “best practices’, empirical data to test these practices is limited. (See original study for all references).

Up on My Box Again: The implications of this study for juries, courts of law, and justice are profound. However, I am not a legal expert and cannot speak to these ramifications. Rather, I am interested in and can speak to the broader issue of human perceptions of and beliefs about dogs. This study demonstrated the existence of a pervasive and unfounded trust in dogs’ ability to accurately seek out and identify the smell of a target odor. Some people apparently also believe that dogs trained for these tasks do not make mistakes and that if they alert, then the thing (drug, bomb, cancer cells, name your poison), simply must be there or have been there in the past, because a dog never lies…..or makes a mistake……or is poorly trained…..or, is simply bad at his job.

Really?

How is it that we humans are willing (tragically) to accept the existence of pilot error in plane crashes, surgical mistakes in hospitals (make sure you label that limb), poor driver judgement on highways, and of course, a myriad of our own personal shortcomings and mistakes…….yet, a substantial number of people seem to have placed the dog upon some mythical pedestal of fault-free existence whenever their noses are in play? Unfortunately, these erroneous beliefs seem to be perpetuated rather than dispelled by many trainers. The use of platitudes such as “trust your dog“, “your dog is never wrong“, and “the dog’s nose never lies” abound in the dog training world. In my view these bromides provide tacit, though perhaps unintentional, support for the baseless belief in the infallibility of the dog’s nose and the unrealistic expectations about canine behavior that follow from it.

Don’t get me wrong. Dogs are the center of my life and I am continually grateful for and humbled by all that they bring to our lives and all that they (and their incredible noses) are capable of doing for us. However, I do have trouble, a lot of trouble actually, with the “dogs are infallible” myths that abound. The reality is that dogs, no matter how well-trained and for what task, just like humans, can make mistakes. Believing in their infallibility not only has the potential for dire consequences for humans (as was demonstrated in the case used in the study; remember that 33 percent of people rendered a guilty verdict for drug trafficking charges, based only on the dog’s alerting and when no drugs were found), but also for the dogs themselves.  What happens when Muffin suddenly stops providing emotional support to the autistic child she is placed with, when Samson misidentifies a friendly visitor as a burglar, when Buffy’s temperament is actually found to be unsuitable for working with the elderly, or when the drug dog stops being motivated to work or makes multiple errors?

When a dog makes a mistake at a job (a job that they rarely select for themselves), or fails to meet expectations, the consequences for the dog may range from retraining to retirement to rehoming or worse.  We place such enormous responsibilities upon dogs and expect so very much of them – find that drug, support that disability, detect that illness, comfort that person – AND  some of us continue to have an unshakable trust in their abilities. As we continue to perpetuate fictions of the dog’s olfactory super power, we conveniently ignore the potentially bad consequences to the dogs for failing to meet our romanticized expectations.

We need a middle ground. We need more evidence-based understanding of the dog’s scenting capabilities, empirically tested training practices, and standardized certification programs for all types of working dogs. We also need to let go of dog myths that elevate and romanticize dogs, and that cause errors in our judgement and decisions about them. Such mythologies serve no one well – neither ourselves or our dogs.

Nuff ‘ said. Off box.

Cited Study: Lit L, Oberbauer A, Sutton JE, Dror IE. Perceived infallibility of detection dog evidence: Implications for juror decision-making. Criminal Justice Studies 2019; https://doi.org/10.1080/1478601X.2018.1561450


New! Science Dog Webinars! Click Here for Information about Upcoming Topics!


Interested in learning more about dog training and behavior? See Linda Case’s newest book – “Dog Smart: Evidence-based Training with The Science Dog“!


 

 

5 thoughts on “The Nose Knows Bias

  1. Thanks for this article. I’ve often wondered about the training techniques used for detection dogs. Are any of them science based? How is a ball motivated dog a better detection dog than a food motivated dog? Is there scientific evidence for this, or is it a hold over from early 20th century military style training that the best dogs don’t need food?

    Like

  2. I might be missing something, but as I’m reading the summary of this study, it seems to me that even though:
    96.2 % believed that detection dogs provide a reliable method for finding hidden items.
    71.2 % believed that detection dogs will only alert when they find their trained odor.
    80.0 % believed that a drug detection dog who alerts in the absence of drugs must be alerting to residual odor.

    There was still only 1/3 of participants that rendered a guilty verdict.To me that suggests that although they are saying they believe that a drug detection dog alerting in the absence of drugs is alerting to residual odor, in their actions the majority are not showing this belief. Or at least, they are finding reasonable doubt regarding the drug trafficking charges.

    Of course, that also means 1/3 did not find reasonable doubt even though no drugs were found. Might this also be akin to the level of confidence jurors also ascribe to eye witnesses, even thought their is plenty of data showing eye witness accounts can be very problematic? My neuroscience training has also found these data fascinating.

    Like

  3. In my experience, both as a jurror and a dog trainer who has trained scent work, it isn’t the dog who makes the mistakes. Handlers often use preconcieved training indea; cannot read the dog; reward for a specific behavior; insist the dog “find it” when :”it” isn’t present. Unless both ends of the leash are properly trained; unless the end of the leash with thumbs doesn’t have any preconcieved ideas; unless proper continued training for both ends is on going, yes, mistakes will be made, but it isn’t the dog’s nose that fails. It’s the human brain.

    Like

  4. “Other than their level of confidence in detection dog infallibility, no other factors (need for cognition score, belief in science, prior experience with dogs, or participant demographics) influenced participant decision-making”

    Agree that’s just scary but not at all surprising. If anything it serves to reinforce my belief we should not be giving snippets of info about which jurors have no other knowledge nor need for them to be sure of what the relevance is (or isn’t) and base their judgements and decision.

    I don’t think we should have jurors full stop but more so when it’s a case reliant on a complex and specific area of expertise made even more difficult thing because every single dog is different and has its own little quirks, traits and behaviours but you don’t, can’t and won’t know about all this unless you’re aware that it’s always a possibility.

    In a former life I worked in law with a team that included some ruthless barristers and specialists whose attitude was always we were there to win our case and secure yet another successful prosecution. How they got it and whether it fair play did not matter. It was our job to present the case and in a way that means another brag and another fat bonus to pay for holidays and shenanigans. That’s when I decided it wasn’t for me.

    The thought of a similar approach when someone’s whole life is hanging in the balance and from such shaky and unreliable stuff to be putting forward at all is frightening.

    All I can picture is my old Springer who had some level of training when she was younger prior to us re-homing her. She was absolutely off her nut for shiny material and foil banners that kind of thing. Bear in mind how much shiny foil you’d expect to find inside a drug dealer’s house she’d never do anything else but have just thousands of innocent party-goers banged up for going off the radar at their homes.

    I’m laughing at the thought now but hopefully you see my point. “Your Honour… she was just nuts and really really liked shiny birthday banners” 😀

    Like

Have a comment? Feel welcome to participate!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s