Have You Double-Blinded Your Dog Today?

It’s hard not to like dogs. Even if you don’t like them, you’d better say that you do. After all, what kind of person are you if you don’t like an animal that is so clearly and completely into humans? Canis lupus familiaris has been with us so long that we can’t help but stick up for it. As a result, I’m a bit hesitant to cast aspersions at the recent spate of feel-good stories about doggie superpowers, but perhaps I’ll get a pass if I frame it in terms of human error. I’ll also start with the good news.

As research news outlets mentioned a few weeks ago, two studies have underscored dogs’ apparent ability to detect human cancer by sniffing clinical samples. In one, researchers tested the ability of a trained black Labrador retriever to differentiate between the breath and stool samples of healthy controls and those of patients with colon cancer. The dog’s accuracy ranged from 95% for breath samples to 98% for stool.


Meanwhile, another group trained a Belgian Malinois shepherd to recognize urine samples from patients with prostate cancer. In that case, the canine sniff test was 91% accurate. Interestingly, the dog even managed to identify one case that the researchers weren’t aware of. After the animal indicated a positive result on one of the biopsy-negative samples, the team re-biopsied that patient, and discovered that he actually did have prostate cancer.

Given the invasiveness and risks of the standard diagnostic techniques for colon and prostate cancer, these “dog scans” could be a huge improvement. Colonoscopy requires a nasty bowel preparation procedure, anesthesia, and a long tube inserted into one’s nether regions. Prostate biopsy carries its own set of rare but potentially serious risks. Both procedures are expensive. Having a dog sniff samples of stool and urine could save money, time, and lives, right?

Unfortunately, it’s not that simple. As Sonoda et al. explain in the colon cancer paper:

It may be difficult to introduce canine scent judgement into clinical practice owing to the expense and time required for the dog trainer and for dog education. Scent ability and concentration vary between different dogs and also within the same dog on different days. Moreover, each dog can only conduct tests for a maximum of 10 years.

There’s another problem, too: the “Clever Hans” effect. A persistent bugbear of animal behavior research, the effect is named after a German horse that was initially thought to have amazing intellectual powers. As investigators later discovered, the horse was really just responding to subtle, unconscious cues from the people asking it questions. Hans wasn’t clever, just observant.

Animal behaviorists control for that by performing their experiments in double-blind fashion, coding their samples so that the experimenters themselves don’t know which is which until after the test, and therefore cannot bias the results. Both of the recent cancer studies used proper blinding and carefully controlled laboratory settings to address this, so we can trust their results. However, none of those controls are likely to be done in the real-world settings where service dogs usually work.

As Lit et al. recently demonstrated, service dogs are highly susceptible to the Clever Hans effect. In this study, the authors concealed various scented items in a series of rooms, then brought in 14 experienced teams of explosive-sniffing dogs and their handlers. In rooms where the researchers had placed a subtle visual cue indicating experimenter activity (a piece of construction paper taped to a cabinet), the dogs were much more likely to detect explosives, even when none were present. The most likely explanation is that the handlers noticed the paper and subconsciously cued the dogs that something was different.

The implications for drug- and bomb-sniffing dog teams are obvious. In the real-world settings in which those teams operate, the dogs are more likely to “alert” when doing so would confirm the biases of their handlers. If the handlers have preconceptions about the criminality of different ethnic, national, gender, or age groups, the dogs might engage in profiling by proxy.

Something similar could easily happen in high-volume clinical labs. Imagine a stressed-out MLS holding a sample cup for a dog to sniff. Based on the patient’s information in the computer, the technician may already have a bias about the likely outcome, and the dog, now sniffing its umpteenth sample in a long shift, would like nothing better than to please the human. The technician gets a coffee break, the dog gets a chew-toy, and a few days later Mr. Jones gets an unnecessary biopsy.

While doggie diagnosticians aren’t likely to work well, the new studies still provide a clue to what would. In both of the cancer studies, the dogs clearly noticed something different about the smells of patient samples compared to controls, so there must be volatile organic compounds (VOCs) in those samples that are unique to cancer. The next step is to identify those VOCs, and develop sensors that can detect them. The dogs may not be qualified to diagnose diseases on their own, but they did throw us a bone.

Sonoda, H., Kohnoe, S., Yamazato, T., Satoh, Y., Morizono, G., Shikata, K., Morita, M., Watanabe, A., Morita, M., Kakeji, Y., Inoue, F., & Maehara, Y. (2011). Colorectal cancer screening with odour material by canine scent detection Gut DOI: 10.1136/gut.2010.218305

Cornu, J., Cancel-Tassin, G., Ondet, V., Girardet, C., & Cussenot, O. (2011). Olfactory Detection of Prostate Cancer by Dogs Sniffing Urine: A Step Forward in Early Diagnosis European Urology, 59 (2), 197-201 DOI: 10.1016/j.eururo.2010.10.006

Lit, L., Schweitzer, J., & Oberbauer, A. (2011). Handler beliefs affect scent detection dog outcomes Animal Cognition DOI: 10.1007/s10071-010-0373-2