It’s a scorcher outside. Sure, you’ve got air conditioning at home and in the office, but no one can stay inside forever. Eventually you’ll have to face the heat. Can the very concept of justice survive such a situation?
In a widely-cited 2019 article in American Economic Journal: Applied Economics a pair of Canadian economists — University of Ottawa’s Anthony Heyes, who holds a prestigious Tier 1 Canada Research Chair in environmental economics, and Soodeh Saberian of the University of Manitoba — declared that as temperatures rise outside courtrooms judges become less able to perform their jobs in an impartial manner. The hotter it gets, the greater the chance they’ll hand down negative verdicts. The specter of climate change looms large and sweaty over these worrisome results.
The economists used the binary nature of final decisions on asylum claims in US immigration courts to test their hypothesis that external environmental conditions impact indoor work. Their data included 207,000 yea or nay verdicts delivered between 2000 and 2004 in 43 cities across the US, matched to nearby outside temperatures. It bears mention that every courtroom in question was air-conditioned to federal government standards.
For each six-degree C increase in outdoor temperature, Heyes and Saberian claimed an asylum applicant was nearly seven per cent less likely to win his or her case. “Do decision outcomes … depend causally on how hot it is outside on the day the decision is made?” the authors triumphantly asked in their paper. “Our answer is a resounding yes.”
To explain this surprising outcome they suggested, “High temperatures may stimulate temper, irritability … and other emotions that might induce one to judge to be less well-disposed toward a typical applicant.” Even more surprising, female judges were found to be significantly more susceptible to this effect. These results were backed by studies of convincing statistical work. Suitably proven, the findings have since been cited dozens of times in other academic journals and offered fawning media coverage.
“For the first time ever we show there is a link between the decisions people make indoors and the temperature outdoors,” Heyes told a University of Sussex publication. “With global average temperatures continuing to rise it is vital we understand these links so we can best work out how climate change will affect human well-being.” Co-author Saberian noted the results “counter the complacent but often heard [view] that as temperatures get hotter and hotter we can simply insulate ourselves from the effects of that by moving indoors and using better quality air-conditioning.” It seems there’s no hiding from hot, grumpy judges, particularly female ones.
In an American Economic Association podcast, Heyes broadened his findings to cover a much wider range of occupations: “If the performance of a judge is compromised, then equally perhaps all of those other high-skilled people working every day in millions of offices around the world — teachers or accountants or surgeons … their performance could be understood.” As temperatures rise outside offices around the world, it seems every profession will slowly become more crotchety and less reliable.
worried? Don’t sweat it. The future is likely to be far less terrifying than these heated results assert. And for that relief, we can thank the replication process.
Replication is a mundane but vital aspect of the scientific method that seeks to independently verify remarkable academic results. Given the predisposition in academia and the media for unusual or shocking findings that support popular narratives such as climate change, replication injects a necessary measure of skepticism to the process. A little digging can often reveal flaws such as deliberate fraud, unintentional errors or unsupportable conclusions.
In 2020 Holger Spamann, the Lawrence R. Grove Professor of Law at Harvard Law School, performed a replication study on Heyes and Saberian’s work using their own publicly available data set. The result is his informatively titled article “No, judges are not influenced by outdoor temperature” to be published shortly in the same journal that published the original study.
Junk Science Week: Net-Zero Edition — Terence Corcoran: The impossible scheme
Junk Science Week: Net-Zero Edition — Tammy Nemeth: Net-zero? New standards aim for absolute-zero emissions
Junk Science Week: Net Zero Edition — Terence Corcoran: A book to save the world from ‘zero’ science
In his investigation, Spamann uncovered numerous mistakes of basic data entry; there is room here to name only a few. The original article claimed all temperature readings were taken within 32 km of the relevant courthouse. Yet Spamann found court decisions from Arlington, Virginia were matched with temperatures from Arlington, Texas. The same mix-up occurred with Oakdale, Louisiana and Oakdale, Wisconsin, as well as a handful of other cities that don’t even share the same name. Temperatures were also not properly corrected for local times, meaning the data included irrelevant nighttime readings.
Of even greater significance, Spamann discovered more than a quarter of the alleged 207,000 judicial decisions were actually withdrawals or abandonments by the applicants themselves. As such, these outcomes cannot properly be ascribed to the tempers of over-heated judges.
“The bottom line is that after correcting the coding and data entry errors… [there is] no evidence of an effect of outside temperature or other weather on judging,” Spamann concluded. He then enlarged the data set to include several extra decades of asylum decisions. The conclusion once again: “The larger sample yields no evidence of a temperature or other weather effect either, and confidently rules out an effect of the absolute magnitude estimated in Heyes and Saberian.” (In a forthcoming erratum note to their now-discredited work, the Canadian economists attempt to salvage their original findings by re-doing the study with a much smaller data set, among other major changes; Spamann dismisses the outcome of this effort as “spurious .”)
Spamann’s convincing refutation is further bolstered by more recent replication work from Australia using an even larger collection of 2.8 million criminal cases covering 1994 to 2019. “We conclude that outcomes of criminal court cases are not influenced by fluctuations in temperature,” write economists Sally Evans and Peter Siminski of Sydney’s University of Technology, calling it “an unsurprising but reassuring result” for anyone worried about judicial fairness.
There are two lessons to be learned from the hot, grumpy judge affair. First, there’s no evidence climate change has destroyed the promise of a fair trial. Second, there’s no substitute for a bracing dose of skepticism.
Peter Shawn Taylor is a senior features editor at C2C Journal.