Mansions of Straw

The Law of Perverse Consequences

A subset of the Law of Unintended Consequences, where an intervention or policy produces effects that directly contradict the original intention, resulting in an outcome that is the opposite of what was intended. Perverse consequences often emerge when the complexity of a system is underestimated, or when key variables or human behaviours are overlooked, leading to counterproductive or harmful results.

One of the problems with scientific discoveries is that most of them are not real. They are false, or pretend, discoveries. And they arise largely because the current system of funding and rewards inadvertently subverts proper science.

How does that happen?

Well, the current systems reward those who work the system and play the game.

And the first casualty?

Truth. Truth is the first casualty of this game.

And, over time, truth-seeking behaviours give way to progression-seeking behaviours.

Nowhere is this more apparent than in medical research.

While the bean counters may struggle to evaluate the quality of the science, they can definitely count beans.

The clue is in the name.

Counting outputs - publications, citations, grant awards, patent applications.

Only when planted in a seedbed incubator or other research enterprise do we find whether our beans will produce. Commercialization is the hardest and most unforgiving of test beds. In the crucible of R&D our beans are often toast.

As the venture capitalists will tell you, most fail.

Still, we do get to cosplay as entrepreneurs for a time.

But that can take quite some time.

In pharmaceuticals, for instance, there can be a ten year, or even longer, wait before our beans are seen to fail.

The secret to professional success? Move on before the past catches up with you.

In medical research, this amounts to a massive waste of money, a waste of time, a waste of lives. And there are massive opportunity costs arising from pursuing fruitless avenues of research.

In pharma and biotech the frustration begins when we can't even replicate the original findings that gave such initial hope. There's a lot of back and forth with the discovery institution. Candidate biomarkers are found to have nothing to do with the disease. Proposed new treatments grossly overestimate effect sizes. The effects are at best marginal, and at worst non-existent.

Readers of Apes in Lab Coats will not be surprised by any of this.

The current system does not lend itself to replication attempts.

Indeed some actively discourage attempts to replicate 'findings'.

Never attempt to repeat a successful experiment.

Better to publish the 'findings' and move on.

A mansion of straw rather than a house of bricks.

It may not be very stable but it looks pretty impressive from the outside.

Of course, in the close-out meeting with the funders we marvel at the sheer majesty of our new mansion. We highlight its use of innovative, ground-breaking new technologies and the new lightweight, state-of-the art building materials. We salute the novel data analytical supports - the statistical scaffolding - developed to make it stay up. We're very proud of those. We had to invent those ourselves - it kept falling over using conventional methods.

This is a world first, people.

But then it only has to stay up between the end of the meeting and the final report.

It can collapse in a heap after that.

Of course, with a bit of luck, it will last a little longer and we can squeeze out another tranche of funding for the next project before this one falls over. But we'll cross that bridge when we come to it. Hopefully, never. We're good with that.

In reality, often what happens is that we collect a ton of data on our biological systems and attempt to predict clinical outcomes. Collecting that data is a massive, massive effort requiring coordination across multiple centres, each with multiple physicians, and laboratory support from multiple sites and with the informed consent of patients and support of patient groups and patient charities.

Pants (adjective, British slang)

Used to describe something that is of very poor quality, ineffective, or disappointing. Calling something "pants" suggests that it falls far below expectations or lacks value, often in a blunt or humorous way.

Example: "This research is pants" implies that the research is poorly conducted, unreliable, or simply not worth taking seriously.

But what if we're looking in the wrong tissues? Or what happens if there is no relationship between the biology and our clinical scoring system. What if, wash my mouth out, the clinical scoring system itself is pants?

Our planned analyses yield zero, nothing, nada.

We spent three years collecting the data, two years in the laboratory, one year in data handling and preparation and we've got zip.

Well, we're not going to stop there, are we?

Time for some p-hacking.

First, we try this, and then we try that, and then we try this and that.

Nothing.

Then Bob comes along with an untried crackpot method published in 'Drosophila Weekly' and we give it a go. It's no better than what we had before, but it throws up some putative biomarkers and uses a novel analysis method. Spinning gold from straw. Result. Or at least enough to build a mansion of straw.

We turn a blind eye to any misgivings about the untried method. We don't know its properties. How likely is to pick up real effects? How likely is to miss them. How often will it generate spurious 'hits'. These are currently unknown.

And these are not abstract concepts. This is not statistical pedantry.

In medical research, these properties have real consequences. How likely are we to detect a patient with cancer? How likely are we to miss one? How often will we deliver unnecessary treatments to patients flagged as at risk?

All models are wrong, but some are worse than useless.

How about we squander one of our PhD scholarships and get them to investigate the operating characteristics of our putative biomarkers?

That might take five years.

We haven't got time for that.

Let's just run with it.

Make sure the University Press Office know.

Let them know we have another 'ground-breaker'.

We secure a patent on the putative biomarkers.

We publish the data in Nature Scientific Data. A nice publication output.

We upload the R code to Github and publish in PLOS Computational Biology. Another nice output.

Or, if we're lucky, we position it as part of the Cancer Genome, Cell Atlas, or other meta-projects and publish in Nature Medicine or Cell.

OMG. We have outputs coming out of our ears.

By now, the ground under the Press Office is shaking.

Our code spreads, like a virus, through the scientific community.

We count the downloads.

We count the tweets.

We count the citations.

We made it into a government report.

We have an Impact Study on our hands, people.

This research is pants.

Instead, a world class scientific approach allows everyone to say 'Look, we tried this, this, this and this. None of them worked. It was a chuffing disaster. With the benefit of hindsight we were looking for biomarkers in the wrong place. We should have spent more time evaluating sources of variability in the clinical scoring system. If we had our time over again we'd do this, this, this and this.'

But that's not how it works.

The best scientists have mastered both.

They may play the game, to win resources.

But the truth is what matters.

I was lucky, I got to work with a number of people that work like that.

The Master Builders of Science.

Not the Cowboy Builders.

Some Useful Definitions

Master Builders of Science (noun)

Scientists and researchers who rigorously adhere to established scientific principles and practices, prioritizing accuracy, reliability, and reproducibility in their work. They follow methodical approaches, validate findings thoroughly, and aim to build knowledge that stands the test of time. Reputable Builders of Science focus on understanding and integrity, creating robust foundations that future discoveries can confidently build upon.

Example: Reputable Builders of Science verify their results across multiple studies and share their methods openly, ensuring other researchers can reproduce or build on their findings.

Cowboy Builders of Science (noun)

Scientists or researchers who, driven by novelty or pro-innovation bias, may sidestep rigorous validation and quality control in favor of rapid or eye-catching results. Often willing to cut corners or rationalize unstable findings, they can produce results that appear significant but lack durability or reproducibility. Cowboy Builders of Science may prioritize quick gains or high-impact publications, sometimes creating scientific structures that collapse under scrutiny.

Example: Cowboy Builders of Science may jump on trendy techniques without fully understanding the limitations, leading to findings that don't hold up when other scientists attempt to replicate them.

Slow (Measured) Science (noun)

A scientific approach that values careful, deliberate research practices over rapid output, emphasizing depth, accuracy, and long-term significance. Advocates of Slow Science prioritize meticulous data collection, thorough analysis, and thoughtful interpretation, often challenging the pressure to publish quickly. This approach aims to build a reliable foundation of knowledge that withstands scrutiny, resists fleeting trends, and fosters meaningful scientific progress. By allowing time for reflection, peer review, and replication, Slow Science supports discoveries that are both innovative and enduring.

Fast (Quick & Dirty) Science (noun)

A research approach driven by speed and high output, prioritizing quick publication and frequent results. Fast Science often responds to the pressures of competitive funding, career advancement, and media attention, sometimes at the expense of thorough validation, replication, or refinement. While it can yield rapid insights and satisfy demands for timely findings, Fast Science may produce results that are less reliable, prone to error, or difficult to replicate, risking superficial discoveries over sustainable scientific progress.

The Project Management Triangle (or Iron Triangle) illustrates the trade-offs between Fast, Cheap, and Quality (or sometimes Good):

1. Fast and Cheap: When speed and cost are prioritized, quality typically suffers. This might mean cutting corners, using less robust methods, or skipping detailed checks to save time and money.
2. Fast and Quality: When speed and quality are prioritized, costs often rise, as premium resources, skilled staff, and more advanced equipment are usually required to meet high standards quickly.
3. Cheap and Quality: When cost and quality are prioritized, it generally requires a longer time frame. Slower processes and thorough verification may be needed to keep costs low while ensuring quality.

The triangle shows how you can optimize for two out of the three, but never all three at once, making it a helpful tool for highlighting the inherent limitations of any project.

In the crucible of R&D, true discoveries endure as precious gold beads. The rest just evaporate.