I ran into this Wired article that tells the interesting story of the Pfizer failure in launching torcetrapib, a potentially revolutionary compound for cardiovascular treatment.
Although the conclusion is too skeptical and the article gets a little bit lost on the “correlation vs. causation” argument, I believe it’s a good example of what can happen when people become overconfident about their “theories of change.”
Cognitive biases like those of the Michotte balls are pervasive, as shown by Daniel Kahneman in his book Thinking, Fast and slow.
Pfizer’s scientists were so sure they had precisely understood the cholesterol pathway, that the company not only had invested more than US$1 billion in the development of this drug, but also US$90 million to expand the factory that would manufacture the compound, even before having the results of a full scale RCT.
Luckily, in the pharmaceutical industry RCTs are a legal obligation: instead of saving lives, the drug would have killed people.
RCTs aren’t perfect and shouldn’t be above criticism, but yes a thousand times. Every time there’s a major trial failure the reporting will have quotes from the people behind it saying they’re baffled by what went wrong; they were that certain it would work. See this recent NYT article for instance: http://www.nytimes.com/2011/09/08/health/research/08stent.html that ends with “Quite frankly, the results were a surprise”. They’re always a surprise to those who think they know the outcome based on theory or priors, which is why we test them (or should).
Thanks for your comment.
RCTs aren’t perfect and shouldn’t be above criticism, but yes a thousand times. Every time there’s a major trial failure the reporting will have quotes from the people behind it saying they’re baffled by what went wrong; they were that certain it would work. See this recent NYT article for instance: http://www.nytimes.com/2011/09/08/health/research/08stent.html that ends with “Quite frankly, the results were a surprise”. They’re always a surprise to those who think they know the outcome based on theory or priors, which is why we test them (or should).
Thanks for your comment.