There's been an interesting discussion this week online about forecasting in pharma, kicked off by David Shaywitz's piece on a recent article in Nature Reviews Drug Discovery highlighting its wild imprecision. I think this paragraph from Shaywitz's piece sums up the key issue:
The underlying problem is that running a complex business depends upon forecasts. Big decisions – what to develop, what to in-license, what to terminate, what to resource – are driven to a large degree by sales forecasts – despite the fact that as bad as these are at the time of drug launch, they are, inevitably, even worse (to the extent this is even possible) when constructed years in advance of a launch. Faux objectivity can be dangerously compelling.
There's no argument to be made against the critical points: forecasts are inaccurate, and using them as a blunt tool to make R&D investment decisions, especially for early-stage programs, is bad management. However, I think the problem in pharma is with how these forecasts are communicated internally and used to inform decisions, not with their precision per se.
I encourage folks to read the original NRDD article, Shaywitz's commentary and the full range of comments, including mine. For those of you who are already up-to-date on the background, I've posted the full text of my comment below.
* * *
Great post and discussion – I’d like to amplify Jeffrey Seguritan’s comments and present a defense of pharma forecasting, flawed as it may be:
Fundamentally, I believe the forecasting process is important in pharma, even if the “answer” (in terms of peak year sales) is imprecise. In my experience, forecasting forces teams and the broader organization to methodically lay out and challenge their key assumptions as part of developing the strategy for individual assets and the portfolio. Which patient populations in lymphoma do we think our drug will address? When we launch our anti-hypertensive drug, what do we think the landscape will look like in terms of generic drugs, branded agents and device- or intervention-based therapies? Do we think the rate of diagnosis of Alzheimer’s disease will increase, decrease or stay the same in the future? How quickly do we think the middle class will grow in China?
Each forecasting input has uncertainty built into it, which leads to imprecision of the final peak year sales number. But by focusing on the key sensitivities and assumptions, we can still use the forecast to make informed decisions about how to strategically balance risk and investment. If the main influence on the forecast is technical risk (i.e., whether the drug will succeed), maybe we should invest in a small, fast trial aimed at a “quick kill”. If it’s competition from other pipeline agents, perhaps we should spend more upfront to set up and execute a Phase 2/3 adaptive trial that will enable us to leapfrog our competitors. If it’s regulatory, maybe we need to have earlier discussions with the FDA based on the data we have in hand. Whatever the factors – clinical, regulatory, commercial – forecasting shines a bright light on them and forces the organization to understand the key drivers of risk and reward.
In large organizations, the forecasting process also helps harmonize assumptions across the company. If we believe the diagnosis rate of early Alzheimer’s will increase over time, or that China’s middle class will not grow as quickly as the most bullish estimates, we should apply those assumptions to all relevant programs. (As an aside, most sell-side analyst forecasts are not nearly as granular as bottoms-up forecasts developed by companies – which not only helps explain their imprecision, but also makes it impossible to compare and contrast the underlying assumptions.) Without a rigorous and transparent process, I’ve seen forecasting assumptions vary widely across the portfolio and get buried in presentations to senior decision-makers.
“Transparent” is a key word in the prior sentence, because as you and several other commenters imply, the key limitation of forecasts is that they’re often presented as a “roll-up” number to executives, without discussing the key inputs in depth. I think this is the key problem with forecasts – not the underlying rationale for doing them or their lack of precision, but the way they’re used in decision-making.
Pharma doesn’t necessarily have a forecasting problem – we have a problem in how we communicate forecasts internally and use them to make decisions. Yes, forecasting is full of uncertainty, but uncertainty is embedded in the drug development business. When done well, communicated properly and thoughtfully embedded into decision-making, the forecasting process can be extremely powerful at informing strategy.