It seems every few weeks there is another report of a major transportation project falling behind scheduling and vastly exceeding its budget. In fact, a 2005 study of rail projects undertaken between 1969 and 1998 found that the average cost overrun of projects was 45%. Additionally, planners overestimated ridership figures in over 90% of projects, predicting 106% over the actual ridership numbers, on average. Why are planners and engineers consistently so far off on the estimating the costs and benefits of transit projects? Psychologist Daniel Kahneman offers an explanation and proposes a simple solution to improve forecasting in his book, Thinking, Fast and Slow.
Kahneman cites two primary reasons for the lack of accuracy in transportation project forecasts. The first is that plans and forecasts tend to be “unrealistically close to best-case scenarios.” The second problem is that forecasts are usually made using what Kahneman calls the “inside view.” The “inside view” of forecasting involves looking at the constituent parts of project (planning, materials, labor, etc.) and assigning costs to each part. In order to create more accurate forecasts, Kahneman suggests using the “outside view,” which is also known as reference class forecasting. Utilizing the “outside view” involves looking at similar projects to the one that is being forecasted, and using the actual costs of the similar projects to create a forecast for the new project. Using the actual cost of other projects limits the biases of forecasters and prevents them from using best case scenario projections. Choosing the reference class can still be problematic, but, used effectively, the “outside view” provides a superior way to predict the costs and benefits of transportation projects.
The Edinburgh Tram line extension in the United Kingdom is the first documented use of reference class forecasting for a transportation project. While both forecasts were still below the actual cost, the reference class forecast was $100 million closer than the more traditional forecast. Since this time this practice has been spreading throughout Europe. Reference class forecasting has yet to catch on in the United States, but if planners embrace the practice expect to see a decrease in the ubiquitous cost overrun headlines of transportation projects. Of course higher projected costs, even if they are more accurate, will likely decrease the political feasibility of many transportation projects.
Edited by Michael McNamee
Edited by Michael McNamee
Sources:
Kahneman, Daniel. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.
I just read a very interesting paper which contradicts the view that cost overruns are the result of a psychological bias towards the "best case scenario". The authors collected data on 258 transportation projects over the last 100 years and found a consistent underestimation of project costs. They contend that psychological or technical explanations don’t hold up to scrutiny because of the long history of underestimation. It is reasonable to assume that biases would be addressed and technical methods would improve accuracy over time, but they haven’t. These authors contend that political and economic self-interest explain underestimation more effectively. Economically, many of the private consultants and contractors involved in forecasting have some interest in underestimating costs in order for the project to be more likely approved. Politically, planners are beholden to politicians that have political self-interest in a project getting approved (so they can take credit). The authors actually captured evidence of some planners stating they deliberately reduce cost estimates to make a project more politically feasible.
ReplyDeleteI tend to agree with the conclusions of the paper, even if they are more cynical. Don’t you think that psychological bias or technical errors would improve over time?
Here’s a link to the paper:
http://escholarship.org/uc/item/98r01936