The Middle Step: The Institutionalization of Cost Overruns

In 1967 the economist Albert Hirschman published a short book called Development Projects Observed. Hirschman had spent time in the field at eleven World Bank projects, among them a paper mill in Pakistan and a railway in Nigeria. The projects had all run into trouble somewhere. Most of them were finished and in use anyway, which is what caught his attention.

Hirschman drew an interesting conclusion from this. He said the planners' ignorance was what saved the projects. If they had known at the start how hard the thing would be, they never would have started. Because they didn't know, they started, hit the wall, and then had to get creative. The creativity is what got them through. He called this the Principle of the Hiding Hand. Providence hides the difficulties from us so we will begin, and then hides our own resourcefulness from us until we need it.

It is an attractive idea because it explains why smart people keep approving projects on numbers that turn out to be wrong. They are not lying, in Hirschman's account. They are doing what humans do, and it works out for everyone in the long run.

Many still believe this, both in public service and among voters. But Hirschman's thesis had a problem.

The test

Bent Flyvbjerg is a Danish economic geographer who spent his career at Oxford building a database of large projects. He held the first chair in major programme management at Oxford's Saïd Business School. The database started at 258 in 2002 and has grown to over 16,000, across more than 100 countries and 90 years. For each one he has the original estimate, the final cost, the original schedule, the final schedule, and where possible the promised benefits and the delivered benefits.

In 2016 he and Cass Sunstein took Hirschman's idea and ran it against the data. Sunstein is a Harvard law professor who ran the White House regulatory review office under Obama. He also co-wrote Nudge, a book about how predictable biases shape decisions, built on Nobel-winning research. So he had spent years thinking about how governments make decisions and how they fool themselves.

Flyvbjerg and Sunstein did find Hirschman's Hiding Hand in about twenty percent of the projects they looked at. Those went over budget and came out ahead anyway, because the benefits grew faster than the costs did. But with the other eighty percent, costs rose, benefits fell short, and no burst of creativity showed up. Flyvbjerg named this the Malevolent Hiding Hand, and it's not hard to see why.

Flyvbjerg calls the broader pattern the Iron Law of Megaprojects: over budget, over time, under benefits, over and over again, with nine out of ten large projects running past their cost estimate. Overruns of fifty percent are common, and overruns bigger than that are not rare. Across the full database, 91.5 percent of projects miss on cost or schedule or both. Only half a percent come in on budget, on time, and deliver the benefits that were promised.

Here is the part I find hardest to get past. Overruns have been roughly constant for the seventy years of comparable data. Better software, better engineering, better project management, and yet the average does not improve. Every country in the data has the problem. The engineering firms that write these estimates have had the same seventy years of results to look at, and the estimates have not changed. So it is not a competence problem. Competence has improved a great deal in seventy years. So something is holding the average where it is.

Flyvbjerg's answer is that the estimates are low on purpose, or at least low in a way that serves a purpose. A realistic estimate gets a project killed but an optimistic one gets it approved. Once concrete is poured, the project cannot be killed, and the real number can come out. He calls this strategic misrepresentation. He has a second phrase for the result: survival of the unfittest. The projects that get built are not the best projects but the ones with the biggest underestimates.

But does that go far enough?

What happens to the eighty percent

So Hirschman said projects that hit the wall get rescued by creativity. Flyvbjerg says most of them don't. But they still get finished, in some form, and there is a ribbon cutting.

So something happens between the moment the original plan is dead and the moment the project is declared done. I have started calling that the middle step, because I don't have a better name and neither does anyone else. The estimate has failed, the money has run short, the schedule has collapsed, and everyone involved has to find a way through that doesn't end in a half-built bridge and a set of resignations. It is easier to recognize than to describe, so I offer three examples, two from California and one from my own back yard.

California high-speed rail. In 2008 voters approved a bond for a train from San Francisco to Los Angeles. The ballot said $33 billion and service by 2020. The California High-Speed Rail Authority's draft 2026 business plan puts Phase 1 at $126 billion. The same document contains a bottom-up estimate of $231 billion that the Authority now describes as a high-end scenario rather than its plan. The current deliverable is 171 miles between Merced and Bakersfield in the Central Valley, no San Francisco or Los Angeles, and no earlier than 2033.

The Wallis Annenberg Wildlife Crossing. In 2023 Caltrans finished replacing four aging overcrossings on Interstate 80 in Placer County for $49 million, about $12 million apiece. The Wallis Annenberg Wildlife Crossing is an overpass for animals across ten lanes of the 101 in Agoura Hills. Caltrans put it at $87 million in 2018. By the time ground broke in 2022 the budget was $92 million, with an opening in 2025. Now $114 million, opening December 2026. The project's leader attributed the overrun to tariffs, inflation, and labor, and said it was "not that bad" given the times.

The I-395 Signature Bridge. Speaking of expensive bridges, six concrete arches over Biscayne Boulevard, the centerpiece of a rebuild of the interchange where I-95, I-395, and the Dolphin Expressway meet in downtown Miami. Contract awarded 2017. Original budget $802 million, finish 2021. Now it's $866 million and 2029. The contractor is suing the design firm, claiming that design errors and a bad batch of concrete cost it more than $400 million on top of that.

All three of these projects are in the middle step right now. All three will open, and when each one opens there will be a press release calling it a success. The high-speed rail one will be a success because a train runs from Merced to Bakersfield. The wildlife crossing will be a success because a mountain lion walks over it. The Signature Bridge will be a success because it is finally done.

That is what the middle step produces. Not the project that was approved, but the project that can be called finished.

Five common ways through the middle step

Intriguingly, this middle step is not improvised. It is a set of mature professional practices, each with its own firms, its own methods, and its own vocabulary. Projects are rescued with consultants and a common rescue practice, not creativity and new engineering.

I count five tools in this industry.

A little more on each.

Claims. When a project goes late and over, the owner and the contractor fight about who eats it. That fight has its own profession. Firms like BRG, HKA, Ankura, and FTI employ people whose only job is to reconstruct the project schedule after the fact, identify each delay event, and assign it to a responsible party, and there are competing methodologies with names like Time Impact Analysis and Windows Analysis. There are published protocols from AACE International and the Society of Construction Law. There are expert witnesses who testify about nothing else. The Signature Bridge is on this tool now. The contractor says the design firm's wind-load calculations were wrong and cost them $400 million, the design firm disagrees, and the bridge is still being built while they argue.

Rebaselining. When cost or schedule blows through the plan, the agency adopts a new plan. From then on, the project is measured against the new number and the old number stops appearing in status reports. In federal defense procurement this is written into law. A Nunn-McCurdy breach triggers a review, and the usual result is a new baseline and continuation. The F-35 had a critical breach in 2010, was certified, and kept going. The GAO has been pointing out for years that this makes the overrun disappear from the books. The project is now "on budget," against a budget written after the money was spent.

Descoping. If the number can't be hit, change what the number buys. California high-speed rail is the clearest current case, with voters having approved San Francisco to Los Angeles.

Program management. Large public agencies usually don't have the staff to run a megaproject. They hire a program management consultant, often on a cost-plus or time-and-materials basis. The California state auditor flagged this on high-speed rail in 2018, noting that the Authority depended on consultants for work its own staff should have been doing. Fees go up when the project runs longer. That is not a conspiracy. It is just the incentive that the people managing the middle step are working under.

Communications. Every overrun comes with a press release, and the press releases all read the same way. Tariffs. Inflation. Labor. Supply chain. Unforeseen site conditions. Weather. The Signature Bridge's 2022 delay was attributed to the pandemic, supply chains, design time, weather, and work restrictions during special events. The wildlife crossing used tariffs, inflation, and labor. None of those is a lie, but notice that none of them is anyone's decision, either. Aside from the pandemic, every one of them was predictable, and every one of them leaves nobody accountable.

It's worth noting that cost overruns leave a paper trail that is visible to auditors. The people running the plan usually know from the start that an overrun is likely, and they watch it approaching as if it were a scheduled part of the project. There are no consequences for anyone who observes this problem and does nothing, regardless of the many professional and legal obligations that exist on paper. Licensed engineers sign codes of ethics that require them to issue public statements only in an objective and truthful manner. Certified project managers sign a similar code. Public officials have a statutory duty to report accurately to the bodies that fund them. All of that is in force on every project in this post.

The one time I know of that consequences actually followed seems to prove the point. In 2000 the head of Boston's Big Dig was fired after the project was found to have been carrying $1.4 billion in known overruns while reporting itself on budget. He was not fired because taxpayers had been misled but because bondholders had been, and the SEC got involved.

What we learned instead

Seventy years of data and the overrun average has not moved, but something did improve. The five tools above did not exist in 1950, or existed as somebody's side job. Now each one is a career. Forensic schedule analysis has professional societies and published standards. Rebaselining has a legal framework. Descoping has a business plan template. There are people who have spent thirty years learning how to carry a broken project from the day the estimate dies to the day of the ribbon cutting, and they are good at it.

So the skill did develop. It just developed on the wrong side of the problem. We did not learn how to estimate a megaproject. We learned how to survive having estimated it wrong.

Hirschman thought the middle step was where the creativity lived. He was partly right. There is a lot of creativity in the middle step. It goes into finding a version of the project that can be declared done, and a version of the story in which nobody made a bad call.

---

Sources: Albert O. Hirschman, Development Projects Observed (1967). Bent Flyvbjerg and Cass R. Sunstein, "The Principle of the Malevolent Hiding Hand; or, the Planning Fallacy Writ Large," Social Research (2016). Bent Flyvbjerg, "What You Should Know About Megaprojects and Why," Project Management Journal (2014). Bent Flyvbjerg and Dan Gardner, How Big Things Get Done (2023). Richard H. Thaler and Cass R. Sunstein, Nudge (2008). California High-Speed Rail Authority Draft 2026 Business Plan. California State Auditor, report on the High-Speed Rail Authority (2018). Caltrans press releases on the I-80 Placer County overcrossing replacements (2023) and the Wallis Annenberg Wildlife Crossing. 101wildlifecrossing.org project FAQ. Florida Department of Transportation project updates on the I-395/SR 836/I-95 Design-Build Project. NBC6 Investigates reporting on Archer Western-de Moya JV litigation, February 2026. U.S. Government Accountability Office reports on Nunn-McCurdy breaches and program rebaselining. ASCE and NSPE Codes of Ethics. PMI Code of Ethics and Professional Conduct. Massachusetts Turnpike Authority and SEC records on the Central Artery/Tunnel Project cost disclosures, 2000.