The Numbers Never Tell the Whole Story

The autopsy of a troubled program is rarely surprising.

The surprise is how much of it was visible while the patient was still walking around.

That may be the single biggest lesson we have learned from more than thirty years spent recovering major programs.

When Boards eventually see costs rising, milestones slipping or investor confidence beginning to weaken, it is tempting to believe the program has suddenly run into difficulty.

In reality, the numbers are usually the last thing to change. Long before they do, the program has already begun telling its story. The challenge is recognising that story while there is still time to change the ending.

We had seen the patterns before

By the time we developed the Mentor Execution Index (MEI), the patterns themselves were already familiar. We had seen them on every major program recovery we had undertaken over more than thirty years.

It made little difference whether the objective was to launch digital terrestrial television, replace a major billing platform, build a national fibre network, deploy a 5G network, deliver a major software transformation or integrate two large businesses.

The industries, technologies and organisations were different, yet remarkably similar execution conditions kept reappearing.

Sometimes they appeared as fragmented ownership. Sometimes as weak dependency management, supplier misalignment, optimistic planning assumptions or governance that produced reports instead of control. The details varied from one program to another, but the underlying execution patterns remained strikingly consistent.

Eventually we stopped thinking of them as telecoms patterns.

They were execution patterns – and that raised a simple question.

If these execution conditions kept recurring, could they be recognised before they developed into delays, cost overruns and loss of confidence?

The Mentor Execution Index was our attempt to answer that question.

It was never intended to replace experienced judgement or become another management methodology. It was simply an attempt to discover whether decades of practical experience could be applied earlier and more consistently.

The challenge was one of scale.

If senior leaders, program teams and major suppliers were all going to describe how a program was really behaving, we would be dealing with a substantial volume of evidence. We needed a way of processing structured assessments and plain-English commentary consistently, then comparing what people were telling us with execution patterns we had seen repeatedly over many years.

That was where AI came in.

Not as a substitute for judgement, but as a way of organising evidence, recognising recurring themes and testing whether execution conditions could be identified earlier and more consistently.

Where the evidence came from

Looking back, however, the technology turned out to be the least interesting part of the exercise.

The real surprise was where the evidence came from.

We expected to contribute most of the insight ourselves. Instead, we discovered that the organisations already possessed most of it.

Using a common framework, we asked senior leaders, program teams and major suppliers exactly the same questions. Responses were anonymous because we wanted people to describe what they were actually experiencing rather than what they thought others wanted to hear.

Initially, we assumed the scores would be the most valuable output.

They weren’t.

The explanations were.

People described assumptions they no longer trusted, dependencies they worried about, suppliers struggling to meet commitments, responsibilities that nobody really owned and improvement activities that looked convincing on paper but were changing very little in practice.

Individually, none of those observations was remarkable.

Taken together, however, they painted an extraordinarily coherent picture.

That changed our thinking completely. We realised the problem was not a lack of information.

The information already existed. It was simply fragmented across the execution system.

Senior leaders understood part of what was happening. Delivery teams understood another part. Major suppliers often saw issues that neither group fully appreciated. Nobody was wrong.

The difficulty was that nobody could see the whole picture.

The same story kept repeating

As we continued applying the approach, our confidence grew that we were observing something much more fundamental than the problems of any individual organisation.

We saw the same execution conditions in Telefónica O2, BT/EE’s high-risk vendor replacement program, the Cellnex and Cornerstone cell build programs, CityFibre’s national fibre rollout and, more recently, the Roma 5G program.

These organisations had different objectives, different leadership teams, different suppliers, different commercial pressures and operated in different countries. Yet the underlying execution patterns remained remarkably familiar.

CityFibre is simply one example where enough time has now passed for the public outcome to be compared with observations made several years earlier.

It is not the only example we could have chosen. We have seen the same characteristics in software programs, media launches, infrastructure programs, network transformations and business integrations.

The setting changes – but the pattern does not.

This is not a criticism of CityFibre. Quite the opposite. The company has transformed the UK’s fibre market and established itself as a genuine challenger to the incumbent operators. That achievement deserves enormous credit.

The public record also shows that the rollout took longer, required significantly more capital and proved more difficult than originally envisaged.

From our perspective, those outcomes were entirely consistent with execution conditions that we believed, at the time, would make the original objectives progressively harder to achieve unless they were addressed.

Whether a different response would have produced a different outcome is impossible to know.

What we do know is that the public numbers eventually reflected execution conditions that had been visible much earlier.

The second discovery

Looking back, I don’t think the biggest discovery was the Mentor Execution Index itself.

The biggest discovery was that major programs usually contain the evidence needed to understand where they are heading. The challenge is bringing that evidence together and interpreting it before the financial consequences become visible.

But there was one more lesson to learn.

Recognising recurring execution conditions turned out to be only half the challenge.

The second discipline was: intervention.

Understanding what those conditions really meant. Deciding whether the organisation’s response matched the seriousness of the execution risk. Changing the trajectory while meaningful choices still remained.

That realisation eventually led us beyond the Mentor Execution Index and towards today’s Independent Program Review.

Because recognising the problem and changing the outcome turned out to be two very different disciplines.

I’ll talk about the second one next week.

 

About the author

David Hilliard is Founder of Mentor, execution specialists in strategic program execution.