What do AI and smart data analytics have to do with program execution?

Harry Baldock here, Editor at Total Telecom. and today it’s my pleasure to be joined by Ian Waters, Partner at Mentor. Ian how are you doing today.

Ian Waters: I’m very well thank you Harry.

Harry Baldock: Today we’re here to discuss the Mentor Execution Index, a program health assessment tool that you recently launched. This tool uses state-of-the-art AI and analytics to help identify key issues that may have been designed into a program earlier in the program’s life cycle.

So my first question for you Ian is about the inspiration for the tool, what exactly inspired Mentor to apply data analytics and AI tools and techniques to the world of program management?

Ian Waters: Mentor has been around a long time, several decades, and over that period we’ve rescued over 100 programs. We’ve got a real understanding of what goes wrong on a complex program. We’ve developed quite a robust Blueprint on the things that you can do right to stop it from happening again. Historically our approach to rescuing programs would be a classic health check – evaluate the status of the program, evaluate it against all the learnings from our Blueprint, and then map out a path and set of recommendations to get it back to health.

It works as a process but we’ve always been conscious that there are drawbacks with this approach. It’s time-consuming it takes several weeks to do, it can be disruptive with consultants on-site with a team that’s often firefighting in the midst of a challenging program. We’re developing views on the program but they’re often qualitative, and inevitably it’s quite a bespoke process, it’s a one-off that isn’t repeatable or it’s hard to justify doing it multiple times. 

We wanted to apply the latest analytical tools, data analytic techniques, statistical modeling techniques, and AI to our Blueprint to really make our health check a much more robust process, and to make it much more quantitative in the way we evaluate the state of a program. We wanted to make it quick, something that we can do in days rather than weeks or months. Obviously, something that’s eminently repeatable and robust in the fact that you can run it periodically through the life of a program.

We looked around and we engaged a company called Predli, some Swedish guys set up by a data scientist out of the University of California in Berkeley, experts in the application of data analytic techniques and AI to business problems. Their claim to fame was the Berkeley Innovation Index, which looks at some innovation in companies on the West Coast of the States and tries to benchmark companies against the best when it comes to innovation.

We did the same really around program execution, to bring our understanding of our Blueprint on what you need to do to get a program to work well and apply those analytical tools and techniques together, so we came up with the Mentor Execution Index [MEI].

Harry Baldock:  can you give me some detail about how you use the Mentor Execution Index to help companies deliver their toughest programs?

We ask a wide range of program participants, we can ask the employees on the program, we can ask customers, we can ask suppliers, so an end-to-end view of a program, to complete an online survey anonymously. We run them through a set of 50 or so structured questions, a mixture of quantitative questions and qualitative questions, and with those responses from that survey, we’re able to use the latest statistical modeling techniques to build some algorithms that score the results to get us a view of that program.

How well aligned to the teams, how strong is the organization that underpins the program, how robust the plans are, how well a key supplier is performing, all those classic issues that drive programs to either success or failure. It’s an approach to quantifying issues that people are used to, for example, employee engagement tools, customer engagement tools, customer satisfaction tools, use these types techniques and we’ve applied it to the world of programs and we’ve been very pleased in the programs we’ve run.

We get very high response rates and we get very high-quality responses that really allow us to drill into what’s going on in a program and it gives us a set of insights on the state of a program, and what’s really going on under the bonnet that you can’t get with typical PMO tools and techniques like log risk registers. All important parts of a program, but they don’t really tell you what’s really driving success or failure. 

What we’re able to pull out of this Execution Index is a series of lead rather than lag indicators. Issues that are going to sink a program if they’re not addressed quickly, so we can make recommendations to take actions on a program before it gets into trouble, rather than waiting for it to fall over, and be months late, tens of millions of pounds over budget before remedial action happens. 

What we really love about this is we can do it quickly, we can do it in days rather than weeks or months, it’s not based on an expert view or an outsider’s view or third-party view, it’s the views and the words the people who really know what’s going on in a program, those right at the coalface of the activity.

What we really love about this approach is it allows us to take a truly end-to-end view of a program. In a world of increasingly complex supply chains when it comes to program delivery, we can look upstream to the clients and customer, downstream to their suppliers and subcontractors and get a real end-to-end view of what’s going on in the program.

We can pinpoint specific areas where problems might lie and we can do it regularly so you can do it at the start of the program and then you can take a pulse of the program periodically over the life of a program that might run 18, 24, 36 months on some of the big programs, and look at where you are, where you’re trending, how remedial actions improve or don’t, the performance of a program. It gives us a very robust framework for evaluating and addressing program problems.

Harry Baldock:  Finally I wanted to talk to you about how the MEI relates to the telecoms industry. What specific challenges do you see facing the telecoms industry right now and how can the MEI help overcome them?

I think it’s fair to say that the telecoms industry has a lot of complex programs underway or facing it. Whether it’s all the fibre deployments that are going on, tens of billions of dollars being deployed, large 5G deployments, companies having to migrate away from Huawei, looking to bring on loads of new suppliers, the open-ran environment, re-engineering their core networks, there’s a lot of complexity that needs to be delivered in terms of the opportunities facing the telecoms industry. The supply chains that people are dealing with are increasingly complex. Lots of new suppliers that people are evaluating and want to bring into the mix, so a lot of complexity to deal with, but they’re dealing with that in typically very tough financial environments.

You can’t afford to mess up these programs, they can’t afford to be written off late with huge overruns, and they’re doing it obviously in a Covid environment where the sort of informal chats that you might have that might flush out issues around a program just don’t happen or are much more infrequent than they might otherwise be.

We think that the Mentor Execution Index is really well-positioned to help with all these challenges. It allows our clients to take a really critical look at a program, to do it quickly painlessly, regularly flush out the issues and get them on the table, get them understood by management and across the program and with their suppliers and customers, and then allow them to address those issues before they cause real long-term damage to a program, and to track the success of those actions and see that you know that program throughout its life cycle.

Harry Baldock: It’s been a real pleasure speaking to you thanks very much for your time.

Ian Waters: My pleasure thank you harry