If you work in service delivery or customer support, you’ve probably heard of “scope creep” or the dreaded “overrun”. Why do projects frequently run over budget and time? Despite having defined requirements, a statement of work, a project plan, and great people, why do we encounter so many changes to our carefully planned architecture and schedule?
In this post, Steph Martin, Data Solutions Architect at ANS, looks at the customer journey and personas to shed some light on why projects are overrun. She doesn’t promise a solution – if it were that easy, we wouldn’t be discussing it – but hopefully, it’ll bring some insight and empathy into our relations within our delivery teams.
I come from a data background, and we talk extensively about ‘data platforms’ and ‘modern data platforms’, and that’s the first problem. No one outside the world of data knows what that means, and to be honest, no one inside does either.
In Systems Thinking terms, we might also refer to this as a ‘system’. For example, I’ve heard customers say, “The system doesn’t let me do that”, or “the system creates some data/reports”. It’s another unhelpfully abstract term lacking a clear definition.
Ray Ison, a Systems Thinking Professor at the Open University, describes a system as “an integrated whole distinguished by an observer, whose essential properties arise from the relationships between its parts”. Ok, that probably did not help, so let’s unpack it.
A system is more than the sum of its parts. A data warehouse doesn’t deliver a single source of truth, nor does Power BI provide data-driven decision-making; all the components, their interaction with each other, and human interactions with them create a ‘system’.
But the vital part of that sentence is ‘distinguished by an observer’; this is an issue of perspectives. Assume we have a customer asking us to build a data platform to ingest, store, and serve some data. Let’s look at some definitions of what this system might be from different perspectives.
I’m going to do this using a technique called a ‘root definition’ methodology. A root definition defines a system to do what by means of how to why.
First, let’s look at it from a project owner’s perspective, such as a project manager, programme manager, or CIO. They might envision:
- What – A system to improve decision-making.
- How – Using good quality data delivered in a timely manner and shared with relevant departments.
- Why – To improve the services we deliver to our customers.
This sounds reasonable and is a common description we get from our customers. How about the perspective of technical leads? For them, this might be:
- What – A system to reduce the unrestricted proliferation of uncatalogued data.
- How – Providing a centrally managed storage area with controlled sharing processes.
- Why – To reduce the risk of data breaches.
Or
- What – A system to facilitate and improve the performance of the reporting layer.
- How – By developing a robust data model.
- Why – To reduce the impact of inefficient or ad hoc queries on the data store.
These perspectives, while not necessarily conflicting, present different viewpoints on the purpose of the system. Let’s explore a third perspective from a data analyst. For them, this is:
- What – A system to produce data for decision-making.
- How – By automating data collection, processing, and analysis.
- Why – To reduce the composition and size of the team needed to produce reports.
This is a problem. Here, we have a perspective that sees the project as a threat to their job through downsizing or skillset changes. It’s crucial to note that there is no such thing as a wrong perspective. Maybe the project is not intended to make staff fear for their jobs, but if that’s what they feel, then it’s entirely valid.
I want to use this to illustrate a very real threat to the success of the project. The project owner likely wants this done fast for customer benefits, but they have no specifics. For the technical leads, things must be considered more deeply, which might add time and complexity. The analyst doesn’t want the project at all, so they may be unwilling to provide examples of processes they follow. Different goals and objectives result in different answers, challenges, and outcomes.
As a final exercise, I want to map this to the customer journey we undertake and the delivery teams involved.
The presales person talks to the project owner, creating a statement of work detailing the purpose of the system, ultimate goals and cost. The architects talk to the technical leads and discover new information, such as changes in services or configuration. This is included in HLD and LLD documents.
Data engineers build the solution and face challenges accessing customer data stores or mapping the data according to the model. Hopefully, someone is talking to the data analyst. But whether they’re willing to provide information in a timely fashion is up for debate.
There is, of course, overlap. Technical leads may be on calls with pre-sales, and architects will talk to the project owner.
This results in three personas in the delivery team talking to different personas in the customer team, getting different information each time. Pre-sales accuse architects of changing scope and not sticking to the statement of work.
Architects accuse pre-sales of not scoping the solution correctly, and data engineers can’t find the answers to their questions in any of the documents produced by the other two teams, which causes delays. No one is deliberately misleading anyone, and no one is to blame for the inconsistency in what the customer tells us at every project stage.
Now you see why there isn’t an easy solution; there never is whenever you introduce people into a system. Sure, we could get all the personas from both sides in a series of meetings, but logistically, that’s almost impossible, and who wants to spend that much time in meetings anyway?
So, I don’t have an answer, but I do have two calls for action.
- Try to identify the perspectives of the people you’re talking to and spot issues of compatibility and communication. We can’t change our customers’ operational model overnight, but we can identify the risks down the line or the scope and timeline impacts.
- Be kind to each other and respect the team. As I said, no person or team alone is responsible for scope changes, delays, and overruns because no one understands the whole picture. We also vary in perspectives, so no two ‘identical’ projects will look the same if the delivery team has different people. If we can recognise and appreciate the differences in our roles and communicate more, maybe we can head off some issues before they occur.
