Here’s something to think about when you’re planning a big data project: are you planning a project or a program? Here’s a simple distinction:
- A project typically has a beginning, middle, and end.
- A program is something ongoing and relatively permanent.
Projects
Relatively self-contained big data projects may be tied to an ongoing process or program that is already developing or delivering a product or service. Defining the project’s requirements and developing its business case will be linked to existing performance measures or metrics. Answering the following type of question should therefore be simplified:
“How will the big data project contribute to the existing program’s success in terms of customer growth, customer satisfaction, profitability, or cost-reduction?”
Introducing big data tools or technologies to support an existing product or service can sometimes be viewed as a variation of “typical” system development project where software are being developed or adapted to support programs already understood and underway. Even when the specific contribution that a new analytical process or service will make is unknown, understanding of the value of the product or service being supported will tend to reduce uncertainty by making it easier to understand whether the big data service is itself useful.
Programs
A project to support development of an enterprise level “data science” program to provide data analytics support to a range of departmental programs and initiatives will be different both in type and scale. How such a service program will be embedded in the organization and how it will be governed will be of strategic concern with potential long term consequences and significant system and process integration challenges. That such an enterprise level initiative will lead to consideration of enterprise-level data architecture and standardization should be expected.
An uncertainty in such an enterprise level project to develop an ongoing support program might be difficulty in developing a convincing ROI analysis. This is due partly to difficulty in predicting what the benefits will be of potentially innovative or untried analyses. Adding to this will be uncertainties involved in making changes to corporate data governance and changes to how data and applications are managed (for example, via shifts to the cloud from internally managed resources).
Discussion
In the real world the distinctions between these two “big data” project types may not be so cut and dried. Even a narrowly targeted big data project that focuses on direct support for a well understood product, service, or process will encounter system and data integration challenges that raise interesting governance questions. Some of these questions will inevitably be political in nature.
This should not be a surprise to anyone planning or managing such a big data project and suggests the need for both project management experience as well as data science and technical knowledge, as discussed in You Need a Project Manager on Your Big Data Team.
Looking at the program level and potential integration challenges, a reasonable question to ask is, “How long till we see some benefits from this program?” I’ll be the first to emphasize the need to understand the relationship between any project and the organizational goals and objectives it serves. In many cases, though, we will be dealing with uncertainties in a variety of areas ranging from how to implement and manage new technologies to how to take advantage of new analytical powers.
Having been through several adoption cycles for new technologies myself I think the need exists not only for technical data science skills on the implementation team but also for leadership, sponsorship, and – most important in my opinion – the ability to demonstrate benefits and value early on.
One of my interviewees suggested the need for a “portfolio management” approach where one can compare and contrast a variety of initiatives in cost benefit terms. A practical example of this is how the GSA’s 18F program selects projects to address, discussed in The Commerce Data Advisory Council’s 2nd Meeting: Storytelling, Staff Recruiting, and Complex Processes.
Given how data as a resource permeates all aspects of an organization it is tempting to propose a program development strategy that is all encompassing – and very expensive. This is why, I think, that any program development strategy needs to focus early on delivering value while at the same time capturing what is being learned and feeding it back into the program development process.
“Boiling the ocean” by attempting to make too many changes early on is tempting. Incremental steps in delivering analytics-based value may actually accelerate change in the long run.
Note: An earlier version of this article was published as Big Data Project Management Research Report #2: What’s In & What’s Out? This is one of a series of posts that examine big data initiatives from a project management perspective.
Related Reading
Assumption of Breach and Defense Planning
