5 Steps for Applying Big Data to Project Controls

Posted in : Blog Author : Adam Kingsbury

The second in a series of blogs for the MPA exploring Big Data in major projects.

The construction industry in the UK has begun to harness big data to better deliver projects and to monitor and maintain the assets produced by them.

Despite the challenges of big data in the project controls space, there is plenty we can do to prepare for its arrival. Most importantly, the construction industry must improve its data management practices from top to bottom. The unique position of project controls supports this process.

Fortunately, we have the advantage of looking outwards to the industries that have paved the way to big data. We can harness the practices they have pioneered along with the modern tools that they have created. From these lessons, we have been able to put together the 5 steps to applying big data to project controls.

1.  Know Your Who, What and Why

The first step in any big data initiative is to know where you are going, what you think you need to measure, and why it’s important. Don’t focus on eventual outputs and the positioning of information in a report. Instead, identify all of the stakeholders’ business objectives and the key metrics needed in their decision-making process.

By building a dictionary of all the measures, it will then be possible to assess your existing sources of data. You can also determine the readiness or identify a roadmap to reach your desired endpoint.

This process will also help your organisation clarify exactly what the roles, responsibilities and expectations are across the organisation and will potentially help to identify misalignments that need to be addressed.

2. Understand the ‘Golden Thread’

Understanding how all your organisation’s or programme’s information relates to each other is also an important step in the process. This process requires you to identify the classifications or breakdowns that segment all your information.

The lowest common denominator across all information sources is formed once several breakdown/classification schemes are agreed upon. It will then be possible to review everything currently in existence for compatibility. Plus, it will form a minimum set of requirements for any new information systems and processes in the future.

This ‘golden thread’ throughout your organisation will simplify the data collection and transformation process for future and current integration projects.

3. Maintain a Structured Systems

Efficient processing and analytics rely on high-quality information. The more repeatable the end-to-end process of inputting, tracking and utilising information across your business processes, the more reliable and trustworthy the analysis results will be.

The construction industry needs to make a much greater commitment to moving away from stop-gap data management solutions such as Excel, Word and PDF as its primary stores of business data.  Because everyone knows these systems and there is no additional cost, they are convenient. However, it can become prohibitively expensive when you factor in maintenance, error-correcting, data collection, summarisation and incompatibility across the wider organisation.

Instead, advances in cloud services and modern rapid application development mean that organisations should have little difficulty whether this is identifying an off-the-shelf product that will meet their needs or implementing a solution at an acceptable cost.

4. Use Automation Technology

The information recorded in a source system is not always fit for purpose within a reporting system. Sometimes information needs to be rolled up higher, sometimes it needs to be filtered. In other scenarios, you will need to join this with other datasets from across the organisation before it will make sense.

In true big data applications, with regular reporting and monitoring, you would not perform these processes manually as it would be too time-consuming. Instead, to support the automation of these processes, Data Transformation or Extract, Transform, Load (ETL) toolsets have been developed.

Project controls teams should pick the ETL tooling as an opportunity to introduce a greater level of standardisation for information gathering and report generation. If you select a toolset with graphical design capabilities, it provides a level of self-documentation on how data is collected and what is done with it. This provides a level of assurance against the analysis processes.

Getting up to speed in this technology also enables the project controls team to prepare for future big data initiatives. It will give the team the capabilities required to act as business analysts and implementers for new analytics requirements

5. Implement Organisational Change

Until programmes and the organisations running the projects embed information management and data analysis as core functions, the construction industry and the project controls function will not be ready for the wave of big data. This extends well beyond just the selection and implementation of software systems. It is also influencing how an organisation should recruit, train and manage its staff. Additionally, it will influence how they should assign management or ownership of their data.

The vision for how data is managed within an organisation must also be pushed up into the corporate space. It is simply not sustainable to allow projects and programmes to implement their own standards and processes without overarching coordination. This does not mean an organisation must completely lock down its systems, but you must implement a well thought out model.

We see project controls and Project Management Offices (PMOs) as having a significant influence in this space. These bodies can sponsor and influence the adoption of better practices within project-driven organisations. Each of these functions has a wide reach across organisational units and collects and reports significant amounts of data. The project controls function should align itself as the business representative for the organisation relating to project and performance data. They may also have a role to play in becoming the initial business analysts for the organisation.


Project controls can learn and apply numerous lessons from the realm of big data. This will enhance data driven decision making and, ultimately, improve project performance. The good news is that fledgling projects can apply this for a fraction of the cost and complexity of big data players and consumers. It will require strong leadership to generate a culture that values a modern technology infrastructure and data-driven decision making. It will also require one or more consummate professionals to own the design, implementation and management of systems, data and related processes. Organisations who desire to remain relevant in the future will need to start investing in the development of staff and technologies within the data management space at a corporate level, not as a “nice to have” project-level expense.

If you’d like to better harness big data on your project, get in touch!

You can call on +44 (0)20 7404 4826 or email us at info@logikalprojects.com

Or you can use the contact us form here.

About Adam Kingsbury

Adam is an Associate Director and the head of LogiKal's Technology Solution Business Unit.

Click here to contact Adam

Awards highlights reel from Project Controls Expo and Awards 2021