Measurement quantifies your success
February 4, 2020  

Supplied by entelect2013 Administrator from entelect2013
Once an idea has made its journey into an implemented solution, it is necessary to make sure the original need has been met. Just because something is in production, it does not necessarily mean it is doing what it should be, and often it’s difficult to tell straight away.


Once an idea has made its journey into an implemented solution, it is necessary to make sure the original need has been met. Just because something is in production, it does not necessarily mean it is doing what it should be, and often it’s difficult to tell straight away.


There are various techniques that can be used to measure the success of an implementation, both from a business point of view as well as technical.


1. Measuring technical success
There are many things that could potentially fail in a live system. Many of these risks would have been considered when making technology stack choices and while implementing the solution. But there is still a very important need to keep a close eye on what is happening in real time and to be prepared to act as soon as something fails.


•    Gathering metrics

A software system has the potential to generate large amounts of data related to the way it is functioning. This data can take various forms:

o    Text log files
o    Database log entries
o    Hardware performance metrics

Gathering all this data presents some unique challenges, substantial amounts of storage are required and there is a risk of overloading support staff to the degree that important information is missed or ignored.


•    Making sense of metrics
Simply gathering data is not useful to the business or technical team unless they are able to glean insights from it. The data needs to be processed and analysed (in an automated tool) so that trends and anomalies are easily identified. Artificial intelligence and statistical analysis have the potential to find things in the data that most humans would miss.


•    Radiating information
Once trends and worrying issues are identified, the relevant stakeholders need to be made aware of them. Simply emailing reports is likely to get them filtered away and ignored. The insights need to be on display all the time and radiating throughout the workspace of the people responsible for the health of the solution.  


Permanent dashboards displayed on TVs are a great way to radiate information. They do, however, need to be well thought out to be the most effective. Make sure that anomalies are instantly visible, show trends over time so that unusual changes stick out. There is also room for showing successful metrics to promote confidence in the health of the system.


2. Measuring business success
As part of the implementation process, a business intelligence (BI) solution would have been chosen and put into place. Now is the time to leverage that effort to measure the business success of the implemented idea. The chosen BI stack will allow stakeholders to slice and dice the data in order to extract key metrics and indicators.


•    The data pyramid
The data pyramid shows how reporting, roles and use cases for the data change in a mature data environment. The insights gained, frequency and granularity of the underlying data, and audience viewing the reports change, depending on the layer of the pyramid.


At the bottom layer, reporting is primarily based on transactional or “near real time” information. This serves the purpose of showcasing the as-is processes within the business and serves to improve visibility of business processes.


On the upper layers, reporting becomes more analytical and strategic in nature, providing a more aggregated view of information which could help better infer trends, which assists management in doing comparative analysis and scenario planning, among others. This helps identify where a business needs to be and what adjustments need to be made in order to better reach business goals. This is achieved by enabling an accurate view of key indicators tracked against organisational targets.  


The capstone of the data pyramid is predictive and prescriptive analytics that come into play through the use of data science. Predictive analytics indicates what is likely to happen in future (i.e. what will happen), whereas prescriptive analytics refers to recommended actions and strategies (i.e. what to do about it).


The BI maturity model and the data pyramid work hand-in-hand. Having a mature data environment naturally enables more analytical data use cases through the insights journey below.


•    The BI Maturity Model
Each organisation or division is on a journey to an optimised data warehouse (DWH). Even a mature and optimised DWH will still change as new analytical requirements are discovered and as new trends emerge in the field.


The reporting and analytical requirements of an entity change significantly as they move through the steps in the BI maturity model, as explained to the right. Please note that an entity could be in more than one level across different facets of their business.


Stage 1
This stage represents the pre-data warehouse environment where an organisation relies entirely on operational reports for information. In general, these reports are static and inflexible and show a limited range of data for a limited set of processes providing little benefit to the organisation. During this stage it is mostly spreadsheets on local machines that are used to support the reporting requirement of the organisation.

Stage 2
The entity’s first attempt at a BI solution, where an initiative is started by a department without fully understanding all the fundamental requirements of a successful BI solution. The first BI tools are introduced - primarily for ad-hoc queries, reporting and basic OLAP - where licences are given to only a handful of power users and analysts. Benefits are starting to be derived, but only by a handful of people within the organisation.

Stage 3
At this point, a business unit recognises the value of consolidating these silos of information into a single data warehouse to save money and gain greater consistency in the information it uses to understand and analyse the business.

Stage 4
An area is entered where BI/DWH delivers a strategic, enterprise resource that enables organisations to achieve their key objectives. This is accomplished through a unified data warehousing architecture defined by a common set of semantics and rules for terms and metrics shared across the business.

Stage 5
This phase completes the cycle by converting core BI/DWH capabilities into services, both technical and commercial, and redistributing development back out to the business units via centres of excellence.


Stage 6
With a well-established curated DWH solution in place, true value can be derived from BI by using predictive and prescriptive analytics that come into play by incorporating data science. The focus shifts from looking at the past, to trying to look into the future by learning from mistakes made or improving on past successes.



Read the full From Here to There publication


Home | About Us | Offerings | Track Record | Giving Back | News | Careers | Contact Us
PAIA Manual | Privacy Policy | As per statute: |
© Copyright by Entelect. All Rights Reserved.