By Jan Van Ansem, consultant at Bluefin Solutions
The formal name for Business Intelligence (BI) applications developed and managed by non-BI professionals is Shadow BI. Such applications are usually created after a business user has lost their patience and/or their trust in an incumbent software solution and so decides to create their own system often by using Excel or Word. The more tech savvy users might even download some BI software and put the data in their own database.
Although Shadow BI applications initially appeal to the business, there is a real danger that in the longer term they will do more harm than good to the enterprise as a whole.
So why do most organisations find themselves with so many Shadow BI applications, and what guidelines could organisations put in place to prevent and manage the uncontrolled growth of such solutions?
The negative reputation of Shadow BI
Despite what some might think, Shadow BI is not the enemy. Sometimes a solution becomes very successful and will make a significant contribution to the BI ecosphere.
Whether it does or does not, at some point in the lifetime of a shadow BI application some professional guidance will be required. Business users deserve the best support BI can give them, either by showing better alternatives to Shadow BI solutions or by adopting a solution so it can be scaled up and rolled out to a wider audience.
There are also dangers associated with Shadow BI. If it is not properly managed the organisation is exposed to risks associated with poorly tested functionality, data integrity, adoption to future (technology and business) developments and last but not least, legal issues if the software is not used within the boundaries of the software license agreement.
Why do business users rely on their own BI applications?
Based on my fifteen years of experience in BI I believe there is one main reason and several smaller causes contributing to the reliance on Shadow BI. The core reason being that any changes to central BI solutions take too long to implement. In addition to this there can be other reasons such as requests which would require a major investment; the costs charged to the requesting business user’s department are deemed too high or, worst of all, business users simply don’t know what services can be delivered through central BI.
In many cases the business users feel they have no choice but to create their own BI applications. They have tried to get what they want through central BI and failed and so turn to their own devices.
The reason why central BI fails is more often than not poor organisation (of the change processes) or poor implementation of the BI system.
In reality, most business user requirements could be met within an acceptable time if there were an efficient governance procedure and if the BI is implemented in accordance with best practice guidelines. Below are some suggestions to enable just that.
Streamline BI governance
BI applications are principally different from data entry applications. So why is it they are often governed by the same rules? The most likely cause is a reluctance to change agreed procedures – despite the fact that business users are turning their backs on central BI and the return on investment on BI applications is often poor.
The most common change requests for analytical application bring significant business value to a small group of people, are simple to implement and can easily be corrected if something goes wrong. On the other hand, changes to data entry systems are usually part of a more complex change in processes within an organisation, impacting several teams, various screens and processes and would cause a disaster if they go wrong once implemented.
When there is a significant risk to the business then it is perfectly legitimate for a request to be reviewed by design authorities, change advisory boards and test teams prior to implementation. But when the risk is negligible and changes can easily be undone the approval process could be simplified and delegated to senior developers and retrospectively reviewed by design authorities.
Finally, it is helpful when business users, BI teams and service partners agree what a ‘reasonable time’ is for implementing relatively simple changes to BI. Once there is a routine, it makes it so much easier to manage expectations within the business.
Responsible Shadow BI
Once governance has been streamlined there will be an immediate reduction of Shadow BI. But there will still be users asking for things that still can’t be delivered. They may require applications a business can’t support or datasets which cannot be easily incorporated in the existing data model. In these cases, a business led pilot for trying out a solution not supported by central BI can be useful.
Rather than leaving the business team to its own devices, the central BI team should still be involved. Perhaps they can’t provide the application, but some input might still be appreciated. Equally important, central BI will learn from the experience and might see new opportunities when new technology becomes available.
Ultimately, as long as there are clear rules of engagement, Shadow BI could be beneficial to both the business user community and to central BI.