This week I was reading the promotional materials for the upcoming Strata+ Hadoop World conference in New York City. I felt briefly excited about the energy, intensity, and optimism of the data scientists working in business today.
But then, I thought about all the BI departments I’ve actually seen operating firsthand in the real world.
These departments have been in a range of industries from financial services to healthcare to telecom. They have varied enormously in terms of capabilities and sophistication. But they have had one thing in common. Not one has had a BI program I’d consider successful, in the sense that it consistently delivered analytic insight that was understood by its intended audience and that improved the performance of the parent organization. What I have seen instead is that most analysts are self-directed (with the randomness that implies), that most business people do not understand their work, and that most businesses would lack the flexibility to respond to their insights even if such understanding were suddenly achieved.
For the moment marquee data scientists attending hip conferences do not need to worry about this gap between expectation and reality. They are in the middle of an extended honeymoon period. The typical data science organization has budget and a few choice anecdotes to keep the money flowing. The reckoning will occur at some indefinite point in the future, but in the meantime, it would be foolish for data scientists to rain on their own parade.
But eventually, rain it shall. Gartner estimates that 70-80% of BI projects fail to meet business objectives. The Data Warehousing Institute reports that BI administrators, those closest to the tools and best-positioned to know how they are being used, estimate BI adoption rates of about 18%. These are not impressive figures and businesses will not continue to invest with these odds indefinitely. As anyone who has witnessed a major BI platform release can tell you: when a system goes live, it is often to a soundtrack of loudly chirping crickets. An analytical system is not exactly forced on users in the manner of a transactional one. The target users of a BI system can engage if they see value; if they don’t see value, they can safely ignore it.
Why are BI projects failing so badly? The fault may be in the two most common approaches to building BI solutions. These could be characterized as “build it and they will come,” and “specify everything in advance.”
Build it and they will come – The hallmark of this approach is to collect every last bit of potentially relevant data, then simply stand it up for analysis. This is the classic approach to building data warehouses taken in the 1990s. To the shock and chagrin of anyone with a sense of technical history, it is being repeated today with “data lakes” which are simply large repositories of data in their native formats. The typical data lake is even more ungoverned and incomprehensible than the traditional data warehouse it is supposed to be replacing. It may be a repository of great insight, but it could just as easily be a repository of great nonsense. It is completely unclear how a business user operating at a remove from the data lake is supposed to tell the difference.
Specify everything in advance – This second approach is rare at the moment, given that improvisation is currently fashionable and that anything that sounds like waterfall development is unpopular. But it still can be found here and there. The “specify everything in advance” approach tries to anticipate every last use of an analytics platform and determine all the aggregations, reports, and even likely ad hoc analyses (sic) that users will likely perform. Predictably, projects run in this manner frequently get hung up in the requirements or modeling phase and run over budget before going live. If such a system does go live, the “exact” requirements used to build it turn out to be inadequate — users discover new needs shortly after launch, and the new system has a backlog of enhancement requests from its first day in production.
We have to emerge from the morass of these approaches.
Rob Lawrence 