Intuitive Analytics

It was a packed house listening to a panel discussion between two CIOs, a CEO, a vendor and an academician. After almost an hour of discussion on various aspects of Business Intelligence challenges and opportunities, the session end requested final words on what they would like to see from vendors in the future ? Leaving aside the Oscar-ian twist on being good to customers, better decision making and paying more attention to talent, the crowd applauded unanimously to the CIOs wish list. The CIO representing a “mature” user of solutions from the sponsor BI vendor, made a passionate appeal:


Has anyone in the audience attended a training program on how to use Facebook, or any other website or messaging system ? If no, then why do we require everyone even with above average intelligence in the corporate world to be provided training on usage of internal systems ? What makes these systems so complex that they cannot be used without handholding ?

I wish that we can all evolve to a level of BI/DW tools such that any user within the enterprise can start using transactional data to convert to information that can assist informed decision making. Anyone who can use a spreadsheet should be able to extract the insights hidden within the sea of information. They should be able to intuitively understand what is expected from them to get to the next step with no prompting or help (online or otherwise). I am talking about Intuitive Analytics, a term coined by me a while back to refer to analytics that is intuitive in its interface; intuitive to the user the way s/he is able to open the browser on the PC, Smartphone or tablet and start the journey of discovery on the Internet.

In recent times there have been multiple initiatives around improvement of how information is presented to the consumers. Evolution from rows and columns to dashboards, drill-downs, pivots, multi-dimensional analytics has evolved; the evolution of mathematical models as well as technological advances on speed of crunching data have pushed the boundaries across enterprise datawarehouse projects. Over the last three years, DW/BI has consistently been in the top 3 technology and business priorities.

The experiences are however inconsistent in their delivery of business value. Some of the barriers include data quality, data model deficiencies, bad ETLs to name a few. The biggest deterrent has however been the complex user experience which has seen lesser evolution as compared to the technological advances. All tools with no notable exception provide the basic building blocks to create the DW/BI foundation and analytical layer; standard templates, internal IT teams and implementation partners have yet to breakout from the mould to provide a rich, consistent, and meaningful capability to the end consumer of information.

I believe that this is an opportunity for one and all, CIO, DW Architects, vendors, implementation partners, to take up this challenge on making BI as easy as getting on any social media site and get started. If you have already crossed this bridge, do write back, but the applause on the floor to my comments, makes me believe that the journey is still more like an uncharted expedition.

The Business Intelligence Challenge

The term Business Intelligence would imply the "intelligence" that "business" can create and use with help from tools and technology orchestrated by the IT team. Many billions of dollars have been spent on this journey across industry verticals and an equal number of technologies, broadbased or niche, with limited success.

Over the last few months with new projects hard to come by, the focus for many enterprises has been to improve decision making with the help of insights that can be delivered from existing data marts or data warehouses. This is off-course expected with no additional funding. Business Intelligence has also featured on CIO agendas researched by many marquee research entities.

Thus vendors have been getting quite aggressive in their sales pitch claiming to have all the magic formulae towards achieving the elusive ROI from BI as well as improving the usage of information towards making effective decisions. Many of these vendors have staff they have hired from the industry, who worked on in-house BI projects, successful or not, but now purportedly have the wisdom on how to make it work.

The question that baffles me is that in most cases the effectiveness same consultants was at best average with a few exceptions. What has changed that now gives then the insights from the outside to create exceptional performance for their customers ? Talking to them rarely gives one the comfort that they will be able to indeed drive through the change and create value irrespective of what their PowerPoints may depict.

The technology or tools do not appear to matter to this brood. The list appears like a menu card in an expensive restaurant which also includes esoteric dishes with fancy prices. And if it is not on the menu, don't worry, the chef will create what you want (custom solution).

Why is it that "wisdom" on how and what on business intelligence is with the consultants and vendors and rarely manifests itself within the enterprise ? I scratch my head and all I get is hair ! I am now losing it faster with the number of BI shops mushrooming everywhere. Maybe I should think of joining the herd rather than trying to beat them at their game.

BI OIC for CIO

Business Intelligence has been on the CIO radar for over 3 years now as per research reports from the major and respected IT research companies. Billions have been spent by companies and as a category, BI projects have seen the least success across almost all IT enabled initiatives. BI consultants will go about advising you that they "know" how to make it work for your enterprise and if and when things start faltering, it's always you and your users who are responsible. I happened to ask a VP turned consultant from a large bank, "How is it that consultants have all the answers, but employees do not ?" and got no answer.

Every organization aspires to create intelligent insights from transactional data and act upon them to increase revenue, optimize profits, retain customers or create efficiencies. In most cases, users are unable to think or visualize what they want to achieve any of the above objectives and thus end up defining extremely complex scenarios and reports with a hope that they will provide some kind of "Eureka" brainwave and they will be heroes. IT organizations takes limited steps to dispell such myths and takes on the task of creating models that enable the complex reports. Maybe because they are not in the bath tub !

Typically such operational data based reports provide limited insight since the insight is driven by humans and not systems. A report viewed by different people creates different inferences and that a technology cannot provide. Technology works on models created by us and is thus limited by the information gathered and understood by the developer. This indeed is the key to actionable insight and the experience and frame of reference of the person which makes the difference.

This is not a great insight, but it is based on the experience of seeing many business leaders responding with differing insights to the same analytical report presented. It comes out of a deep domain expertise and lateral thinking. Such individuals if enrolled into the program can make the difference between a successful and not so successful BI project.

Left to IT organizations, which is the typical case with BI projects, the end result is multi-dimensional reporting which is finally used to review business but with limited insights. It is evident and obvious that right people make the difference, but the right people are normally too busy to spend long periods of time to understand the possibilities and list them down. If by chance that were to happen, the adoption by the rest of the organization suffers and gets blamed on bad "change management".

So what is the conclusion ? A few "actionable insights" based on 3 BI projects.

1. Find the "right" person within the company who can help create insights. It may be your CEO ! If you cannot find such a person, don't start the project

2. Start small and scale up as you taste success; before you attempt to build the Taj Mahal, practice some smaller buildings

3. Tools and technology matter, but in the end, the data quality makes the difference

4. Do not be averse to restarting from scratch in case the first model or the second model does not deliver. It's quicker to recreate than attempting to patch a bad one

5. Keep on asking questions at every stage, "Why do you want this ?", "How will it help you or the company ?", "Who else can benefit from this ?", you get the picture .....

6. Whatever you do in stage 1 may need to be discarded by the time you are in stage 3 and that's okay.