KPI Partners Blog

by Ron Cruz

Technology has finally gotten to a point where data discovery tools are powerful, easy to use, and available in many markets that were previously out of reach.  With the ever growing social media and big data sets, business intelligence managers are increasingly finding themselves tasked with data discovery initiatives.  However, many experienced business intelligence experts stumble in their path to implementation of a data discovery system.

Here are three simple tips to keep in mind when beginning your journey into data discovery.

Data Discovery Is Not Business Intelligence And Vice Versa

The first mistake many make when undertaking data discovery projects is thinking, "I've done BI for 12 years, this will be easy!" 

Data discovery is related to business intelligence but it is not the same task.  Business intelligence is about a set of report templates that are standardized and repeatable.  Even the ad hoc section of a business intelligence platform is run against a carefully constructed data set where users can seek answers to pre-defined questions.

Data discovery is much more free form.  Many times data discovery users don't actually know what questions they need to ask.  Even if they did, users will often find something that will make them say,  "Huh. That's interesting." 

In data discovery, users can use their initial question as a springboard into further free form analysis.  This free form analysis and data discovery provides an opportunity for a user to have a conversation with the data.

A good way to think of the difference between business intelligence (BI) and data discovery is to think of BI like a Bach Cantata; structured, disciplined, predictable.  Data discovery is more like jazz; flowing, improvisational, and personal.

BI & Data Discovery Are Not Mutually Exclusive.  They're Complimentary.

Some decide to segregate BI and data discovery into exclusive buckets in terms of technology and users.  Ideally, data discovery can feed into business intelligence and vice versa.  

A good example of this unity is a standard business intelligence report that may show a dip in sales for a certain time period.  This insight can launch a data discovery session to try and find out WHY there was a dip in sales.  If a cause is identified, BI reports can be built to alert users that a dip might be coming and trigger steps to mitigate that risk.

The most successful organizations leverage business intelligence and data discovery in a cyclical manner, each feeding into each other when appropriate.  

WATCH: DATA DISCOVERY SESSION - COMPANY PROFITABILITY

In Data Discovery, Speed Is King: Speed Of Thought. 

In business intelligence, the speed of the extract-transform-load (ETL) process and the speed at which screens can be painted with dashboard information is critical.   In data discovery, it is not impeding the speed of thought which is most important.

When entering a data discovery session, a user doesn't know what they will find or if they will find anything at all.   Employing the old paradigms of a structured release cycle with weeks of design, development, testing and deployment can be hugely detrimental to data discovery.

For a larger release of a data, a traditional release cycle is often necessary.  But many times when taking in a data source, such as a third party file, it is unknown if anything will be found. In these cases, it should be cheap and easy for a user to build a data set and prepare for quick discovery.

An effective data discovery tool will make data exploration quick and simple.   An effective data discovery tool will also allow for easy recovery from failure.   In data discovery, there will be moments where a user says, "Hmm....I wonder what happens if I cut the data this way."   These ideas are often followed by a, "Oh, that doesn't tell me anything."  It should be easy for a user to quickly track back to something that was successful.

Sometimes the technology tool can get in the way and sometimes IT process can get in the way.  Data discovery requires a fair amount of imagination and nothing kills imagination quicker than obstacles.  Empower your users as much as possible by clearing a path where users can have a conversation with the data.

Ron Cruz is a Solution Architect and Project Management Professional (PMP) at KPI Partners, specializing in business intelligence tools and applications.  Check out Ron's blog at KPIPartners.com.