by John Thuma, Teradata
I see a lot of awesome data science being done when I visit companies. And that is not necessarily a good thing because I’m talking about “awesome” in its old school, fall on your knees, cower in fear definition.
You see, what I witness is a lot of complexity being heralded as analytics victory—cluttered dashboards divided into multiple sections, each heaped with a dense thicket-like graphic. It is impressive to see the incredible amount of work that went into creating these things. It’s also an easy trap to think that those creations represent some kind of victory. They do not.
True achievement comes at the end of the line, and what is being overlooked in such scenarios is the fact that these types of analytics products are not making things simpler for the people in the business. The answers may be hidden in those visualizations, but the user is expected to work too hard to find the relevant bits and take action on them.
I recently saw a demonstration where a company was trumpeting an analytics victory. They had moved a huge volume of data into Hadoop, then further wrangled it onto another product, from there it went into Spark, and then finally a model was created to predict fraud. The data made 4 or 5 hops before it could finally be used for analytics. They were rightfully proud, but as I listened I noticed some very important details were overlooked, like how this was going to be implemented by the business. They were so focused on the analytics and technical wizardry that they never once mentioned how all this was going to be used by the actual people who operated the business. Literally not a single word about the end users.
In contrast, twenty years ago I was working at Microsoft when we came out with a predictive analytics solution that used point of sale data to predict in real time when items would go out of stock on grocery store shelves. But it wasn’t algorithms, reports, and visualizations that made our team proud. It was the fact that we could hand a device to someone working in a local supermarket that made their eyes light up. At a glance they could immediately see whether a given SKU was moving quickly or slowly, and the date and time when it would need to be restocked. They got it. We’d taken something fairly complex and created a simple, empowering tool that met the real needs of people who had actual jobs to do.
With the lessons of these two cases and many others, separated by time and technology but ultimately with similar goals, I offer five recommendations to help analytics make the jump from the laboratory into the hands of real people:
- Focus on the last mile. The best delivery of data science does not expect the consumer to be a data scientist. Those end users already have jobs. Their role and their time demand respect. It is the responsibility of the data scientist therefore to deliver the analytics in a way that is clear, concise and usable. People don’t need to know how the clock works; they just need to know what time it is.
- Give advice and concrete actions. Even the most polished, accessible visualization only serves to set the scene. It still needs to be consumable, and that means it needs to tell the user what best actions are available to take. Without advice, even the best analytics are a dead end from an operational perspective–interesting, but not functional.
- Segment and personalize. Not all the consumers of analytics are the same. It is essential for the people building the models and creating visualizations to understand the different people the analytics will serve. That insight provides a foundation for personalizing the methods of delivery and the actions to best suit varying roles.
- Use all the data. If the goal is to equip end users and the business at large, success ultimately relies on providing them with the most accurate analysis and reliable options. Systems are big enough and powerful enough to handle petabytes of data. Data science needn’t be limited to samples anymore. Using the whole population of data, for example the entire customer base, rather than just a sample, reduces error rates and provides more accurate and refined results.
- Keep score. The project doesn’t end when the model is built and easily implemented results are delivered to the end user. Not only can models be improved over time, but users can provide feedback to track how well the system is working and what can be done to enhance its utility further.
Put the trumpets down! It’s time to check our egos and think about what the humans want. The humans you work with will love you for it. When data science shakes off the perspective of “data science for data science’s sake” and starts connecting with the people and business it serves, a new, more human-facing dynamic is created that can effectively turn the science into positive action.