Organizations face many potential issues when trying to make decisions. Those issues are more frequent and have more negative impacts if the organization is not data literate and if they have not adopted a data-informed decision-making process and culture. These issues include the following:
These issues cost organizations millions of dollars in lost opportunities and added costs. Below are some of the common symptoms we see in organizations related to this.
A very common scenario when an organization is trying to make a data-informed decision is that the data needed for the analysis is at some level not correct or trusted. Maybe there is no standardized data definitions or standardized calculation used for a measure. As a result, the data is not accurate or trusted. Another common symptom seen is that decision makers within an organization do not even have access to the right data needed to make appropriate decisions.
Individuals and organizations who are not data literate will be more likely to use less than ideal analytics and visualizations as part of their data-informed decision making process. This could be as simple as analyzing some measure and calculating it using the wrong aggregation method. Or, it could be more complex like excluding information (data or context) that is vital to making the best decision. In some cases, the correct calculation on a given measure is appropriate for the analysis, but maybe the measure itself was not appropriate to use for the current situation. That measure may not be an accurate Key Performance Indicator (KPI) that ties to organization goals. Organizations then end up driving behaviors and performance to a measure that is not positively impacting the organizational goals. In fact, in most cases, it ends up hurting the organization.
One example of this would be a store that is trying to measure sales performance of their various locations. They created a KPI that measures sales performance and compares the performance to other locations. Rather than driving the right behaviors, it caused locations to pursue and aggressively market the same customers, even offering higher discounts. This had an overall impact of lowering sales, and also higher costs (as the products being bought now had to be delivered from the warehouse to locations that are farther away).
In many organizations, different people are responsible for building analytics and visualizations than those who actually need to consume them to make decisions. In these cases, the consumers of analytics can misinterpret the data or analytics, leading to incorrect conclusions and insights.
One famous example of this is when the engineers in charge of a part on the U.S. Space Shuttle Challenger tried to get the launch stopped because they believed the part would get damaged during the take-off, which could then lead to a massive explosion. Those engineers showed a visualization to NASA that included only a subset of the data they used to gain their insights. That visualization was misinterpreted by NASA and deemed that the launch was not at risk. The launch happened and unfortunately the space shuttle exploded during take-off. The after-action report highlights multiple lapses in process and also groupthink, so this is just one example from that report.
In some cases, the interpretation of a given analysis could be incorrect due to a wrong deduction or having an incorrect mental model. Maybe a visualization is created that is accurate, but it is void of context which can lead to misinterpretation. Or, maybe an individual interprets something with a correlation and believes that means there is a causation and acts on it. Potentially an individual is looking at descriptive analytics and incorrectly infers a root cause from this (ignoring the exploratory analytics process). Potentially the analytics are correct, but the individual has a cognitive bias that causes them to have a blind spot in their reasoning which leads to a less than ideal decision. Or, the group of people making the decision suffers from groupthink and a lack of a decision-making process.
One famous example of groupthink is the decision of the United States to invade Cuba during the 1960s in the Bay of Pigs conflict.
Another common symptom of lack of data literacy and lack of a data-informed decision-making process is when decisions struggle to take hold within an organization. This could be due to a lack of understanding how to communicate the decisions to all the stakeholders. One common mistake is to use the same visualizations and analytics used for gaining the insights to also communicate them. Another common mistake is to not communicate the thought process for the decision. In this case, an organization with a culture that does not trust data is likely to not embrace the decision.
One famous example of ineffective communication is the failure of project managers and engineers which then resulted in a deadly collapse of a Hyatt Regency Walkway.
There are many situations where an organization will make a decision with not enough data, but another common situation is when an organization over relies on data alone. Data alone is not a magic bullet for the best decisions. Even if an organization has good data, decisions are typically made with assumptions. If those assumptions are not correct, you will make a bad decision regardless of the data. This is why it is important to marry data with intuition and other factors. Data cannot replace intuition, but when data is used along with intuition and other factors, flawed decisions are minimized.
All these symptoms occur in organizations all the time, leading to ineffective decisions and ultimately to worse performance of the organization as a whole. How can organizations improve on this? Follow these 4 steps: