(The below article is adapted from a recent episode of Data Chats with Claravine CEO Verl Allen.)
Organizational entropy is the collective understanding that there is a data quality problem, but the response is learning to live with it.
Why are companies failing to improve their data quality? I think it’s simply the fact that it’s almost easier to live with the problem than to make the necessary changes.
Simply put: The opportunity is the data; the challenge is the data.
For example, we may know that 30 percent of our data is unusable in its current form, so we discard it and use what is left over to make our decision.
Even more problematic is that most companies lack an organization-wide data strategy. And, companies that do have a plan in place, have varying degrees of maturity in their approach. In other words, this problem with data quality is incredibly pervasive.
But, rather than address this issue, organizations will continue to exist with the problem and silently hope it is never identified because solving it requires a massive time investment.
More importantly, it takes an element of courage to say “we have a problem” and then work on fixing it without looking for a place to put the blame.
Here are four strategies to help companies manage and use data while avoiding organizational entropy.
Strategy #1: Approach the data problem proactively, not reactively.
At the very beginning of the creation process, you should establish data standards because you’ll end up with a much better data set.
Everyone facing the data quality problem will think that it’s an issue with the data pipeline, or it’s a problem that requires artificial intelligence or software to solve. However, when you implement data standards on the front end, many of these downstream problems go away.
It almost seems too simplistic to be effective.
We’re not trying to boil the ocean. We’re just trying to help the company organize their teams and improve their ability to understand data and implement it on the business side.
The best approach is to solve this problem before it’s ever a problem.
It’s uncomfortable to implement this proactive approach because change is hard and it can expose bad practices that have been going on for a long time. But the more you shine a light on data quality problems, the more you can find effective solutions you can implement broadly.
This old way of operating in silos because “that’s the way we’ve always done it” just doesn’t work anymore; that is a recipe for disruption.
Strategy #2: Extend data quality controls to your agencies and your partners.
Partners and agencies have historically existed on their islands; in some ways, they remained insulated from the problem of data quality.
Global deployment of data quality controls means everyone involved with the company’s success must adhere to these standards. This strategy is often meant with resistance because brands are pushing for more transparency from their partners.
Strategy #3: Decide what you’re trying to accomplish.
There are a number of large enterprises looking to build machine-learning infrastructure. However, as they begin the work they realize they need a certain level of data quality in order to train or build a model off of the data set.
So they begin the project under the assumption they have a massive amount of data when actually they can only utilize a fraction of it for machine learning; expectations don’t meet reality.
Essentially, they think they are going to solve all these problems, but the reality is, you have to decide where there is the greatest opportunity for gain.
When you make it to this intersection, you have to decide to go for a big win or a handful of smaller wins. You can certainly make the case for either approach depending on the situation.
This type of work is often completed by an analyst who is trying to assess the scale and value of the problems that need solving.
There are many organizations that lack an analyst role who make it to the other side of the problem and realize that ROI was insignificant; It wasn’t worth the effort.
Strategy #4: Think of data quality as an organization-wide effort.
The entire organization should be invested in the data strategy, not just the data team.
There is not one person or one team responsible for data quality and using the insights effectively. The executive team cares deeply about data quality because they want to trust it to make decisions. The data team cares about data quality too, but they may lack an understanding of the role of data in the business strategy.
Over the last few years, the role of chief data officer has emerged. However, it’s not about putting data in a silo. Success with data starts when this senior leader operates in a cross-organizational function that works closely with executives leading other departments.
In addition to senior-level leadership, creating an enterprise-level data strategy includes having a strong data team, which consists of a data engineer, data scientist and data analyst.
Case Study: Here’s what happens when you put these data quality strategies to work.
One of our clients was a large technology company trying to automate the creation of experiences. Their challenge was they had content siloed across the organization in a myriad of digital asset management solutions and content management solutions.
They were struggling to automate the creation and delivery of experiences because they couldn’t efficiently locate content in their libraries or quickly identify customers and where they were at in the buying journey.
This problem created broken customer experiences that weren’t ideal for their brand. When they stepped back and invested in a holistic approach and set their data standards, they were able to quickly locate content and tie that asset to the specific stage of the funnel.
It wasn’t about efficiency. This approach improved the types of experiences they were delivering to their users, which was foundational to their continued growth.
At my company, we’ve challenged our clients to flip the normal paradigm on its head by saying,”let’s be proactive to data quality.” That approach is different from most folks in the industry and has changed the way we think about solving the data quality problem.
