Resources > Articles

5 Common Bias Affecting Your Data Analysis

Author
  • Pragmatic Institute

    Pragmatic Institute is the transformational partner for today’s businesses, providing immediate impact through actionable and practical training for product, design and data teams. Our courses are taught by industry experts with decades of hands-on experience, and include a complete ecosystem of training, resources and community. This focus on dynamic instruction and continued learning has delivered impactful education to over 200,000 alumni worldwide over the last 30 years.

professionals working on computer analyzing reports

Data analysts play a critical role in the success of organizations and it’s important to be responsible at each stage of analyzing data. Did you know biases influence every piece of data that we analyze? It affects everyone, even the most talented analysts. 

On average, there is a decision being made every 2.5 seconds. And in order for us to reach decisions at speed, our brains begin to take shortcuts. Most often than not, these decisions are cognitive biases, a way for our brains to simplify what is being processed and reach decisions faster. 

Reducing bias in your data analysis is important as biases can lead to false conclusions and can be potentially misleading. In this article, we’ll dive into the five common biases that may be affecting your data analysis and ways you can avoid it. 

  1. Confirmation Bias
  2. Historical  Bias 
  3. Selection Bias 
  4. Survivorship Bias 
  5. Availability Bias 

 

1. Confirmation Bias 

Confirmation bias is when we favor information that confirms our preconceptions. We encounter underlying bias pretty much every day of our lives without noticing. This often happens subconsciously and often focuses on information that supports our arguments or way of thinking. 

It’s important to go through all the data and evidence presented to assure no one is jumping to conclusions and confirming their own beliefs. We often see this when someone has a predetermined assumption about something and uses data analysis to prove their point.   

So, how can we reduce the impact of confirmation bias? We should constantly challenge and question the data at hand. It’s important to seek evidence that may contradict preconceptions. 

Most often than not, this type of bias can lead to bad business outcomes as data analysts tend to lean towards data that is aligned with their:

  • Views
  • Beliefs
  • Opinions

 

2. Historical Bias 

Historical data bias occurs when systematic culture prejudices and beliefs influence decisions. However, this type of bias can be avoided by regularly auditing the data collected and ensuring there is someone with enough domain knowledge assigned that can audit the data on a cadence.  

It can become particularly challenging when the data sourced from historical bias is used to train machine learning models. Therefore, it’s important to ensure inclusivity is established within frameworks for underrepresented groups. We must acknowledge and identify biases in our historic and contemporary datasets. 

When the historical data is used for AI and ML models, historical bias might occur when: 

  • The data involves bias already (such as human discrimination and prejudice) 
  • The data is not correct or incomplete 
  • The data is no longer a valid representation of reality  

 

3. Selection Bias 

Selection bias is an error that occurs when the population samples are not accurately representing the entire target group or representing skewed insights. Meaning, the data is selected subjectively rather than objectively – making the sample non-random and ultimately, not reflective of real-world data distribution. 

Selection bias can arise due to poor study design if the sample taken was too small, or the sample is simply not randomized. To avoid selection bias, it’s important to also address historical bias in data sources to increase inclusivity.  

Here are three types of selection bias to keep in mind: 

  1. Sampling bias 
  2. Convergence bias 
  3. Participation bias 

 

4. Survivorship Bias 

Survivorship bias typically occurs due to lack of visibility from counterparts. This bias has a tendency to draw conclusions based on things that have survived and ignore things that have not survived in the past. Furthermore, this common bias tends to lead to false beliefs on the impact given. 

In order to avoid survivorship bias, we must be extremely selective with the data sources being utilized. Additionally, we must ensure the data sources we are using have not omitted observations or sets that are no longer existing for one reason or another. 

The two main ways data analysts reach conclusions through survivorship bias are: 

  1. Inferring causality
  2. Inferring a norm 

 

5. Availability Bias 

Availability bias has a great influence on how we view the world around us. This bias tends to overestimate the likelihood of something occurring with the greater availability in memory. And depending on what has happened, there tends to always be some top-of-mind items we tend to analyze and put most effort towards.  

One way to overcome availability data bias is to broaden our horizons. It’s important to keep an open mind and be welcoming of ideas that might not align with our initial thinking. Being aware of this cognitive bias will reduce the chances of making poor decisions. 

Additionally, here are a few items that can help avoid being swayed by availability biases:

  • Keep an eye out for trends and patterns 
  • Make a conscious effort to look for different points of views
  • Avoid making impulse decisions or judgments 

 

Final Thoughts 

Cognitive biases are embedded when working with data. Although data is a critical tool and can greatly help organizations, it’s important to remember to be aware of data biases when presenting insights that drive decisions and move the business forward. It can be difficult to avoid bias, but as data analysts, it’s important to be aware and minimize its effects. 

Here are a few items to keep in kind that can help reduce the impact of bias in your data analysis: 

  • Always keep an open mind 
  • Challenge your preconceptions and beliefs 
  • Don’t be afraid to be skeptical 
  • Use all data available to reach a decision 

 

Elevate Your Career in Data Science

Change your approach to data analysis and advance in your career with Business-Driven Data Analysis. This hands-on course will teach you a proven, repeatable approach you can leverage across data projects. Gain the skills you need to deliver actionable insights and serve as a better partner to stakeholders. 

Learn More

Author
  • Pragmatic Institute

    Pragmatic Institute is the transformational partner for today’s businesses, providing immediate impact through actionable and practical training for product, design and data teams. Our courses are taught by industry experts with decades of hands-on experience, and include a complete ecosystem of training, resources and community. This focus on dynamic instruction and continued learning has delivered impactful education to over 200,000 alumni worldwide over the last 30 years.

Author:

Other Resources in this Series

Most Recent

The Ultimate Guide to Explaining Your Data Findings
Article

The Ultimate Guide to Explaining Your Data Findings

Without context or explanation, data is little more than numbers and statistics on a computer screen, right? Or, if you’re really retro, figures on a paper spreadsheet! As a data practitioner, you need to describe...
Category: Data Science
Article

10 Ways to Communicate Data Findings Effectively

There’s a popular joke in data circles that you might have already heard: Data practitioners spend 80% of their time preparing data and 20% complaining about preparing data. The truth is, there’s much more to...
Category: Data Science
How to Communicate Data Insights to Business Stakeholders
Article

Comprehensive Guide: How to Communicate Data Insights to Business Stakeholders

Data analysis is about more than crunching numbers. You need communication skills as sharp as your technical abilities to drive real impact. When you’re presenting findings to stakeholders, making complex insights clear and credible is...
Category: Data Science
Flying the Profitable Skies: A Sneak Peek at How Airline Loyalty Programs Really Work
Article

Flying the Profitable Skies: A Sneak Peek at How Airline Loyalty Programs Really Work

“The data is there; it’s about making it actionable.” – Mark Ross-Smith  Have you ever wondered how you could help airlines extract more value from their loyalty programs? Or questioned how they rake in billions...
Category: Data Science
Article

Harmonizing Data and Storytelling: Miriam Quick's Creative Approach to Data and Visualizations

In a world overflowing with data, Miriam Quick believes it is crucial to interpret this information in ways that are meaningful and accessible. Quick is helping to democratize data through data and visualizations.
Category: Data Science

OTHER ArticleS

The Ultimate Guide to Explaining Your Data Findings
Article

The Ultimate Guide to Explaining Your Data Findings

Without context or explanation, data is little more than numbers and statistics on a computer screen, right? Or, if you’re really retro, figures on a paper spreadsheet! As a data practitioner, you need to describe...
Category: Data Science
Article

10 Ways to Communicate Data Findings Effectively

There’s a popular joke in data circles that you might have already heard: Data practitioners spend 80% of their time preparing data and 20% complaining about preparing data. The truth is, there’s much more to...
Category: Data Science

Sign up to stay up to date on the latest industry best practices.

Sign up to received invites to upcoming webinars, updates on our recent podcast episodes and the latest on industry best practices.

Subscribe

Subscribe

Training on Your Schedule

Fill out the form today and our sales team will help you schedule your private Pragmatic training today.