The Dangers of Small Data Sets for Decision-Making

Small Data Sets and Big Decisions

The Dangers of Small Data Sets for Decision-Making

Small Data Sets and Big Decisions

Have you heard the one about the marketing director and the CEO?

After some analysis, a marketing director presents an enthusiastic report to the CEO.

The director exclaims, “We’ve increased traffic to the site from social media networks 400% this quarter!”

The CEO, thrilled to hear this improvement, replies, “Well, that’s fantastic! We should move budget to support that effort!”

What the CEO doesn’t realize is that in all actuality, 400% could mean increasing from 20 visits to 80 visits. While the data might look great on paper, this small increase is hardly worthy of a strategic shift in budget and resources.

Waiting for the punchline? The only punch here went to the CEO’s budget.

Size Matters

Sample size is a count the of individual samples or observations in any statistical setting.

Small numbers raise statistical issues, and as a result, they alter the accuracy and usefulness of the data. Many problems with reliability arise with small numbers. These issues are due to the fact that rates and percentages are subject to random variation. Thus, these numbers often fluctuate.

Small samples yield unreliable results. The smaller your sample size, the more likely outliers — unusual pieces of data — are to skew your findings.

Context is Everything

Percentages don’t always mean much if you remove the context.

We recommend CEOs look at percentages with the corresponding hard numbers. Growth is relative. All that data might appear impressive until you assess the context.

Think about it.

If 80% of a company’s employees claim to exercise five times a week, you might think that business has healthy employees. But what if you discover there are only 5 employees at the company? Those numbers aren’t very impressive anymore, are they?

Placing too much emphasis on small data sets may lead to poor decisions about budget allocation and personnel resources.

Digital marketing numbers fluctuate, and that’s okay! Although all data is important, the most significant guideline is how to look at your data.

Avoid Blowing Your Budget on Poorly Informed Strategies

So how do you avoid making big decisions based on small numbers? Shift the conversation away from impressive looking percentages and focus on real numbers.

A client of ours recently asked for recommendations on how to adjust content on their website. We dove into their Google Analytics account and searched for the biggest opportunities for the company.

Unfortunately, their company only receives about 100 hits to their site each month. If you take away all spam traffic and the business IP address, we’re down to about 80 hits. That’s barely any data to work with.

This small data set is not worth recommending an expensive restructure of their site. 80 visitors are simply not enough of a sample from which to base recommendations.

The Takeaway

If you are given a report with only percentages showing – push back and ask for the hard numbers. The findings may surprise you. Finally, ask questions and get explanations of numbers that look too good to be true.

Want More Information? We Can Help!

TG Marketing Inc can help ensure you’re not wasting time or money focusing on small numbers. Contact us to schedule your free consultation today!

Michelle Tresemer

Michelle is the owner and founder of Tresemer Group, creating and implementing effective, data-driven digital marketing strategies for our clients. Michelle brings expertise in SEO/SEM, web analytics, social media, lead generation, and conversions. Connect with me on LinkedIn