How to analyse a statistic

What is Statistical Analysis?

Statistical analysis is the process of collecting, interpreting, and analyzing data to identify patterns, trends, and relationships. It involves using statistical methods to summarize large amounts of information into a more understandable format. This process is vital in various fields, from scientific research and business intelligence to public policy and everyday decision-making, helping us make sense of complex information.

Why is Analyzing Statistics Important?

In today's data-driven world, the ability to analyze statistics effectively is a critical skill. Statistics are used to inform decisions, evaluate the effectiveness of interventions, predict future outcomes, and understand societal trends. Without proper analysis, statistics can be misleading, leading to poor decisions, misinformation, and flawed conclusions. For instance, a news report might present a statistic about crime rates without providing context, potentially causing undue alarm. A proper analysis would involve looking at the time frame, the geographical area, the types of crimes included, and how the data was collected.

Steps to Analyze a Statistic

1. Understand the Context and Source

The first and most crucial step is to understand where the statistic comes from and how it was generated. Ask yourself:

• Who collected the data? Is it a reputable organization, government agency, research institution, or a biased source?
• What was the purpose of collecting the data? Was it for academic research, market analysis, political campaigning, or something else?
• When was the data collected? Is it recent and relevant, or outdated?
• How was the data collected? Was it through surveys, experiments, observations, or existing records? What was the sample size and sampling method? A small or unrepresentative sample can lead to unreliable statistics.

For example, if a company releases statistics about the success of its own product, this data should be viewed with caution due to potential self-interest. Conversely, data from a government census or a peer-reviewed scientific study is generally considered more reliable.

2. Identify the Population and Sample

Statistics are often derived from a sample of a larger population. It's essential to know:

• What is the population? This is the entire group that the statistic is intended to represent.
• What is the sample? This is the subset of the population from which the data was actually collected.
• Is the sample representative? A random or stratified sample is more likely to be representative of the population than a convenience sample.

If a survey on voting preferences only polls people in one specific neighborhood, the results might not accurately reflect the preferences of the entire city or country.

3. Examine the Methodology and Definitions

The way data is collected and defined can significantly impact the results:

• What specific definitions were used? For example, how is 'unemployment' defined? Does it include discouraged workers? How is 'success' or 'effectiveness' measured?
• What were the limitations of the methodology? Were there any constraints that might have affected the data collection process?

Consider a statistic about 'poverty.' Different organizations might use different thresholds and criteria to define poverty, leading to vastly different numbers.

4. Look for Bias

Bias can creep into statistics in numerous ways. Be aware of:

• Selection Bias: When the sample is not representative of the population.
• Confirmation Bias: When data is interpreted in a way that confirms pre-existing beliefs.
• Reporting Bias: When certain results are more likely to be reported than others.
• Measurement Bias: When the method of measurement is flawed.
• Survivorship Bias: When focusing only on things that 'survived' a process, ignoring those that didn't.

If a study on the benefits of a new drug only recruits participants who are already in good health, it might show exaggerated positive effects due to selection bias.

5. Analyze the Data for Trends and Patterns

Once you understand the context and potential issues, you can start looking at the numbers themselves:

• What is the central tendency? Look at the mean (average), median (middle value), and mode (most frequent value). These can give different pictures of the data.
• What is the dispersion? How spread out is the data? Measures like range, variance, and standard deviation are important.
• Are there outliers? Extreme values can significantly influence averages.
• What are the trends over time? If you have historical data, look for increases, decreases, or stability.
• Are there correlations? Does one variable seem to change in relation to another? Remember that correlation does not imply causation.

For example, seeing that ice cream sales increase when the number of drowning incidents increases is a correlation, but the actual cause for both is likely the warmer weather (a third variable).

6. Consider the Margin of Error and Significance

Most statistics derived from samples have a margin of error, indicating the range within which the true population value is likely to lie. Statistical significance tells you whether the observed result is likely due to chance or represents a real effect.

• Margin of Error: A poll with a +/- 3% margin of error means the actual result could be up to 3% higher or lower than reported.
• Statistical Significance (p-value): A low p-value (typically < 0.05) suggests the result is unlikely to be due to random chance.

If a study finds a small difference between two groups, but the margin of error is large or the result is not statistically significant, it's difficult to conclude that there's a real difference.

7. Compare with Other Data

No statistic exists in a vacuum. Compare the statistic you are analyzing with:

• Historical data: How does it compare to previous periods?
• Data from similar populations or contexts: How does it compare to other cities, countries, or similar products?
• Data from different sources: Do other reputable sources report similar findings?

If one source claims a dramatic improvement in a particular metric, but multiple other reliable sources show no such change, it warrants further investigation.

8. Draw Conclusions Cautiously

Based on your analysis, form a conclusion, but always acknowledge the limitations and uncertainties. Avoid overgeneralization. Remember that statistics are tools for understanding, not absolute truths. They provide probabilities and likelihoods, not certainties.

Common Pitfalls in Statistical Analysis

• Misinterpreting Correlation as Causation: Assuming that because two things happen together, one must cause the other.
• Ignoring Sample Size: Drawing strong conclusions from very small sample sizes.
• Using Anecdotal Evidence: Letting a few personal stories override statistical trends.
• Cherry-Picking Data: Selecting only the statistics that support a pre-determined conclusion.
• Confusing Percentages: Misunderstanding percentage changes, especially with small base numbers.

By following these steps and being aware of common pitfalls, you can develop a more critical and informed approach to understanding and using statistics in your daily life.