What is statistical significance?
Statistical significance helps determine whether the differences in your survey results are due to real variations in opinions or behaviours, rather than random chance. It ensures the insights you gather are meaningful and reliable.
Does population size matter?
Once a population reaches a sufficiently large number (around 1 million or more), adding more people to the potential sample doesn’t improve statistical accuracy. Instead, what matters is having a large, representative sample.
Sample sizes and margin of error
n=400 respondents: ±4.6% margin of error.
Most results will fall within this range if the survey is repeated.
n=800 respondents: ±3.5% margin of error.
Doubling the sample size slightly reduces the margin of error, but the improvement diminishes with further increases.
Detecting significant differences between concepts
When you’re testing two or more concepts, how different do the results need to be before you can say there’s a real difference?
At n=400 respondents: A difference of at least 6.7% is statistically significant.
At n=800 respondents: A smaller difference of 4.4% is enough to confirm significance.
Why it matters for decision-making
Understanding statistical significance helps you focus on meaningful variations in your data, avoiding overreactions to minor differences. This approach enables more confident, data-driven decisions based on reliable insights.