A Well-stirred Sample

A typical national survey will sample only about 1000 people.  In this problem, we will try to understand why.

• If you're trying to check whether a large pot of soup has enough salt or herbs in it, how much would you need to taste?
• What does this tell us about sampling to find out about the whole population?

Many surveys want to know the proportion of the population who think or do something.  Let's say that we want to know the proportion of the population who would vote for the Fantabulous political party in an election tomorrow.  So we sample $n$ people to find out.

• What question should we ask in our survey?

Let's say a proportion $p$ of them (so $np$ people) say they would vote for the Fantabulous party.

What proportion of the whole population would vote for the Fantabulous party tomorrow?

We don't know the answer, but we can say that it is probably around $p$, and in fact $p$ would be our best estimate.

• If we assume that the true proportion is $p$, how many people out of a sample of $n$ would say they would vote for the Fantabulous party?  This is not a fixed number, so what is the probability distribution of $X$, the number of people?
• The expected number of people who say they would vote for the Fantabulous party is $\mathrm{E}(X)=np$ (under this assumption).  What is the standard deviation of the number of people, that is $\sqrt{\mathrm{Var}(X)}$?
• We are interested in the proportion who say they would vote.  If we call this $Y$, so $Y=\frac{1}{n}X$, what are $\mathrm{E}(Y)$ and the standard deviation of $Y$, $s=\sqrt{\mathrm{Var}(Y)}$?

A 95% confidence interval for the true proportion is a range of possible proportions around our observed proportion.  The range is chosen such that the probability of our calculated confidence interval containing the true proportion is 95%.  A 95% confidence interval is approximately given by: $$[p-2s, p+2s]$$ where $p$ is the observed proportion and $s$ is the standard deviation that we worked out earlier.  (More properly, we should use $p\pm 1.96s$ as the limits, but the difference is small.)

The number $2s$ is called the margin of error.  It gives a single number which indicates how reliable our estimate is.

• The margin of error $2s$ depends on the value of $p$.  What is the maximum possible margin of error for a given sample size, and what value of $p$ gives this?
• What is the maximum possible margin of error for a sample size of 1000?
• How big would the sample size have to be for the margin of error to be 1%?
• Reflecting on your answers to the above questions, why do you think that most national surveys have a survey size of about 1000 people?

This resource is part of the collection Statistics - Maths of Real Life