sample design

Probability Sampling Techniques

These are sampling techniques where each member of the target population has a known probability of selection.

A typical probability sample selected by Bluewave Geographics would be a systematic random sample of addresses from the Postcode Address File (PAF) for a specified survey area.

Non-Probability Sampling Techniques

These are sampling techniques where the probability of selection for each individual in the target population is unknown and cannot be calculated.

A typical non-probability sample selected by Bluewave Geographics would be a random-location quota sample in a local authority. The probability of selection for each individual in the target population cannot be calculated as the final selection of the respondents is made by the interviewer.

Simple Random Sample (SRS) versus Systematic Sample

With a simple random sample (SRS) members of the target population are selected completely at random with each member having an equal probability of selection.

With a systematic sample the population members (or sampling units for multi-stage samples) are first ordered in a particular way and then a random-start-and-fixed-interval procedure is used to select every nth member.

Depending on the choice of sort order, systematic samples can actually have a smaller variance (and therefore a smaller confidence interval) than simple random samples, making the estimates more accurate. A common technique used in the UK is to sort population members by a combination of geographical and demographic variables prior to selection. This is known as implicit stratification. For example, for a local authority sample the first stage sampling units (eg census OAs) may be sorted within each ward by the %AB households. This ensures that the sample is broadly representative in terms of both wards and %AB households.

Multi-stage Clustered Sample

For face-to-face surveys some level of clustering is usually used in order to reduce fieldwork costs. The majority of samples are two-stage clustered samples. At the first stage small primary sampling units (PSUs - such as postcode sectors or census OAs) are selected systematically from a list making up the universe of PSUs. Prior to selection the PSUs are sorted so as to introduce the implicit stratification discussed in the previous paragraph.

At the second stage the respondents themselves are drawn from within the selected PSUs. In the case of probability samples the second stage may involve a further systematic sample of people or addresses from a list within each PSU sampled at the first stage.

In the case of non-probability, random-location quota samples, respondents are selected at the second stage to fulfil quotas on key demographic variables. The quotas for each sample point are set so as to be representative of the small area making up the sample point. The sum of the quotas across all of the sampled PSUs will then be a close match for the profile of the whole survey area, something that would always be checked, along with the geographical distribution, prior to accepting the sample.

Design Effects

When calculating approximate confidence intervals for the survey results using the sample size and survey percentage (see our Sample Size Calculator), an assumption is usually made that the sample is a simple random sample (SRS). However, in practice most survey samples are not SRSs but are more complex in design. Even when we select probability samples we are likely to use a multi-stage, clustered design employing a systematic sampling technique (see above).

There are a number of factors that introduce design effects, which can increase the variance of the survey estimates and therefore widen the confidence interval associated with the result.

In the case of clustered samples, if the within-cluster variance of the measured variables is smaller than the overall variance then this will introduce a design effect that increases the confidence interval making the result less reliable. The more homogeneous the sample units are the greater the design effect.

Although there are rules of thumb that can be used to estimate likely design effects, the true effects can only be calculated after the survey has been completed and a comparison can be made between the within-cluster variance and the between-cluster variance of each estimate. The design effects will be different for each survey question. For examples people that live near one other are likely to share a characteristic such as deprivation but not one such as disability.

The introduction of survey weights will also cause a design effect.

Effective Sample Size

Once the overall design effect is known for each variable, this can be applied to the sample size to derive the effective sample size. This reduced figure is the sample size for an equivalent SRS with no design effects.

© Copyright Bluewave Geographics Ltd