SCOTTISH CRIME SURVEY 2003
APPENDIX C: SAMPLING ERROR AND DESIGN EFFECTS
Data collected in surveys always provide estimates of the true proportions in the population. The accuracy of these estimates - the sampling error - can be calculated for any estimate in the survey using information about the proportion of people giving a response and the number of people in the sample (or sub-sample). The sample error can be expressed as a 'confidence interval', which can be added to and subtracted from the survey estimate to give a range within which it is fairly certain that the true value lies.
The precision of estimates derived from a sample is normally measured using standard errors. Essentially standard errors are calculations of the standard deviation of the sampling distribution. They differ from standard deviations, though, in that while standard deviations are a measure of variability derived from actual observations of a sample, standard errors refer to the variability of possible values that could be obtained from a series of samples.
Usually, the formula used for calculating standard errors assumes a simple random sample (SRS). However, as the SCS sample was both stratified and clustered, the standard errors must take into account the sampling methodology in order to calculate confidence intervals.
Tukey's Jack-knife technique was used to calculate the complex standard errors for a series of results in the study. This is a well-established technique for working through the effects of stratification and clustering. Basically, Tukey's Jack-knife creates a series of sub-samples of data, recreating the original methodology of sampling the "clusters" of areas. The sub-samples preserve the stratification and clustering techniques used in the original survey. The technique then goes on to simulate a further series of samples based on evidence observed from the sub-samples and the estimate of the error - a complex standard error - is based on the degree of variation of the data from simulated sample to simulated sample.
Finally, the design effect was calculated as the ratio between the complex and the SRS standard errors. This indicates the effect of the sampling design (stratification and clustering) on the standard error and hence confidence intervals. A design effect of 1 indicates that the effect of the sampling methodology used in the survey is equivalent to that of a simple random sample. A design effect of 1.2 would indicate that the sampling methodology is 20% less efficient than a simple rate sample, while, conversely, a design effect of 0.8 indicates that the sampling methodology improves the sampling efficiency.
Table C.1 shows the standard errors, the design factors, and the confidence intervals for the victimisation rates. As can be seen from the table, the design factors ranged from 0.76 to 1.30. The overall average is 1.03, but that should not be taken as a 'typical' value, given the distribution of values across different variables. However, it suggests that using a value of 1.2 as a 'rule of thumb' for adjusting the standard errors of the survey data would safely account for the design factors associated with most estimates from the survey.
Table C.1: Victimisation Rates and Sampling Errors for 2003 SCS
SRS Standard Error
Complex Standard Error
COMPARABLE WITH POLICE
Theft of motor vehicle
OTHER SURVEY CRIMES
Theft from a motor vehicle
Attempted theft of/from m.vehicle
Other household theft *
Theft from the person
Other personal theft
Motor vehicle vandalism
ALL HOUSEHOLD CRIMES
ALL PERSONAL CRIMES
1. For violence, theft from the person, assault, robbery, other personal theft and all personal offences, rates are quoted per 10,000 adults. For acquisitive crime, vandalism, housebreaking, vehicle offences, bicycle theft, other household theft and all household offences, rates are quoted per 10,000 households.
2. For the distinction between crimes which are 'comparable with police' and 'other survey crimes', see Appendix D.
3. Source: 2003 Scottish Crime Survey, unweighted n=5,041.