We have a new website go to gov.scot

The Scottish Executive Central Heating Programme: Assessing Impacts on Health

Listen

SECTION 3: OUTLINE OF THE EVALUATION

3.1 Objectives of the evaluation

In order to assess the effect of the CHP on the health of households which received heating, the Scottish Executive commissioned an evaluation focused explicitly on the health impacts of the Programme. This was carried out by a team from the Research Unit in Health, Behaviour and Change and the General Practice Section (School of Clinical Sciences and Community Health, University of Edinburgh) and The Medical Research Council Social and Public Health Sciences Unit (University of Glasgow). The evaluation had the following specific objectives:

a) To measure change in health status among Programme recipients up to two years after installation

b) To establish the reasons for these changes, clearly separating out any effects directly or indirectly attributable to the Central Heating Programme, including changes in temperature and humidity, living conditions and use of the house, and reduced fuel poverty

c) To identify the particular aspects of health (physical, mental, well-being) and the types of household and occupant (children, adults, elderly, very elderly) that are affected by the Programme

3.2 Design of the evaluation

The design of the evaluation was shaped by two main principles. Firstly, it was required to be longitudinal - that is, to collect data on respondents' experiences at multiple points in time. Secondly, it was specified that the evaluation should be controlled: the experiences of respondents who received heating under the Programme were to be contrasted with those of a comparison group who did not receive facilities under the CHP. The longitudinal element (specifically, the collection of data both before and after the installation of heating in recipient homes) permitted the identification of changes (for example, in respondents' health status) which took place over the period when heating was provided. The inclusion of a comparison group assisted in isolating changes which might reasonably be attributed to the CHP from those which might have occurred for other reasons.

While, as stated, the evaluation featured a comparison group, it was not a true experiment, as the allocation of individuals to the recipient and comparison groups was not carried out by the research team. The recipients in the study were an 'opportunistic' sample, reflecting varying degrees of willingness among public-sector landlords and tenants / householders to participate. Moreover, the comparison group (which would ideally have consisted of households which lacked central heating throughout the period of the evaluation) necessarily included a proportion of homes which possessed central heating systems, for the simple reason that it would have been impossible to find a sufficiently large body of 'heating-less' households which were not enrolled in the CHP. In the event, the comparison group included dwellings both with and without central heating. This being so, the comparison group was viewed as a body of households whose status in respect of their domestic heating arrangements was expected to remain broadly static across the period of the evaluation, thus providing a fairly constant base against which any changes experienced by the recipient group could be contrasted. This expectation was not, in the event, wholly met (see Section 3.4).

To provide data for the evaluation, a sample of households eligible to receive central heating under the Programme was contacted at three different time points. First, an initial interview with the head of household (or her / his partner or spouse) was conducted in the respondent's home shortly before heating was scheduled to be installed. This initial data collection phase gathered information on a range of topics, including (but not limited to) household heating arrangements and the health of the respondent 6. One year after the initial interview, a short postal questionnaire was sent to participating households. This included a subset of the questions presented at the initial interview stage (some of them modified), together with a number of new items. Finally, one year after provision of the postal questionnaire (and thus two years after the initial interview), a final interview was conducted, again in the respondent's home.

The interview schedule for this final data collection phase was almost identical to that used in the initial contact. The final questionnaire differed from that used at the baseline point in two respects. First, the final questionnaire included some additional items - presented to all respondents - about the receipt of central heating and related thermal efficiency measures. These additional items sought to establish whether heating had been installed since the initial interview, how long it had been in the property, and whether additional measures such as loft insulation or cavity wall fill had been fitted since the first interview. Respondents in the CHP recipient group who had not actually received heating were also asked to provide the reason(s) for non-installation. Second, a series of questions on key life events (such as bereavement, or the experience of serious illness) within the past year was included. These questions were not included in the initial questionnaire (though they were included in the interim postal questionnaire).

In parallel with the process described above, a sample of comparison households was subjected to an identical data collection regime ( i.e. initial interview, interim postal questionnaire and final interview). Comparison households were, as far as possible, individually matched to recipients in terms of certain key characteristics, the matching criteria being described in the Fieldwork Report ( Appendix A).

3.3 Data collection timetable and achieved sample numbers

3.3.1 Initial interviews

The initial data collection took place between November 2002 and February 2004. A total of 3,849 households provided data. Of these, 1,977 (51.4%) were central heating recipients, the remaining 1,872 (48.6%) being comparison households.

3.3.2 Interim postal questionnaire

Postal questionnaires were sent out over the period November 2003 to March 2005, i.e. approximately one year after the initial interviews. Responses were received from 2,131 participants, representing 55.4% of the sample interviewed at the start of the evaluation.

3.3.3 Final interviews

Final interviews were conducted between December 2004 and March 2006. Successful interviews were achieved with 2,365 households, representing 61.4% of the original sample. The results presented in this Report are based on data from these households. Of the 2,365 respondents who contributed to both the initial and final interviews, 1,281 (54.2%) were heating recipients, the remaining 1,084 (45.8%) being from comparison households. The 1,281 recipients represented 64.8% of the total number of recipients interviewed at the initial point, while the 1,084 comparison respondents made up 57.9% of the comparison households which provided initial interviews.

3.4 Stability of the respondent groups

As explained earlier, the comparison group was not a 'control' element in the accepted sense, but rather a set of respondent households - some with central heating at the start of the evaluation, some without - whose domestic heating arrangements were expected to remain broadly unchanged across the two-year period covered by the evaluation. In the event, it was found that a substantial proportion of comparison households did experience a change in their heating status (specifically, the acquisition of central heating) in the course of the evaluation. Of the 1,084 comparison respondents who provided data both at Wave 1 and Wave 3, 279 (25.7%) appear to have acquired central heating at some point during the period examined by the evaluation. Conversely, of the 1,281 recipient households who yielded successful interviews at Waves 1 and 3, evidence from the interviews suggests that 92 (7.2%) did not actually receive heating or had a central heating installation which predated the start of the evaluation.

Despite this 'contamination' of the two groups, the original classification of respondents as either recipients or comparison group members was retained for the analyses reported here. This approach corresponds to the 'intention to treat' approach applied in clinical trials, under which subjects are retained for analysis purposes in the treatment group to which they were originally allocated, even if for some reason they did not in the event receive the intended treatment option. In the present case, the decision to adopt the 'intention to treat' approach was motivated by the following considerations. The remit of the evaluation was not to assess the health impact of receiving central heating in general, but rather to determine the health-related effects of a specific initiative - the Central Heating Programme - with its own unique client base, and financial and administrative characteristics. One feature of such a real-world initiative is that, for a variety of reasons, some intended clients of the Programme will not in the event actually receive heating systems. A second feature is that households outside the Programme will, via a variety of routes, acquire central heating independently of the initiative. Thus, to assess the specific health impacts of the CHP - as distinct from the more general effects of 'receiving central heating' - in the actual context within which it operates, it was considered appropriate to retain the original respondent groupings (even if subject to 'contamination') on the grounds that this approach faithfully reflects the experience of the CHP as actually implemented.

3.5 Comparability of recipient and comparison groups in final achieved sample

Table 3.5.1 compares the recipient and comparison groups in the final achieved sample, in terms of a number of key characteristics: age; sex; socioeconomic group; household composition; housing tenure (simplified representation); and physical property type.

Table 3.5.1 Characteristics of CHP recipients and comparison group

characteristic

CHP recipients
n = 1,281

comparison group
n = 1,084

Age, years: mean ( SD)

61.9 (16.5)

62.4 (16.8)

Sex: % female

64.7

63.5

Socioeconomic group (%): [1]

AB

3.6

2.6

C1

17.2

14.9

C2

18.1

22.0

DE

61.2

60.5

Household composition (%):

single adult

8.7

9.5

single parent

8.5

6.7

single pensioner

37.3

35.8

couple with no children

11.1

11.1

couple with children

7.1

7.7

pensioner couple

22.7

22.4

multiple adults

4.6

6.9

Tenure (%):

owner-occupier

42.5

40.7

Rented from local authority/housing association

51.4

56.4

Rented from private landlord

4.7

1.9

Other

1.4

1.0

Property type (%):

detached house

13.7

14.9

semi-detached house

14.0

15.7

terraced house

27.7

23.4

tenement

22.5

19.2

four-in-a-block

12.3

15.6

flat in converted building

1.5

1.1

high-rise flat

2.6

7.6

other

5.7

2.6

Notes
(1) The characteristics shown are as measured at the initial interview point.
(2) The cell content represents the percentage of the group (column) total e.g. 'AB' respondents comprise 3.6% of all recipients. This form of representation is also used for the 'household composition' and 'property type' variables.