5 MAKING A DIFFERENCE
5.1.1 The central question is: Are ADATs making a difference? This can only be answered by effective monitoring of impacts through data collection by ISD and others and by communication and consultation with the very wide range of people affected by substance misuse. We have considered this question under the following sections:
- Public Engagement
- Service Delivery and Outcomes
- What difference do ADATs make?
5.2 Public Engagement
What is public engagement?
5.2.1 We have used the term public engagement as a generic description of the whole field of activity including communities, families and carers, service users and wider public information. Where we are discussing one of these groups we have used the specific description.
5.2.2 At the inception of AATs and DATs it was envisaged that alcohol and drugs forums would be established. Their purpose was to bring in advice and input from communities, families and carers, and service users. They might also include voluntary sector representatives. The chair of the forum would ideally have a seat on the AAT or DAT.
5.2.3 Over time and especially in the last decade, there has been a significant shift in the relationship most professionals working in public and voluntary services expect to have with individuals for whom they provide a service, whether they are described as service users, patients, clients or customers. This change is exemplified by the following comment of a service manager working in one of the ADAT areas: "When I did my social work training I was taught that as social workers we are the change agents for clients but through my training and experience in counselling I have come to realise that the client is the change agent and we facilitate them. It is now more the client's view that defines the service, rather than the professional's."
5.2.4 There has also been a transformation, partly due to Best Value and other similar approaches in the relationship between public services and the communities they serve involving consultation and sometimes participation in the decision making processes. The guidance by the Scottish Executive to the NHS in Scotland is a typical articulation of official policy in relation to public engagement. The approach is described in the Scottish Executive document Patient Focus and Public Involvement ( PFPI) which was published in December 2001. Its stated aim is to achieve culture change in the way the service interacts with the people it serves and the way services are delivered. It says that "it is no longer good enough to simply do things to people; a modern healthcare service must do things with the people it serves."
5.2.5 Another example of official policy in relation to public engagement is contained in Communities Scotland's National Standards for Community Engagement. Like the PFPI policy for the NHS it seeks to change the way public services interact with communities. The Communities Scotland document sets out principles, behaviours and practical measures that underpin effective engagement.
What we found and commentary
5.2.6 We found forums in some ADAT areas but not in all. With the passage of time since the DAT guidance of 1995 and the 2002 Plan for Action on Alcohol and the changed role from AATs and DATs to ADATs, it is not surprising that we found that the idea of forums has evolved in different ways in different parts of the country. In some areas there were geographically based sub-groups that worked in similar ways to the forums originally envisaged. The distinction between a forum for consultation and a sub-group of the ADAT was sometimes not clear. However, as a method of getting input from communities, families and carers, and service users, forums were often of little benefit and in some areas no longer existed.
5.2.7 What has emerged from our Stocktake is that even where there were forums there were different ideas about their purpose, their composition and the benefits they brought to the ADATs' work. In some areas forums were seen as the practitioners' networking meetings; in others they were criticised as having become the property of a single interest group; while in other areas they were a successful mechanism for two way communication with the strategic ADAT, providing both a forum for consultation and a fertile ground for ideas and energy to develop local initiatives.
5.2.8 Forums can be a useful mechanism but they cannot be the only response to the need for public engagement which is a complex area requiring a range of responses to suit different needs and purposes.
5.2.9 A comment that was typical of the responses we found in most parts of Scotland is: "more needs to be done to involve service users and the wider community in drugs and alcohol issues". Some ADATs with geographical forums or sub-groups regarded these as having close community connections. An example of another approach was an ADAT having a community engagement officer within the support team who had a specific role to work with communities.
5.2.10 There was no uniformity of approach in consulting the wider community but the range of approaches included:
- Conducting special surveys (youth, street, etc) and getting results from wider quality of life surveys organised by local authorities and community planning partnerships.
- Attending local events, fairs, freshers' weeks, tenant/resident groups, community meetings, etc.
- Parent awareness sessions at family centres and young people forums in schools.
- Citizens' Panels established under Community Planning arrangements.
- Councillor members of ADATs feeding in the views of the wider community.
5.2.11 Generally it was felt that this was a difficult area which lacks a structured and strategic approach. Rurality was often cited as a difficulty because of stigma and confidentiality but some ADATs operating in rural areas also felt that the closeness of the community was an advantage in having informal knowledge of the wider community views.
Families and Carers
5.2.12 We found little evidence that ADATs consult families and carers about the services in their area. There is no doubt that families and carers are knowledgeable and experienced about the problems of substance misuse in a way that could inform strategic planning and commissioning and yet their potential seemed to be largely overlooked by most ADATs.
5.2.13 Families can offer vital support for a person who is a substance misuser and family members can become the victim or sufferer of a substance misuser's chaotic lifestyle. We met families whose lives were totally dominated by the need to care for a substance misuser. This ranged from obvious concerns about the health of the family member and the risk of early death to a fear of violent outbursts, as well as concerns about criminal behaviour.
5.2.14 In some ADAT areas there were family support networks, often run by local voluntary organisations, where people came together to gain mutual support. We met some who were dissatisfied with the level of support they received from the statutory services and who felt unrecognised by the ADAT. In recent years the need to support carers who care for older and disabled people has received increasing government support and recognition. By comparison, the needs of families and carers of substance misusers appears to be an area of need that is under-recognised and under-resourced by ADATs. ADATs should address this when drawing up their Strategic Plans.
Recommendation 26: ADATs should give greater emphasis to meeting the needs of families and carers and the potential for their participation in shared care arrangements and in informing service development.
5.2.15 We found few examples of service user involvement in the ADAT and its subgroups. Where it did exist, such as when a former service user was a member of a forum, it was likely to be dismissed as tokenistic or unrepresentative. We often heard that it was difficult to get beyond the "usual suspects" when engaging service users in planning and designing services and that the views they bring are not necessarily representative of the client group.
5.2.16 There was some evidence of public consultation being designed to engage a wider group of service users in order to inform the strategic processes. This sometimes took the form of open days and surveys of service users and evaluations of services. SDF often played a key role in survey and evaluation design and implementation. In order to overcome the challenge of substance misusers often being a hard to reach group, some surveys were carried out as peer research.
5.2.17 At service delivery level it was commonly reported to us that the service provider would have arrangements to capture service user feedback about the quality of their service, although this was not always in documented form. This was supported by an Alcohol Focus Scotland survey of service users which found "almost all interviewees felt that they could raise concerns freely" and concluded that services were responsive to ad hoc issues. SDF or other organisations were sometimes commissioned to provide evaluation of services which was a useful input for service providers.
5.2.18 In some areas it was a requirement of the Service Level Agreement for the service provider to obtain feedback from the service users and this would feed into the commissioning process but in other areas collecting this information relied on the initiative of the service provider with no link into commissioning. Where service users' views were sought and acted upon, it could make a real difference to the service. (See Case Study 5.)
5.2.19 Service users undoubtedly have a useful role to play in making services more responsive to client need. While we recognise it can be difficult, for a variety of reasons, to capture the views of service users, we believe this is an area where ADATs generally could do more and they should include a commitment to this in their Strategic Plans.
Case Study 5: Involvement of drugs service users in a peer research project
In one ADAT area forum, concern was raised about the experience of drug users in their communities and how this experience might inform the development of services locally. The ADAT applied to the Scottish Community Action Research Fund ( SCARF) and was granted funding to develop a peer led appraisal of the needs of drugs and alcohol service users in its area. Providing service users with a voice was felt to be important and the resulting profile of drug service users has begun to inform the way services are being developed. These profiles have been considered by the ADAT forum who have identified a number of areas where key improvements in service delivery could be made to address gaps in general service provision as well as issues around methadone prescribing and needle exchange services.
Recommendation 27: ADATs should improve the involvement of service users and put in place mechanisms to capture and act on their views.
5.2.20 ADATs used a variety of means to communicate with the public including radio, articles in local newspapers, press releases, newsletters, annual reports, leaflets, advertising, directories of local services and holding ADAT meetings in public (although these did not appear to be well attended). Some ADATs used several of these approaches and appeared to be particularly conscious of their role to provide public information both by raising awareness of issues and by informing the public about services. But there were also some who did little or nothing. Even though some ADATs put a fair amount of resources into providing public information there was little evidence of them obtaining structured feedback or evaluation of this work.
5.2.21 In order to raise their profile and improve communication of their activities, most ADATs had developed websites to advertise their work and the services available as well as local contact numbers. Only two appear to have no presence on the internet while the others either have their own website or webpages within a section of one of the partners' websites. However, not all of the websites had been kept up-to-date and the standard of information and content is variable. Some websites mention only services and not the ADAT itself. Websites can be a useful means of communication, especially in rural areas.
5.2.22 ADATs also used open days, information stands in public buildings such as sports centres and libraries, community meetings and focus groups. Most ADATs recognised they could make a contribution to educating young people by linking in with programmes at local schools as well as taking information out to dances and night clubs. The need to catch people's attention had also led to novel methods of communicating messages about substance misuse such as the Pink Handbag campaign on safe alcohol use. (See Case Study 6.)
5.2.23 Significant effort is going into local campaigns but ADATs thought that national campaigns were not always relevant at a local level or that they replicated work already being done locally. Many ADATs felt that they needed greater communication with the Scottish Executive with regard to the focus and content of national campaigns for drugs and alcohol. They wanted to ensure that through good planning arrangements, their local campaigns would dovetail with the national agenda , ensuring consistently powerful messages across the country. Such improved communication processes would enable the partners within each ADAT to jointly plan and organise their local role within the national agenda. An example quoted by one police authority was of a national campaign on safe drinking targeted at younger people. The distribution of posters and leaflets was determined nationally and did not tap into local knowledge of venues used by young people. As a result, many of these were not included in the campaign whereas venues not usually used by younger people, such as ex-servicemen's clubs, were included.
5.2.24 The development of national campaigns does seem to be an area where the expertise and knowledge of ADATs could make a useful contribution. Although there is a Scottish Executive National Communications Group on drugs and alcohol which includes a representative from SAADAT, and a further two places for ADATs, the ADATs did not seem to be aware of this. It would be helpful if SAADAT and the Scottish Executive took steps to raise awareness of the National Communications Group and encourage more ADATs to participate.
Recommendation 28: The development of relevant national campaigns about drugs and alcohol should be undertaken in partnership with ADATs. This would include decisions about the focus and content through to planning and organisation. Steps should be taken to raise awareness of the National Communications Group and encourage participation by ADATs.
5.2.25 ADATs' development of policies and practices for public engagement was patchy. Although at their best we came across ADATs making serious efforts in a difficult field with limited resources, even they admitted that they relied on fragmented coverage and immature methods. Most respondents recognised that their ADAT should be doing more and doing better in this area. Few ADATs had developed a communications strategy and while most were clear about which approaches they prioritised and preferred, there was a lack of a strategic approach to communications with the public.
5.2.26 There is a clear need to develop clarity about whom the ADATs should be engaging with, for what purposes and by what means. Guidance such as Patient Focus and Public Involvement and National Standards for Community Engagement are useful guidance and available to ADATs to draw upon in this area. ADATs need to develop a strategic approach and a structured plan of action and should draw on guidance that will help them to tackle these issues in a consistent way.
Case Study 6: Pink Handbag Women & Alcohol Communications Campaign
One ADAT developed an alcohol health promotion resource targeting young women aged between 16 - 25 years old. The need for this campaign emerged because of an increase in the number of young women drinking harmfully.
The ADAT was successful in bidding for £10,000 from the Scottish Executive alcohol communications budget and worked with a design firm to develop an innovative, eye-catching leaflet to educate young women on the health and personal safety risks associated with binge drinking. A member of the ADAT support team liaised with the design company throughout and provided the text input for the resource. The draft leaflet design, in the shape of a pink handbag, was circulated to all ADAT members for comment and then market tested with young women. The ADAT organised a formal launch of the campaign and 20,000 copies of the pink handbag leaflet were distributed to young women via pubs, nightclubs, schools, specialist services, health centres and homeless units.
The campaign was evaluated 6 months after the formal launch, with extremely positive results. The evaluation results indicated a very high recall rate of the campaign amongst young women and specialist alcohol services in the area noted a significant increase in referrals from young women.
As a result of the resource's success, the relevant ADAT support officer was invited to make a presentation at two international health-related conferences and the leaflet was selected as a top finalist in the Association of Public Service Excellence ( APSE) awards 2006. Moreover, the Pink Handbag has been rolled out to several other ADAT areas in Scotland, England and Europe. The ADAT has since developed a male equivalent resource to the Pink Handbag, which is also proving to be extremely popular within the target group.
Recommendation 29: ADATs should develop a public engagement strategy within their 3 year Strategic Plan. It should encompass plans in relation to communities, families and carers, and service users. Implementation should be through the ADAT Annual Delivery Plan.
5.3 Service Delivery and Outcomes
What is Effective Service Delivery? What are Relevant Outcomes?
5.3.1 The core function of ADATs is to ensure the delivery of services, activities and projects which meet Ministerial targets and commitments in relation to tackling drug and alcohol misuse. It is beyond the scope of the Stocktake to assess directly the quality and effectiveness of these services - but the Stocktake did examine the processes undertaken by ADATs to ensure that the right services are delivered to the right standard and as cost effectively as possible.
5.3.2 All services - statutory, voluntary and private - are struggling to develop relevant, measurable outcomes where personal perceptions and human condition are significant factors. At present the emphasis in performance management tends to be put on the easily measurable - numbers treated, waiting times, etc. This can distort the real priorities of a partnership or single agency and where proxy measures are used these need to be linked by evidence to a sound understanding of better outcomes. An increasing focus on outcome measurement should not deflect attention away from the relevance of effective processes since the impact of a service on people largely depends on the way it is delivered.
What we found and commentary
5.3.3 All of the ADATs felt that the existence of the drugs and alcohol partnership had resulted in improved services within their areas - extending the range and diversity of services and service providers.
5.3.4 ADATs were asked to identify their success criteria in terms of service delivery. The most commonly cited aspects were:
- Customer focus - services which are appropriate and accessible and which address need
- Sustainable, quality services
- Consistency of standards across service provision
- Impact assessment - knowing what difference the services are having
- Value for money
5.3.5 Many of these success criteria were being met resulting in the range and diversity of services within ADATs having increased and almost all of those interviewed regarded this as a direct result of partnership working and ADAT activity. This was the single most reported benefit of partnership working by interviewees.
5.3.6 We encountered a number of examples, some highlighted within this document, of good practice and a number of examples where there is a move towards being much more customer focused and organising services around client need - rather than the more traditional model of suiting organisational need. However this appeared to be confined to pockets of good practice rather than an overall deliberate attempt to put the customer at the heart of service delivery. One door approaches, and single point of service entry were in evidence in a number of areas.
Consistency of services
5.3.7 There were examples of ADATs contributing to more consistent services. Shared protocols had been developed for prescribing, information sharing, drugs and alcohol policy for schools. The National Quality Standards were also beginning to be embraced which, in time, should lead to a standardised level of service which customers can expect. There were a few areas where the voluntary sector was an equal partner on the ADAT. This had resulted in less unproductive competitiveness for funding between voluntary sector organisations and more co-operation between the statutory and voluntary partners.
5.3.8 However, we also encountered examples where practice was not consistent or client centred. Most significantly there were examples of Health Boards and local authorities taking unilateral decisions, apparently for reasons of internal policy, which were inconsistent with practice elsewhere in Scotland and which acted against a client based approach.
5.3.9 For example, one Health Board had set the budget for methadone treatment at a level which ensured that some patients, mainly single men with no dependants, would never receive a treatment service. This increased the burden on other service providers such as social work and the police as well as neighbouring Health Boards. It also meant that offenders being released from prison in another Health Board area where they had been stabilised on methadone, could find themselves no longer able to access a treatment service. ADAT partners had tried, unsuccessfully, to change the Health Board's policy.
5.3.10 In another Health Board, we heard that clinicians had decided on a policy to provide methadone treatment at clinically sub optimal dosage levels. This approach, compounded with a strict approach to clients who topped up with other substances, meant that many clients complained of a "revolving door" approach to treatment and social work staff were concerned about the additional burden on them.
5.3.11 A third example is of a social work department which had adopted a policy of opposition to residential rehabilitation - it seemed on grounds of cost and conviction rather than evidence - even though this was not an approach shared by the other local authorities in that ADAT.
5.3.12 Waiting times for drug and alcohol assessment and treatment services varied widely across Scotland and this may in part be due to the absence of relevant targets in the Health Efficiency Access Treatment ( HEAT) key targets set by the Scottish Executive to secure effective performance by Health Boards.
5.3.13 We heard from a number of ADATs that they had been unable to influence what was seen as inconsistent standards relative to the practice elsewhere in Scotland or, in a very few cases, non-compliance with clinical guidance. While, we believe that some of these inconsistencies can be addressed by compliance with the National Quality Standards, ADATs have few other means at their disposal. Their position could be strengthened by the introduction of national targets and we propose that the Scottish Executive should seek to improve waiting times for treatment services; and improve consistency between treatment services by considering the development of appropriate HEAT targets. The Scottish Executive should also consider the need to work with the clinical community such as the medical Royal Colleges to ensure compliance with guidelines and good clinical practice.
Recommendation 30: The Scottish Executive should: ensure that compliance with the National Quality Standards is monitored; consider the use of HEAT targets to deliver improvements in drug and alcohol treatment services; and explore the need to work with the clinical community to secure consistent and equitable compliance with Clinical practice.
Primary Care and General Practitioners
5.3.14 Many ADATs expressed concern that the changes in the General Medical Services ( GMS) contract for GPs in 2003 had led to significant problems in the availability of treatment services. However, in some ADAT areas Health Boards had used a mix of funding and support to persuade GPs to offer shared care to patients with less complex needs and we heard that this worked effectively. In other areas, Health Boards had increased capacity of centrally provided services.
5.3.15 Problems of access to treatment for service users often arose where there was little choice of general practitioner and poor access to specialist services - usually but not always in rural areas. Poor public transport links compounded this. We heard from some GPs and others in the NHS that the engagement of GPs might be aided by greater clarity of national policy, with regard to, for example, the evidence underlying a harm reduction approach as compared with the evidence underpinning an abstinence programme. However, most ADATs who raised GP services as a concern, indicated that GPs were unwilling to engage with substance misusers as a client group.
5.3.16 Recent changes in NHS structure place responsibility for the development and management of primary care services with Community Health Partnerships ( CHPs) and we consider that close links between CHPs the ADAT structure are essential to resolve this issue. While CHPs are the appropriate mechanism for engaging GPs in service planning and delivery, some ADATs might find it helpful also to build a relationship with local GPs.
Moving from inputs and outputs to relevant outcomes
5.3.17 There was a strong sense across all ADATs that the determination of relevant outcome measures was key to ensuring the effective delivery of services. Some ADATs had invested a significant amount of time and effort into this work, often with external facilitation. However, all ADATs had struggled with developing outcome measures and would welcome - and need - support from the Scottish Executive to achieve this.
5.3.18 There is no doubt that across Scotland organisations and partnerships of all kinds find it difficult to measure outcomes. For this reason there is often a heavy reliance on measuring inputs and outputs - measuring activity rather than impact. ADATs are thus far from unique in finding outcome measurement a challenge. In other parts of the UK there are helpful examples of a national approach to performance management and outcome measurement.
5.3.19 One such example is the Welsh Assembly Government's Key Performance Indicators for Substance Misuse Treatment in Wales. These were developed in consultation with commissioners and service providers and implemented from August 2006. The KPIs were intended to tackle high "do not attend" rates and long waiting lists. They are the first phase of the development of an outcome measurement framework to ensure a comprehensive and consistent approach across Wales. The aim is that eventually all clients in substance misuse services will have their experience measured against four key outcomes:
- Effect on offending behaviour
- Effect on substance misuse behaviour
- Improvement in relationships with family
- Improvement in general health
5.3.20 The National Treatment Agency for Substance Misuse has recently launched its Treatment Outcomes Profile tool ( TOP) which is intended to measure improvements in the health and wellbeing of drug misusers across a range of factors including substance use, crimes committed, health, employment and education and housing.
5.3.21 It is our view that a national approach to outcome measurement should be developed in Scotland. In doing so it would be sensible for the Scottish Executive to work with ADATs and to draw on the experience of developing outcome measurement elsewhere in the UK.
Recommendation 31: The Scottish Executive should work with ADATs to develop national outcome measures, including the identification of robust proxy measures for less tangible outcomes. This could be informed by the experience of developing outcome measures elsewhere in the UK.
5.4 What Difference do ADATs make?
Why does this matter?
5.4.1 Any partnership has an opportunity cost. It takes time, money and effort to work collaboratively across agencies that may have different philosophies, priorities and cultures. At worst, the existence of a partnership to address issue or issues may offer an excuse to individual agencies to avoid focusing on the very topic that is considered important enough to have a partnership in the first place. Therefore it is essential that we can support our conclusion - that the ADAT approach to tackling the issues of substance misuse is appropriate with robust evidence from the Stocktake.
What we found and commentary
5.4.2 The majority of ADAT partners that we interviewed felt that the drug and alcohol partnership was making a difference to services, and hence to service users. They considered that the time and effort required to sustain partnership working was rewarded by successes that could not be achieved by individual agencies. There was an overwhelming view that the complexities of the issues relating to substance misuse demanded a collaborative effort from a wide range of agencies.
5.4.3 It is important to note that most of the respondents were giving time to the ADAT partnership that was over and above their "daytime job". There was no evidence of that view being influenced by vested interests. Indeed, we might have expected ADAT partners to welcome the opportunity to reduce their workload.
5.4.4 The overwhelming message from all ADATs was that focus on substance misuse would be lost if the ADAT agenda were to be subsumed by an alternative partnership. The evidence found throughout the Stocktake supported the continuation of a dedicated partnership for substance misuse. The benefits of a partnership approach were identified as:
- Being able to focus clearly on the issues of drugs and alcohol and their impact on the community
- Having a multi-agency response to complex issues that did not sit within the remit of a single agency
- Raising the profile of substance misuse locally
- Providing a conduit for co-ordinating service interventions - cutting down duplication and overlaps
- Identifying gaps in provision - through needs and gap analysis
- Integration of services and single point of entry for service users
- Enabling the re-design of services that focused on the needs of service users
- Providing a mechanism for joint commissioning and allocating finance to the services
- Protecting resources dedicated to tackling substance misuse problems
- Creating a mechanism for sharing information and having a shared perspective of the area
- Closer working between the voluntary and statutory sectors
- Sharing training needs analysis and training plans and opportunities.
5.4.5 Many ADATs were undergoing significant change and development - generally they recognised the need for their partnerships to be more focused, to generate more resources and to sharpen up commissioning, contracting and performance management.
Characteristics of Effective ADATs
5.4.6 There is a number of characteristics displayed by the more successful ADATs which have had an impact on their success and effectiveness. These include:
- Strong leadership - the ability of the Chair and support team to facilitate and lead the development of the partnership.
- A commitment to effective partnership working at senior level in the partner organisations.
- Having the right partners at the right level to be able to commit and deploy resources.
- Clear lines of accountability
- An interlinked relationship with other strategic partnership bodies at a local level.
- Having a clear understanding of the strategic role of the ADAT.
- Good analysis and information about local need
- Effective engagement with service users, families and carers and the wider community.
- Effective performance management.
5.4.7 There is an important role for the Scottish Executive to provide strong leadership at Ministerial level and to ensure that its executive functions are organised and managed to support the joined up working expected of ADAT partners at Strategic and Implementation levels of the proposed structure.
5.4.8 We are firmly of the view that the fundamental principles and purposes of ADATs as set out in the 1995 and 2002 circulars and the respective drug and alcohol strategies are as relevant now as they were then.
Recommendation 32: On the evidence gathered in the preparation of this report we believe that ADATs are, in the main, adding value to the substance misuse agenda. ADATs should continue as discrete partnership bodies, albeit in a form modified by the recommendations in this report.