We have a new website go to gov.scot

Evaluation of the Assessment is for Learning Programme: Final Report and Appendices - February 2005

Listen

5.5 Support and collaboration

Contributions of stakeholders and sources of support to teachers

Local authority co-ordinators and HE representatives were asked to indicate the extent to which a range of stakeholders had made an impact in introducing change to schools. Respondents were asked to rate each group of stakeholders listed in Table 5.28 according to how they perceived their impact.

Respondents did not feel able to take a view on all stakeholders, with the HE representatives in particular being reluctant to rate other stakeholders for a number of reasons: collaboration was the main focus and people collaborated in different ways, so a view could not be taken on a whole group; they had insufficient evidence to make any judgement as they did not know enough about the roles of others, eg the LA co-ordinators, and it was difficult to get an overview of the various contributions. It was generally agreed by the HE representatives that all groups had had some impact.

The responses from the local authority co-ordinators are given in Table 5.28.

Table 5.28: Views on impact of stakeholders on changing practice in schools ( LA co-ordinators)
(1 = high impact to 5 = no impact)

High impact ? No impact

1

2

3

4

5

LT Scotland Development Officers

3

9

2

9

3

Local Authority Assessment Co-ordinators

11

11

7

0

0

Local Authority Assessment Development Officers (where relevant)

11

6

1

1

0

HE representatives

1

2

3

5

10

Expert speakers at conferences/in-service events

11

13

2

1

1

In the survey of LA co-ordinators carried out in May 2003 there had been strong agreement that the LT Scotland DOs provided strong and committed support for the developments. At September 2004 views were more ambiguous with respect to the LTSDOs, with a perception that they had a reduced impact. However, their role had changed since the early phases of the programme and different DOs were in post. During 2003-2004 they had more of a co-ordinating than a delivery role. Therefore the change in the extent to which they were perceived as having a high level of impact on changing practice in schools is unsurprising. One assessment co-ordinator reported that the LT Scotland staff continued to support the co-ordinators in their roles and therefore this in turn affected practice in schools, but the relationship is not as direct as it was at first.

The role of the HE representatives has been more of an advisory one to SEED, with only a few becoming involved in delivery of in-service or in direct involvement with the authorities. It is therefore unsurprising that they have not been perceived as contributing directly to change in practice in schools. One respondent commented on the impact on Initial Teacher Education, suggesting that probationers were now arriving with knowledge of the programme. Interviewees had emphasised that the HEIs' role in preparing new teachers for the use of formative assessment was a key aspect of sustainability.

The impact of expert speakers was seen as high (confirming the view on effectiveness of their role in staff development - see p76), and the role of the authority staff themselves was very important in introducing change.

Teachers were asked to indicate how important they perceived various players were in taking forward AifL developments during 2003-2004. Their responses are reported in Table 5.29.

Table 5.29: Importance of various contributors to AifL developments during 2003-2004 (Teachers)

Very important

Important

Of little importance

No role to play

No response

%

%

%

%

%

School management

68

22

5

2

4

Fellow teachers in own school

62

29

6

0

3

Teachers from other schools

16

42

22

15

5

Local authority co-ordinator

19

31

24

16

10

LT Scotland development officers

19

35

21

14

11

Expert speakers at conferences/in-service events

22

48

15

4

11

Faculty of Education representatives linked with projects

6

32

25

22

15

The most important sources of support for teachers remained within their own schools, but the input of experts was acknowledged as being the next most important influence in taking forward developments.

Effectiveness of approaches to dissemination

Views on the effectiveness of various communication and dissemination activities within AifL were sought. Responses from the local authority co-ordinators are reported in Table 5.30.

Table 5.30: Views on effectiveness of AifL communication and dissemination ( LA co-ordinators)
(1 = very effective to 5 = ineffective)

High impact ? No impact

1

2

3

4

5

LT Scotland website

0

9

15

6

0

Publications, eg AifL Newsletter/Update

2

14

11

3

0

Open Space events

3

11

8

5

0

Regional seminars/conferences/ dissemination events

6

20

2

1

0

National seminars/conferences

9

16

2

2

0

The local authority assessment co-ordinators clearly viewed the regional and national events as being highly effective in impact.

The more remote authorities mentioned not being part of Open Space events. Some suggested that national events were more useful for co-ordinators but that the main dissemination to schools was effective only at local level and through local network meetings. Four respondents raised concerns over the national events: two indicated they were repetitive; one suggested it was too much sharing among the 'converted'; and one was concerned about the cost due to the expensive venues. Once again the issue of travel and cost for more remote authorities to attend national events was raised and it was suggested that progress could be made through the use of video-conferencing.

Comments from LA respondents were that the website needed greater publicity, it was not updated regularly, it did not provide much useful help, and it was not user friendly. The limited usefulness of the website reflects responses given by both headteachers and teachers. Less than a quarter of headteachers had used the website to share information with other schools (see Table 5.31) and both teachers and headteachers had agreed that finding time for this was a challenge (see Table 5.29).

The HE representatives shared the views that regional and national events had the greatest impact, with the website and newsletters being less effective. Few commented, but one respondent summarised the position as follows: 'dissemination is limited by the time of people to participate. The website was fine, but many teachers lacked the time to check it out. National seminars were inspirational but addressed a captive audience which was small. Properly organised regional events have more potential, but will only work if there is subsequently time, support and resources (and maybe a carrot').

One respondent saw the benefit of being able to use the case studies as they appeared on the website as a useful teaching tool for students, but regretted that there was little in the way of video resources for university teaching. It was felt that slow development of the case study materials was hindering the opportunity for wider dissemination and also for giving assessment issues a Scottish context (for teacher education purposes).

Working collaboratively

The AifL Programme was designed to encourage and support collaboration between key players including local authorities, higher education institutions, LT Scotland and SEED. Respondents were asked to comment on the progress they thought had been made to date on this aspect of the programme.

The majority of local authority co-ordinators reported that effective collaboration had been established in working with SEED and LT Scotland, with closer links being formed than previously existed. Three authority respondents indicated that this was less than it might have been because they themselves had not particularly sought support or had only had contact at national meetings; one said it was an area they needed to develop. A remote authority indicated that it was difficult to maintain after initial enthusiasm. Another respondent suggested that, at times, tensions had arisen between authorities and SEED which could have been managed more effectively, and two experienced the relationship as authoritarian and challenging rather than collaborative.

The aspect of collaboration most frequently mentioned as beneficial was the way in which authority co-ordinators had come together at the assessment co-ordinator meetings which had provided a forum for them at a national level. This had led to collaborative working and sharing of ideas, and several mentioned networking of neighbouring authorities. There was mention of the benefits of sharing materials between authorities.

The majority of respondents stated that they had had little or no contact with the HE representatives and that they were unclear as to what their role was or what they had to offer. Three suggested that this might improve in the future and one reported plans for collaboration on research.

The HE representatives gave a wide range of responses to this, ranging from those who thought progress had been limited and superficial to those who thought substantial progress had been made:

  • Four mentioned formal collaboration through meetings, with one especially mentioning the Toolkit focus group, but more spontaneous collaboration was more difficult to develop, partly through lack of time and opportunity for professional exchange. One commented 'that we do not always know each other as well as SEED think we should'
  • One respondent reported only very superficial collaboration: 'it will take time for trust and relationships to build'. Another said it had been limited: 'barriers still exist between different agencies, especially in respect of philosophies; differing agendas in terms of the purposes of assessment mean that people speaking the same language and using the same terminology can often mean very different things. However, there has been progress and at least people are talking to one another'
  • One reported significant progress: 'there is a community in development, with a better understanding of each other's goals, aspirations, constraints and requirements in the area of assessment for learning rather than assessment for summative purposes'
  • One person suggested that there had been little progress between local authorities and HEIs, but that this was improving. Others mentioned links with local authorities for specific purposes, such as research and CPD provision
  • One person noted that real progress had been made but that it was not uniform. They made the important observation that we tend to treat local authorities and HEIs as if they were single bodies but they are not and therefore collaboration will vary between organisations: 'the numbers who are part of the learning community are growing'.

A question to headteachers focused on the issue of liaison between schools, locally and as part of national networking. Respondents were asked to indicate from a list what type of liaison they had been involved in and their responses are reported in Table 5.31.

Table 5.31: Inter-school liaison and networking (Headteachers)

Primary

Secondary

Special

5-14/16

Total

n

n

n

n

(Primary schools) with other primaries

37

0

0

1

38 (70%) 1

(Secondary schools) with other secondaries

0

11

0

1

12 (39%) 2

Liaison across the local cluster involving both sectors

24

13

0

2

39 (51%)

National networking, eg contact with other schools involved in the same project within own local authority or in other authorities

17

6

3

1

27 (35%)

Using LT Scotland Assessment is for Learning website to share ideas electronically

12

6

0

0

18 (23%)

1 % of primary schools; 2 % of secondary schools

One respondent reported networking through conferences for Gaelic-medium teachers, while another mentioned that liaison and networking took place during the project activities, but that these were no longer happening. One Project 4 respondent indicated that no one else was working in the same area of curriculum development and they felt they were 'ploughing a lonely furrow'. A Project 5 primary headteacher reported that there were no plans to develop the work further and that the excellent partnership that had been developed with the secondary school had ended with the final report. The related secondary commented: 'unfortunately the primary secondary liaison in our project was temporary and has now lapsed'.

Clearly AifL developments have brought different groups together to work in new relationships, though not all within the same frame of reference. Authorities were working with each other and sharing, some of the HE representatives were working cross-institutionally, but relationships between local authorities and HEIs were still developing. Schools were more likely to work with other schools, with the support of the authorities. However, cross-sector and wider networking was less developed for schools.

5.6 Funding and sustainability

Funding

Substantial funding was made available to schools to take forward development and this was noted by various stakeholders as a major strength in the programme. School managers were asked to indicate the extent to which the funding was essential in taking forward developments and the main uses of that funding. Responses are given in Tables 5.32 and 5.33.

Table 5.32: Extent to which funding was important to schools in implementing the programme

Primary

Secondary

Special

5-14/16

Total

n

n

n

n

It was not important. We would have carried out the work anyway

3

2

0

0

5 (7%)

It helped. We were able to make progress with developments which would have taken us longer without the funding

18

12

2

1

33 (43%)

It was essential. Without the funding we would not have undertaken any of the developments

24

9

3

1

37 (48%)

The funding was clearly fundamental to taking developments forward. One headteacher reported: 'the funding was valuable in ensuring that the teacher responsible for the project had time out of class and resources. We really valued this' (Project 9).

Table 5.33: Use of funding by schools

Primary

Secondary

Special

5-14/16

Total

n

n

n

n

Obtaining supply teachers/cover

42

19

4

2

67 (87%)

Paying teachers to work in their own time

21

7

2

1

31 (40%)

Sending teachers on staff development activities

23

8

2

2

35 (46%)

Purchase of equipment, eg computers, video-cameras, software

26

15

3

2

46 (60%)

Purchase of other materials, eg books, folders, storage

24

9

4

0

37 (48%)

Travel costs to national events

20

11

3

1

35 (46%)

Other reported uses of funding included photocopying, inviting guest speakers to school, paying for 'lets' for meetings with parents, and lunches!

Thirty-six respondents indicated that they had received funding to continue developments during 2003 to 2004. The majority (20) reported ongoing AifL funding from SEED/ LT Scotland; 10 received funding from their local authority; and the remainder referred to funding related to other initiatives such as Building Bridges and CPD developments or the school's own budget.

Sustainability

Local authority co-ordinators and school managers were asked about the sustainability of developments.

Fourteen of the LA respondents referred to the fact that practices and standards advocated by AifL were, or would be, embedded within learning, teaching and assessment policies and plans and would therefore be monitored through the normal quality assurance processes. Some described it being 'embedded' or 'mainstreamed' and, as such, authority funding would be allocated to ongoing developments.

However, a number of LA respondents expressed different views. Ten respondents (including 4 who referred to 'embedding' developments) suggested that developments, including in-service and staff development, required dedicated staff. Funding had made this possible, but without funding it would be difficult to maintain dedicated staff. The developments and in-service would continue, but at reduced levels.

While formative assessment might be 'mainstreamed', it was considered that other aspects of developments such as PLPs, ICT aspects of PLPs and reporting would continue to need additional funding.

Five respondents suggested it would be difficult to sustain developments at all without additional funding and two indicated that as yet they were unsure about future plans.

Forty-seven headteacher replies were given:

  • The majority (28) indicated that the main resources would be from existing school budgets, particularly staff-development and CPD budgets. One said this could be achieved through 'good use' of school budgets, while another commented that they 'would find it somehow'. At least half a dozen suggested that the development would, of necessity, be limited and less than that supported via the AifL Programme
  • 7 suggested that there would be local authority funding available, particularly for the development of PLPs, as they had become/were becoming an authority priority
  • Another 7 suggested that without funding there would be no developments, again with specific reference to PLPs
  • 5 respondents suggested that funding for other initiatives would also be relevant to taking forward AifL developments: for example, Building Bridges and FLaT funding.
5.7 Developing a unified and coherent system

Awareness of wider programme

An important aspect of moving towards a coherent system of assessment is the drawing together of the contributions of the separate projects. For the system to be recognised people need to be aware of and understand its constituent parts.

Headteachers and teachers were asked how much they knew about projects other than their own. In Table 5.34 the responses relating to each respondent's original project have been excluded. All respondents know 'a lot' about their original project, and this question was designed to show awareness of other projects in the programme. The exceptions are Projects 6 and 7, for which all respondents have been included. These data are illustrated in Charts 5.13 and 5.14.

After formative assessment, headteachers/ SMT members were most likely to know about PLPs and the new National Assessments, though one-fifth and one quarter, respectively, reported knowing nothing about these two areas. Respondents reported knowing little or nothing about Projects 3, 4, 5 and 8, re-emphasising previous findings that wider awareness of the programme was slow to develop.

Teachers were somewhat reluctant to agree that they knew a lot about the work of other projects, though the much lower percentages indicating that they knew nothing about formative assessment and developing PLPs compared to the previous survey (see page 67) are in keeping with the ongoing developments which were reported.

Table 5:34: Awareness of other projects

Know a lot

Know a little

Know nothing

HT

Teach

HT

Teach

HT

Teach

%

%

%

%

%

%

1. Support for Professional Practice in Formative Assessment

33

19

47

43

20

27

2. Personal Learning Plans

27

9

53

62

20

20

3. Supporting the Management of Personal Learning Plans

11

6

41

29

48

56

4. Gathering and Interpreting Assessment Evidence

2

7

54

35

44

46

5. Local Moderation: sharing the standard

8

8

30

11

62

69

6. new National Assessments ( NNA)

28

12

47

43

25

35

7. Assessment of Achievement Programme ( AAP)

11

9

54

43

35

35

8. ICT Support for Assessment

4

3

43

27

53

57

9. Reporting to Parents and Others

19

6

46

35

35

45

10. Meeting the needs of pupils with Additional Support Needs

8

6

45

19

47

61

Chart 5.13: Awareness of other projects (Headteachers)

Chart 5.13: Awareness of other projects (Headteachers)

Chart 5.14: Awareness of other projects (Teachers)

Chart 5.14: Awareness of other projects (Teachers)

Differences in responses between those who had been involved from the beginning and those who had become involved during 2003-04 were investigated using a chi-square test. The only significant difference was in relation to Project 9, where a higher than expected proportion of the new participants said they knew nothing, compared to the others. Other differences, although not statistically significant, may be worth noting. Of those from schools which were not originally involved in Project 1, a higher percentage of teachers who had recently become involved indicated that they knew a lot about Project 1 compared to those involved from the beginning (36% compared with 20%). This would reflect schools' keenness to develop formative assessment. The responses of the two groups of teachers were almost exactly the same with respect to Projects 2 and 4; in relation to Projects 3, 5, and 8 a higher percentage of those involved from the beginning noted that they knew a little compared to the percentage of recent teachers who noted that they knew nothing.

Understanding how projects link together

Headteachers were asked about their understanding of how the AifL projects linked together to contribute to the development of a unified and coherent system of assessment, and whether participating teachers had a clear understanding of this:

  • 4% strongly agreed they had a good understanding of how the projects linked together
  • 66% agreed
  • 26% disagreed
  • 4% strongly disagreed.

With respect to their teachers:

  • 5% strongly agreed that their teachers understood how the projects linked together
  • 53% agreed
  • 28% disagreed
  • 10% strongly disagreed.

This suggests that almost one-third of headteachers were not clear on the 'bigger picture' of how the projects relate to each other and have potential for developing into a coherent system, and that almost two-fifths thought their teachers did not understand this.

The teachers were slightly less positive than the headteachers:

  • 5% strongly agreed
  • 44% agreed
  • 37% disagreed
  • 6% strongly disagreed.

Of those who had been involved from the beginning, 54% agreed they had a clear understanding of how the projects linked together, while 42% disagreed. Teachers who had become involved more recently were slightly more negative in their response: 37% agreed they had a clear understanding, while 48% disagreed.

It was considered by some of the assessment co-ordinators that too many projects were begun simultaneously and had not been drawn together to form a unified system; it was suggested that 'explanations in the beginning were complex in the extreme - too complex'; another described it as 'scary'. One stated: 'it is like a 3 year programme for 10 projects and at the end we have one and a half - formative assessment and a bit of PLPs'.

Progress towards aims of the AifL Programme

The AifL Programme has two broad aims of providing a streamlined and coherent system of assessment and ensuring that parents, teachers and other professionals have the feedback they need on pupils' learning and development needs. All respondents were asked to indicate the extent to which they thought that progress had been made towards the following three aspects, ( ADP Action Plan, SEED, 2002) which contribute to these aims:

  • The development of a unified system of recording and reporting (the PLP)
  • The bringing together of current arrangements for assessment, including the AAP, National Tests and the 5-14 survey of attainment
  • The provision of extensive staff development and support (to develop understanding of assessment for learning, improve assessment practice in schools and to improve recording and reporting of achievement).

They were invited to add further comments.

Each of these aspects will be presented giving an overview of the responses of each stakeholder group. Both numbers and percentages are presented for comparison purposes, though some groups ( HE representatives and LA representatives) are small and would not normally be presented as percentages. The ASG co-ordinators' responses have been included here to form part of the wider picture.

A cautionary note is given here. The respondents to school surveys represent about half of the original schools involved in the programme. These schools, for the most part, have shown a commitment to ongoing development of their original project involvement, with more than half of them also taking on further developments connected to other aspects of the AifL Programme. Some of the schools who did not respond to the survey indicated that this was because there had been no further AifL developments during 2003 to 2004; therefore, we might assume they would have responded more negatively regarding overall progress. The ASG respondents also represent less than half the total ASGs identified as existing in 2003 to 2004; they were targeted because they appeared to be working collaboratively and making progress. It is difficult to generalise to the wider population, but with a bigger sample we might anticipate less positive responses.

The development of a unified system of recording and reporting (the PLP)

The responses of the various groups are presented in Table 5.35.

Table 5.35: Progress towards a unified system of recording and reporting

Good progress

Some progress

Little progress

No progress

No response

Local authority co-ordinators

4 (13%)

5 (17%)

20 (67%)

1 (3%)

0

HE representatives

0

5 (63%)

3 (37%)

0

0

Headteachers

10 (13%)

23 (30%)

28 (36%)

8 (10%)

8 (10%)

Teachers

20 (15%)

51 (39%)

26 (20%)

9 (7%)

24 (19%)

ASG co-ordinators

2 (4.5%)

17 (38%)

15 (33%)

2 (20%)

2 (4.5%)

Overall, only a small proportion reported that they thought good progress had been made in this area; local authority assessment co-ordinators were most likely to think that little progress had been made.

Some local authorities had taken forward the development of PLPs as identified in the role of the authority personnel (p75) and in the focus of ASGs (see section 5.8). About half had PLPs as part of the improvement planning process. However, developments in this area appeared to be slow. One LA respondent commented that this seemed to be the least developed aspect of the programme. In a question seeking views on limitations of the programme, 12 of the assessment co-ordinators expressed concerns about PLPs. For some the concern was a lack of sharing of good practice and the outcomes of the original PLP projects and their view was that national guidance would be beneficial. There was concern that there was pressure to develop PLPs without having a clear picture of what they were to achieve or sufficient understanding of formative assessment to support their development. Others noted that there was potential for conflict between locally developed approaches to PLPs and any national guidance which may emerge.

The issue of the purpose of PLPs continued to be questioned, with one LA co-ordinator asking: 'is a PLP a recording and reporting tool, or is it a means of encouraging learning and enabling pupils and their parents to understand their learning processes, strengths and needs? Can it serve both functions?'

One HE respondent expressed the same concern: 'This for me is where some serious tensions exist. The developing of a unified system of recording and reporting is not what a PLP is. The PLP is a plan that is about learning and is personal. The very notion that it is about recording and reporting is contrary to the spirit of assessment is for learning and moves the whole process back to assessment is for measurement. This also causes anxiety amongst teachers who see it, in this form, as another workload issue'.

On the same point, one headteacher commented that PLPs were for pupil use and that reports to parents were a different issue. This was strongly supported in a response from an ASG co-ordinator, who indicated they had reports which were 'distinctly separate from PLPs. These are two separate entities and … they should stay that way. The pupil report is by the teacher of the pupil; the PLP is by the pupil for his/her personal development and is therefore biased. Both can work side by side and complement/supplement each other, for the pupil's benefit'.

Another headteacher commented that parents wanted a report as well as the PLP. One added a further comment: 'the most important aspect is not recording results/levels but in using AifL strategies to assess learners' needs DURING lessons and adapt teaching strategies to meet pupils' needs and next steps'.

The underlying conflict of the purpose of PLPs and the delay in the production of a national framework (which was circulated for consultation after this evaluation survey had been completed) undoubtedly contributed to the views that there had been limited progress in this area.

Bringing together current arrangements for assessment

The responses of the various groups are reported in Table 5.36. These issues were the focus of the consultation on Assessment, Testing and Reporting: 3-14 which was launched in September 2003, but whose findings were not published until November 2004. The evaluation surveys took place before publication of the consultation outcomes, and this is reflected in the responses and comments made.

Table 5.36: Progress towards bringing together current arrangements for assessment including AAP, National Tests and the 5-14 survey of attainment

Good progress

Some progress

Little progress

No progress

No response

Local authority co-ordinators

2 (7%)

18 (60%)

7 (23%)

3 (10%)

0

HE representatives

3 (63%)

5 (37%)

0

0

0

Headteachers

18 (23%)

28 (36%)

20 (26%)

4 (6%)

7 (9%)

Teachers

17 (13%)

48 (37%)

23 (18%)

7 (5%)

35 (27%)

ASG co-ordinators

5 (11%)

18 (40%)

14 (31%)

6 (13.5%)

2 (4.5%)

Overall, respondents were more positive regarding progress in this area than in progress towards a unified recording and reporting system.

However, 7 LA respondents noted that the Consultation on Assessment, Testing and Reporting: 3-14, part way through the programme, and waiting for the outcomes of the consultation, had made it difficult to determine what progress should be made; this was noted as a particular limitation of the programme.

The HE representatives also expressed concern over the timing of the AifL Programme with respect to related consultations and reviews (viz: Consultation on Assessment, Testing and Reporting: 3-14 and the Curriculum Review) and how they would link together. Furthermore, it was considered that there had been a 'rushed approach' to implementing National Assessments, which had caused anxiety in schools. Several local authority co-ordinators also referred to the National Assessments and 'the heartache they had caused'.

However, an HE representative suggested that closer collaboration between the AAP team and the National Assessments team at SQA had laid the groundwork for developing a more coherent approach (pending the Ministerial announcement on assessment and testing).

In this context, one of the local authority co-ordinators raised the point that there were major issues to be resolved between a system of reporting in the 5-14 curriculum which is heavily norm-referenced and an approach based on formative assessment and PLPs. This was an issue raised by local authority co-ordinators in identifying challenges to local authorities and schools, and also by HE representatives and teachers. Ongoing work is required to clarify the relationship between formative and summative assessment and how the whole teaching, learning and assessment process can build into a coherent system including enhanced learning and nationally-delivered assessment.

The provision of extensive staff development and support

The responses of the various groups are reported in Table 5.37.

Table 5.37: Progress towards the provision of extensive staff development and support

Good progress

Some progress

Little progress

No progress

No response

Local authority co-ordinators

20 (67%)

10 (33%)

0

0

0

HE representatives

2 (25%)

6 (75%)

0

0

0

Headteachers

17 (22%)

44 (57%)

8 (10%)

2 (3%)

6 (8%)

Teachers

39 (30%)

52 (40%)

21 (16%)

2 (2%)

16 (12%)

ASG co-ordinators

13 (30%)

20 (44%)

8 (18%)

1 (2%)

3 (6%)

The high level of positive responses from local authority assessment co-ordinators reflects the amount of staff development undertaken by authorities and reported in section 5.3 of this report.

Few additional comments were made in relation to staff development. One HE respondent suggested that a new understanding of staff development was emerging that focused on practitioner-led developments and action research. More generally, the view was expressed that, while those who had been involved had greatly benefited, there was still much to be done to reach the wider teaching community.

Strengths of programme

Local authority assessment co-ordinators and HE representatives were asked to indicate what they saw as particular strengths of the programme.

For LA co-ordinators, the allocation of sustained, sufficient funding to both schools and authorities was the most frequently mentioned strength of the programme. It was appreciated that it was 'ring-fenced' but, beyond that, schools could use it in ways which they perceived as relevant to their needs. One respondent appreciated the 'creative use of funding via ASG projects'. HE respondents agreed that supplying resources to the pilot schools was a major strength.

Both LA and HE respondents agreed that the leadership given by SEED was important. The HE representatives focused on the inclusive approach taken by SEED, involving different groups, encouraging dialogue between the groups, consulting with and developing communication with and between groups. The LA respondents noted the support from the Minister for Education and Young People, the appointment of dedicated staff, the high-level promotion of the programme and the establishing of good communication between SEED and authorities. This was partly achieved by the establishing of strong networks (assessment co-ordinators' meetings) and by the encouragement to work collaboratively at all levels.

Both groups also mentioned the 'bottom-up' approach which valued the practitioner, practitioner development and practitioner research as a major strength of the programme. The process of 'growing policy', that is allowing it to emerge and develop rather than imposing it from the top down, was appreciated.

Concern

A concern was expressed regarding the management of the programme. Ten LA respondents referred to the issue of the mismatch between academic and financial years and the conflict of priorities caused by initiatives being introduced outwith the development planning cycle. This was a point reinforced by interviewees who reported that money was received in January and had to be spent by the end of March.

5.8 Associated Schools Groups ( ASGs)

Local authority perspective

A key feature of developments during 2003-2004 was to increase the number of schools involved in the programme, with a particular emphasis on members of Associated Schools Groups ( ASGs) working together.

Differences in size and geography of local authorities mean, of course, that there is huge variation in the number of school clusters within authorities, from 3 in Clackmannanshire to 23 in Edinburgh and 29 in Glasgow and Highland. Three authorities reported including all their clusters in ASG developments. Fifteen authorities reported involving 3 or 4 clusters as ASGs, representing between a half and a third of secondaries and their associated primaries in these authorities. The respondents identified a total of 427 school clusters, 128 (30%) of which had become involved in AifL as Associated Schools Groups.

Alternative arrangements to geographical clusters (ie secondary and related primaries) were reported. Groups of primaries were working together but not in collaboration with the related secondary; in rural authorities it was not possible to bring all cluster primaries to work together because of distance and so sub-groups were formed; some worked together on the basis of size, eg fewer than 5 teachers; some ASGs worked on the basis of common interest, eg maths or language developments; secondary subject-based groups were formed, with all secondaries in the authority collaborating.

Data collected from assessment co-ordinators as a separate exercise indicated that 78% of ASG activity focused on formative assessment; 6% were focusing on formative assessment and PLPs together; 5% were developing PLPs; and 10% were considering moderation issues/'sharing the standard', in particular from P7 to S1.

The LA respondents were asked to give an indication of the progress they thought had been made by the ASGs. This is reported in Table 5.38.

Table 5.38: Progress made by Associated Schools Groups ( LA co-ordinators)

Good progress

Some progress

Little progress

No progress

Developing Assessment Action Plans

12

13

4

1

Achieving objectives of Action Plans

12

14

3

1

Working collaboratively as ASGs

16

11

3

0

Raising awareness of assessment issues in schools

20

8

2

0

Improving assessment practices in schools

12

15

2

1

Improving liaison between sectors

10

17

2

0

The negative ratings came from a small group of authorities, who explained their responses. In one case it was indicated that an ASG had found it difficult to plan as a group and it was felt that they would have liked the authority to develop the plan for them. In another authority it was also mentioned that those involved tended to look continuously for guidance rather than taking ownership of the project. A third authority co-ordinator indicated that the ASGs would have benefited from more central support which could not be given at the time. Other respondents indicated that progress had varied, with some ASGs 'floundering' or 'not getting off the ground', some 'raising awareness' and others 'embracing the project'. The authority co-ordinator who indicated that no progress had been made in improving assessment practices in schools indicated that the work developed in the ASG schools would be shared within the ASG clusters in 2004-05 and then disseminated to other clusters in the authority.

It is clear, however, that the majority of authority staff with AifL responsibility are confident that progress is being made. Some respondents pointed out that ASG developments did not fit in with cycle of improvement planning and, while schools were keen to be involved, it was difficult to organise, develop and implement plans within the prescribed timescales. The developments in the ASGs would continue into 2004-2005. As with those who recorded negative responses, respondents who identified that progress was being made also noted that some ASGs had made better progress than others.

Responses from ASG co-ordinators

The sample

A list of 2003-2004 ASGs was supplied by SEED - there were 111 ASGs on the list. Local authority co-ordinators were contacted to supply information about key contacts for each ASG. Information received from assessment co-ordinators indicated that a number of ASGs 'had not got off the ground' during 2003-2004 and that in some cases there had been no collaborative working between schools in the ASGs. It was decided to focus on ASGs where there had been collaborative working between schools and across sectors and where something had 'got off the ground' during 2003-2004. One assessment co-ordinator did not supply any information regarding the ASGs in their authority and therefore they were not included in the survey.

Questionnaires were sent to 81 ASG contacts (73% of ASGs on list). Forty-five (56%) responses were returned, with a further 3 contacts indicating that they could not complete the questionnaire as they felt they had not made sufficient progress to answer the questions. Responses represented 40% of the ASGs on the original list. Twenty local authorities were represented in the returns.

The contacts were asked to complete the questionnaire on behalf of the whole ASG and, if possible, to consult with colleagues from other schools on responses.

Organisation of ASGs

The size of the school clusters from which the ASGs were formed varied from 3 schools to 25 schools. The respondents were asked how many schools from the cluster group formed the ASG for AifL developments and how many of those nominated were active. (Eight of the returns had incomplete data and so are not included in this summary, which therefore reports on responses from 37 ASGs.)

  • 19 reported that all schools were involved and all actively participated; 15 of those were from clusters of 6 or fewer schools; the others had 7, 8, 9 and 12 schools.
  • 8 reported three-quarters or more of the cluster schools actively participating, with 6 reporting just one school not involved.
  • 8 reported between a half and three-quarters of the cluster schools active; all of these included 7 or more schools in the cluster.
  • 3 reported less than half of the schools active - in one case this was one school out of 3 (therefore, in effect, not a cluster).
  • in 8 cases it was reported that some schools began working with the ASG but did not remain active.
  • in 6 cases a smaller number started out and more became involved over time.

Overall a total of 232 schools were reported as participating across the 37 ASGs for which data was provided.

Although there were some larger clusters that engaged all schools and retained participation (7, 8, 9, 12 representing all schools in the cluster and 22 out of 25 schools in one case), it would appear that groupings of 6 or less were more likely to gain and retain the involvement of all.

Six reported working in ASGs formed on a non-geographical cluster basis. These were focused on subject developments across the authority - for example secondary schools working together on home economics, maths, aspects of English and modern languages.

Twelve (26%) reported that there had not been any cross-sector working; the majority 32 (72%) reported cross-sector collaboration (one missing response). In some cases, cross-sector working was not intended; for example, a group of secondary schools working on one subject area; or a group of primary schools working together on 'sharing the standard' on maths levels. However, it was reported in at least one case that the secondary school could not participate as originally intended because of other commitments and therefore the primary schools had worked on their own. The number of times that the groups met to discuss AifL developments varied widely from once to 11 times. However, 60% of the respondents indicated that they met between 3 and 6 times.

Focus of ASG developments

The respondents were asked to give a brief outline of the main focus of their ASG developments:

  • The majority (34) reported focusing on formative assessment strategies, either as the sole focus (26) or in conjunction with other aspects of AifL developments: 4 with PLPs, 2 with moderation/evidence gathering and 2 with a particular focus on bridging the primary to secondary curriculum
  • For 7 ASGs the prime focus had been moderation/sharing the standard, with 2 specifically mentioning the primary to secondary transition. One authority-led cluster focused on developing maths criteria for levels A to C for sharing with parents
  • Three ASGs (in the same authority) reported focusing on developing 'a model of effective teaching and learning from P6 to S2' across subject areas. Although not clear initially, later in the questionnaires it was explained that formative assessment is a core element of this
  • One ASG reported developing resources for the new National Assessments (5-14) in Writer's Craft.

Table 5.39 reports on the year groups and aspects of the curriculum that were targeted across the sample.

Table 5.39: Year groups and curriculum areas targeted in the ASG developments

Year group 1

No of ASGs

Curriculum areas 2

Pre 5

3 (7%)

Combination of different areas of curriculum

P1

22 (49%)

All (4); aspects of English (3); maths/numeracy (2); social subjects (1); combinations of curricular areas (11).

P2

23 (51%)

All (4); aspects of English (3) maths/numeracy (3); social subjects (1); science (1); combinations (10).

P3

24 (53%)

All (4); aspects of English (3); maths/numeracy (2); social subjects (1); combinations and other topics (12).

P4

24 (53%)

All (4); aspects of English (3); maths/numeracy (3); social subjects (1); combinations and other topics (12).

P5

27 (60%)

All (4); aspects of English (3); maths/numeracy (3); social subjects (1); combinations and other topics (15).

P6

37 (82%)

All (4); aspects of English (5); maths/numeracy (5); social subjects (1); combinations and other topics (17).

P7

40 (89%)

All (5); aspects of English (6); maths/numeracy (4); social subjects (2); modern languages (2); science (2); combinations and other topics (18).

S1

30 (67%)

All (2); aspects of English (5); maths/numeracy (3); social subjects (2); modern languages (2); science (2); combinations and other topics (13).

S2

26 (58%)

All (2); aspects of English (6); maths/numeracy (3); social subjects (1); modern languages (2); science (2); combinations and other topics (9).

S3

10 (22%)

All (2); aspects of English (3); science (1); combinations and other topics (4)

S4

7 (16%)

All (2); aspects of English (1); combinations and other (2).

S5/S6

6 (13%)

All (1); aspects of English (1); social subjects (1); combinations and other (2).

Note 1: The number of year groups targeted in the work of the ASGs varied from 1 to 14, with 22 ASGs (50%) targeting between 6 and 9 year groups; 29 worked across P7 to S1 year groups.
Note 2: Some respondents did not detail the areas of the curriculum being targeted. Therefore the curriculum area numbers do not add up to the number of ASGs.

The main target groups within the ASGs were clearly P6 and P7 pupils, with very low levels of pre-5 and upper secondary participation. Developments were most likely to be occurring across the curriculum - if not in its entirety, certainly in more than one area (noted as 'combination of curriculum areas' in the table).

Key objectives for ASG developments

The statements of objectives varied from high level objectives such as 'raising pupil attainment', 'improving motivation of pupils', 'raising staff awareness of formative assessment strategies' and 'developing school/cluster policy on teaching and learning' to more specific objectives such as 'introducing learning intentions' and 'introducing traffic lighting and two stars and a wish'. Examples of objectives include:

Example 1:

Obj1: raising attainment through formative assessment

Obj2: marking less to achieve more

Obj3: sharing learning intentions "targets and goals"

Example 2:

Obj1: adoption of core formative assessment strategies

Obj2: develop as school policy

Obj3: share strategies with cluster schools

Example 3:

Obj1: using success criteria/learning intentions

Obj2: use of traffic lighting

Obj3: thinking time

Example 4:

Obj1: to produce resources and develop pupils' experiences in self- and peer-assessment across the cluster

Obj2: to increase class teacher awareness of assessment strategies

Obj3: to increase opportunities for cluster colleagues to work together

Example 5:

Obj1: increase range of effective teaching/learning methodologies in use in classrooms

Obj2: improve attainment

Obj3: improve motivation of pupils and active participation in learning

Example 6:

Obj1: to train pupils in self-assessment

Obj2: pupils can accurately identify their strengths and needs.

The extent of achievement of these objectives was broadly split between achieved and partially achieved, with very few indicating that any of their objectives had not yet been achieved.

Formative assessment

The respondents were asked to indicate which formative assessment strategies were being incorporated into developments. They were asked to identify strategies under generic headings and give examples of specific practice in the named areas. The responses have been categorised into the projects which reported the main focus as formative assessment and those which did not (see Table 5.40).

Table 5.40: Aspects of formative assessment being introduced through ASG activities

Aspects of formative practice

FA main focus (34)

FA not main focus (11)

Sharing learning outcomes with pupils

32

7

Sharing/agreeing assessment criteria with pupils

29

3

Discussing/agreeing criteria with other teachers

22

10

Questioning

31

3

Feedback

28

3

Self-assessment

28

4

Peer-assessment

23

3

Discussing and agreeing criteria with other teachers was recorded more in relation to 'sharing the standard' and less by those focusing on formative assessment strategies, though one might expect that teachers working collaboratively would also be discussing such things with colleagues. Peer-assessment appears to have been adopted less widely than the other strategies.

The number of strategies being adopted across each ASG varied. Some respondents indicated that each school was trying out some aspects so that across the cluster all strategies would be attempted. Others were introducing only one or two aspects before expanding to include more. The number of strategies reported as being used is given in Table 5.41.

Table 5.41: Number of formative assessment strategies being developed within ASGs

No of FA strategiesFA main focus (34)FA not main focus (11)

1

0

1

2

0

1

3

0

6

4

4

0

5

8

1

6

5

2

7

16

0

Examples of practice

Sharing learning outcomes with pupils: The majority of respondents indicated that sharing learning outcomes/intentions was a feature of developments, though only about two-thirds gave examples. The most frequently named strategies for sharing learning outcomes were ' WALT' (We are learning to …) and ' WILF' (What I'm looking for …), though respondents focused more on the process of sharing: for example, discussion at the beginning of the lesson, sheets with learning outcomes, use of whiteboard or wall displays with targets outlined. In relation to sharing learning outcomes, 10 respondents either referred to WALT and WILF together or only to developing success criteria. Only 4 respondents seemed to make a clear distinction, noting WALT as part of sharing learning outcomes and referring to WILF in relation to sharing assessment criteria with pupils.

Sharing/agreeing assessment criteria with pupils: Fewer respondents indicated that this was part of practice compared to sharing learning outcomes/intentions and only a half gave examples. As noted above, both WALT and WILF were named, though only a few, as indicated, distinguished WILF in relation to assessment criteria. The process of sharing was referred to, eg class discussion, discussion with other pupils ('buddies'), use of poster displays. One respondent noted that for secondary pupils the criteria might be class/teacher generated or the 'official' SQA criteria for S3 to S6. Several indicated that discussing criteria with pupils was being piloted or was still to be developed.

Practitioners may be used to thinking of the two aspects together and the question therefore might have made an unnecessary distinction; the numbers in the sample are small and therefore conclusions cannot be drawn. However, further investigation might be useful into the extent to which there is lack of clarity between learning intentions and the criteria by which judgements about the achievement of learning at the required level are made.

Discussing/agreeing criteria with other teachers: This was of slightly less importance to ASGs where formative assessment had been the main focus, but for the others, in particular moderation (sharing the standard), this was one of the prime purposes of the development. Again about half the respondents gave examples. For both 'formative assessment' and 'non-formative assessment' ASGs, discussion of criteria happened most often in cluster meetings, with several (12) mentioning that working groups had been established for that purpose. Discussions had also taken place within schools between teachers teaching the same stages or in secondary schools at departmental meetings (6 mentions). Two ASG representatives referred to discussion particularly in relation to 'National Tests, new National Assessments and their relationship to the 5-14 guidelines'. Other approaches mentioned included preparing a folder of exemplars to be shared among teachers, discussions over examples of pupil work to agree levels and shadowing across sectors.

Questioning: Just under half the respondents gave examples of strategies being adopted in relation to questioning. These were:

  • Increased think/answer/wait time (11)
  • Increased use of open questions (5)
  • 'No hands' (4)
  • Discussion/use of 'fat' and 'thin' questions (3)
  • Development of 'key' or 'clever thinking' questions related to subject (2)
  • Peer/group discussion before answering
  • Linking questions to learning intentions
  • Share-it boards
  • Traffic lights.

Feedback: A small number of respondents (4) indicated that this was an aspect still being developed. Less than half the respondents gave examples of approaches being developed in relation to feedback. These were:

  • 2 stars and a wish (6)
  • Comment only marking (4)
  • Comments specifically linked to targets/criteria (2)
  • Next steps feedback
  • Using constructive feedback (rather than good or well done)
  • Review at end of lesson
  • Notes home
  • Formative use of summative tests
  • Thumbs up
  • Individual discussion
  • Comment bank for homework.

Self-assessment: Four respondents indicated that self-assessment was just being tried out/piloted in some of the cluster schools and so could give no examples. Less than half commented on the use of self-assessment. The following approaches were given:

  • Traffic lights (9)
  • Thumbs up (2)
  • Making use of learning outcomes and criteria to check their own work
  • Checklist to tick if they have achieved target and show work to support it
  • Learning logs
  • Spots on jotters
  • 2 stars and a wish
  • Commenting on their own work before giving it to the teacher
  • Prompts on posters in class to remind pupils
  • Teacher/pupil interview for PLPs.

Peer-assessment: It was noted that some schools were just starting to develop peer-assessment. Less than a third of respondents (13) gave examples of approaches to peer-assessment. These were:

  • Marking in pairs or buddies (using criteria) (6)
  • Exchange jotters for comments (2)
  • Traffic lights (2)
  • Discussion in pairs/groups (2)
  • Group feedback
  • Testing each other's knowledge with prompt cards
  • Role play.

Influence of formative assessment on use of National Assessments

Ten respondents indicated that the use of formative assessment had influenced decisions about when pupils completed National Assessments, though 5 indicated that they could not comment for all schools in the cluster; this issue had not been discussed at ASG meetings.

Responses reflected teacher and pupil development. In-service and ASG meetings with ongoing discussion between teachers had increased teacher awareness and they were now 'more able to make a judgement about when pupils were ready to move on to the next level', while 'using writing criteria has highlighted to staff the need for specific criteria to be met' (both comments from respondents based in primary sector). One respondent suggested that there must now be evidence in written work and in assessment logs before National Assessment is allowed. It was noted that 'self-assessment has enabled teachers to highlight areas within which pupils are not confident' (secondary respondent).

Some indicated that pupils were now more aware and able to make judgements about their own readiness for example, 'pupils now more involved in planning their own targets and projecting when they will sit tests' and 'pupils more aware of what is expected in reaching/covering the criteria in responses' (both comments from respondents based in secondary sector).

Impact of ASG developments

Staff development

It was reported that members of the ASGs had taken part in a variety of assessment-related staff development activities. The responses are reported in Table 5.42. Over half of the respondents reported that members had participated in 4 or more of the undernoted activities.

Table 5.42: Staff development undertaken by ASG teachers during 2003-2004

No. of ASGs

Peer staff development (ie events which allowed sharing between teachers in own school)

37 (82%)

Joint events organised for ASG members with other practitioners, eg teachers involved in pilot projects, local authority staff, LT Scotland development officers

40 (89%)

Joint events organised for ASG members with other guest speakers

19 (42%)

Local authority in-service, seminars, courses etc. with high profile guest speakers, eg Dylan Wiliam, Shirley Clarke, Ian Smith

39 (67%)

Other local authority events

20 (44%)

Nationally organised AifL activities

19 (22%)

Peer- and ASG-focused development were the most frequent types of staff development, though participation in local authority events, especially with high profile speakers, was important. Members were less likely to take part in national events.

Other staff development mentioned was regular contact with and advice from the local authority assessment development officers.

Progress of ASG activities

Respondents were asked to give an indication of the degree of progress they felt had been made by the ASG 'to date', that is up to around October 2004. Responses are given in Table 5.43.

Table 5.43: Progress made by ASGs (number of responses)

Good progress

Some progressLittle progressNo progress

n

n

n

n

Developing Assessment Action Plan(s)

24

11

2

4

Achieving objectives of Action Plan(s)

16

14

6

4

Working collaboratively as an ASG

27

12

3

1

Raising awareness of assessment issues in the ASG schools

24

16

3

1

Improving assessment practices in schools

10

28

4

1

Improving liaison between sectors

19

17

4

2

Twenty-seven ASG contacts added comments on progress. The majority of these were positive, highlighting the strength of collaboration in some groups and that the above points were the focus of ongoing developments for 2004-05. Examples of comments are:

  • 'In the session 03-04 all schools dipped into the ideas suggested in the training days … but the work lacked structure. We now have a cluster action plan … and this has given a boost to the project and will making monitoring progress and sharing ideas much easier'
  • 'The ASG has worked really well and has completed its initial task. This is now to be delivered to another cluster group'
  • 'While progress in schools may be variable, a positive response to strategies is common. New approaches now firmly embedded in many teachers'/schools' classroom practice'
  • 'The high school departments and the primary schools involved now all see formative assessment strategies as being "everyday" methodology. Strong cross sector links have been formed which are ongoing'
  • 'Materials developed will be used with the whole cluster in 2004-05'.

Some indicated that work began in individual schools and collaboration and cross-sector development was only beginning to occur. Others indicated that although AifL is on the ASG plan, the work was still in initial stages. The one respondent who, it was noted at the beginning, was from the only school in the cluster, reported 'no progress' on all items.

Monitoring of developments

Twenty-six ASGs had arrangements for monitoring AifL developments. The responses varied in detail, with some outlining levels, eg in school ( HT responsibility), within the cluster (cluster co-ordinator responsibility) and within the authority across the clusters (assessment co-ordinator) - each gathering and passing on information on progress being made. Others referred to some aspects of the responsibility at the different levels.

The most frequently mentioned level of monitoring was at cluster level (19), with representatives of each school reporting at cluster meetings on progress. Some reported that this was informal; in one case the cluster had produced a monitoring and evaluation sheet to be used by everyone.

The next most frequently mentioned level of monitoring was related to local authority responsibility (11). This was through reporting to the assessment co-ordinators and other quality improvement officers/educational development officers. For some it was part of the normal improvement planning and review process rather than a special focus on AifL.

Some mentioned monitoring within the schools (7). Several mentioned that developments were monitored as part of the normal school monitoring and evaluation process; two mentioned classroom observation by senior management and peers and one mentioned teachers keeping diaries indicating both progress and setbacks.

Benefits

Forty-three of the ASG contacts identified benefits of working as an ASG to take forward AifL developments.

Two main themes emerged: sharing good practice, expertise and ideas and collaborative working, though the two are clearly very closely linked.

Sharing good practice (23): Many of the responses did not elaborate on the benefits of sharing good practice, but where they did it related to the exchange of ideas and learning from each other. The opportunity to discuss was important. Six mentioned the importance of sharing leading to the development of a uniform approach or cohesiveness in teaching and learning across the schools involved in the ASG.

Collaborative working (22): Nine of the respondents specifically mentioned the development of new and/or improved links between primary and secondary teachers in terms of 'creating links', 'removing barriers', 'real advances in primary-secondary liaison' and 'new professional relationships'. Five specifically mentioned the benefits of collaboration in terms of continuity of the teaching and learning experience of pupils: for example, 'working to reduce the discontinuity between primary/secondary in terms of practice, methodology and curriculum'. Other benefits of collaborative working included the mutual support it provided and the increased motivation of staff to start and stay on track.

Six mentions were made of benefits to pupils, including increased motivation and involvement in learning, pupil skill development, better understanding, greater awareness of how they learn and how to self-evaluate.

Other benefits given included:

  • Funding allowed time to discuss the programme (4)
  • Increased awareness of AifL
  • Focus on improved teaching and learning
  • Change in learning and teaching practice
  • Refreshing to go over 5-14 guidelines
  • Production of materials which would be used in other schools
  • Sharing of financial resources.

Challenges

Forty-three of the ASG contacts named challenges in working as an ASG. The majority of these were logistical challenges, though some related to issues of professional practice.

Logistical challenges: Unsurprisingly, time was the most frequently mentioned challenge (24). This was primarily related to finding time and agreeing times to meet, due to the complexity of timetables and difficulties in getting everyone out of their schools at the same time. A further 6 mentioned the problem of getting together as a group, though did not relate it to time. There was also a lack of time for school representatives to feed back within their own schools what they had discussed/learned at the cluster meetings and events. One person said that they would have liked more time to be able to visit other schools and observe what others were doing.

Distance was a hindering factor in rural communities, with implications for time and cost.

ASG organisation and planning was seen by some as challenging. One person reported difficulties in getting someone to agree to be the ASG co-ordinator, with a resulting lack of leadership and agreement on a joint focus. Three others also reported a lack of agreed focus; this was because of the difficulty in balancing whole-group and individual school priorities and also having no cluster agreement with individuals choosing their own priorities. One reported that their cluster grouping was too large and that this made it difficult ('and arguably less than appropriate') to develop a common approach.

Three respondents mentioned conflict with other priorities and 'top down' demands. With respect to three clusters in one authority, it was reported that the timing of the receipt of information and funding did not allow schools to include the work in their development plans.

Staff changes were also mentioned as a factor that led to AifL having lower priorities in some schools.

Professional practice : At the individual level a small number (3) mentioned that taking on board new research and its implications for practice in the classroom was challenging, but more (10) expressed concern about convincing other teachers: for example, maintaining the motivation and enthusiasm of teachers who had been introduced to new ideas, involving more staff and convincing them of the benefits.

Two primary-based contacts referred to the challenge of working with the secondary schools - one because it required seeing 'a different view', and another who said, 'Primaries are used to working together, the challenge has been to persuade the secondary to participate in a two-way relationship rather than be the "Big Brother" '.

Next steps

Forty-one ASG contacts responded to the question about 'next steps' in relation to the ASG for 2004-2005:

  • One respondent indicated that the ASG had been disbanded due to staff changes and that another cluster would be using the materials and assessing the work (moderation of maths)
  • 10 respondents indicated only that they would be continuing the work of the ASG, and for some plans had still to be discussed/developed
  • 23 respondents reported that the work would be extended in a variety of ways: for example, develop more materials; introduce more/new strategies; extend to other areas of the curriculum, other levels, other subjects, other departments (in secondary schools), further year groups; spread the work to other clusters or schools (2 specifically mentioned 'rolling out of PLPs' to other schools)
  • 12 respondents mentioned ongoing staff development, both for existing participants and also to involve more teachers
  • 2 mentioned the development of cluster policies for teaching, learning and assessment.

At the time of the questionnaire survey some groups were clearly more advanced in their planning than others, though all were planning ongoing developments.

Key issues from ASG survey

Local authority perspective

  • Across the country around 30% of school clusters had become involved in the AifL Programme as Associated Schools Groups, with some authorities including all clusters and others between a half and a third (depending on authority size). A small number of non-geographically based clusters were formed: for example, around a common interest such as maths or language developments, or on a subject basis at secondary level.
  • The majority of ASGs were focusing on developing formative assessment, with around 10% developing PLPs (some jointly with formative assessment) and 10% focusing on 'sharing the standard'.
  • Progress of the ASGs was variable, from 'floundering' to 'embracing the project'. Overall, authority co-ordinators thought progress was being made, despite the difficulties of not fitting in with the planning cycle and tight prescribed timescales.

ASG survey

  • The survey targeted ASGs where collaborative working across sectors had been reported by assessment co-ordinators. Responses represented 40% of the total list of ASGs and 20 authorities.
  • Cluster size varied from 3 to 25 schools; while some large clusters retained the involvement of all or most of the schools, the optimum size for gaining and retaining involvement was groupings of 6 or less.
  • Formative assessment was the main focus of developments, with a small number developing aspects of moderation. ASGs in which the main focus was not formative assessment were also developing some formative assessment strategies. The main stages involved were P6 and P7 and, to a lesser extent, S1.

Impact

  • Peer in-school staff development and joint cluster activities had been the main types of staff development, with over 80% of ASGs participating in such events.
  • Progress towards ASG objectives and the extent of collaboration varied, with over half reporting good progress in developments and around 10% reporting little or no progress. Collaborative working had made good progress in over two-thirds of the ASGs and improved cross-sector liaison in about half. Developments would continue in 2004 to 2005.
  • Sharing learning intentions/outcomes and questioning strategies were the most frequently reported formative assessment strategies being developed, with peer assessment being the least frequent.
  • For a small number, formative assessment practices were influencing decisions about when pupils completed National Assessments.
  • The main benefits of being involved in the ASG developments were sharing good practice, expertise and ideas, and collaborative working. These had led, in some cases, to cohesiveness in teaching and learning across the ASG schools and to continuity of the learning experience for pupils across the primary-secondary transition. Benefits to pupils included increased motivation and involvement in learning.
  • The main challenges were logistical issues in terms of time, organising and planning, and in some rural communities, distance. Changing professional practice was also a challenge, both in terms of maintaining the motivation and enthusiasm of teachers and involving more staff and convincing them of the benefits.
5.9 Summary of key points emerging from the second phase of the evaluation

Developments and progress during 2003 to 2004

  • Local authorities had appointed additional staff to take forward AifL developments: 14 had appointed development officers during 2003 to 2004, with a further 6 in 2004 to 2005. Roles varied, but the majority were responsible for taking forward operational aspects such as organising in-service and supporting schools, while assessment co-ordinators focused on strategic management. In some authorities these roles were shared. Ten had not appointed additional staff.
  • All authorities had delivered a programme of staff development on assessment-related issues. Participation in staff development across authorities varied from all schools in some authorities to one-fifth of primary and one-tenth of secondary schools in others. In about one-third of authorities more than 40% of primary teachers were involved. However, in the majority of authorities, fewer than 20% of secondary and special needs teachers have participated. The main focus was formative assessment, with PLPs mentioned by a few authorities.
  • The majority of authorities had engaged clusters of schools in Associated Schools Group developments, with 30% of identified school clusters becoming involved. Again the main focus was formative assessment.
  • The HE representatives had all engaged in assessment-related research and all identified ways in which AifL had been introduced into Initial Teacher Education and other teacher education provision.
  • In response to the school survey, 16% of the targeted schools reported that no further developments had taken place for a variety of reasons and so they could not complete the questionnaire.
  • Only 53% of the original pilot schools responded to the survey. In the majority expansion of the original work had occurred or was planned, mainly through involving more teachers, year groups and other areas of the curriculum. 74% of these schools reported undertaking developments of aspects of AifL other than the work of their original projects.
  • Schools involved originally in Project 1 (formative assessment) were more likely to be expanding their original work. For those involved originally in other projects, formative assessment was the most frequently mentioned new development, followed by PLPs.

The impact on policy and practice

  • Two-thirds of headteachers thought that the AifL Programme had substantially influenced their development planning, though primary headteachers thought this more than secondary headteachers.
  • With respect to National Priorities, headteachers were more likely to indicate that AifL had impact on Achievement and Attainment than on other priorities, although primary headteachers were more likely than secondary headteachers to see AifL complementing all aspects of the curriculum.
  • Involvement in AifL was encouraging the development or revision of school assessment policies.
  • Headteachers and teachers agreed that there was increased awareness of research related to teaching, learning and assessment, clearer understanding of assessment, changes to classroom practice, more varied approaches to assessment in use, improved feedback to pupils and more meaningful discussion with pupils about their learning. Generally, this agreement was stronger for those who had been involved in Project 1 from the beginning.
  • The greatest challenges to introducing change were time and engaging all staff. Time was at a premium both for preparing materials and engaging in dialogue with colleagues, due to competing priorities and also, in some cases, lack of supply cover. Agreement that there was resistance to new developments and difficulty in changing practice had increased since the first survey. Maintaining enthusiasm and engaging new staff was more challenging as the programme progressed.

The impact on pupil motivation and attainment

  • Some teachers thought it was still too early to comment on benefits to pupils but, for those who did, there was broad agreement that pupils had become more actively involved in their learning, were better equipped to assess their own learning, had shown increased confidence and self-esteem and were themselves positive about the changes. There was less confidence in the ability of pupils to set targets and engage in peer-assessment. Generally, there was stronger agreement on all of these issues from those involved originally in Project 1.
  • About a quarter of teachers indicated that they had evidence of improved pupil attainment, but the majority thought it was too early for this. This had changed little from the 2003 survey.

Staff development

  • As noted above, local authorities had been actively involved in promoting staff development in assessment during 2003 to 2004.
  • The main types of staff development in which people had participated were peer-delivered 'in-house' events, joint events with cluster schools and local authority-delivered courses. From the local authority perspective, peer development and events involving national experts had the greatest impact, followed by local authority-delivered events.
  • There was broad agreement across all participants that progress had been made in providing extensive staff development, though with the cautionary note that much work was still required to reach the wider teaching community.
  • The style of staff development encouraged in AifL and, in particular, by Project 1, took the form of 'action research' which involved recall days, discussing with colleagues, reflection and writing case study reports. There was evidence that not all headteachers and teachers had attended national events or contributed to case studies.

PLPs and meeting the needs of pupils, parents, teachers and others

  • The development of PLPs had been slow, with only 15 authorities indicating they were in the authority improvement plan. There were different opinions as to the purpose, and hence content, of PLPs, with local authority, HE and school representatives all contesting their suitability for recording and reporting purposes. Rather they were a tool to support learning and therefore related more closely to developments in formative assessment.
  • The specific focus on PLPs in meeting information needs was not addressed in the second survey, due to the lack of widespread PLP development. However, a general question was asked on how all aspects of AifL met these needs. The greatest contribution was in meeting the information needs of pupils and the class teacher, with teachers, in particular, agreeing that developments had improved these aspects. About 40% of headteachers and about a third of teachers thought that substantial progress was being made with regard to meeting the information needs of parents. Some thought it was still too early to make judgements on how AifL contributed to meeting information needs.
  • Benefits to parents in terms of increased contact with the school and teacher, and better understanding of how they can help their child, were project-specific, with participants in Projects 2, 3 and 9 recognising these benefits and others indicating that this was not a focus of the development. Some indicated that relationships had always been good and AifL had had no impact; others recognised that improvements had occurred but not necessarily because of AifL.

Convergence of assessment arrangements

  • An important aspect of moving towards a coherent system of assessment is the drawing together of the contributions of the separate projects. At the beginning of session 2004-2005, both headteachers and teachers reported a lack of awareness of projects other than the one in which they had been involved. They knew more about formative assessment and PLPs but reported knowing little or nothing about Projects 3, 4, 5, 8 and 10; 70% of headteachers and 49% of teachers indicated that they had a clear understanding of how the projects linked together to form a coherent system.
  • The issue of bringing together classroom assessment and more formal means of assessment such as National Assessments, AAP and 5-14 testing was less clear. At the time of the second phase of the evaluation there was a lack of clarity regarding the latter three, as the outcomes of the 'Consultation on Assessment and Testing: 3-14' had not been published. There was evidence of perceived conflict between formative assessment and the summative approaches of National Assessments and other tests and examinations.
  • Local authority co-ordinators expressed concern about difficulties schools had experienced in the use of the online National Assessment Bank. Almost all headteachers thought that using the online National Assessment Bank meant additional workload and costs to schools, with only a quarter agreeing that it was easier to manage than the previous approach.

AifL Programme issues

  • Collaboration and community of practice: AifL developments had brought different groups together to work in new relationships, though not all within the same frame of reference. Both LA co-ordinators and HE representatives identified the opportunity for networking as a major strength of the programme. However, while authorities were working with each other and sharing ideas and some of the HE representatives were working cross-institutionally, relationships between local authorities and HEIs were still developing. The main sources of support for teachers remained within their own schools (management and other teachers), though schools were working more with other schools. However, cross-sector and wider networking was less developed for the original pilot schools. The Associated Schools Groups were beginning to encourage greater links within clusters and across sectors.
  • Practitioner-led developments: The central role of the classroom teacher in taking forward developments and engaging in practitioner research was seen as a major strength of the programme, contributing to successful outcomes in many schools. This had led to high levels of commitment and enthusiasm. The process of 'growing policy', that is, allowing it to emerge rather than imposing it from the top down, was appreciated.
  • Funding: The provision of funding to the pilot schools was recognised as a major strength of the programme, with almost 50% of headteachers saying they would not have undertaken any of the developments without it. The main use was for the purchase of human resources - for supply cover or to pay teachers to work in their own time.
  • Sustainability: About half of the authority representatives indicated that AifL would be 'embedded' into teaching and learning policies and plans and that authority funding would be allocated to ongoing developments. Others thought that additional funding was necessary to maintain the level of development and to continue with developments in PLPs, use of ICT and reporting. Headteachers were also divided between those who thought they could sustain developments from their existing school budgets and those who saw the need for ongoing additional funding.
  • Monitoring progress: Around half of the local authority co-ordinators indicated that AifL issues were integrated into existing quality assurance procedures.
  • Planning: A recurrent theme from local authorities and headteachers was the mismatch between improvement and development planning and the funding cycle.