1 - Why Focus on Implementation in Education Reform?

Summary

There is constant pressure on governments and policymakers globally to raise the standard of education and to develop the appropriate curriculum and pedagogies to enable students to fit the world they will enter post-school; there are also international comparisons. There is a body of scholarship in the leadership field on change and reform, largely focused on the processes and ways of working. The policy and academic world have also focused on these matters, largely in the form of theoretical discussions or critical debates about issues of transnational work, school effectiveness or school improvement. There has been less focus on the implementation of reform or change. This chapter synthesises the literature on implementation in the fields of public policy and education and reviews existing thinking and scholarship on reform and implementation. The authors identify the common understandings, different approaches and gaps in the field, thus providing a rationale for the book and for the choice of case studies.

1.1 Current International Educational Context

Around the world there is a constant pressure on governments and policymakers to raise the standard of education and to develop the appropriate curriculum and pedagogies for students which will fit them for the world they will enter post-school. There is also much competition due to the new methods of international comparison, such as PISA and TIMSS, and much writing about change and frameworks for bringing about reform (e.g. Oates 2017; RAND 2018). There is a body of scholarship in the leadership field on change and reform too, which largely focuses on the processes and ways of working (e.g. Fullan et al. 2018). The field is also one where economic and academic organisations mix. Political life cycles are short. With notable exceptions, such as Richard Riley who served all eight years of President Clinton’s administration, education ‘ministers’ last about twenty months. This encourages a culture which proclaims reform and seldom implements it.

1.2 Aims of the Text

There has been much discussion of educational reform in the policy and academic world too. Much of this has taken the form of theoretical discussions or critical debates about issues of transnational work, for example, Salhberg (2016) on the global education reform movement. Others have concentrated on school effectiveness or school improvement (see Robinson et al. 2017). Some scholars have explored particular features of or vehicles for reform, such as Peurach’s (2016) work on networks. Hargreaves and Goodson (2006) is an example of the examination of perceptions and experiences of educational change.

When we look at reform, we see that the underlying logic model of educational reform is basically four steps:

1. 1. Design

2. 2. Proclaim, sell or promote

3. 3. Implement

4. 4. Evaluate the effect.

There is an abundance of designed reforms (what Elmore (1996) called ‘steady work’), some context-specific and some generic. There are plenty of ‘effect studies’, ranging from gross measures of student knowledge like PISA and TIMSS to national test scores to intervention-specific studies. There is little written about the different rollout measures (step 2) and even less about the process of implementation, exceptions are Pressman and Wildavsky (1973), Odden (1991a, 1991b) and Stringfield’s (1995) work on high reliability organisations. There are also syntheses of research studies, e.g. sixteen studies of school reforms by Datnow and Stringfield (2000) and a survey of a sustained programme of school improvement in a high-poverty area in Wales using propositions and practices generated by the principles of high reliability organisations, ones that ‘are assigned the very challenging task of operating without critically cascading errors the first time, every time’ (Stringfield et al. 2012: 45).

There are also different strategies that have been employed to create innovative approaches to change. These include examples we explore in our text. A highly successful one is the London Challenge model, which operated on collaborative groupings of schools with high external support. Another reform implementation strategy is the practice of creating a ‘free space’ where existing rules and regulations are put aside to allow for the adoption of new practices. Often these are referred to as Special Economic Zones (SEZs). In an SEZ the ‘rules for doing business are different from the rest of the country’. SEZs are different from the surrounding economic environment because they make it easier for companies to get access to reliable infrastructure; offer freedom from or deferral of taxes and customs charges and controls; and provide some fiscal incentives like free movement of capital and subsidies (World Bank 2017: 11–13). In effect they have an ‘extraterritorial status which enables them de facto immunity from domestic civil laws and government controls’ (Jayawardena 1983: 428). The network of Nazarbayev Intellectual Schools and its companion institution, Nazarbayev University, operate under a legal framework exempting them from many regulatory constraints. This is also one of our case studies. Similarly, Qatar’s Education City is an enabling environment for a number of branch campuses of foreign universities. The effectiveness of these arrangements in facilitating educational reform is understudied, as is the impact of practices in these zones on the rest of the nation.

There has been some work on teaching standards (by the National Board of Professional Teaching Standards in the United States and by the Department of Education in the United Kingdom); on instructional leadership; on particular initiatives, e.g. literacy in the United Kingdom; and on models of reform and the importance of teacher professional development. In economic terms it is a bit like focusing on inputs and outputs, with no attention to the throughputs. There has also been little study of ‘successful’ systems, apart from the Finnish miracle, which has also been described as a myth and a folk story (Oates 2017).

This book brings together detailed case studies of implementation over time and mostly written by those who were involved or were close observers. It deliberately represents a range of different models of reform in a range of different cultures and countries; and it includes evidence on the effectiveness, or lack of it, of specific reforms. It uses a grounded approach to study the implementation of reform. In Section 1.3 we outline some of the previous studies of implementation in social programmes and education and the current state of thinking about implementation, and then go on to present the case studies. We then undertake a cross-case analysis exploring aspects already established and those that are not. Some of the themes that we describe come from the literature, like the tension between ‘fidelity’, the accuracy or rigor with which a particular programme or intervention is applied, and unintended consequences and the professional adaptation of practice to individual student needs and particular contexts or environments. We also look at the ‘take-up rate’ for reforms that are not mandatory and even for those that are presented as mandatory.

1.3 Why Focus on Implementation?

This section begins with a synthesis of the literature on implementation in the fields of public policy and education. It then reviews existing thinking and scholarship on reform and implementation. We identify the common understandings, different approaches and the gaps in the field. We believe this provides a rationale both for the book as a whole and for the choice of case studies.

While evaluations of the US ‘War on Poverty’ spurred some interest in implementation (Odden 1991a; Weiss et al. 2008) the work of Pressman and Wildavsky on US urban reform programmes launched in the 1960s seems to be the first sustained study of the challenges of implementing social interventions. Initially, Pressman and Wildavsky, in their 1973 edition, thought that it was ‘flawed’ to separate design and implementation but also acknowledged that once implementation began the action of participants shaped the design. This led them to see implementation as part of a complex system of ‘reciprocal interactions’ (xxv). Six years later, Majone and Wildavsky, in an essay included in Pressman and Wildavsky, third edition (1984), wrote of implementation as evolution. They were rejecting a highly rational, linear, model of implementation; ‘implementation as control’, because it leaves out the ‘lumpy stuff of life’ (165) which includes resource constraints and the preferences and actions of individuals. Majone and Wildavsky also find the interactive model of implementation wanting. It ‘minimizes the importance of goals and plans’ and sees policy as no more than the starting point ‘for bargaining among implementers’ (166). But they acknowledge that this model has an element worthy of development, the notion that policies evolve as they are implemented. Majone and Wildavsky develop this idea, observing that ‘policies are continuously transformed by implementing actions that simultaneously alter resources and objectives’ (170). This leads them to label a model of interactive implementation where the act of implementing the policy changes the policy (177) as an ‘evolutionary model’. This is a model that allows for actors to ‘learn from experience … correct errors’ and even change ‘policy ideas’ (177).

This more nuanced and more realistic model of implementation was the backdrop for Browne and Wildavsky’s (1983) essay examining the significance of evaluation in implementation. The essence of their argument is that, while implementation and evaluation are both ‘concerned with the relationship between resources and objectives’ in the evolutionary framework of Majone and Wildavsky where implementation reshapes the desired outcomes, and objectives ‘cannot be held constant’, evaluation becomes a relative rather than an absolute process (204).

The evolutionary framework evokes the interaction between living organisms and the environment, sometimes discussed in terms of adaptation, or in Browne and Wildavsky’s case ‘mutual adaption’, an idea they borrow from Berman and McLaughlin’s (1974) Rand study. McLaughlin adds to the evolutionary metaphor by describing stages of development in implementation studies. She observes that ‘Implementation’ joined the working vocabulary of policy analysts in the early 1970s when ambitious, sweeping federal reform efforts followed ‘prevailing theories of governmental action and organizational behaviour (which) assumed away implementation issues or overlooked them altogether’ (McLaughlin 1987: 171). Evaluators of these programmes found that implementers ‘did not always do as told’, that ‘local factors such as size, intra-organizational relations, commitment, capacity, and institutional complexity moulded responses to policy’ and that the problems to be addressed also varied by location (172). These lessons were the base for the next generation of social programmes which focused on the linkage ‘between policy and practice’ because we ‘have learned that policy success depends critically on two broad factors: local capacity and will’ (172). Training, recruitment and resources might address capacity but motivating local actors was not just shaped by a policy or nicely designed programme. Context matters and many local factors shape the willingness of individuals to act, including ‘environmental stability, competing centres of authority, contending priorities or pressures and other aspects of the social-political milieu’ (173).

Pointing to a third generation of programme design, McLaughlin observes that ‘change ultimately is a problem of the smallest unit. At each point in the policy process, a policy is transformed as individuals interpret and respond to it’. This shifts attention away from institutions and their priorities to ‘individuals and individual incentives, beliefs, and capacity’ (McLaughlin 1987: 174). Spillane et al. (2002), drawing primarily on US and UK research on education policy implementation, argue that ‘implementing agents’ interpret a policy message by triangulating their ‘including knowledge, beliefs, and attitudes’ with their environment or context and ‘the policy signals’ (388).

In practice this means the differences between actors and between settings produce different problems for implementation, which are addressed through an iterative process of negotiation and adjustment. This leads to a model of implementation that is more nuanced than regarding policy change or a reform programme as an event rather than a process (Hall 1992: 104). It extends implementation past the act of proclamation and the exercise of authority and the distribution of incentives to an explicit acknowledgement that success is likely to depend on some degree of negotiation and adaptation at the site level. Supovitz (2008) draws out this idea; beginning with the proposition that variability is to be expected in implementation as there are many factors which cause ‘refraction’ and which are beyond the control of designers, policymakers and even supervisors (164). He describes implementation as an interactive process of ‘iterative refraction’ where ‘reforms are adjusted repeatedly as they are introduced into and work their way through school environments’ (153). Individual actors adjust their behaviour as they interpret a ‘policy signal’ using their professional judgement and practice knowledge and taking into account their own circumstances (Spillane et al. 2002: 420). Commenting on curriculum standards in Massachusetts, McDermott (2006: 48) linked successful implementation with instances where ‘policies interact with implementers’ understandings of their work and day-to-day needs’. We will use these ideas as we review the cases to examine the extent to which stakeholders and practitioners were involved in designing and enacting interventions because, as McDonnell (2004) observes in her study of US student testing and standards reforms, ‘involving those who implement a policy develops a sense of ownership’ (136) and is likely to increase effectiveness.

This negotiated and adaptive conception of how policies and programmes are implemented often raises questions about the fidelity of implementation. Some argue for flexibility and others for conformity to ensure the right dose. Lytle (2002), drawing on his experience as a US school superintendent, comments that developers of comprehensive school reform models are ‘often … overly concerned about implementation “purity”, and not adequately respectful of the need for mutual adaption (and) … slow to learn from the experience of implementation’ (166). They were dismissive of ‘practitioner knowledge’ and did not consider local conditions. Instead they tended to plan and design centrally and expect schools to act like franchise holders who adhere to ‘corporate policies and regulations’ (Lytle 2002: 166).

The need for fidelity and consistency in implementation arises when interventions or programmes are scaled up (McLaughlin and Mitra 2001; Bradach 2003) or applied to whole districts or systems. This is sometimes described as organisational replication. School reform efforts in the United States from 1990 onwards were often designed by a single external agency or corporation and enacted by existing schools or by new schools. The emphasis is on schools adopting externally developed programmes rather than developing their own programmes independently or in collaboration with other schools. Peurach and Glaser (2012) identify two assumptions underlying this approach. The first is that innovation, or a change in practice, follows a sequential path of research, design, communication and enactment. The second is that this type of replication is fast and effective because it delivers proven, ready for use materials or strategies. Both are questionable but provide themes that we will examine from the cases set out here.

An alternative to the apparent uniformity of replication and fidelity is the idea of coherence put forward by Robinson et al. (2017). Prompted by findings about the effectiveness of ‘joined up’ school improvement initiatives they draw the notions of coherence, coordination and orchestration from organisational literature. They use coherence to refer to instances where the interdependent parts of a system ‘are connected in ways that enable’ it to produce a desired end or outcome (2–3) and note that there are various forms of connectedness, not just one consistent or logical way to interact or address an issue. A strength of this approach is that coordination sits comfortably with three realities of school life; there is a shared purpose, to educate the next generation, which is a collective endeavour, and while many activities are done independently the process as a whole is based on interdependence. Orchestration is used to describe deliberate leadership acts that aim to align the efforts of actors in the school community. Studying the work of five high schools in a school improvement programme in an environment where school leaders have a lot of discretion, including in decisions to participate in such programmes and in the choice of support services, Robinson et al. found that schools with a high degree of coherence and tight coordination were more likely to realise improved student performance than those schools with lesser degrees of coherence. We will look for these themes in our review of the cases. Similarly we will look for the three characteristics that Hopkins et al. (2014) see to be antecedents of successful school improvement efforts: a ‘strategic … medium term approach’, the ready transfer of effective practices across sites, extensive professional support and ‘mentoring’ (274).

Coherence is often evoked by commentators advocating a systems approach to educational reform and is often linked to the notion of alignment (see for example World Bank 2018: 14). The proposition is that if four elements (learning objectives, assessment, finance and incentives) are all focused on effective teaching or ‘towards learning’, student outcomes will be improved (World Bank 2018: 174). While this is an appealing notion it overlooks Baker’s (2004) caveat that things fall in and out of alignment because the elements are not static. There are people involved, which immediately creates variation. Nonetheless we searched for alignment in the case we present here, be it attempts to harmonise actions or events which foster a sense of purpose. We do so because while ‘getting many people to work together on a common problem’ is seldom easy, this is the most likely pathway to success (Jochim 2018: 65).

In summary, the rationale for the book is that educational policymakers and reformers need a body of research on the implementation of reform on which to base their decisions and programmes. This is not a new idea. Elmore (1979) observed that there is a ‘noble lie’ in public administration that policymakers can or should be able to exercise some control over implementation. We know they cannot do so, even in the most autocratic environments, nor do we believe they should be able to dictate professional actions from afar. But we do wish, hope, for policies which are better designed with the realities of implementation in mind. As such we make the case for focusing upon implementation because it is the most significant phase of sustained and sustainable reform and the most ignored. To begin to address this gap we have collected a set of cases of relatively recent reforms and used them to identify some emerging issues in educational reform at a practical and theoretical level.

We have selected cases of reform implementation in a variety of stages, contexts and scales. We present case studies of major and minor reforms and of successes, of failures and of reforms still underway. The cases have been written by knowledgeable participants in the main; they are research-informed and they represent a range of different approaches to reform.

The cases are drawn from different geographies; the United States, the United Kingdom, Singapore, Kazakhstan, Hong Kong, Vietnam and Qatar. This gives us cases in large and small nations, in centralised and locally controlled systems. There are cases in elective democracies, a communist controlled society, in post-colonial and post-soviet states and a monarchy. We have reforms which are mandated and some that are locally adopted. Most cases are in relatively well-resourced systems with reasonably high levels of school completion.

The first case examines a systemic approach to equity. Equity is a major concern within UK education systems and the last ten years have seen efforts to address this issue through a series of ‘challenge’ programmes, the first of which took place in London. This chapter examines the evidence regarding what has been called the ‘London effect’, before going on to focus on the work of the Greater Manchester Challenge. This was a follow-up project that involved a partnership between national government, local authorities, schools and other stakeholders, and had a government investment of around £50 million.

The decision to invest such a large budget reflected a concern regarding educational standards, particularly amongst children and young people from disadvantaged backgrounds. The approach adopted In Greater Manchester, which was influenced by the earlier initiative in London, was based on an analysis of local context and used processes of networking and collaboration in order to make better use of available expertise. An independent evaluation concluded that it had been largely successful in achieving its objectives. The evaluators suggested that the strategic factors contributing to its success were the timescale; the focus on specific urban areas; flexibility of approach; use of expert advisers and bespoke solutions; school staff learning from practice in other schools; and the programme ethos of trust, support and encouragement.

Reflecting on the impact and the difficulties involved, plus subsequent efforts to create similar challenge programmes in Wales and Scotland, the chapter draws out lessons that are relevant to other contexts. Mel Ainscow, the author, was involved in these projects as an adviser working on a part-time basis, wherever possible using knowledge of evidence from relevant research to guide decision-making. This involvement provided privileged access to information regarding the way decisions are made within an education system, from the levels of government ministers and senior civil servants, through to that of teachers in the classroom. All of this provided frequent reminders of the cultural, social and political complexities involved when trying to bring about changes in the way that an education system does its business.

The ‘city’ focus of Ainscow’s chapter is echoed to some degree in Mary James’s case study of ten years of reform in Hong Kong. In 2002, Hong Kong embarked on a carefully planned and enormously ambitious ten-year reform of its education system of primary, secondary and tertiary education. The central aim of the reform was to promote all-round (whole person) development of students and a disposition towards lifelong learning in order to meet the needs of life and work in the twenty-first century. Changes in curriculum, assessment and pedagogy were thought to be necessary. Far-ranging structural changes were also introduced. Most significant is the introduction of the Hong Kong Diploma in Secondary Education (HKDSE), for all students, awarded at the end of secondary schooling, now at 17, thus replacing the old British system of examinations at 16+ and 18+. The reforms have been successful in increasing access of students to senior secondary studies, whilst maintaining or improving standards of achievement.

These root-and-branch reforms required thorough, on-going coordination, evaluation and renewal. Inevitably this was costly and government expenditure increased. Support for the recruitment and training of teachers and school leaders has been important. There have been worries, particularly about workload for students and teachers, but there is evidence that much has been gained in terms of students’ broader knowledge and skills, and enhanced self-confidence.

Hong Kong demonstrates that it is possible to introduce a more broad, balanced and coherent curriculum and assessment system whilst preserving or enhancing excellence. The crucial condition has been the opportunity to plan and implement a long-term, publicly agreed, reform programme protected, thus far, from too much political interference.

In contrast to the Manchester and London challenges, the reforms in Hong Kong were system-wide and this is also true of the case of Kazakhstan, where efforts have been underway for a similar ten-year period. Colleen McLaughlin and colleagues focus on the reform work that began in 2011. It is a large scale, comprehensive reform of the educational system covering the curriculum, assessment, teacher development, language policy, funding mechanisms, leadership, teacher appraisal and teacher working conditions. The authors were partners to the establishment of a group of pilot schools or schools of innovation, which served as models for the later translation to the whole school system that was completed in the 2019/20 school year. The authors have systematically studied this since 2012 and draw on their work and other data to explore this model and examine different perspectives on implementation: the teachers and school leaders, local leaders of education and the national stakeholders and policymakers.

The system-wide theme is taken up by Matt Hartley and Alan Ruby, who document reforms in the governance of higher education institutions in Kazakhstan, focusing on the challenges and opportunities of greater autonomy. One of the predominant strategies many countries have used to foster higher quality higher education systems is granting greater institutional autonomy.  The hope is that by moving from centralised systems controlled by Ministries to ones where institutions can pursue their destinies, innovation will inevitably result and performance improves. In reality, moving towards a more autonomous system comes at a cost.  Being free to set institutional strategies brings the possibility of making mistakes, something many leaders who have been trained in a compliance-based system find daunting.  Further, if leaders have never operated in a more market-based system, their ability to scan the environment to determine and launch new initiatives can be a challenge.  Such pressures can result in institutions following practices similar to the old compliance-based model in order to demonstrate to the larger society that they are being responsible and faithful to prior norms of behaviour.  Finally, autonomy requires constructing new, alternative systems of accountability (for example, the establishment of boards of trustees). Kazakhstan’s higher education reforms offer insights into both the challenges and possibilities of greater institutional autonomy.

A system-wide reform on school autonomy in Qatar is an interesting contrast to the higher education work in the context of Kazakhstan. Asmaa Alfadala, Stavros Yiannouka and Omar Zaki explore the theory and practice on school autonomy reform over almost twenty years. In 2001, Qatar embarked on comprehensive education reform – Education for a New Era – to meet the country’s changing needs and aspirations. The reform programme was based on a comprehensive study and recommendations made by the RAND Corporation, which envisaged a K-12 system, modelled in part on the US Charter Schools experience, that would offer autonomy and accountability for schools and variety and choice for parents and students. In its implementation the policy reform effort did not meet the objectives of the original design. Evidence reveals that the failure to meet the objectives can be attributed in part to the fact that important conditions were not present to enable schools to effectively practice autonomy. In particular, school leaders were not equipped to interpret and act on their mandates. As a consequence, the reform policies were partially reversed in 2014–15 and elements of the K-12 system recentralised.

They present evidence and explore in detail the underlying reasons for the lack of success of the initial reform effort and discuss some of the solutions being piloted to address the leadership capacity gap at the school level and promote greater professional autonomy. They ground their observations in the Qatari context, notably the enabling conditions for the exercise of autonomy, what the exercise of autonomy meant in practice, the impact of school autonomy on learning outcomes and student well-being and the strengths and limitations of capacity-building programmes to address leadership gaps in implementation.

Issues of leadership and the impact of system-wide reforms on student learning are also examined in a case study of promising practices in the government schools in Vietnam developed by Tony McAleavy and Rachael Fitzpatrick, who have been observing reform and development in the country very closely. They note that many people were astonished by the performance of Vietnamese students in the OECD-PISA tests administered in 2012 and 2015. Vietnamese students did well relative to students in most other countries, performing well in the science tests, although Vietnam was the poorest participating jurisdiction in terms of per capita income. The significance of the PISA results has been contested, but there are other signs that suggest that Vietnam has an effective school system with good student outcomes in core subjects.

McAleavy and Fitzpatrick outline the way the Vietnamese school system has evolved and works today, drawing upon research undertaken by the Education Development Trust and the Vietnam Institute of Educational Sciences. The case is grounded in an extensive analysis of Vietnamese education policy since the 1990s which has underpinned qualitative fieldwork, involving discussions with a wide range of stakeholders in four contrasting provinces: Ho Chi Minh City, Hanoi, Binh Dinh and Ha Giang. The respondents, unsurprisingly, confirmed that there are powerful cultural forces at work in Vietnamese society which are conducive to good performance in tests. Schools cannot take all the credit for Vietnamese performance in PISA. While recognising the importance of culture, and making no definitive causal claims, the case suggests that particularly ‘promising’ features of the development of the Vietnamese school system include a high level of consistency of policy priorities over the last three decades. This enabled local middle-tier agencies to play a key role in mediating and implementation of national policies. This was reinforced by a high level of professional accountability through different forms of in-school and external monitoring. This in turn fosters a degree of teacher professionalism and McAleavy and Fitzpatrick discuss the way teachers and others perceive government attempts to change classroom practice in order to encourage more ‘student-centred’ pedagogical practice. Finally they examine the role of the school principal as the interface between the school community and the external authorities and the unusually high level of parental involvement in the life of Vietnamese schools, with government regulations giving a prominent role to parents in school governance.

Continuing the theme of learning from high-performing systems, Saravanan Gopinathan and Edmund Lim draw implementation lessons from Singapore. They describe Singapore as an improbable success story, not least in education. Small, resource starved, hemmed in by large, occasionally unfriendly neighbours, it had to become a state and a nation, indispensable to first the region, then globally. Today, measured by GDP per capita, it ranks amongst the five richest countries in the world.

Education was central to this transformation. Post-war realities posed major challenges to policy formulation and implementation. How was a school system, segmented by media of instruction, to be unified? How could an academic grammar school curriculum be redesigned to aid rapid and transformative industrialisation? While the school-building programme and enhanced access to education was achieved, a hastily conceived and poorly implemented policy of school bilingualism created major problems in the 1970s.

These problems were overcome by the mid-1980s. Policymakers realised that a hitherto successful education-economy strategy had to give way to emergent globalisation’s challenges and opportunities. This in turn unleashed a wave of reform initiatives, at one stage promoting within a strongly nation-oriented system, choice, competition and branding as key drivers. Rapid changes to curricular and pedagogic frameworks, enhanced TVET, re-positioning the universities and upgrading teacher education have completely transformed the system. Singapore ranks highly in all international comparisons of educational quality.

In this case they examine the policies and processes that were responsible for this transformation. But they also caution against a too simplistic reading of the Singapore success story and suggest that a radical rethinking of the aims and purposes of Singapore education is due.

Counterbalancing these system-wide studies, Brian Rowan examines six externally developed ‘Instructional Improvement Programmes’ in the United States which have been subjects of a sustained programme of intervention studies. Over the past fifteen years, Rowan and colleagues have conducted large-scale quasi-experimental and experimental studies of six different, externally designed programmes that seek to change instruction and improve student achievement in US schools. The programmes were developed by not-for-profit and for-profit organisations, sought to change instructional practice in both English Language Arts and mathematics and were adopted by schools both as a result of government incentives and normal ‘market’ processes. In each of the six studies, they gathered data on how the externally developed programmes were designed and how they worked with school personnel to promote instructional change in schools. In each study, they carefully measured patterns of instructional practice and student achievement in order to assess the extent to which the programmes succeeded in changing teaching and improving student learning. Across the six studies, they found programmes that change teaching and improve student learning (success cases), programmes that change teaching but do not improve student learning and programmes that neither change teaching nor improve student learning. From this, Rowan developed a set of theoretical ideas about how to successfully change teaching and improve student learning when change occurs from the ‘outside in’ in American schools. One key finding is that successful external programmes of instructional improvement have well-specified designs for instruction and provide strong pressures and supports to encourage faithful implementation of these instructional designs in classrooms. But they caution that simply implementing pre-planned instructional designs is not enough to improve student learning. In their work, the central finding is that only when the instructional designs being implemented are different from – and better than – normative practice does improvement occurs in student achievement. Rowan illustrates these points by briefly laying out a theory of externally promoted instructional change and showing how different programmes fit within this theory. He concludes by discussing some of the challenges that governments and markets face in promoting the development of externally designed programmes for instructional improvement in US education. This confirms some of the findings from earlier research on implementation (see Odden 1991b, for example) that good design and thoughtful and sustained support increase effectiveness.

After the cases we offer some closing observations about what we have learned about implementation. We engage in a cross-case analysis of the cases of reform implementation drawing out the cross-cutting themes and lessons learned. We are not looking for consistency or an ‘Iron Law of Implementation’, but identify some points of commonalities and present the dilemmas and different implementation design options that are presented in the cases. We tie this back where apposite to previous work and sketch out ideas for more work on implementation.