Conference Agenda

Long periods of policy stability are rare in education. The intensification of external pressures over educational systems that we have witnessed in the last two decades has changed the focus and the rhythm, but also the very nature of educational reform, which currently mainly gears around the goal of strengthening the effectiveness, performance and competitiveness of educational systems. Beyond inputs, current reforms are guided by the definition of learning and other performance goals, and outcomes-based management approaches seem to enjoy bipartisan support in the sector of education. Despite equity and innovation tend to be part of reform intentions as well, they are always articulated with, subordinated to and assessed against performance standards.

Regarding this trend, school autonomy and accountability have become two central policy principles in the contemporary educational reforms we are describing. We label this reform approach as school autonomy with accountability (SAWA), given its emphasis on simultaneously promoting school-based management among public schools and strengthening outcomes-based accountability relations between schools and educational authorities. Issues of centralization, power and control in school decisions are at the core of SAWA as a model of educational governance. To a great extent, school actors have increasing capacity to decide on school management and pedagogy, as far as these decisions are taken to address centrally-defined performance outcomes, and feed a course of continuous school improvement. Within the SAWA model, schools are conceived as autonomous managerial units that enter into contractual relationships with public authorities and are encouraged to engage in performance data-use, (self-)assessment, innovation and improvement virtuous cycles (Grek et al 2022). Despite their piecemeal approach, SAWA reforms have great transformatory potential. They can alter schools’ ethos and organizational routines, change the interactions and decision-making capacity of key educational actors, and modify the main objectives and practices of teaching work.

However, and paradoxically, countries are adopting SAWA policies despite only weak evidence on their benefits. Empirical research in this area reaches very different and even contradictory conclusions. There is a body of literature that reports the positive effects of SAWA policies (especially in student learning outcomes) that clashes with another current that points to negative results (especially in relation to educational inequalities and segregation) and numerous adverse effects. Some of these impact studies conclude that the difficulty of reaching more conclusive and generalizable results is related to the context-dependent nature of SAWA interventions.

The main purpose of the symposium is to examine the enactment of SAWA instruments and their regulatory effects on school actors’ behaviors and their practices. The panel will combine single-case studies with comparative papers that will address enactment’s similarities and differences in different countries, allowing for a better understanding of the role of regulatory factors in mediating enactment processes and their effects.

The cases covered include countries and regions with different administrative traditions and educational arrangements on school autonomy, teachers’ regulations and accountability: The Netherlands, Spain (Catalonia), Italy, Norway, England and Chile. The papers presented in this symposium will allow for a better understanding of the mechanisms and circumstances that explain variegated effects in schools. Thus, this session will shed light on the implementation of recent education reforms across diverse contexts by paying attention to different components of the global education reform agenda -i.e., performance-based accountability, school autonomy, curricular standards, educational innovation- in local contexts. Furthermore, the papers included in the panel will provide insights on the role of local education markets, and sense-making processes together with an account of the administrative designs of the cases studied, which also mold the policy mandates in its implementation phases, bringing to the surface tensions and dilemmas (Pollitt & Bouckaert, 2017).

Often, contemporary educational reforms are not structural in nature, but rather follow a piecemeal approach, advancing through the adoption of new policy instruments. Such instruments are data-intensive, they set standards of appropriate behavior and best practice among teachers, school principals and school providers, and they are tied to incentives and sanctions. Schools, conceived as autonomous managerial units, often enter contractual relationships with public authorities and are encouraged to engage in continual data-use, assessment and improvement cycles (Authors, 2021). Despite their piecemeal approach, contemporary reforms have a great transformatory potential (Author et al., 2019). They can alter schools’ ethos and core organizational practices, the interactions and decision-making capacity of key educational actors, and change organizational routines, and modify main objectives and practices of teaching work. The use of such instruments is often enforced by regulatory power, e.g. when routines and procedures embedded in quality assurance are regulated. Thus, governance instruments connected to routines and procedures can gain importance and develop a regulatory status themselves even if they only represent means that aim to fulfil quality assurance. In this paper, we analyse how policy instruments structure and regulate school practices enable combinations of performance-based, contractual and legal accountabilities. We apply a political sociology of policy instruments which implies analyzing the logics of instrument choice, as well as tracing the development of policy instruments. We pay particular attention to how policy instruments incentivize the generation of new constituencies, which are comprised of actors “oriented towards developing, maintaining and expanding a specific instrumental model of governing” (Simons & Voß, 2018. p. 31). The analysis draws on data from empirical studies on datafication in education and the enactment of accountability policy in various educational settings, such as Chile, England, the Netherlands, Spain, and Norway. Based on existing case studies, the use of policy instruments in schools are re-analysed and compared, and thus providing a rich basis for generating new knowledge and discussing implications for theories of education governance. The preliminary analysis shows that accountability systems have become central in the governance of multiple public sectors and domains but are becoming increasingly sophisticated and adaptable to a complex and fluid world. In education, most of accountability systems assemble different policy instruments such as large-scale assessments, targets, data infrastructures and contracts between public administrations and schools as well as between actors within the schools such as teachers and students, and regulated routines and procedures.

Authors (2019) Authors (2021) Lascoumes, P., & Le Gales, P. (2007). Introduction: Understanding public policy through its instruments—from the nature of instruments to the sociology of public policy instrumentation. Governance, 20(1), 1–21. Simons, A., & Voß, J.-P. (2018). The concept of instrument constituencies: Accounting for dynamics and practices of knowing governance. Policy and Society, 37(1), 14–35.

Over the last two decades, schools are, nearly worldwide, increasingly expected to use various data sources to change and improve education. As a result, discussions about what data is and how and for what it can be used animate education debates in many countries. Sources such as student performance data, are at the core of school autonomy with accountability regimes that have developed to steer growingly fragmented, complex and multi-layered education systems (Verger & Skedsmo, 2021). Most of the research on the use of student performance data comes from the United States, which has a longer and stronger tradition of output-oriented school governing linked to high-stakes accountability. This research has also influenced how we talk about and understand the use of student outcomes in Europe and Norway (Prøitz et al., 2017). With more countries adapting policies and practices of data use, what we understood as data a decade ago, may be too reductionist if compared to what we consider as data today. In addition, digital technology has increased access and ability to combine data on different aspects of education. To understand conceptualisations of data (Coburn and Turner, 2011; Spillane, 2012) and how they are used by school actors in different times and places means enabling informed debates about data use in the education sector. In this paper, we present the results of a mapping review (cf. Grant & Booth, 2009) of international research on data use published during the period of 2000-2022.In the analysis, we concentrate on tracing development trends with respect to data sources and purposes of data use across country contexts. Methodologically, systematic literature searches for peer-reviewed publications were conducted in the Scopus and ERIC databases. After initial searches, a search string was developed and criteria for inclusion/exclusion were formulated. The identified publications were first screened based on title and abstract, then based on full-text versions. In cases of disagreement, the publications were critically examined to reach a joint decision, leading to a refined attainment of decisions and increasing the agreement rate. Preliminary results indicate a geographical spread of research in this area moving from Anglo-Saxon countries, to include continental, Nordic, and Mediterranean European countries, and other parts of the world. Moreover, the analysis shows an expanded understanding of what data is, who can produce it and how it should be viewed by school actors, a more holistic approach to quality indicators, and increased awareness of ethical issues.

Coburn, C. E., & Turner, E. O. (2011). Research on Data Use: A Framework and Analysis. Measurement, 9(4), 173–206. DOI: https://doi.org/10.1080/15366367.2011.626729 Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x Prøitz, T.S., Mausethagen, S. & Skedsmo, G. (2017): Investigative modes in research on data use in education. Nordic Journal of Studies in Educational Policy, 3(1), p. 42-55. https://dx.doi.org/10.1080/20020317.2017.1326280 Spillane, J. P. (2012). Data in practice: Conceptualizing the data-based decision-making phenomena. American Journal of Education, 118(2), 113–141. DOI: https://doi.org/10.1086/663283 Verger, A. & Skedsmo, G. (2021). Enacting accountability in education: exploring new policy contexts and theoretical elaborations. Educational Assessment, Evaluation and Accountability, 33(3), 391-402. https://doi.org/10.1007/s11092-021-09371-x

Performance-based accountability (PBA) policies have become central in the governance of educational systems worldwide. In the field of education, these policies commonly consist of the evaluation of students’ performance through large-scale standardized tests, to which consequences of a different nature are tied (material and symbolic, individual and collective), which are faced by school actors (Verger et al., 2019). The idea behind this policy approach is that being held accountable and facing consequences will generate pressure, which will move actors towards accomplishing the requested performance goals more efficiently (Chiang, 2009). Research on PBA in education has long considered material consequences as generating more pressure, and therefore as having stronger behavioral effects, compared to symbolic ones, i.e., consequences merely associated with the visibility of test results (Au, 2007). Nonetheless, more recently, studies have found that symbolic consequences also have the potential to generate high levels of performative pressure and can trigger similar behavioral responses as material ones (Thiel et al., 2017; Authors, forthcoming). Reputational concerns seem to play a crucial role in explaining this. Research on public accountability in general has recently also highlighted the importance of deepening our understanding of the “reputational basis” of public accountability (Busuioc and Lodge, 2015; 2016). Reputational concerns, however, remain underexplored in the educational literature, in particular in non-marketized contexts. In this study, we conduct a comparative study of the role of reputation in explaining accountability concerns in different policy contexts: Catalonia, Chile and Norway. These three cases differ in terms of the consequences associated with PBA, but also in other aspects (school choice, students´ enrolment and/or teachers´ recruitment policies, school actors´ working conditions, etc.). Our aim is to explore and analyze to what extent, how and why reputation matters for school actors by taking into account the complex interplay between accountability policy settings and other issues related with contextual factors and other educational policies. Our comparative analysis relies primarily on qualitative data based on in-depth interviews with school principals and teachers in all three policy settings. The findings provide key insights in terms of the extent to which, how and why reputational concerns are related to accountability matters. By highlighting both a number of general/universal trends in terms of how reputational concerns are experienced, and by identifying various context-specific mechanisms that influence the extent and impact of reputation concerns in different policy settings, the paper contributes to the existing literature on reputation and accountability.

Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36(5), 258–267. Busuioc, E.M., Lodge, M. (2016). The reputational basis of public accountability. Governance, 29(2), 247-263. Busuioc, E.M., Lodge, M. (2017). Reputation and Accountability Relationships: Managing Accountability Expectations through Reputation. Public Administration Review, 77(1), 91-100 Chiang, H. (2009). How accountability pressure on failing schools affects student achievement. Journal of Public Economics, 93, 1045–1057. Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: how public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40. Thiel, C., Schweizer, S., & Bellmann, J. (2017). Rethinking side effects of accountability in education: insights from a multiple methods study in four German school systems. Education Policy Analysis Archives, 25(93), 1–32. Verger, A., Parcerisa, L., & Fontdevila, C. (2019). The growth and spread of national assessments and test-based accountabilities: a political sociology of global education reforms. Educational Review, 71(1), 5–30.

Most education systems are trying to combine external and internal school evaluation as well as school autonomy to foster authentic improvement processes (Eurydice, 2015). Through these instruments, schools are expected to improve and adapt to the learning contexts and needs of students. Following these premises, schools are expected to engage in a virtuous circle of improvement, which should benefit all students, particularly the most disadvantaged. Yet, the meaning and goals of improvement remain obscure and its ways of addressing it in schools are also unexplored. The existing evidence suggests that unexpected effects tend to emerge, often resulting in superfluous school changes (Lubienski, 2009), instrumental behaviors (Au, 2007) and increasing competition between schools (Falabella, 2020). Recent reforms in Catalonia (Spain) follow these trends. Improvement and innovation are core elements of the governance system, characterized by being a low-stakes accountability, where internal and external evaluation―through standardized tests and inspections―coexist. The apparatus, based on a continuous improvement process, should lead to reflexivity and change in professional practices. Improvement discourses from the government push towards a student-centered, competence-based teaching, and are associated with pedagogical and organizational change, and the strengthening of school innovation networks. However, structural conditions seem not supportive of such changes (EU Commission, 2018). The goal of the paper is to analyze how improvement materialize in schools, looking at how schools interpret and interact with the policy expectations according to the local context and school results. How do school actors negotiate between the education mandates? What organizational and educational practices does this translate into? What aspects facilitate or hinder the adoption of such practices? What is the role of school context and performative pressure? We embrace a mixed methods approach that combines in-depth semi-structured interviews with teachers (n=25) and principals (n=8) from 8 schools, with descriptive analysis of survey responses. Preliminary findings show that school actors adopt a range of responses to the improvement mandate. Pressure to perform shapes the adoption of improvements strategies in most schools. In some cases, innovation is sustained by schools’ capacity to use autonomy and is mainly associated with project-based planning and pedagogical changes, but in other occasions is related to 'innovation shopping' or to performance-oriented practice (such as intensive test preparation). Finally, we also find resigned schools in socially segregated contexts, renounce to play the innovation game. Professional, organizational, and material factors emerge as enabling conditions to explain these variegated school responses.

Falabella, A. (2020). The ethics of competition: accountability policy enactment in Chilean schools’ everyday life. Journal of Education Policy, 35(1), 23-45. European Commission, Directorate-General for Education Sport and Culture (2018). Study on supporting school innovation across Europe: final report. Report, European Commission: Publications Office 2018. Lubienski, C. (2009). Do Quasi-markets Foster Innovation in Education? A COMPARATIVE PERSPECTIVE. OECD Education Working Papers. Malen, B., Croninger, R., Muncey, D., & Redmond-Jones, D. (2002). Reconstituting schools: “Testing” the “theory of action.” Educational Evaluation and Policy Analysis, 24(2), 113–132. https://doi.org/10.3102/01623737024002113 Paletta, A., Basyte Ferrari, E., & Alimehmeti, G. (2020). How Principals Use a New Accountability System to Promote Change in Teacher Practices: Evidence From Italy. Educational Administration Quarterly, 56(1), 123–173.