23. Policy Studies and Politics of Education
Paper
Accountability, Datafication and Sense-Making in Disadvantaged School Contexts: A Comparative Analysis of Spain and Chile
Lluís Parcerisa1, Marcel Pagès2
1University of Barcelona, Spain; 2Autonomous University of Barcelona, Spain
Presenting Author: Parcerisa, Lluís
Standardized tests and performance metrics are becoming increasingly widespread as key components of global education reform (Ball et al. 2017; Sahlberg, 2016). Countries with different teaching models (Voisin & Dumai, 2020) and diverse education policy approaches are adopting performance-based accountability (PBA) as a policy solution to improve the quality of education systems (Lingard, 2013).
As a response to general concerns about education quality, standardized tests are used to hold teachers accountable for students’ results, with the expectation that performative pressures will induce teachers to align their instructional practices with learning standards and utilize the achievement data for school improvement purposes. According to this theory of change, external accountability can be a suitable instrument for enhancing teacher quality, ensuring learning and improving school performance. Moreover, the test data are expected to be employed as a part of an informational system to identify areas that need further attention and eventually implement improvement plans and corrective strategies (Lingard et al., 2017).
Interestingly, existing research on PBA shows mixed results. While some investigations observe an active policy appropriation by teachers (Hardy, 2014), other investigations suggest that performative pressures, far from reinforcing virtuous circles of improvement and policy alignment, tend to erode the professional autonomy of teachers and educators (Daliri-Ngametua et al., 2021; Holloway & Brass, 2018). This mismatch between policy design and actual practices is observed in numerous education systems where accountability mechanisms result in policy decoupling, ritualistic implementation, and instrumental responses (Reinhorn et al. 2017; Thiel et al. 2017).
Indeed, schools and teachers may embrace strategic practices to escape pressure and cope with performance expectations. School competition, teaching to the test and curriculum narrowing or cheating are only some of the undesired responses that schools might adopt to dilute the external pressures associated with testing and accountability (Falabella, 2020; Koretz, 2017). These results are observed in contexts with different accountability models, but appear to be very frequent in disadvantaged school contexts (Candido, 2019; Diamond, 2012).
Still, little is known about under what conditions such instrumental practices emerge and how they become institutionalized in different education systems. In order to understand this process, we suggest that we need to better understand teachers’ interpretations of accountability mandates. We aim to unpack teachers’ discourses about testing and accountability in order to shed light on the sense-making of accountability policies, with a particular focus on vulnerable school contexts. This investigation focuses on the interpretation of the accountability mandates of schools in disadvantaged contexts because within these institutional environments, school actors appear to be more prone to adopt instrumental and undesired responses. Our argument is that by analysing teachers’ interpretations of accountability policies, we can better understand how and why instrumental practices emerge and become the norm in certain schools.
Accordingly, the research goal of our work is to unpack the different components of teachers’ discourses on PBA in vulnerable schools to better understand how school actors’ sense-making sustains instrumental practices. To do so, we conduct a comparative case study with a qualitative approach, analysing the discourses of teachers working in vulnerable school settings in Spain (Madrid) and Chile. These are interesting contexts for investigating the role of performative pressures since they combine high levels of marketization with different approaches to PBA (Falabella, 2020; Prieto & Villamor, 2012).
Methodology, Methods, Research Instruments or Sources UsedThe study adopts a qualitative comparative analysis (QCA) to better understand the teachers’ sense-making of accountability policies and datafication in disadvantaged schools’ contexts in Spain and Chile.
The selection of the two cases was made on purpose, according to the accountability policy design and the structure of the educational supply. While Chile and Spain differ in terms of their accountability policy approach, both countries have some similarities in terms of the structure of their education provision, including market-oriented models. In terms of the accountability policy model, Chile has a high-stakes PBA model, which is deeply consolidated in the education system with a long trajectory and relative stability. In contrast, Spain, and particularly the case of Madrid, has adopted accountability mechanisms quite recently, following a lower-stakes model with erratic policy trajectories. Interestingly, both cases share similarities in the structure and governance of education provision. Accordingly, the two cases compared share a market-oriented education system with important levels of private-subsidized schools and salient levels of school competition. In short, we suggest that these are particularly interesting contexts for investigating the role of performative pressures since they combine high levels of marketization with different approaches in relation to PBA (Falabella, 2020; Prieto & Villamor, 2012)
To conduct our analysis, we purposefully selected small-n cases (Spain and Chile) to enhance the external validity of our study. We mobilized context-sensitive knowledge for each case to analyse and compare our data in order to ensure internal validity, and developed an inductive and explorative mode of reasoning to interpret our results (Thomann & Maggetti, 2020).
Our data are based on a sampling of public and private-subsidized schools with low socio-economic status. We conducted 26 semi-structured in-depth interviews with teachers and school leaders to develop a systematic comparison of teachers’ enactment of PBA in both countries. To analyze the interviews, we combined deductive and emerging codes. We first applied a list of structural codes (Saldaña, 2021) defined in a codebook to share the same criteria to code and analyse interviews in Chile and Spain (Parcerisa & Verger, 2023). With the comparison of the codes, new themes and topics were identified and we iteratively built new labels to classify, interpret and examine these emerging results.
Conclusions, Expected Outcomes or FindingsThis article shows the importance of school context and meaning-making processes in the enactment of educational policy, and more particularly the key role of interpretation and sense-making as a mediating factor explaining policy decoupling and opportunistic behaviours. Based on a comparative case study, the paper illuminates the similarities (and also some differences) in the policy reception and interpretation of accountability policies by teachers working in disadvantaged contexts. Although the discourses analysed share important ideas and critical understandings of PBA, we do not aim to suggest that the discourses of teachers are univocal or homogenous. Some teachers give more importance to social justice arguments, whereas others highlight pedagogical or professional discourses. Moreover, some discourses of appropriation and negotiation are also found, despite not being the norm. Our results do not suggest that similar discourses imply similar school responses to PBA. Indeed, as we have analysed elsewhere, the school responses to PBA are multiple and diverse (Authors, 2023; Authors, 2021). This suggests that the school's policy responses cannot be understood as a mechanical and linear process from interpretation to translation, but a conflicting and negotiated process mediated by organizational, professional, and contextual factors that modulate different translations within a range of similar forms of policy interpretation.
Despite the differences in the characteristics of the educational systems and the design of accountability instruments, our research suggests that teachers working in vulnerable school settings in Madrid and Chile share important arguments when they identify negative components of PBA for disadvantaged schools. However, the article shows that critical discourses on PBA are complex, interwoven, and multifaceted.
ReferencesBall, S. J., Junemann, C., & Santori, D. (2017). Edu. net: Globalisation and education policy mobility. Routledge.
Candido, H. H. D. (2019). Datafication in schools: enactments of quality assurance and evaluation policies in Brazil. International Studies in Sociology of Education, 29(1–2), 1–32. https://doi.org/10.1080/09620214.2019.1656101
Daliri-Ngametua, R., Hardy, I., & Creagh, S. (2021). Data, performativity and the erosion of trust in teachers. Cambridge Journal of Education, 1-17. https://doi.org/10.1080/0305764x.2021.2002811
Diamond, J. B. (2012). Accountability policy, school organization, and classroom practice: partial recoupling and educational opportunity. Education and Urban Society, 44(2), 151–182. https://doi.org/10.1177/0013124511431569
Falabella, A. (2020). The ethics of competition: accountability policy enactment in Chilean schools’ everyday life. Journal of Education Policy, 35(1), 23-45. https://doi.org/10.1080/02680939.2019.1635272
Hardy, I. (2014). A logic of appropriation: enacting national testing (NAPLAN) in Australia. Journal of education policy, 29(1), 1-18. https://doi.org/10.1080/02680939.2013.782425
Holloway, J., & Brass, J. (2018). Making accountable teachers: The terrors and pleasures of performativity. Journal of Education Policy, 33(3), 361-382. https://doi.org/10.1080/02680939.2017.1372636
Koretz, D. (2017). The testing charade: pretending to make schools better. University of Chicago Press
Lingard, B. (2013). Historicizing and contextualizing global policy discourses: Test-and standards-based accountabilities in education. International Education Journal: Comparative Perspectives, 12(2), 122-132.
Lingard, B., Sellar, S., & Lewis, S. (2017). Accountabilities in schools and school systems. Oxford Research Encyclopedia of Education, 3, 155. https://doi.org/10.1093/acrefore/9780190264093.013.74.
Parcerisa, L., & Verger, A. (2023). Researching ‘Autonomy with Accountability’ in Schools: A Qualitative Approach to Policy Enactment and Practice. REFORMED Methodological Papers No.3, 1-33. doi: 10.5281/zenodo.1036169
Prieto, M., & Villamor, P. (2012). Freedom of choice, competition and quality: educational policies of the Autonomous Region of Madrid. Profesorado, Revista de Currículum y Formación del Profesorado, 16(3), 127-144.
Reinhorn, S. K., Johnson, S. M., & Simon, N. S. (2017). Investing in development: Six high-performing, high-poverty schools implement the Massachusetts teacher evaluation policy. Educational Evaluation and Policy Analysis, 39(3), 383-406. https://doi.org/10.3102/0162373717690605
Sahlberg, P. (2016). The global educational reform movement and its impact on schooling. In K. Mundy, A. Green, B. Lingard, & A. Verger (Eds.), The handbook of global education policy (pp. 128–144). Wiley-Blackwell. https://doi.org/10.1002/9781118468005.ch7
Saldaña, J. (2021). The coding manual for qualitative researchers. London: Sage.
Thiel, C., Schweizer, S., & Bellmann, J. (2017). Rethinking side effects of accountability in education: insights from a multiple methods study in four german school systems. Education Policy Analysis Archives, 25(93), 1–32.
Voisin, A., & Dumay, X. (2020). How do educational systems regulate the teaching profession and teachers’ work? A typological approach to institutional foundations and models of regulation. Teaching and Teacher Education, 96, 1-16. https://doi.org/10.1016/j.tate.2020.103144
23. Policy Studies and Politics of Education
Paper
Participation in Adult Learning and Education in the UK: The Scarcity of Consistent Statistical Evidence in the Abundance of Data
Ellen Boeren1, Betul Babayigit2, Zyra Evangelista1, Sharon Clancy2, John Holford2
1University of Glasgow, United Kingdom; 2University of Nottingham, United Kingdom
Presenting Author: Babayigit, Betul;
Evangelista, Zyra
The participation of adults in learning and education (ALE) has been subject to research for many years, and attempts to make it more prevalent have been going on throughout the world due to its value in the labour market and the empowering role of ALE in individuals’ social relationships, social mobility, job prospects, finances, health, and wellbeing. These innumerable benefits of ALE require the genuine dedication of countries to making participation in ALE more prevalent and accessible for all adults, regardless of their educational or socio-economic background. Inevitably, this dedication necessitates effective policymaking that aims to involve more adults in learning, especially those with lower educational attainment and from lower social classes who are traditionally less prone to participate in or access ALE opportunities (Boeren, 2009; Kersh & Laczik, 2021). One of the most crucial factors influencing the effectiveness of policymaking is taking scientific evidence into account during the decision-making process concerning ALE. The consistent decrease in ALE participation rates in the United Kingdom since 2010s makes evidence-based policymaking more indispensable and urgent than ever. Through the use of scientific evidence, policymakers can better target the groups who don’t participate in ALE or those who are deprived of ALE opportunities, which can result in an increase in total participation rates and more equitable proportions of learners within those rates based on their economic and social backgrounds. However, the effectiveness of evidence-based policymaking is partially bound to the amount and quality of the scientific evidence available. It is ideally expected that the data on ALE should provide a good measurement of the people who participate, for what reasons, and in what type of learning activities as well as the benefits of ALE (Boeren, 2016). It is equally important that the data on ALE should depict a very accurate picture of who does not participate and why.
This contribution will present findings from an ongoing research project funded by the UK’s Economic and Social Research Council (ESRC). The project aims to investigate the statistical evidence-base in ALE in the UK, reveal the potential reasons behind the decreasing participation rates, and unpack how policymakers benefit from the available evidence-base during their decision-making process. In this paper, we aim to explore how participation in ALE is measured by large-scale surveys that collect data from the UK. We also aim to investigate how major determinants of participation (motivations for ALE, barriers to ALE, and the benefits of ALE) have been encompassed by these surveys. While approaching the participation questions in the surveys, we will adopt the Total Survey Error paradigm to reveal potential sources for varying participation rates. In terms of motivations and barriers, we will mainly rely on the Bounded Agency model (Evans, 2007) and Boeren’s (2017) layered model of participation, along with other theoretical frameworks such as Cross's (1981) typology of barriers and Houle’s (1961) typology of adult learners’ motivations.
Methodology, Methods, Research Instruments or Sources UsedEmploying a qualitative approach, we conducted an extensive text-based content analysis on the questionnaires of 16 national and European surveys that collect data on participation in ALE from the UK context. The surveys under our scrutiny were the Adult Participation in Learning (APiL) survey, the Survey of Adult Skills (PIAAC), Adult Education Survey (AES), Labour Force Survey (LFS), European Social Survey (ESS), European Quality of Life Survey (EQLS), European Company Survey (ECS), European Working Conditions Surveys (EWC), Continuing Vocational Training Survey (CVTS), National Child Development Study (NCDS), British Cohort Study (BCS), Next Steps (NS), Understanding the Society (UtS), and the UK Time Use Survey (UK-TUS). The documents for these surveys were downloaded from their websites, as most of them were already available for public use. The questionnaires that were not publicly available were shared with us by the relevant institutions. During the data analysis, we systematically coded and categorised the questions for participation, motivations, and barriers to reveal their compatibility with the theoretical frameworks mentioned above by following the steps proposed by Zhang and Wildemuth (2009).
Conclusions, Expected Outcomes or FindingsThe findings indicate that the surveys differ from each other in terms of their methodology, their ways of measuring participation, and how they approach to motivations and barriers, which can complicate further secondary statistical analyses. Most of the surveys in our sample do not collect detailed data about ALE as it is not their primary focus. Another major finding is the scarcity of consistent and comprehensive longitudinal data underpinning ALE. It is also revealed that data on motivations and barriers are rare and the scope of them vastly differs across surveys and from the theoretical models. Most surveys do not adopt a comprehensive understanding of barriers, failing to acknowledge the layered nature of the factors affecting participation behaviour (Boeren, 2017) since the surveys usually focus on individual (micro) level factors rather than meso-/macro-level factors and seem to fail to depict the intertwined relationships between them. Therefore, they may create the illusion that nonparticipation is largely caused by individuals’ time constraints and family commitments by ignoring the role of how economic volatility may require individuals to work for longer hours or how insufficient childcare policies may impede participation. In conclusion, the data underpinning ALE is mostly piecemeal and divergent in nature, which undermines future secondary analyses and comparisons across different surveys. Although the available ALE data do tell us which groups of people tend to participate, we are still in the dark when it comes to answering more intricate questions: When do former non-participants switch to the state of participation? When do former participants stop learning? How are the switching states of (non)participation affected by micro, meso-, and macro-level determinants? The lack of answers to these questions may jeopardise effective policymaking by preventing policymakers from addressing the most relevant factors and cause ALE policies to be tautologous, generic, or deflective.
ReferencesBoeren, E. (2009). Adult education participation: the Matthew principle. Filosofija-sociologija, 20(2), 154-161.
Boeren, E. (2016). Lifelong learning participation in a changing policy context: An interdisciplinary theory. London: Palgrave Macmillan.
Boeren, E. (2017). Understanding adult lifelong learning participation as a layered problem. Studies in Continuing Education, 39(2), 161-175.
Cross, K. P. (1981). Adults as learners. San Francisco: Jossey-Bass.
Evans, K. (2007). Concepts of bounded agency in education, work, and the personal lives of young adults. International Journal of Psychology, 42(2), 85–93.
Houle, C. O. (1961). The Inquiring Mind. Madison: University of Wisconsin Press.
Kersh, N., and Laczik, A. (2021). Towards understanding of policy transfer and policy learning in adult education in the context of United Kingdom. Research in Comparative and International Education, 16(4), 384-404. https://doi.org/10.1177/17454999211061236
Zhang, Y. and Wildemuth, B. M. (2009). Qualitative analysis of content. In B.M. Wildemuth (Ed.), Applications of Social Research Methods to Questions in Information and Library (pp. 1-12). Libraries Unlimited.
23. Policy Studies and Politics of Education
Paper
Enhancing Adult Education Policy Through Data Collection and Registries – Experiences of the Visegrad Countries
Horacy Debowski1, Wojciech Stęchły1, Erzsebet Szlamka2, Zdenka Simova3, Marketa Tuckova3, Ľubica Gállová4
1SGH Warsaw School of Economic, Poland; 2Governmental Agency of IT Development, Hungary; 3National Training Fund, Czechia; 4The State Institute of Vocational Education, Slovakia
Presenting Author: Debowski, Horacy;
Stęchły, Wojciech
The integrity and quality of information in registries are fundamental to all stages of policy development – from conception and design to implementation and evaluation – especially in fields like adult learning, where the needs and conditions can be diverse and dynamic (Roumell & Roessger, 2019). Such data can reveal trends, needs, and gaps in the current education system, enabling more targeted and effective policy interventions. Data from registries can offer insights into adult learners' demographics, learning preferences, and career trajectories. Big Data and advanced analytics are vital in creating responsive and adaptive workforce development systems (Williamson, 2017). There are expectations that education policymakers will need to be plied with quality data in the form of predictive analytical patterns (modelling, machine learning, and data mining of historical data) and knowledge about global educational predictions of future outcomes and trends (Soskil, 2018).
Reliable registries, which include information about the accreditation status of training providers and programs, are also important from learners' and employers' perspectives as they might reduce the asymmetry of information and assure the quality and the potential for a return on their investment in education and training. They might also enhance the efficiency of the search for the appropriate training offer and, therefore, contribute to better investments in human capital. The Council Recommendation on individual learning accounts of 2022 recommends developing public registers of training offers. The Council Recommendation states: “There is also a need for up-to-date public registries of recognised training through dedicated single national digital portals accessible to all, including people with disabilities, and, preferably, interconnected with the Europass platform”. Establishments of public registers in many European countries is also linked with the development of national qualifications frameworks for lifelong learning (Markowitsch & Dębowski 2022)
In the article, we aim to analyse solutions adopted in the four Visegrad countries, namely the Czech Republic, Hungary, Poland and Slovakia, regarding developments of registries and data collection in the education sectors with particular attention to the vocationally oriented adult education sector. Following Desjardins' adult training systems typology (2017), we aim to identify how data in adult learning subsystems (sectors) is collected and used by policymakers and stakeholders. We distinguish between data (registries) regarding learners and data (registries) regarding the training offer. The analysis of data collection systems will be conducted against the background of policy frameworks that underpin adult learning in the Visegrad countries, noting the interplay between European Union recommendations and national priorities, including the structure of governance of the adult learning and financing.
The findings aim to contribute to the broader discourse on adult education systems and inform future policy development within and beyond the Visegrad region. The article draws on evidence from the international project: Digital Individual Learning Accounts In The Visegrad Countries (D-ILA in V4) financed within the Erasmus+ framework. The project used mixed research methods, including literature and policy documents analysis of public and private registries of data collection as well as in-person interviews (44 interviews in total) with the key stakeholders: training providers, employers, policymakers, trade union representatives, policy researchers.
Methodology, Methods, Research Instruments or Sources UsedThe article draws on evidence from the international project: Digital Individual Learning Accounts In The Visegrad Countries (D-ILA in V4) financed within the Erasmus+ framework and has been conducted by four institutions from Visegrad countries. Authors of the article have been involved in the D-ILA in V4 project.
The article draws on mixed research methods, including literature and policy documents analysis, analysis of public and private registries of data collection, as well as in-person interviews (44 interviews in total) with the key stakeholders: training providers, employers, policymakers, trade union representatives, and policy researchers.
The article compares and synthesises solutions and practices from the four Visegrad countries.
Conclusions, Expected Outcomes or FindingsThe Visegrad countries decentralised their education systems at the beginning of the transformation in the 1990s, and adult learning sector was viewed as market oriented and was essentially left to private providers and voluntarist initiatives of NGOs. In the absence of governmental regulation and support, the institutionalization of adult education policy was slow. However, in Hungary since 2010 there has been a strong move to centralisation within education in general and within VET in particular, and a resurgence in top down, system-wide policy initiatives.
In the Visegrad countries, similarly as in other EU countries, adult education and training takes place mostly in the non-formal education setting, and this sector has been growing over the years while the share of adults participating in formal education is decreasing. At the same time this sector is largely unregulated and not monitored, with Hungary to be an exception.
The functioning of the adult education system in Hungary is regulated in detail by laws and in recent years, there has been an expansion and tightening of data collection related to: a) persons participating in adult education and training, b) training courses; c) data related to the organisation of examinations and organisations providing. In other Visegrad countries there is no one training database for adults, and data regarding persons participating in non-formal education is generally not collected. However, all of the Visegrad countries introduce new policy initiatives and tools, including registers, in order to better monitor and coordinate adult education sector. In Hungary, Czechia, Slovakia and Poland some forms of accreditation are being introduced for providers willing to be included in public registries and this often is linked with public funding. However, the scope and thoroughness of accreditation varies in all of the countries. Public registries are functioning along with numerous private initiatives.
ReferencesDesjardins, R. (2017). Political economy of adult learning systems: Comparative study of strategies, policies and constraints. Bloomsbury publishing.
Markowitsch, J., Dębowski, H. (2022). Education systems and qualifications frameworks, [in:] Tutlys, V., Markowitsch, J., Pavlin, S., Winterton, J. (eds.). Skill Formation in Central and Eastern Europe, Berlin, Germany: Peter Lang Verla. DOI: 10.3726/b19799
Roumell, E. A., & Roessger, K. (2019). Humanistic, Innovative Solutionism: What Role do Data Analytics Play in Developing a More Responsive and More Intelligent Adult and Workforce Education Policy?. In The Educational Intelligent Economy: BIG DATA, Artificial Intelligence, Machine Learning and the Internet of Things in Education (Vol. 38, pp. 127-142). Emerald Publishing Limited.
Soskil, M. (2018). Education in a time of unprecedented change. In Teaching in the Fourth Industrial Revolution (pp. 8-24). Routledge.
Williamson, B. (2017). Big Data in education: The digital future of learning, policy and practice. SAGE Publications.
23. Policy Studies and Politics of Education
Paper
Beyond global discourses of data: A cross comparative analysis of schooling data in Australia, Bangladesh, England and Singapore
Louise Phillips1, Ian Hardy1, Obaid Hamid1, Vicente Reyes2
1University of Queensland, Australia; 2University of Bristol, UK
Presenting Author: Phillips, Louise
Schools around the world increasingly rely on a range of different modes of evidence of student learning, often described as ‘data’. In keeping with Pangrazio and Sefton-Green’s (2022) call for increased attention to more local, vernacular responses to datafication processes, this paper draws upon perspectives of students, teachers and school-based administrators as they are influenced by global school data trends and seek to be more responsive to engagement with myriad forms of data. We argue there is a need to ensure meaningful aspects of education do not become marginalised. This is always a risk because numbers convey a sense of ‘objectivity’ and ‘authority’ (Desrosières, 1998), and can be difficult to challenge such perceptions, especially in meritocratic contexts in which numbers are focal measures. These pressures to focus upon numeric conceptions of data are central to the quantification of social processes more broadly (Mau, 2019), which are becoming an increasingly constitutive part of social life. However, more dominant, quantified forms are not the only forms of data that exist in school settings and responses to such data are not simply passive.
After more than 20 years of international large-scale assessment (PISA, TIMSS, PIRLS), and national education policies and practices that centre standardised assessment, we seek alternatives. We present from our four-year study which critiques the effects of standardized test data, and associated processes of the quantification of education, on the work and learning of system personnel, teachers and students in schools, and looks for emerging alternative approaches in Australia, Bangladesh, England, and Singapore. Inspired by Collyer, Connell, Maia and Morrell’s (2018) work of creating knowledge beyond ‘northern’ contexts alone, we have deliberately engaged with varied national contexts, including from ostensibly wealthy, ‘northern’/dominant contexts (England), more peripheral ‘northern’ settings (Australia), seemingly ‘successful’ east-Asian settings (Singapore), and ‘southern’ settings (Bangladesh). In this way, we seek to reveal the richness, diversity and plurality of the types of data, and engagement with data that occur in marginalised communities in these settings at a more genuinely ‘global’ scale. Mirroring how high stakes testing at the local, micro-level is then deployed to criticise schools at a more political, macro-level, we focus on students and educators’ stories of critique and engagement at the local level to challenge more reductive accounts that seem to characterize more ‘global’ discourses of data.
Our inquiry seeks to understand:
- the principal forms of qualitative and quantitative data (‘global’, national, local) drawn on in different national and local contexts;
- the multifarious ways in which educators engage with these data and how the current
focus upon data (‘datafication’) impacts on the lived realities of students, teachers and system personnel;
- the role of various modes of data in this work, and how these are mediated by teachers and system educators; and
- how these practices compare with more dominant, ‘global’ perspectives about data use.
We draw upon storying in marginalised settings to make visible how students and educators in schools and systems in varied policy contexts make sense of data at a more genuinely representative ‘global’ scale. We lean into storying, because humans have long ‘read’ the world through stories, and by making visible the experiences of those typically marginalised, it enables accessibility to theorizing beyond the elite and highly educated (Phillips & Bunda, 2018). Storying claims voice in the silenced margins and counters metanarratives, such as “‘monovocal’ stories about the low educational achievement and attainment of students of color” (Solórzano & Yosso, 2002, p. 27).
Methodology, Methods, Research Instruments or Sources UsedWe inquire through storying – that is, the act of making and remaking meaning through
stories (Phillips & Bunda, 2018) – to reveal the meaning-making that students and educators give to learning in the context of increased use of national and international standardized data for accountability purposes. Through storying methodology we have taken time to build relationships with school leaders, teachers and students at four schools in each nation, so that there is trust to share and cocreate stories, by interweaving past and present archives and experiences, through story-sharing, observing and document analysis. Through such a situated relational methodology, we highlight the human, lived experiences of datafication in schools.
The participating schools have been sourced through recommendations from central ministry staff who identified the schools as having a notable approach to data, along with recommendations through personal networks. School leaders have self-selected staff who have significant roles with school data, and students to reflect different age group experiences of data on their learning. At each school we spent one to two days each year to immerse in the culture of the schools and co-produce stories on data in schools through:
(a) conversations with students and educators (individually and focus groups) in a variety of roles (teachers, principals, system/regional personnel) to uncover how they make sense of data and student learning;
(b) observations of classroom practice and environments to develop better insights into how this data sense-making is practised;
(c) immersion in relevant meetings and professional development activities to understand how educators are informed on how engage with school data;
(d) review of systemic and school policies, and associated documents, pertaining to student learning and data, mentioned in conversations and identified in observations and students’ work samples, tests and other documents (e.g., bookwork) to further flesh out our understandings school data in action.
Our research team of four have endeavoured to all attend each site visit and online interview so that our diverse ways of the reading the world are brought to our inquiry. We co-write reports for each school that are a basis for discussion and storying for subsequent visits. Further, we visually story through mapping global and local viewpoints as features of education ‘datascapes’, a notion suggested by Lingard (2021, 3), as a possible addition/extension to Appadurai’s (2001) theoretical ‘scapes’ to arrive at a greater understanding and appreciation of the global historical cultural flows and complexities situated in education datascapes.
Conclusions, Expected Outcomes or FindingsIn Australia, we learnt that there is a growing shift away from emphases on national literacy and numeracy testing data (NAPLAN). Data-rich cultures of wondering-with-data for the sake of better outcomes for students were evident through visualizing data in meaningful ways and collaborative data meaning-making for different stakeholders (i.e., school leaders, teachers, students and parents).
Bangladeshi education has its own system of standardized testing, introduced during British colonial rule and expanded in the postcolonial era. From stakeholders, we heard the welcomed impact of recently introduced reforms to reduce examination pressure and prevent students from relying on after-hours private tutoring while seeking to enhance their wellbeing. These include a new curriculum which emphasizes experiential learning across the classroom, school, and society.
In England, we heard how Ofsted school inspections have increasingly shifted away from predominantly quantitative data measures towards explicit attention to curriculum and how teachers can evidence their students’ learning (Ofsted, 2019). At the same time, schools that are ‘unperforming’/‘inadequate’ against more traditional measures continue to be under pressure to improve and may be allocated to ‘Multi-Academy Trusts’ with which they have little affinity.
Singaporean students are globally known as forerunners on International Large Scale Assessments (ILSAs). The current Singaporean education system agenda has recently shifted to “learn for life” with emphases on values, social and emotional competencies, student well-being and flexibility with subject-based banding (MoE, 2023). However, we heard how broader social pressures (e.g., competition for college places; parental expectations; ‘fear of missing out’) continue to challenge these more holistic and educationally-oriented approaches to student learning.
Across these four nations, we see a growing trend toward more holistic approaches to data on students learning including advocacy for well-being, experiential learning and lifelong learning. However, the legacy of high stakes school performance data continues to exert influence.
ReferencesAppadurai, A. (2001). “Disjuncture and Difference in the Global Cultural Economy.” In Media and Cultural Studies: Key Works, edited by M. Durham, and D. Kellner, 584–603. Blackwell Publishing Ltd.
Collyer, F., Connell, R., Maia, J., & Morrell, R. (2018). Knowledge and global power. Monash University Publishing.
Desrosières, A. (1998). The politics of large numbers: A history of statistical reasoning. Harvard University Press.
Goh, K.S & Education study team (1978). Report on the ministry of education (Goh Report). Singapore.
Lingard, B. (2021). Globalisation and Education. Routledge.
Mau, S. (2019). The metric society: On the quantification of the social. Polity.
Ofsted (2019). Inspecting the curriculum. Revising inspection methodology to support the education inspection framework. https://assets.publishing.service.gov.uk/media/5d1dfeba40f0b609dde41855/Inspecting_the_curriculum.pdf
Pangrazio, L., & Sefton-Green, J. (2022). “Learning to live well with data: Concepts and challenges.” In L. Pangrazio, and J. Sefton-Green (Eds.), Learning to Live with Datafication: Educational Case Studies and Initiatives from Across the World, (pp.1-16). Routledge.
Phillips, L.G. & Bunda, T. (2018). Research through, with and as storying. Routledge.
Ministry of Education (MoE), Singapore (2023, March 1). Learn for Life: Forging Our Collective Future. https://www.moe.gov.sg/news/press-releases/20230301-learn-for-life-forging-our-collective-future
Solorzano, D., & Yosso, T. (2002). Critical race methodology: Counter-storytelling as an analytical framework for education research. Qualitative Inquiry, 8(1), 23-44.
|
