Conference Agenda
Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).
|
Daily Overview |
| Session | ||
99 ERC SES 03 I: Assessment for driving impact
Paper Session
| ||
| Presentations | ||
99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper Relational Feedback: Listening to the Experiences of First-Year Students Encountering the “Power” of Teacher Feedback Vrije Universiteit Amsterdam, Netherlands, The Presenting Author:Assessment feedback is widely recognised as central to learning (Black and Wiliam, 1998; Nicol and Macfarlane‐Dick, 2006; Hattie and Timperley, 2007; Shute, 2008). Yet students do not always engage with feedback (Dawson, Carless, and Lee, 2020) or take action on it (Evans, 2013; Postmes et al., 2024; Winstone et al., 2017), rendering teachers’ time investment obsolete. Consequently, a key challenge in feedback research concerns not only what feedback says, but how dialogic feedback moments are intentionally designed to support meaningful engagement. Traditionally, assessment research has conceptualised feedback as information transmitted from teacher to student, an intervention some describe as monological and transactional (Winstone and Boud, 2019). Popularised by Hattie and Timperley (2007), this framing positions feedback as a message whose effectiveness depends on how it is crafted. With the advent of Generative Artificial Intelligence (GenAI), feedback risks becoming further reduced to an input-output model, reinforcing transmission logics in which feedback is treated as data assumed to produce learning gains when consumed (Corbin, Tai and Flenady, 2025; Flenady and Sparrow, 2025). More recently, feedback has been reconceptualised as a process (Winstone and Carless, 2020), informed by humanist epistemologies and focused on students’ perspectives, self-regulation, and feedback recipience. This shift has foregrounded the design of dialogic feedback moments and led to practical recommendations around feedback literacy for students and teachers (Sutton, 2012; Carless and Boud, 2018; Molloy, Boud and Henderson, 2019). However, this approach has also been critiqued for following a logic of equipment (Biesta, 2007; 2010; 2016), reinforcing individualism (Nieminen and Carless, 2022; Gravett and Carless, 2023), and insufficiently accounting for context (Biesta, 2007). As Carless (2020) notes, feedback remains “a contested terrain” (p. 426). Despite their differences, both feedback-as-information and feedback-as-process risk reinforcing performative notions of learning focused on outcomes, often at the expense of broader educational purposes such as socialisation and subjectification (Biesta, 2020; 2022). Within these paradigms, feedback is rarely questioned; disengagement or rejection is framed as maladaptive (Boud and Molloy, 2013), reinforcing measurement cultures and the entanglement of feedback with grades (Winstone and Boud, 2022). While feedback can be transformative, engaging with it is neither neutral nor context-free. Drawing on the stories of eight first-year students, my doctoral research shows that feedback inter- and intra-actions (Barad, 2003) are woven into surrounding assessment cultures and must be understood as socially and materially situated (Gravett, 2022). One guiding research question was: “How do assessment cultures influence students’ engagement with feedback?” This study examines how interrelated layers of assessment culture (from macro and chrono levels to micro and nano levels) shape students’ lived experiences of feedback. Using a sociomaterial lens, the study connects everyday student-teacher interactions with the broader forces that configure what feedback is, how it is enacted, and how it is received. Sociomateriality reframes educational practices as relational encounters shaped by human and non-human actors, including time, space, artefacts, power, and personal ecologies (Fenwick, Edwards and Sawchuk, 2011). In this view, feedback engagement unfolds within assemblages rather than residing solely within individual students (Gourlay, 2015). The findings illustrate how assessment cultures shift attention away from individualised explanations of disengagement and toward the conditions under which feedback encounters occur. Students’ engagement is shaped by access to resources such as time, by competing responsibilities, and by whether feedback is experienced as supportive or identity-threatening. Understanding feedback as sociomaterial and culturally situated offers a richer account of how students navigate feedback within complex assessment environments, and why disengagement may sometimes be a rational, situated response rather than a deficit. Methodology, Methods, Research Instruments or Sources Used The onto-epistemological positioning of the research was informed by critical theory, which is absent in feedback research (Nieminen, Bearman and Tai, 2023), and the theoretical framing was inspired by the work of Gravett (2022) in situating feedback literacies as sociomaterial practices. The study adopted a two-stage, qualitative mixed-method design: Stage one involved 16 students completing a narrative frame (Barkhuizen and Wette, 2008) through an anonymous questionnaire, designed to elicit stories of encountering teacher feedback (Passeport, 2023). Stage two involved in-depth semi-structured interviews with a smaller group of eight students from the stage one sample (three domestic, five international) and integrated the use of photo-elicitation to support participants in visualizing their experiences of teacher feedback. This abstract zooms into the data collected stage two (interviews) which was analysed through thematic coding in NVivo, using Interpretative Phenomenological Analysis (IPA) (Smith, Flowers and Larkin, 2022) which facilitated a close, idiographic examination of the data, representing both the individual accounts and shared experiences. Conclusions, Expected Outcomes or Findings Examining how undergraduate students experience teacher feedback revealed the influences of students’ contexts-for-(in)action, moving from a cognitive understanding of feedback as a sense-making process requiring feedback literacy skills, to a situated experience informed by the ‘intra-actions’ of various sociomaterial elements. Through the data, feedback talk (Heron et al., 2021) and dialogic course design (Winstone and Carless, 2020) constitute promising ways to deliberately create a new micro-culture that influence students’ uptake positively and sustainably (Carless et al., 2010), and to resist a transmission culture whereby feedback does not land. The results of the study point towards a shift from the logic of effectiveness, homogeneity, and individualism (Biesta, 2007; 2010b; 2016b; Orón Semper and Blasco, 2018; Gourlay, 2019), and towards more relational and embodied approaches that honour plurality and contexts, for more holistic feedback encounters (McLean, Bond and Nicholson, 2014). Relational pedagogies recognise that student engagement with feedback might be inhibited by factors beyond the control of individuals (Gravett, 2022; Gravett and Carless, 2023), and that students’ act by means of their contexts-for-(in)action. The lens of feedback literacies as relational events (Gravett and Carless, 2023) and sociomateriality (Gravett, 2022) can further support the refining of our understanding of feedback encounters, by situating them socially and materially, intertwined with many elements that enter in relationships with one another. People (students, teachers) are not free-floating but entangled with the material in dynamic and ever-changing contexts (Fenwick, Edwards and Sawchuk, 2011; Gourlay, 2015; Gourlay, 2019; Gravett, 2023; Quinn, 2024). This approach recognises that dialogue is not neutral and that contextual elements matter greatly to the feedback encounter (Gravett, 2022; Gravett and Carless, 2023). References Barad, K. (2003) Posthumanist performativity: toward an understanding of how matter comes to matter. Signs, 28(3), pp. 801–831. Biesta, G.J.J. (2007) Why “what works” won’t work: evidence-based practice and the democratic deficit in educational research. Educational Theory, 57(1), pp. 1–22. Biesta, G.J.J. (2016) Resisting the seduction of the global education measurement industry. Ethics and Education, 10(3), pp. 348–360. Biesta, G.J.J. (2020) Risking ourselves in education: qualification, socialization, and subjectification revisited. Educational Theory, 70(1), pp. 89–104. Black, P. and Wiliam, D. (1998) Inside the black box: raising standards through classroom assessment. Phi Delta Kappan, 80(2), pp. 139–148. Boud, D. and Molloy, E. (2013) Rethinking models of feedback for learning. Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. Carless, D. (2020) Longitudinal perspectives on students’ experiences of feedback. Higher Education Research & Development, 39(3), pp. 425–438. Carless, D. and Boud, D. (2018) The development of student feedback literacy. Assessment & Evaluation in Higher Education, 43(8), pp. 1315–1325. Corbin, T., Tai, J.H.-M. and Flenady, G. (2025) Understanding the place and value of GenAI feedback. Assessment & Evaluation in Higher Education, pp. 1–14. Evans, C. (2013) Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), pp. 70–120. Fenwick, T., Edwards, R. and Sawchuk, P. (2011) Emerging approaches to educational research: tracing the sociomaterial. London: Routledge. Gravett, K. (2022) Feedback literacies as sociomaterial practice. Critical Studies in Education, 63(2), pp. 261–274. Gravett, K. and Carless, D. (2023) Feedback literacy-as-event. Assessment & Evaluation in Higher Education, 49(2), pp. 142–153. Hattie, J. and Timperley, H. (2007) The power of feedback. Review of Educational Research, 77(1), pp. 81–112. Winstone, N.E. and Boud, D. (2019) Exploring cultures of feedback practice. Higher Education Research & Development, 38(2), pp. 411–425. Winstone, N.E. and Carless, D. (2020) Designing effective feedback processes in higher education. London: Routledge. 99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper The Politics of Assessment in Early Childhood Education and Care University of Tampere, Faculty of Education and Culture, Finland Presenting Author:Assessment plays a significant role in contemporary life, permeating all aspects of society. Peter Dahler-Larsen (2012) charactetizes this ubiquity by describing modern society as an ”evaluation society”. This increasing prominence of assessment is evident across various policy domains, including education. Within the field of education, this trend is exemplified by the rise of standardized assessments, which are widely regarded as key indicators of educational effectiveness and quality (Camphuijsen, Møller & Skedsmo, 2021; Verger, Parcerisa & Fontdevila, 2019). The growing emphasis on assessment also extends to Early Childhood Education and Care (ECEC) influencing both national and global policies and affecting the experiences of individual children and educators (Siippainen & Pitkänen, 2024). In Finnish ECEC assessment has traditionally emphasized play rather than children’s educational performance and formal assessment (Karila, 2012). However, recent studies have identified the increasingly prominent role of learning-oriented assessment in Finnish ECEC (Keski-Panula & Niemelä, 2022; Siippainen, Toivonen & Paakkari, 2023). While the emergence of learning-oriented assessment has been acknowledged, its underlying drivers have not been thoroughly examined. To be able to understand the changes the earlier research (e.g. Paananen, Länsmans & Räikkönen, 2025; Paananen & Siippainen, 2025) has identified in the assessment of Finnish ECEC, this study integrates the fields of childhood studies and policy research. From a policy research perspective, this study will first focus on the Multiple Streams approach (MSA). MSA primarily seeks to identify the conditions that allow a policy idea, such as the use of child assessment, to emerge at the right moment, however, over the past two decades, this approach has been modified and applied in a variety of contexts (Moulton & Silverwood, 2018; Santos & Kauko, 2022). The policy-making process consists of three streams: problems, policies and politics, and when these three streams converge, a policy window opens allowing for change (Moulton & Silverwood, 2018). This study aims to understand how the three streams (problems, politics and policies) of MSA have converged to create a policy window for changes in assessment policies and practices within Finnish ECEC. The second part of this study employs the doppelganger approach, a methodological tool for understanding assessment and datause in educational contexts (Pierlejewski, 2024). Earlier research points out that data production is often prioritized over other teaching responsibilities, reshaping professional priorities and contributing to the emergence of data-driven subjectivities among both children and educators (Pierlejewski, 2024). It alters pedagogy by shifting attention from engaging with children to collecting data (Bradbury, 2021). This approach considers child assessment practices as technologies of power that alter subjectivity and change pedagogy. It directs the focus on ”data doubles”, the images of children produced in data, and their performative force, the way in which these partial images of children change ECEC practices. The first research question is based on multiple streams approach with three sub-questions. The second and third questions are informed by doppelganger approach. The research questions are as follows: 1. Which factors influence the adoption of learning-oriented assessment practices? 1A What kinds of constructions of challenges or deficiencies have been used for justifying the need for learning-oriented assessment? 1B What kinds of epistemic authorities shape the development of learning-oriented assessment practices? 1C How have political dynamics and institutional agendas influenced the adoption of learning-oriented assessment policies? 2. What kinds of “data doubles” emerge in current local child assessment practices in early childhood education? 3. To what extend and how does the “data double” function as a technology of power that shapes subjectivity and transforms pedagogy? Methodology, Methods, Research Instruments or Sources Used This doctoral research employs a multiple case study approach to examine assessment policy in ECEC, focusing on five cases. Data for the dissertation will be collected through interviews, document analysis and ethnographic observation. The dissertation will consist of three sub-studies. The first sub-study addresses the first research question. Data for this sub-study will comprise interviews from early childhood education teachers and municipal officials from five different municipalities. The municipalities are selected from a large-scale national survey conducted by VADA-project in 2021, which mapped assessment practices in Finnish ECEC. Ethnographic methods will be employed to answer second and third research questions. Ethnography allows the researcher to gain an in-depth understanding of the social, cultural or organizational context in which a phenomenon occurs (Cohen, Manion & Morrison, 2018). Ethnographic observation data will be connected during field visits to kindergartens located in municipalities where learning-oriented assessment practices are implemented. Preparations for the fieldwork focus on both practical arrangements and ethical considerations. This study adheres to ethical guidelines of the Finnish National Board on Research Integrity (TENK, 2023) as well as European principles of good scientific practice, which include reliability, honesty, respect and responsibility. I am committed to following these principles at all stages of research. A fundamental ethical principle guiding this research is the obligation to avoid causing harm to participants at any stage of the process (Simons, 2009). Given the ethnographic approach, I will continuously reflect critically on my role as a researcher and integrate my observations into an interpretative analysis within a clear and consistent framework (Simons, 2009; Cohen et al., 2018). Conclusions, Expected Outcomes or Findings Based on this study, we can better understand how changes in assessment policy happen. We will better understand which factors influence the adoption of new policies and practices. Throughout the research process, this case in ECEC offers valuable insights to assessment research within the broader field of education. References Bradbury, A. (2021). Ability, Inequality and Post-Pandemic Schools: Rethinking Contemporary Myths of Meritocracy (1st ed.). Bristol University Press. https://doi.org/10.46692/9781447347019 Camphuijsen, M. K., Møller, J., & Skedsmo, G. (2021). Test-based accountability in the Norwegian context: exploring drivers, expectations and strategies. Journal of Education Policy, 36(5), 624–642. https://doi.org/10.1080/02680939.2020.1739337 Cohen, L., Manion, L., & Morrison, K. (Keith R. B.). (2018). Research methods in education (Eighth edition). Routledge. Dahler-Larsen, P., (2012). The Evaluation Society. Stanford University Press. Karila, K. (2012). A Nordic Perspective on Early Childhood Education and Care Policy. European Journal of Education, 47(4), 584–595. https://onlinelibrary.wiley.com/doi/10.1111/ejed.12007 Keski-Panula, H., & Niemelä, E. (2022). Lapsiin kohdistuva arviointi varhaiskasvatuksen paikallisella tasolla: Lapsiryhmään tai vastaajaan liittyvien tekijöiden yhteys demokraattiseen ja performatiiviseen arviointiorientaatioon. Master’s Thesis. Tampere University. Moulton, J. F. G., & Silverwood, J. (2018). On the agenda? The multiple streams of Brexit era UK climate policy. Marmara Journal of European Studies, 26(1), 75–99. Paananen & Siippainen (2025). Johdanto: Varhaiskasvatuksen arviointi datafikaation aikakaudella. In. A. Siippainen & M. Paananen (Eds.), Mitattu Lapsuus – Varhaiskasvatuksen arviointi Suomessa (pp 9–23). Gaudeamus. Paananen, Länsmans, Räikkönen (2025). Kyynisyyttä ja innokkuutta: Arvioinnin monenkirjavat paikalliset käytännöt. In. A. Siippainen & M. Paananen (Eds.), Mitattu Lapsuus – Varhaiskasvatuksen arviointi Suomessa (pp 81–111). Gaudeamus. Pierlejewski, M. (2024). Doppelganger as method: A framework for examining datafication. Contemporary Issues in Early Childhood, 25(3), 322–332. https://doi.org/10.1177/14639491241273927 Santos, Í., & Kauko, J. (2022). Externalisations in the Portuguese parliament: analysing power struggles and (de-)legitimation with Multiple Streams Approach. Journal of Education Policy, 37(3), 399–418. https://doi.org/10.1080/02680939.2020.1784465 Siippainen, A., Toivonen, H., & Paakkari, A. (2023). Toward a democracy of translations? Local evaluation actor networks in Finnish early childhood education. Global Studies of Childhood, 13(3), 217–231. https://doi.org/10.1177/20436106231175026 Siippainen, A. & Pitkänen, H. (2024). On the surface and below: A genealogical look at the waves of evaluation in early childhood education and care. Journal of Education Policy 1–24. https://www.tandfonline.com/doi/full/10.1080/02680939.2024.2344099#abstract Simons, H. (2009). Case study research in practice (1st ed.). SAGE. TENK.(2023). Hyvä tieteellinen käytäntö ja sen loukkausepäilyjen käsitteleminen Suomessa. Tutkimuseettisen neuvottelukunnan HTK-ohje 2023. https://tenk.fi/sites/default/files/2023-03/HTK-ohje_2023.pdf Verger, A., Parcerisa, L., & Fontdevila, C. (2019). The growth and spread of large-scale assessments and test-based accountabilities: a political sociology of global education reforms. Educational Review (Birmingham), 71(1), 5–30. https://doi.org/10.1080/00131911.2019.1522045 99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper Evaluating Junior High School Student Literacy and Numeracy Outcomes under Kurikulum 2013 and Kurikulum Merdeka: A Curriculum Reform Impact Assessment (2021-2025) 1: National Taiwan Normal University, Taiwan; 2: Universitas Pelita Harapan, Indonesia Presenting Author:Over the past decades, countries across the world including Asia have undergone profound social and political transformations. Education systems have been central to these transitions, with large-scale curriculum reforms aiming to redefine "official knowledge" and align schooling with national development (Apple, 2014; Gopinathan, 2007). While traditionally top-down, contemporary reforms increasingly emphasize inclusive stakeholder engagement and teachers' pivotal role in enacting change (Gouëdard et al., 2020). Within this context, curriculum reform has been a persistent feature of Indonesia’s education system. However, student learning outcomes remain a challenge. PISA results consistently place Indonesia below the OECD average (ranking 70/78 in 2018 and 69/80 in 2022). International reports indicate these issues extend beyond curriculum design, pointing to systemic weaknesses in instruction, teacher development, and support for inclusive learning (UNESCO, 2022; World Bank, 2020). In a significant departure, Indonesia has implemented Kurikulum 2013 (K13) and Kurikulum Merdeka (KM) in parallel between 2021 and 2025. K13 emphasizes structured competency development, while KM promotes flexibility, learner agency, and character education through the Pancasila Student Profile Strengthening Project. Empirical evidence comparing their effects on core learning outcomes remains critically limited. Drawing on theories of educational change that frame reform as a multi-level process (Fullan, 2007), this study analyses Indonesia's reform through the phases of initiation, implementation, and institutionalization.
Initiation Reform was initiated through Ministry-led policy reviews under the 2020–2024 Education Strategic Plan (Kemdikbud [MoEC], 2020). Evaluations of K13 revealed persistent challenges: declining PISA performance, an overly dense curriculum, rigid sequencing, and limited instructional flexibility (BSKAP Kemdikbudristek [MoECRT], 2024). Teachers often equated mastery with content completion over conceptual understanding. Notably, the Emergency Curriculum used during the COVID-19 pandemic, emphasizing essential learning, yielded better literacy and numeracy outcomes (numeracy: 517; literacy: 570) than K13 (numeracy: 482; literacy: 532). This success provided a strong rationale for KM, launched in 2021 to simplify the learning load and center on the six dimensions of the Pancasila Student Profile.
Implementation (2021–2025) Consistent with Fullan and Park's (1981) conception of change as shifts in materials, methods, and beliefs, KM differs substantially from K13. While K13 was centrally prescribed, content-heavy, and assessed through high-stakes national examinations to determine the entrance to the higher level of education, KM emphasizes flexible, student-centered learning, project-based pedagogy, differentiated instruction, and low-stakes system-level assessment through the National Assessment (AN) for diagnostic and formative purposes. Administratively, K13 relied on a highly centralized, top-down structure in which schools acted mainly as curriculum implementers. In contrast, KM adopted a more decentralized support model through pilot schools, teacher coaches, and digital infrastructure via the Merdeka Mengajar digital Platform, enabling greater instructional support and flexibility at the school level.
Institutionalization The recent administrative restructuring of the Ministry of Education has reinforced implementation continuity, allowing schools flexibility between K13 and KM. Concurrently, it discontinued the pilot school and pilot teacher programs and reconfigured the national assessment. This assessment is now integrated with the Academic Test Ability for grades 6, 9, and 12, with grade 12 results impacting eligibility for high-achiever university tracks.
Research Gap and Questions Despite this extensive reform process, a critical evidence gap persists. There is no rigorous, national-level analysis comparing the effectiveness of the parallel K13 and KM in achieving literacy and numeracy outcomes. This study directly addresses this gap to inform future policy by answering:
Methodology, Methods, Research Instruments or Sources Used Design This study employs an explanatory sequential mixed-methods design, beginning with a quantitative phase followed by a qualitative phase to deepen the findings (Creswell & Creswell, 2018). This approach integrates the generalizability of quantitative analysis with the contextual depth of qualitative inquiry, enabling a more comprehensive evaluation of curriculum reform impact (Greene et al., 1989). The analysis focuses on student literacy and numeracy outcomes at the school level. Literacy is measured across content (informational, fictional), context (personal, sociocultural, scientific), and cognitive levels (access, interpret, evaluate). Numeracy is measured across domains (number, geometry, algebra, data), similar contexts, and cognitive levels (knowing, applying, reasoning). Quantitative study The quantitative phase utilizes dataset from the Ministry of Education’s (MOE) National Assessment (AN) for 2021–2025. The sample comprises junior high schools under the MOE’s jurisdiction, excluding schools managed by the Ministry of Religious Affairs and those with inconsistent AN participation, resulting in a final sample of N=11,281 schools (2025 data await the MOE’s official release). Schools are categorized as: 1. Early Adopters (KM from 2021) 2. Late Adopters (KM from 2022-2025) 3. Never-Adopters (K13 only) Using a Two-Way Fixed Effects (TWFE) Event Study design (Callaway & Sant’Anna, 2021), separate regression models will estimate the dynamic effect of curriculum adoption on literacy and numeracy. Models control for time-varying school-level aggregates of student socioeconomic status and parental education/occupation. With the Variables as follow: 1. Dependent variable: Student learning outcomes on literacy and numeracy. 2. Independent variables: Dummy variables for each year relative to each school's adoption time (event-time dummies). 3. Time-Varying Control Variables: Student composition characteristics aggregated at the school-year level: a. Student SES b. Mother education c. Father education d. Father occupation Qualitative study The qualitative study will be conducted through document analysis and in-depth semi-structured interviews conducted online, expected to last approximately 45–60 minutes and will be audio- and video-recorded with the participants’ consent as well as researcher realtime field notes to capture participants’ impressions, emphases, and nonverbal cues. Interview data will be analyzed using the six-step Thematic Analysis framework (Braun & Clarke, 2006, 2022): 1. Familiarization with the data 2. Generating initial codes 3. Searching for themes 4. Reviewing themes 5. Defining and naming themes 6. Producing the report. To ensure rigor, methodological triangulation with documents, limited member checking, and a maintained audit trail of coding decisions and reflexive notes will be employed (Creswell & Miller, 2000). Conclusions, Expected Outcomes or Findings This study is significant, as it is expected to contribute in three primary ways. First, it serves as an impact evaluation of the curriculum reform implemented between 2021 and 2025 on junior high school students’ literacy and numeracy outcomes in Indonesia, as measured by the National Assessment (AN). By doing so, it provides empirical evidence on the effectiveness of this reform, thereby contributing to the broader literature on education policy evaluation in developing countries where large-scale reforms are often contested. Second, the study is particularly important because the two curricula under examination, K13 and KM, represent fundamentally different approaches. K13 emphasizes structured competency development with a heavier learning load, whereas KM adopts a more flexible, student-centered model with a lighter load. Evaluating their comparative effectiveness offers critical insights into which pedagogical approach may be more suitable for Indonesia’s diverse educational contexts. Third, the study not only measures reform outcomes but also identifies implementation challenges and contextual factors by incorporating stakeholder interviews. This dual perspective facilitates evidence-based recommendations for policy refinement and targeted interventions, ultimately supporting the goal of improving educational quality and equity in Indonesia. Limitation of the study This study has several limitations that should be acknowledged. First, the concurrent implementation of the Emergency Curriculum alongside KM13 complicates the isolation of the latter’s independent effect on learning outcomes. Methodologically, while the National Assessment (AN) data provide contextual relevance and scale, they offer less international comparability than assessments like PISA. The Two-Way Fixed Effects (TWFE) design also relies on the parallel trends assumption, which may be tenuous in this context. Furthermore, schools under the Ministry of Religious Affairs are excluded due to incomplete data and ambiguous reform implementation. Finally, although qualitative interviews enhance explanatory depth, their interpretive nature means findings are indicative rather than definitive. References Apple, M. W. (2014). Official Knowledge. Routledge. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa Braun, V., & Clarke, V. (2022). Thematic Analysis: A Practical Guide. SAGE Publications Ltd. BSKAP Kemdikbudristek [MoECRT]. (2024). Kajian Akademik Kurikulum Merdeka [Academic Review of the Merdeka Curriculum]. Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed). SAGE. Creswell, J. W., & Miller, D. L. (2000). Determining Validity in Qualitative Inquiry. Theory Into Practice, 39(3), 124–130. https://doi.org/10.1207/s15430421tip3903_2 Fullan, M. (2007). The New Meaning of Educational Change. Teachers College Press. Fullan, M., & Park, P. (1981). Curriculum Implementation: A Resource Booklet. Ontario Inst for Studies in. Gopinathan, S. (2007). Globalisation, the Singapore developmental state and education policy: A thesis revisited. Globalisation, Societies and Education, 5(1), 53–70. https://doi.org/10.1080/14767720601133405 Gouëdard, P., Pont, B., Hyttinen, S., & Huang, P. (2020). Curriculum reform: A literature review to support effective implementation (OECD Education Working Papers, No. 239). OECD Publishing. https://doi.org/10.1787/efe8a48c-en Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a Conceptual Framework for Mixed-Method Evaluation Designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. https://doi.org/10.3102/01623737011003255 Kemdikbud [MoEC]. (2020). Rencana Strategis Kementrian Pendidikan dan Kebudayaan [Ministry of Education and Culture’s Strategic Plan] 2020-2024. Kementrian Pendidikan dan Kebudayaan [Ministry of Education and Culture]. UNESCO. (2022). National Consultations on Transforming Education in Indonesia. UNESCO. https://media.unesco.org/sites/default/files/webform/ed3002/Indonesia_NC_final_Report_0.pdf World Bank. (2020). The Promise of Education in Indonesia. © World Bank. The World Bank. https://documents1.worldbank.org/curated/en/658151605203420126/pdf/The-Promise-of-Education-in-Indonesia.pdf 99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper The Impact of Criteria-Based Assessment on the Motivation of Mainstream Secondary School Students in Kazakhstan 1: University of Bristol, United Kingdom; 2: AEO "Nazarbayev Intellectual Scools" Presenting Author:A major development in Kazakhstani education was the implementation of an updated curriculum for secondary education. This initiative began with a program for gifted students in 2008, leading to the creation of Nazarbayev Intellectual Schools (NIS) in 2009. These schools pioneered modern management to test innovative programs before spreading practices to mainstream schools (Yakavets, 2014). In 2016, the Ministry of Education and Science (MoES) introduced nationwide reforms encompassing curriculum, teaching methods, and assessment. A key component was replacing the traditional five-point grading system with criteria-based assessment (CBA). By the 2021-2022 academic year, the reform affected all grade levels. This research aims to uncover secondary school students’ views on CBA and how it influences their motivation. The introduction of CBA marks a significant shift from traditional systems. It aimed to modernize practices through a transparent and developmental approach. Despite ambitious goals, implementation has faced challenges. Students transitioned from five-point grades to a system emphasizing specific criteria and continuous feedback. This shift was intended to foster an engaging environment. However, early observations suggest mixed reactions, with some experiencing a decline in motivation due to the lack of conventional grades. There is limited empirical research on how CBA affects student motivation in the Kazakhstani context. This study fills that gap by investigating students' views on CBA's effectiveness. The guiding question is: What are mainstream secondary school students’ perspectives on and experiences with criteria-based assessment? CBA evaluates students based on specific criteria provided beforehand, allowing for a focused process. Clear criteria move focus from a teacher-led system to a student-centered approach (Otter, 1995). CBA fosters development through constructive feedback (Carlson et al., 2000). Feedback is crucial, helping students improve and assess academic levels. Assessment serves three purposes: for, as, and of learning. Summative assessment (SA) captures knowledge at a point in time. Formative assessment (FA) identifies needs and gaps to adjust teaching. FA is highly effective in raising standards. Key forms include peer and self-assessment, where students judge their work quality using explicit criteria (Andrade et al., 2010). Motivation is a dynamic process propelling individuals toward action (Pintrich & Schunk, 1996; Dörnyei, 1998a). This study is grounded in Ryan and Deci’s (2000a) Self-Determination Theory (SDT). SDT distinguishes between autonomous (intrinsic) motivation and controlled (extrinsic) motivation. Intrinsic motivation is supported when three basic psychological needs are met: autonomy, competence, and relatedness. Autonomy involves feeling in control of decisions. Relatedness involves a sense of belonging (Martens et al., 2004). Competence involves mastering tasks. Motivation exists on a continuum. External regulation occurs when students are driven by rewards or fear. Introjected regulation involves internal pressures like guilt. FA is a tool for boosting motivation (Cauley & McMillan, 2010). Effective FA helps students internalize mastery goals—focusing on understanding rather than outperforming others. Involving students in goal-setting makes them active participants. Teachers play a key role in choosing assessments that inspire effort and reduce anxiety. FA helps students understand strengths and weaknesses. In Kazakhstan, existing studies focus on theoretical foundations. Little research examines CBA's direct impact on student motivation. Findings indicate students' motivation to prepare has declined. Teachers and parents attribute this to the absence of traditional grades. Some students stopped completing homework without failing grades ("2") or top grades ("5") (Ixanova, 2018). Students also struggled to understand the purpose of FA. Some resorted to dishonest practices for SA, which led teachers to lower scores, often backfiring and diminishing motivation (Turganalina, 2020). Methodology, Methods, Research Instruments or Sources Used This research follows a constructivist approach, acknowledging that reality is shaped by the interactions between the researcher and participants (Denzin & Lincoln, 2005). Given the limited literature on students' perspectives of Criteria-Based Assessment (CBA) in Kazakhstan, a qualitative research design was chosen as it is best suited for exploring the central phenomenon in depth (Creswell, 2014). Specifically, the study employs a phenomenological design to describe the "lived experiences" of the participants regarding the new assessment system (Patton, 2015). The research site comprises mainstream secondary schools in urban southern Kazakhstan, excluding pilot schools to focus on standard implementation contexts. The study involved 20 students selected through homogeneous purposive sampling, a strategy that involves choosing individuals based on shared defining characteristics—in this case, experience with CBA (Creswell, 2014). Snowball sampling was utilized to recruit participants, helping to maintain confidentiality and encourage open responses (Creswell, 2014). Data were collected via individual semi-structured interviews. This method allows for direct, flexible interaction and real-time clarification. Conducting interviews one-on-one fosters a trusting atmosphere, conducive to obtaining genuine responses (Lopez & Whitehead, 2013). The interview protocol consisted of "broad and general" open-ended questions to allow participants to articulate their own understandings (Creswell, 2007). For data analysis, interviews were transcribed and translated into English. The data were coded and organized into themes using NVivo software. This process involved labeling and categorizing data to identify recurring themes (Creswell, 2014) and analyzing these patterns to cluster shared experiences (Cohen, Manion & Morrison, 2007). Ethical considerations were paramount; fully informed consent was obtained from all participants and their parents (Cropley, 2015) , and pseudonyms were used to ensure anonymity. Conclusions, Expected Outcomes or Findings This study demonstrates that the transition to Criteria-Based Assessment in Kazakhstani mainstream schools has generated profound disconnect between the reform's developmental goals and the students' established learning habits. The investigation revealed that the removal of the traditional five-point grading system created a significant "motivational vacuum" for students who had been historically conditioned to rely on external validation. Participants consistently indicated that without the immediate reinforcement of daily marks, they struggled to perceive the value of ongoing classwork, often categorizing ungraded formative tasks as "optional" or inconsequential. This reliance on extrinsic rewards resulted in a notable decline in lesson preparation and homework completion, confirming that the shift in assessment structure disrupted the students' traditional mechanisms for self-regulation. Furthermore, the data highlighted a critical misalignment in how students conceptualized the purpose of assessment. Rather than viewing Formative Assessment (FA) as a tool for "assessment for learning" to guide improvement, students predominantly associated value only with high-stakes Summative Assessments (SA). This perception fostered a paradox where the absence of daily pressure led to disengagement during the learning process, which in turn spiked anxiety during summative periods. In several instances, this pressure to perform without adequate preparation drove students toward academic dishonesty, prioritizing the final grade over content mastery. However, the findings also suggest that this "motivation gap" is not an inherent failure of the CBA model itself, but rather a symptom of its partial implementation. In classrooms where educators successfully substituted grades with high-quality, actionable feedback, students reported a restored sense of competence and autonomy. The study concludes that the implementation of CBA requires more than a structural change to grading criteria; it demands a cultural shift where feedback becomes the primary driver of motivation. Consequently, future policy must prioritize teacher training to effectively bridge the gap between policy intent and student engagement. References Andrade, H. L., Du, Y., & Mycek, K. (2010). Rubric-referenced self-assessment and middle school students' writing. Assessment in Education: Principles, Policy & Practice, 17(2), 199-214. Carlson, T., Macdonald, D., Gorely, T., Hanrahan, S., & Burgess-Limerick, R. (2000). Implementing criterion-referenced assessment within a multi-disciplinary university department. Higher Education Research & Development, 19(1), 103-116. Cauley, K. M., & McMillan, J. H. (2010). Formative Assessment Techniques to Support Student Motivation and Achievement. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 83(1), Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education. New York: Routledge. Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches. Thousand Oaks, CA: Sage Publications. Creswell, J. (2014). Educational research: Planning, conducting, and evaluating quantitative and qualitative research: International edition (4th ed.). Boston: Pearson. Cropley, A. (2015). Introduction to Qualitative Research Methods. Denzin, N. K. & Lincoln, Y. S. (2005) Introduction: The discipline and practice of qualitative research, in: N. K. Denzin & Y. S. Lincoln (Eds) Handbook of qualitative research (3rd edn) (Thousand Oaks, CA, Sage Publications). Dörnyei, Z. (1998a). Motivation in second and foreign language learning. Language teaching, 31(3), 117-135. Ixanova, U. (2018). Fifth and seventh grade mainstream schoolteachers' perceptions of the renewed curriculum: a qualitative single case study in Central Kazakhstan (Unpublished Master's thesis), Nazarbayev University Graduate School of Education, Astana. Lopez, V., & Whitehead, D. (2013). Sampling data and data collection in qualitative research. Nursing and Midwifery Research: Methods and critical appraisal for evidence-based practice, 124-140. Martens, R., Gulikers, J., and Bastiaens, T. (2004). The impact of intrinsic motivation on e-learning in authentic computer tasks. J. Comput. Assist. Learn. 20, 368-376. Otter, S. (1995). Assessing competence: the experience of the enterprise in higher education initiative. Assessing competence in higher education. London: Kogan Page, 43-64. Patton, M. Q. (2015). Qualitative research & evaluation methods (4th ed.). Thousand Oaks, CA: Sage. Pintrich, P. R., & Schunk, D. H. (1996). Motivation in education: theory, research, and applications. Englewood Cliffs, NJ: Prentice Hall. Ryan, R., & Deci, E. (2000a). Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp. Educ. Psychol. 25, 54-67. Turganalina, Z. (2020) Criteria-Based Assessment in Kazakhstani Mainstream Schools: Teachers' Perspectives and Experiences, Nazarbayev University Graduate School of Education, Astana. Yakavets, N (2014) Educational Reform in Kazakhstan: The First Decade of Independence. In Educational reform and internationalization: The case of school reform in Kazakhstan, Cambridge: Cambridge University Press. | ||