Conference Agenda

Note: Participants should bring a laptop. Lecturers will hand out USB sticks with published PIRLS material for group work. Participants could practice analysis if they have the IEA IDB Analyzer and either SPSS, SAS, or R installed on their laptop.

The primary objective of this workshop is to explore how data from international assessments can be used for research regarding outcomes and contexts of reading literacy. The workshop will put emphasis on how data from studies conducted by the IEA (International Association for the Evaluation of Educational Achievement) could provide further insights for policy and practice.

As a leading organization in the field of educational research for more than 60 years, the IEA promotes capacity building and knowledge sharing to facilitate innovation and foster quality in education. IEA studies approach the reality of educational learning outcomes in all its complexity by collecting a huge variety of background information that can be related to students’ achievement, knowledge, and attitudes.

This course will introduce participants to the IEA Progress in International Reading Literacy Study (PIRLS) 2021. The PRILS 2021 database will be published in June 2023 and will provide a fresh and rich source for secondary research of outcomes related to reading literacy across the world, and in particular in Europe with more than 30 European countries participating. PIRLS 2021 is the 5^th cycle of IEA’s flagship study in reading literacy, following the administrations in 2001, 2006, 2011, and 2016.

The course will include an overview of PIRLS, covering its background, conceptual framework and design. It will present some key findings from the 2021 data collection. Participants will be introduced to the survey instruments and database, and be provided with access paths to data sources, technical documentation, analysis guides and software tools. There will also be a presentation about available variables such as students’ achievement, their attitudes towards reading, characteristics of their teachers who teach reading, and class- and school-level learning contexts.

With this information, participants will formulate and discuss research questions that can be addressed with PIRLS 2021 data. The instructors will be available to mentor the development of research ideas and design as well as to answer data related and technical questions. Research questions from individual attendants will be presented to all participants in order to provide opportunities to share ideas.

No prior knowledge about large-scale international studies is required. Basic knowledge about statistical analysis is not required but is an advantage.

Draft Agenda:

Introductory session – 10 min

• Introduction of participants and their research interests

• IEA – mission, studies, topics, audiences

PIRLS – 20 min

• Introduction

- Background

- Main research focus, framework

- Design

• PIRLS 2021

- Highlighted results

- Instruments, outcome variables and scales

- Access and availability of data files, technical documentation, analysis guides and software tools

Group work – 35 min

• Participants will form working groups

• Each group will receive selected questionnaire materials and information on the corresponding variables (e.g., perceptions and background) of PIRLS 2021

• Participants will develop their own research questions that could be answered with the information collected in PIRLS 2021

• Each group presents one or two research questions and gets feedback from instructors and audience

• If participants have the IEA IDB Analyzer and SPSS, SAS or R installed, they can practice analyzing the PIRLS data with support by the instructors

Example Analysis – 20 min

• Live demo of analysis of example research questions

• Discussion of analysis and results

Closing – 5 min

• Questions, summary & conclusions

• Invitation to advanced data analysis seminars and initiation of collaborative work

The Education Preparedness Index in Covid-19 (EPIC) is one of the outputs of the project Enabling Learning to Happen for All Children in Emergency Crisis. The project is funded by the Global Campus of Human Rights in partnership with the Right Livelihood Award Foundation. In 2019, people all over the world were faced with new realities and adopted new rules of life. One of those realities also affected the education system. Due to the COVID-19 pandemic, many countries in the world initiated emergency remote education applications and platforms to continue providing education without interruption, aiming for students to continue their learning. Under the crisis of COVID-19 remote/distance learning became a viable alternative to ensure the continuity of the educational process. Under these circumstances, the research aimed at studying the preparedness of the education system to adapt to new realities and to act in crisis conditions. To this end, a group of Armenian experts took the initiative to develop a model for assessing the education system in emergencies - Education Preparedness Index in COVID-19 (EPIC). EPIC assessment of preparedness for education in emergencies is a set of indicators, tools, and methods aimed at measuring education system readiness for emergencies and analyzing the effectiveness of education policy responses in times of crisis. The four main thematic areas that the framework covers are as follows:

• Policy and Legal Framework

• Coordination and Cooperation

• E-readiness

• Capacities and Resources

Each thematic area incorporates a set of indicators and sub-indicators that allow uncovering the level of achievements and the efficiency level of preparedness within each recommended section.

EPIC is applicable in all emergencies entailing physical distancing and education through online means. The emergency context was retrieved from the conditions and limitations appeared through COVID-19 period combined with other crises such as war, internal unrest, context of disability and some other characteristics. The framework is flexible to changes of individual country cases and these are specific characteristics are considered during the assessment.

The research question of this study is as follows:

What is the level of preparedness of the country/countries in provision of education in emergency entailing rapid shift from conventional education to distant/ online education?

EPIC basis on the international principles of child rights. Among them the framework of 4As, Availability, Accessibility, Acceptability and Adaptability of Education of the UN Human Rights Office of the High Commissioner is the corner stone of the study. The methodology is based on the UN CRC Commentary N13 description and statements of the 4As framework. Furthermore, the EPIC grounds its target group that primarily on children of 6-14 age group following the CRC General Comments N13, World Declaration on Education for All, as well as on SDGs, the Minimum Standards for Education in Emergencies, the Global Study of Children Deprived of Liberty (Nowak, 2019), Chronic Crises and Early Reconstruction (2004). the Dakar Education for All (EFA) framework, and the Sphere Project’s Humanitarian Charter.

The COVID-19 pandemic has led to near-universal closing of schools at all levels worldwide, remaining negative consequences for all participants in the educational process. School support plays a key role in mitigating the negative effect of school-closure during the pandemic period on student learning achievement. The present study assesses the possible mediating role of school support in the effects of COVID-19 disruption on primaty students learning achievement through analysis of data collected through the IEA’s Progress in International Reading Literacy Study (PIRLS 2021).

In Kazakhstan, as in many countries, regular schooling was disrupted since the COVID-19 pandemic broke out in March 2020. In 2021, 77% of all school students started the new academic year via distance learning. In response to the pandemic, the government of Kazakhstan undertook set of systemic measures: distance learning was implemented using online platforms and services, as well as using audio and telework; computer equipment and Internet cards were presented freely to students in need; professional development courses on distance learning were offered for about 347 thousand teachers.

So far, some studies have pointed to significant losses in students' knowledge in Kazakhstan during school-closures led by the pandemic (IAC, 2020; Dzhaksylykov, 2020). Researchers in the USA found that many children in Year 2 and Year 3 in their study lost momentum on fundamental skills such as reading, with the difficulty in creating a language-rich environment on Zoom being one of the primary reasons (Domingue et al., 2021). In another study conducted in the UK, attainment gaps were found for both Year 1 and Year 2 students, with the most profound effects on students from a disadvantaged background (Rose et al., 2021).

The following research questions will guide the study:

1. How is the school closure during the pandemic period resulted in reading achievement of Kazakhstani primary grade students?
2. Does school support (providing access to digital devices, delivering printed and online learning materials, organizing online activities, providing technical and methodological support for teachers) mediate the relationship between school closure due to COVID-19 and reading achievement?
3. Does the mediation effect of school support vary across different socio-demographic groups?

COVID-19 led to strict lockdown measures, which included school closures in most countries. As a result, more than 1.5 billion students were out of school for weeks or months (UNESCO, 2022). The loss of schooling is expected to negatively impact children's cognitive development, even if distance learning modes are enacted. The loss of in-person teaching could also lead to inequality since the only remaining relevant input is parental involvement during school closures (Agostinelli et al., 2022). Most studies document significant learning loss. In Europe, the average learning loss is almost a quarter of a school year, but the estimates are available mainly for Western European countries (Donnelly and Patrinos, 2021). Worldwide the loss is even greater, especially in lower-income countries (Patrinos et al., 2022). Poland is an interesting case because it represents countries in Eastern Europe where school closures lasted longer, and research on learning loss is scarce.

One of the basic skills for success in the knowledge society is the ability to learn. The Finnish national learning to learn (L2L) assessment program was launched in the mid-1990s as a result of a worldwide interest in the measurement of cross-curricular competences (Hautamäki et al., 2013).

The Learning to learn longitudinal assessment brings significant value and gathers sufficient information about learning outcomes and monitor changes in students’ competence during the basic education (Hoskins & Deakin Crick, 2010).

In Finland, L2L is assessed by administering cognitive tasks measuring general reasoning and thinking skills, and self-evaluation scales measuring beliefs and attitudes towards learning (Hautamäki & al., 2002). The concept of mathematical thinking is a traditional part of Learning to learn assessment.

Children’s learning-related beliefs, self-concept and interest in a particular subject play an important role in their school performance, particularly in mathematics. In educational research, academic self-concept has been defined as students' perception of themselves within the academic environment (Marsh, 1990, Marsh and Scalas, 2010).

In this regard, it is of interest how the development of mathematical thinking occurs in schoolchildren during schooling and what other factors can influence the development and improvement of mathematical thinking.

Of particular interest in presented study was the development of mathematical thinking skills and mathematical self-concept (Marsh et al.,1988), as part of learning-related beliefs from the third grade to the ninth grade during the completion of basic education.

The main purpose of this study is to answer the following questions:

1. How do pupils` mathematical thinking skills and mathematical self-concept develop during the comprehensive school years from the third to the ninth grade?

1. Do mathematical self-concept on the third and the sixth grade predict the level of mathematical thinking skills on the sixth and the ninth grade?
2. How do gender and mother’s education explain the level differences and change of mathematical self-concept?

The study's overall purpose is to explore the formation of student academic self-concept (ASC) in the subjects of English and mathematics. ASC is commonly defined as self-perceived academic ability and is related to cognitive and non-cognitive outcomes such as academic engagement, goal-setting, task choice, persistence and effort, intrinsic motivation, strategy use, academic achievement, and future career selection (Bong & Skaalvik, 2003; Marsh et al., 2019). When students perceive their previous experiences of academic activities to be positive and when they perceive that they are capable of managing future academic activities, it is thus an advantage that goes beyond immediate academic success. Rather, ASC has been shown to have prolonged effects (Marsh et al., 2001).

Because ASC frequently has been shown to be important for student success, much research has been dedicated to explaining how it is formed. The main explanation is the big-fish-little-pond effect (BFLPE), which posits that equally abled students perceive their abilities differently depending on their context (Marsh et al., 2008). A student in a high-achieving context would rate their ability to be lower than a student in a lower-achieving context, even if both students have the same abilities.

In 1962, tracking was introduced in the subjects of English and mathematics in all secondary schools in Sweden (Grades 7-9). With recommendations from teachers, students were to choose between advanced and general courses in the two subjects (Marklund, 1985). The general courses were easier and given at a slower pace than the advanced courses and tended to have lower class-average achievement. Tracking is no longer a formal practice in Swedish compulsory education, but it commonly occurs when teachers organise education in Sweden and internationally (Trautwein et al., 2006).

ASC is a well-researched area, but so far, only a few studies have conducted longitudinal analyses to investigate effects over time. There is also a need for studies that look at how ASC is affected by school systems with some form of tracking (i.e., ability stratification, ability grouping etc.). The specific purpose of the study is to explore the effects of non-tracking and tracking in secondary school on ASC in upper secondary schools. With longitudinal data from the 1980s and 90s, ASC will be measured in Grade 6 (pre-tracking) and Grade 10 (post-tracking).

Previous Research

In a longitudinal study, Marsh et al. (2001) compared students from former East and West Germany (N = 2 778). They found that when East and West Germany reunited and the schools merged, the students who had attended the selective and ability-stratified schools in West Germany were more strongly affected by the negative BFLPE when compared to the East German students. Before the reunification, East German students had not experienced an ability-stratified school system. The difference between the merged students decreased with time when the former East German students became integrated with the more selective school system. Overall, the findings of Marsh et al. (2001) indicate that school policies and systems may have an impact on the formation of student ASC.

Similarly, Liem et al. (2013) found that compulsory school students in low-ability streams in English and mathematics had higher self-concepts than students in high-ability streams when student achievement was controlled for. However, Herrmann et al. (2016) investigated the German within-school track system (N = 1 330) and found that the negative BFLPE for students in the advanced mathematics track disappeared when they controlled for positive assimilation effects. The positive assimilation effect is similar to the basking-in-reflected-glory (BIRG) effect, which both refer to the notion that attending a high-achieving class or school positively affects ASC.

Achievement gaps in mathematics can be found among education systems all over the world in international large-scale assessment studies (ILSAs). In almost all education systems, students’ socioeconomic status (SES) has been documented as one of the most important factors associated with achievement, known as the “socioeconomic achievement gap” (Chmielewski, 2019), while in other education systems, achievement gaps can be accounted for by gender, immigration background, ethnicity and/or urban-rural locations of schools and students (e.g., Bondy et al., 2017; Brozo et al., 2014; Song et al., 2014). In Sweden, remarkable differences can be observed between public and independent schools and the differences might be explained by a larger share of students with well-educated parents in independent schools than in public schools (Klapp Lekholm, 2008). Taking mathematics as an example, students in independent schools perform better than students from public schools in Programme for International Student Assessment (PISA), even after controlling the background variables, and the crucial difference in achievement holds consistent from PISA 2003 to PISA 2012 regardless of the sharp decline, and the advantage of independent schools has emerged over time (OECD, 2019).

The types of schools (private or public as categorised in PISA) are generally differentiated by the ownership of schools. Private schools refer to schools managed directly or indirectly by a non-government organisation (such as a church, trade union, business or other private institution), while public schools are managed by a public education authority, government agency, or governing board appointed by the government or elected by a public franchise (OECD, 2020). In the Swedish context, instead of private schools, it would be more accurate to use the term independent schools, which can be run by private organisations to operate educational activities through a publicly funded voucher system (Yang Hansen & Gustafsson, 2016) and could be running for profit (Wiborg, 2015).

Research also indicates that students’ motivational beliefs seem to be important for academic achievement in the Swedish education system (Klapp, 2018). Previous research has established that student self-beliefs could predict and impact academic achievement, among which self-concept and self-efficacy are the most identified ones (Bong & Skaalvik, 2003; Multon et al., 1991). Mathematics self-concept is an individual’s perceived competence in mathematics (OECD, 2013), and was found strongly related to students’ general mathematics achievement (Bong & Skaalvik, 2003; Ma & Kishor, 1997). Mathematics self-efficacy measures students’ expectations and conviction of what can be accomplished when they need to solve pure and/or applied mathematics tasks. Students’ mathematics self-efficacy had a strong direct effect on mathematics problem-solving despite their general mental ability (Pajares & Kranzler, 1995).

It is well established that mathematics self-concept and self-efficacy to a varying degree are associated with students’ mathematics achievement. It has also been observed for many decades that student gender, socioeconomic status and immigration background influence academic achievement, directly and indirectly (e.g., Bondy et al., 2017; Schleicher, 2006). There is still uncertainty, however, regarding how the relations among mathematics self-concept, self-efficacy, student characteristics (SES, gender, immigrationbackground) and mathematics achievement may vary for students in different types of schools (public or independent) in the Swedish education system and over the years.

The main aim of the study was to investigate the relative importance of student mathematics self-concept and self-efficacy for mathematics achievement across Swedish public and independent schools over time, concerning student characteristics such as SES, gender and immigration background.

Recent studies indicate that Sweden faces issues of decreasing educational equity (Siebecke & Jarl, 2022; Yang Hansen & Gustafsson, 2019), suggesting that the impact of socioeconomic background on achievement has increased. However, some students achieve high despite disadvantages in their socioeconomic background that place them at risk for low achievement. These students are often referred to as academically resilient and yield hope for a more equitable future. In general terms, resilience is grounded in the recognition that individuals’ responses to adversities differ (Rutter, 2012). While some struggle or fail in the face of adversity, others seem to adjust just fine. Those, who demonstrate positive adaptation despite being exposed to adversities, are usually considered resilient (e.g., Masten & Obradovic, 2006). The identification of supportive and risk factors can help socioeconomically disadvantaged individuals in becoming academically successful and, thus, improve educational equity.

Previous studies have indicated that individual and external resources, such as supportive adults and peers (Fergus & Zimmerman, 2005) and a student’s sense of belonging at school (Gonzalez & Padilla, 1997) can promote a student’s resilience. According to a framework by Borgonovi and Pál (2016), these indicators - that is a student’s sense of belonging at school and their relationship with their teachers, parents, and peers – also act as subdimensions of social well-being. This may imply a relationship between academic resilience and social well-being. Yet, research on the (social) well-being of academically resilient students is scarce, especially in Sweden. While the relationship between social well-being and academic resilience is underexplored, previous research does indicate a positive albeit small relationship between well-being and achievement (Bücker et al., 2018; Kaya & Erdem, 2021). However, this relationship is not straightforward and a multidimensional conceptualization of well-being is needed to assess which aspects are particularly important for achievement (Clarke, 2020). In general, well-being is hypothesized to be a multi-dimensional construct consisting of social, physical, and mental/psychological dimensions, which can further be structured in subdimensions (Colombo, 1984). The social dimension of well-being, for instance, can be measured by including subdimensions such as the students’ relationship with peers, parents and teachers and their sense of belonging at school (Borgonovi & Pál, 2016). These subdimensions have been found to be interrelated. For instance, a student’s sense of belonging is closely related to their relationship with peers and teachers (Govorova et al., 2020).

Thus, the main objective of the present study is to investigate whether and how students’ social well-being predicts their academic resilience. The present study focuses on social well-being, as one important dimension of student well-being, and attempts to capture its complexity by not only modeling its’ possible relationship to academic resilience but by also considering the interrelationship between subdimensions of social well-being. The study is anchored in Bronfenbrenner’s ecological theory and specifically focuses on the students’ microsystem, that is their close interaction with their immediate environments, as well as the mesosystem, which describes the interrelation among the environments in which the student participates (Bronfenbrenner, 1979).

Student well-being in school can be considered as a condition that enables positive learning outcomes but also as an outcome of successful learning and students’ satisfaction at school (Morinaj & Hascher, 2022). Students’ well-being in school refers to an emotional experience characterized by the prevalence of positive feelings and cognitions towards school, persons in school and the school context over the negative ones towards school life (Hascher, 2003). According to Hascher (2003), it consists of six dimensions, three positive, i.e., positive attitudes to school, enjoyment in school, and positive academic self-concept, and three negative, i.e., worries in school, physical complaints in school, and social problems in school, that can be used as indicators of well-being.

In Finland, the trends in students’ academic well-being (e.g. Helakorpi & Kivimäki, 2021; Salmela-Aro et al., 2018, 2021) and learning performance (e.g. OECD, 2019; Mullis et al., 2020) have been descending in the last decade. For example, grade 4 students’ performance in mathematics and science has decreased from 2011 to 2019 as evidenced by the Trends in International Mathematics and Science Study (TIMSS) (Mullis et al., 2020). The performance in mathematics declined by 10 points from 2011 to 2015 and by three points from 2015 to 2019. In science, the decrease from 2011 to 2019 was 15 points. When examining the international mathematics and science benchmarks (Mullis et al., 2020), these declines in learning outcomes mean that the percentage of high achievers has dropped from 49% to 42% in mathematics and from 65% to 56% in science during this period. Meanwhile, the percentage of the students below the low international benchmark has grown from 2% to 5% in mathematics and from 1% to 3% in science.

Previous research has detected the interrelation between student well-being and learning performance but also the need for examining this relation with possible associated factors in more detail (e.g. Bücker et al., 2018; Nilsen et al., 2022; Pietarinen et al., 2014). For example, the Programme for International Student Assessment (PISA) shows the relationship between students’ socio-economic status (SES), well-being and achievement (OECD, 2017). Further, the study using TIMSS data by Nilsen, Kaarstein and Lehre (2022) shows that a safe environment, as an aspect of school climate, and student self-concept, both indicating students’ well-being in school, declined from 2015 to 2019 and mediates the changes in mathematics achievement over time in Norway.

The above statements point out that both students’ well-being in school and academic achievements may have declined in the last decade in Finland. Thus, in this study, we ask the following research questions, using the TIMSS fourth grade assessment data:

1) How has students’ well-being in school changed, if any, from 2011 to 2019?

2) What is the relationship between students’ SES, well-being and achievement in mathematics and science?

This panel discussion will be led by representatives of Scotland's Council of Deans of Education with input from Deans / Heads of School of Education from other contexts who will be participating in the conference. Professor Lynn M.Gangone, President of the American Association of Colleges for Teacher Education (AACTE), will contribute to the panel.

As well as setting out the role and contribution of the Scottish Council of Deans of Education, a key focus will be on discussing the challenges and opportunities experienced by those directly involved in leading teacher education faculties in universities. Themes and questions will be framed to encourage audience participation and dialogue, with each other and with the panel. While the current education reform programme in Scotland will be a focus, other areas to be explored will include shared challenges relating to teacher recruitment and retention and research priorities for teacher education.

With the introduction of a mandatory teacher license in 2011, Sweden had an aim to raise teacher quality in Swedish schools (Jarl & Rönnberg, 2019). However, the increasing need for more teachers as well as difficulties to find adequate positions that matched teachers’ certification introduced challenges to fulfil that aim (Hansson and Gustafsson, 2016). The current study investigates how different measures of teacher quality developed in Sweden during the past decade, particularly focusing the distribution of teacher quality across disadvantaged and privileged schools.

The Swedish Education Act 2010 emphasizes equality in education. Pupils should be provided equal education opportunities in terms of access and resources to promote their academic success, and education should work compensatory for those with special needs and disadvantaged backgrounds (Holmlund et al., 2020). The idea of compensatory allocation of resources can be related to the opportunity to learn (OTL) model which address the essential inputs and processes within a school context for students to achieve intended outcomes (Elliott & Bartlett, 2016). While socioeconomic status (SES) has been shown to affect students’ achievement, there is a growing interest among scholars regarding what factors can compensate those with low achievements and disadvantaged home backgrounds (Nilsen et. al., 2020). One aspect of OTL framework may be the access to qualified teachers. However, students with low SES had been shown to have consistently less access to teachers with high qualifications than their more advantaged peers (Luschei & Jeong, 2018; Glassow & Jerrim, 2022).

In a Swedish context, Hansson and Gustafsson (2016) studied how teacher quality was distributed among Swedish schools using teacher and student register data between 1994 and 2011. The study showed a significant variation in the number of qualified teachers, with respect to the student-teacher ratio and the number of teachers whose teaching subjects were not matched with their teacher education among schools. While the schools whose students were eligible for mother tongue language learning, which indicated their immigration background, had a higher teacher density, they had a higher number of unqualified teachers, and teachers teaching subjects were more likely to be different from their subject specialization. The study also showed that teachers with high competence tended to work at schools where students had a high level of academic achievement, and which provided a higher salary and better working environment. Hansson and Gustafsson (2016) argued that results was influenced by decentralization and marketization reforms in 1991, which allowed municipalities and schools greater freedom employing teachers.

In 2011, another teacher education reform was introduced to increase the quality and specialization of teacher education. The reform divided the one common teacher education program of 2001 into separate programs for different subjects and age groups (Åstrand, 2017). In parallel, the reform introduced a teacher license, and the teacher education qualification requirement for employment has been stricter since then (Jarl & Rönnberg, 2019). What the results are of the policy intentions regarding increasing teacher quality and the need for more teachers has not been studied. An especially salient issue is how highly qualified teachers have been distributed across schools in the past decade regarding teacher qualification and subject specialization. Against this background, we stated the following research questions:

1. What is the proportion of teachers with a teacher license during 2013-2020 in Swedish secondary schools?
2. What proportion of teachers have a matched position, i.e., a teacher license relevant to the subjects and age groups they are educated for?
3. Are any differences with respect to the proportion of teachers with a license and matching as regards the distribution across schools (school areas, school types, and student social background)?

The cultural and linguistic diversity of classrooms continues to increase across OECD countries significantly and steadily (OECD, 2020). In addition to the ongoing achievement gap, it has also been found that students from culturally and linguistically diverse backgrounds are experiencing lower levels of belongingness in schools (Cerna et al., 2021). This trend is concerning given that research has demonstrated students’ relatedness to be connected to students’ engagement, academic self-concept, motivation, willingness to seek help from peers, and academic achievement (Flook et al., 2005;Goodenow, 1993; Shim et al., 2013). In addition to individual characteristics, classroom composition has also been shown to affect the well-being and academic outcomes of students (Van Ewijk & Sleegers, 2010). However, results are mixed as to whether these compositional effects on academic achievement are positive (Cho, 2012; Rjosk, 2014) or negative (Van Ewijk & Sleegers, 2010; Rjosk et al., 2017). It is also unclear as to whether all students are affected similarly (Rjosk et al., 2017). Investigations into how culturally and linguistic class composition relates to students’ socioemotional outcomes have similarly produced inconclusive results (Thijs & Verkuten, 2014; Veerman et al., 2022).

It has on the one hand been theorized by Putnam’s (2007) constrict theory that the presence of cultural and linguistic diversity threatens social cohesion and increases social disorganization (Putnam, 2007). Yet, that belongingness appears to be a function of cultural and linguistic distance (Cerna et al., 2021) and orientation towards a majority norm reference group (Veerman et al., 2022) suggests that these in-group out group distinctions may not be so clear cut. Indeed, in a contrast to constrict theory, Allport’s (1954) intergroup contact theory suggests the mixing of various ethnic groups can serve to reduce prejudice and result in greater understanding and empathy among members of varying ethnic groups over time. The extent to which these positive outcomes are realized, however, may be conditional to Allport’s (1954) criteria that group members get adequate opportunity to know each other, share similar status position, are in a situation of collaboration, and are supported by the institution to which a person belongs. Applied to the context of education, both the teacher’s role, as well as a positive cooperative classroom climate, can be seen as central components of Allport’s (1954) criteria.

To assess the extent to which these theories hold within the context of Norwegian elementary classrooms, the present study aims to assess how classroom cultural and linguistic diversity relates to students’ perception of classroom climate and reading achievement over the 2^nd, 3^rd, and 4^th grades and the role of teacher quality on these relations. In doing so, this study aims to address the inconclusiveness of research findings to date, to further expand our understand of these relations beyond the US to the Nordic context, and to contribute the limited body of longitudinal studies on these relations. Guided by Jennings & Greenburg’s Prosocial Classroom Model (2009) and drawing upon Deci & Ryan’s (1985) theory of self-determination, this study therefore aims to address the following research questions:

1) How does cultural and linguistic diversity in the classroom relate to students’ perceptions of classroom climate and reading comprehension over time in Norwegian elementary schools?

2) Does teacher quality moderate relations between classroom cultural and linguistic diversity and students’ perceptions of classroom climate over time?

The Trends in Mathematics and Science Study (TIMSS) study measures students’ competence based on the participating countries’ curricula. Changes in students’ TIMSS achievement over time within a country is hence of interest to educational policy and practice as they may be related to, or possibly could reflect, changes in contextual factors important for student learning.

In Norway, students’ performance in mathematics at grade 9 decreased from 2015 to 2019 as evidenced by TIMSS. Seeing how teachers’ competence and their instruction are the most proximal to students and key to their learning outcome (Baumert et al., 2010; Darling-Hammond, 2000; Klieme et al., 2009; Praetorius et al., 2018), the present study seeks to investigate whether changes in teacher variables may be related to changes in the students’ mathematics.

Teachers’ competence is shaped by their formal level of education, the degree to which they have subject specialization, and through participation in professional development (e.g. Desimone et al., 2013). Teacher competence has proven to be important for teaching quality and for the students’ learning outcome (e.g. Baumert et al., 2010; Jentsch & König, 2022): A competent teacher tends to provide high quality teaching.

Teaching quality reflects the teaching going on in the classroom (i.e. teachers’ behavior) (Praetorius et al., 2018). It is a broad concept including different dimensions that have been found to promote student learning (e.g. Pianta & Hamre, 2009). According to the framework of the Three Basic Dimensions (TBD) (Klieme et al., 2009; Praetorius et al., 2018), teaching quality includes the following three dimensions: 1) Classroom management, which is about arranging for effective learning in the classroom (e.g. managing noise and disruptions). 2) Supportive climate, in which the teacher for example shows interest in and respect for all students, gives (individual) feedback and helps connecting new topics to what has already been learned, and 3) Cognitive activation, which is about challenging the students cognitively (e.g. students are asked to reason, interpret, solve problems).

As the TBD framework captures the main aspects of teaching quality, is extensively used in Europe and in the TIMSS contextual framework (Senden, Nilsen, & Blömeke, 2021; Mullis et al., 2020; Pianta & Hamre, 2009), it is used as the theoretical background for teacher quality in this study.

However, the quality of the teaching also depends on who is taught, on the background and composition of students (Praetorius et al., 2018). The quality of the teaching may be limited by students who for instance lack basic previous knowledge, who are uninterested, or lack basic language skills.

Knowing how important teachers and their teaching are to student learning, it is plausible that changes in teacher competence, their teaching quality and limitations to teaching quality, may be related to changes in student outcome. This study hence seeks to answer the following two research questions:

1. How have teacher competence, teaching quality and limitations to teaching quality changed from 2015 to 2019?

2. What is the relation between the changes in the predictors (i.e. teacher competence, teaching quality and limitations to teaching quality) and the changes in students’ mathematics achievement from 2015 to 2019? In other words, do changes in the predictors mediate the changes in students’ mathematics performance over time?

In the Professional Enquiry and Decision-Making course at the University of Glasgow, part of the MEd in Professional Practice, students write a 1500-word assignment, which they often find challenging. The task is new and complex; their work must be original, and they are often unclear about requirements associated with each assessment criterion. Rubrics, descriptions of what is required, are of limited help. Existing support, where students received peer and tutor feedback prior to assignment submission improves outcomes but incurs high staff workload and does not necessarily foster independence. Therefore, a complementary intervention was devised in which students generated their own feedback (Nicol, 2021).

International studies have shown that feedback is an area of European or even global concern for students even though they can create their own feedback by comparing their work against rubrics, exemplars, or peers’ work (e.g., Lipnevich et al, 2014; Nicol and McCallum, 2022). Indeed, Nicol (2021) has developed a model to explain this in which the core feedback generation mechanism is comparison, thus arguing that capacity-building for self-regulation requires student development of inner feedback capability via explicit comparisons. (Nicol and Selveretnam, 2022).

Prior research gives exemplars before student work to clarify requirements, although recently some have argued for their use after student work: a form of post-production feedback (To, Panadero and Carless, 2021). However, we argue that both modes support self-feedback production. Exemplars can be similar in presentation format and subject topic to the work the student has produced or similar in format but different in topic. With this assignment, the latter enables a focus on writing (e.g., structure, argument) without distraction from content.

Five aspects of the MEd assignment served as focus for feedback improvements: the writing of literature search strategies, literature review, ethics application, research dissemination and limitations in research designs. For each, students: (i) compared exemplars of quality work (different topic/similar format) selected from students in previous years and identified common principles; (ii) produced their own work; and (iii) compared their findings from (1) with own work. The tutor guided students through the first comparison in class with second completed individually out of class (Nicol, 2021).

A key assumption of established models of school effectiveness and improvement is that factors of the teaching quality significantly affect the development of students’ competencies and attitudes (Kyriakides & Creemers, 2008). In particular, the power of feedback as a component of teaching quality has been stressed (Hattie & Timperley, 2007; Lipnevich & Smith, 2018; Seidel & Shavelson, 2007). Drawing on the feedback theory of Hattie and Timperley (2007) three dimensions of feedback quality are distinguished: On the task level feedback informs the learner on their actual state of learning and/or performance. On the process level feedback provides information on the progress students have made toward meeting and gives hints on how to improve. On the self-regulation level feedback encourages students to regulate and evaluate their own learning process. Additionally, Willems and Dreiling (in press) suggested a dialogue related dimension of feedback involving peers as a source of feedback in evaluating students’ performances. Existing studies show that the quality of feedback affects learning outcomes on both cognitive (e.g., achievement) and motivational levels (e.g., intrinsic motivation) (Rakoczy et al., 2008; Wisniewski et al., 2020). The impact of feedback, however, is not necessarily positive which indicates that individual students differ considerably in the ways that they perceive and use the feedback they receive (Wisniewski et al., 2020). In current social constructivist models, the learner is assumed to be an active agent in receiving, perceiving, and processing feedback information (Thurling et al., 2013). Recently, Lipnevich et al. (2016) proposed a student interaction model of feedback that highlights how feedback is received by the learner and how subsequent action on feedback is influenced by the learner’s individual characteristics. Hence, examining students’ perception of feedback and its determinants has been the focus of much recent feedback research (Lipnevich & Lopera-Quendo, 2022; Winstone et al., 2017).

There is also evidence that points to gender differences in the perception and processing of feedback (Chen et al., 2011; Hoya, 2021). Yet, studies on gender differences in perceptions of feedback are limited, primarily because of the lack of existing instruments that measure multiple dimensions of feedback perception (Lipnevich & Lopera-Quendo, 2022). Against this background, we aim to investigate whether boys and girls differ in their perception of feedback in German language classes. We adopt a multidimensional view on feedback (Strijbos et al., 2021) by differentiating simple and elaborated dimensions of feedback quality that influence how the feedback is perceived and used for further learning. In order to make meaningful comparisons of means across gender groups, measurement invariance of the instrument must be established (Millsap, 2011). Thus, the purpose of the current study is threefold. First, we discuss the validation of an instrument to measure multiple dimensions of perceived feedback quality. Second, we examine the measurement invariance of the feedback perception questionnaire across gender and investigate mean differences in the feedback perception scales between gender groups. Third, we explore whether the assumed relations between gender and feedback perception exist even under control of individual performance as well as the students’ intrinsic learning motivation.

Background and theoretical framework

Research in assessment recognize formative feedback as crucial to enhance student learning and achievement (Black et al., 2004; Black & Wiliam, 1998; The Assessment Reform Group, 1999). Feedback can be related to giving information about the gap between actual level of performance and the desired outcome of a learning process, aiming to reduce the gap through formative feedback to the learner (Ramaprasad, 1983; Hattie & Timperley, 2007). Thus, feedback comprises information to the learner about performance as well as information about how to reach the learning aims. This points to an understanding of assessment as both formative and summative and the well-known concepts of assessment for and of learning (Wiliam, 2011). These perspectives have been developed in a comprehensive body of research in assessment internationally and integrated in classroom practice in numerous countries (Baird et al., 2014; Hattie & Timperley, 2007). Research in formative feedback has shown promising result as regards student learning. However, effective formative feedback must be practiced within a teaching design promoting students` use of feedback, develop common understanding of learning aims and provide quality feedback, such as timely and specific feedback, and detailed information about next step in the learning process (Hattie & Timperley, 2007).

In the Norwegian context, formative assessment and assessment for learning have been implemented in the curricula and assessment regulations since 2006, including students` self-assessment. Formative assessment as a concept is an essential part of the assessment regulations, describing when and how to assess formatively. Several national initiatives to support implementation of formative assessment in schools have been developed since 2007 (Norwegian Directorate for Education and Training, 2018). Despite this effort, there is still need for further development of formative assessment to support learning, and the annual student survey reveals decrease in formative feedback the higher in the education system (Norwegian Directorate for Education and Training, 2023). Rambøll Management Consulting (2020) shows lack of enough time to give feedback and that grades overshadow students` awareness of the learning potential in formative feedback.

Practical and pedagogical challenges related to implementation of formative feedback in teaching design call for new tools for working with formative assessment as a resource for learning in classrooms (Heitink et al., 2016) and several studies have investigated use of digital tools as useful for effective formative feedback in education (Henderson & Philip, 2015; Dawson et al., 2019; Mahoney et al., 2019). More specific, several studies show that use of video in formative feedback to students provides new opportunities for quality feedback and demonstrate more timely, detailed and personalized formative feedback which students use in further learning (Dawson et al., 2019; Kay & Bahula, 2020; Mahoney et al., 2019).

The literature review indicates that former research in VF is related to higher education and higher education institutions outside the Nordic countries. Research in VF also focus mostly on VF in language studies and studies where students receive feedback on written text (Bakla, 2017; Mahoney et al., 20219; Kay & Bahula, 2020). To our knowledge, there are few studies of use of video feedback (VF) in formative assessment in upper secondary school and from teachers` perspective. This study therefor aims to investigate upper secondary teachers` experiences with the use of video feedback to enhance students` learning. The study focus on whether VF provides new conditions for formative assessment. The research questions are:

a) How do teachers use video in formative assessment?

b) What kind of advantages and challenges can be identified in using video feedback?

To participate in today’s society, one has to be able to read and write. For a person with intellectual disability (ID), this requirement may be insurmountable and many never become proficient readers (Cawley & Parmar, 1995; Di Blasi et al., 2019; Lemons et al., 2013; Ratz & Lenhard, 2013a; Wei et al., 2011). Thus, we must provide effective reading instructions early. This study aimed to investigate reading development in individuals with ID enrolled in the Swedish compulsory school system for students with ID using a, for the field, methodologically rigorous design. The study focused on the effects of two reading instruction strategies, phonics-based and comprehension-based, on reading development in beginning readers with ID. The Simple View of Reading (Gough & Tunmer, 1986; Hoover & Gough, 1990; Tunmer & Hoover, 2019) describes reading comprehension as the product of word recognition and language comprehension. The model is supported by meticulous research (e.g., Lervåg et al., 2018; Lervåg & Melby-Lervåg, 2012). Word recognition is the process of correctly identifying meaningful units in text and is associated with several different pre-literacy skills, including phonological awareness. Language comprehension refers to the process of binding together multiple lexical units into coherent semantic representations, using contextual, syntactical, and inferred information. Research has found that individuals with ID have poor pre-literacy skills, word recognition, and reading comprehension. The difficulties for a person with ID also include poor executive functions and working memory, and the difficulties often increase with the severity of the ID. There is a large body of research on reading ability in the population without ID, but the research for persons with ID is lagging (Dessemontet et al, 2019). Recent studies found similar variables associated with word recognition and reading comprehension in individuals with ID as for students with a typical development, suggesting that reading manifests similarly in both groups which in turn would indicate that the methods that are known to support reading development in typically developing children should also produce positive effects on reading development in children with ID. Literature has found that phonic-based interventions are effective for teaching literacy to students with mild ID and students with severe cognitive disabilities (Ainsworth et al., 2016; Dessemontet et al., 2019). Ainsworth et al., (2016) investigated teaching phonics to students with autism, intellectual disabilities, and complex communication needs and found that children increased their letter-sound-knowledge. Dessemontet et al., (2021) performed an RCT phonic-based instruction for students with ID and found positive results. Additionally, studies have shown promising results in combining the instruction of phonic-based and comprehension-based strategies for children with ID (e.g., Browder et al., 2012; Gustafson et al., 2007). However, the samples in the studies are often small without using a control group. The current project builds on these studies using a large-scale controlled study, by implementing a comprehension- and a phonic-based approach (and a combination thereof) using digital media. The aim of the current study is to help children with ID in need of AAC to reach as fluent reading capacity as possible using three different intervention strategies: a phonic-based, a comprehension-based, or a combination of the two. In the present study, we tested three pre-registered hypotheses (Palmqvist, Samuelsson, et al., 2020):

1. A phonics-based or a comprehension-based reading strategy improves phonological awareness.
2. A phonics-based or a comprehension-based reading strategy improves reading ability (word recognition and reading comprehension).
3. The combination of both reading strategies is more effective than either strategy on its own.

The hypotheses will be tested on these outcome variables: phonological awareness (1, 3), word recognition (2, 3), and reading comprehension (2, 3).

The opinion of people with intellectual disabilities (ID) and communication difficulties can be challenging to capture. Pictorial support to enable communication, such as the methodological framework Talking Mats^TM has been successful to solicit the views of both adults and young people with ID (Murphy & Cameron, 2008). We believe that a constructionist way of thinking supports the idea of listening to children and trying to understand their thoughts, likes, fears, hopes, and problems, with the goal of forming a partnership. The process of guided participation, as presented by Rogoff (2003), involves children engaging in communication and acquiring knowledge through close collaboration with their peers and surroundings. This process views children as active and capable agents of change, as described by Rogoff (2003), Tomasello (2013) and Vygotsky et al. (1978).

According to the United Nations Convention on the Rights of the Child (UNCRC) and The Convention on the Rights of Persons with Disabilities, (CRPD), all children have the right to express their views and the views shall be given due weight. The children shall be provided with the opportunity to express their views in accordance with their age and maturity. To provide all children to express their views, also those with ID and communication difficulties, environmental support and adaptations are needed.

It is important to carefully consider the challenges that individuals with ID may face when expressing their views and opinions and to use strategies to support their participation and ensure that their perspectives are included and valued. Many individuals with ID and communication difficulties need Augmentative and Alternative Communication (AAC) to understand and to be understood (Beukelman, 2020). Using interviews and questionnaires developed for the general population is often complicated for individuals with ID and communication difficulties (Santoro et al., 2022). The addition of pictures in conversations, interviews, and questionnaires can support cognitive and communication difficulties as well as executive functions and working memory (Boström et al., 2016). Resources for pictorial support facilitate comprehension and supports expression and Talking Mats^TM has been widely used together with people affected by communication difficulties for both research and clinical purposes (i.e. Breeze, 2021; Stans et al., 2019).

Research supporting literacy interventions for individuals with different diagnoses and for all ages has shown promising results indicating that everyone should be given the opportunity to get a literacy education (Yorke et al., 2021). The enjoyment of reading and its positive relationship to reading ability has been well researched among students with typical development (e.g., Rogiers et al., 2020; Smith et al., 2012), but not in children with ID. The conclusion cannot be drawn that the simple fact of enjoying reading leads to increased literacy skills for neither students with typical development nor students with ID. Research in comparing children´s perspective have mostly been focusing on participation and goal setting and in comparison with proxy raters (Stans et al., 2009). The children´s views compared to corresponding test results is sparse.

In the current study, we interviewed students with ID in Swedish special needs schools. They all attended a reading intervention with digital apps and were interviewed before and after the intervention about their own communication and reading. The overall aim was to determine the relationship between students’ own views and their corresponding formal test results on speech sound production and word reading ability.

The study posed two research questions: (1) What are the students' own views of their speech and reading activities? And (2) Is there a positive correlation between the students' views of their speech and their speech sound production, and between their views on reading and their word reading ability?

While research on multilingualism has shown both, positive (e.g. inhibition; Coderre et al., 2013), and negative (e.g. vocabulary; Bialystok et al., 2008) effects on cognition and language proficiency, its influence on scholastic achievement appears to be largely negative (Hoffmann et al., 2018; Martini et al., 2021). Children in Luxembourg are educated in a multilingual educational system. In Kindergarten, the main teaching language is Luxembourgish. This switches to German for literacy acquisition in elementary school, with French taught as a second language. Despite its small size, Luxembourg is also highly multi-cultural, boasting 170 nationalities (The Government of the Grand Dutchy of Luxembourg, 2023). Thus, many of the children in the school system do not speak the language(s) of instruction at home. Data from the Luxembourgish national school monitoring program reveals significant differences in German reading comprehension in grade 3 depending on the language spoken at home. Because Luxembourgish is linguistically close to German, Luxembourgish-speaking children generally perform better than children who do not speak Luxembourgish at home (Hoffmann et al., 2018; Martini et al., 2021).

Furthermore, the language-based differences in children’s scholastic performance complicate the diagnostic process of children with potential learning disorders, such as dyslexia and/or dyscalculia. In Luxembourg, the language in which children are screened and diagnosed for potential learning disorders is usually identical to the main language of instruction at school, which at time of diagnosis (typically grade 3) is German. It is therefore difficult to distinguish poor performance based on potential difficulties with reading/writing or mathematics from poor performance based on low language proficiency in the test language. Furthermore, the diagnostic tools currently employed in Luxembourg are developed in countries with primarily one language of instruction, challenging the validity of the diagnostic process in a multilingual population (Ugen et al., 2021).

We have thus developed a comprehensive reading/writing test battery adapted to the Luxembourgish educational curriculum and multilingual environment. Children’s potential language proficiency differences in the test language (German) are taken into account using simplified instructions with reduced language load, multiple examples, varying degrees of difficulty of the test materials, as well as the construction of distinct language-group norms, depending on the language(s) spoken at home. This helps avoid over-diagnosis of reading and writing disorders in children who do not speak the language(s) of instruction at home and underdiagnosis of children who do. The developed test battery assesses children’s performance in key domains relevant for reading and writing comprising phonological skills, (non)word and text reading (fluency and accuracy), reading comprehension, writing, and vocabulary. Furthermore, we link children’s performance in the newly developed test battery to their performance in the Luxembourgish national school monitoring program.

Both IEA and the Swedish National Agency for Education (Skolverket) stresses the importance of reading literacy as they recognize it as an individual right and a prerequisite to participate in society, to gain knowledge of school subject and being able to argue and participate in decision making situations (Skolverket, 2016; Mullis, Martin & Sainsbury, 2015). The definition of reading literacy has evolved over time from the simple view of reading as a decoding and word recognition to involve further skills as comprehension and meta-comprehension (Gough & Tunmer, 1986; Chall, 1989; Sefton-Green, Marsh, Erstad & Flewitt, 2016) and with the entrance of digital means in everyday and school life has grown further to involve digital and social skills as well (Coiro et al., 2008; Leu et al., 2009). The differences between paper-based and digital reading have been under close scrutiny in recent past indicating that the two modalities differ in several aspects besides the obvious format, such as the skills required, the processes needed and the consequences in comprehension and concentration (Delgado et al., 2018; Baron, 2017). Previous research has explored and confirmed the paper-based reading ability as a predictor of later achievement (Butler et al., 1985; Sparks et al., 2013;). Less is known though about the role digital reading ability as a predictor for later achievement. The present study will shed light to the possible effect of difference between digital reading assessment and paper-based reading assessment as a predictor of later reading achievement and general achievement. So far, most studies have focused on the individual factors that may influence achievement in reading. The present study examines the two modalities in comparison along with individual factors, such as SES and immigrant background.

IEA’s PIRLS and ePIRLS both measures reading literacy in paper-based and digital form respectively. In 2016, Swedish 10-year-olds completed both the paper-based and digital version of the reading assessment PIRLS. The correlation between students’ test scores amounted to 0.79, thus a high, yet not a perfect, relationship. The correlation revealed that there likely are differences between the two formats and that some students perform better in paper-based reading whereas others perform better on the digital test. IEA analyzed the differences that emerged between PIRLS and ePIRLS results for all countries. Swedish 10-year-olds were generally better in digital, but the difference was not statistically significant.

The main purpose of the current study is to investigate the predictive validity of the paper-based and digital reading scores in PIRLS by comparing these scores with the core subject marks and the overall marks in grade 6 for the students that participated in PIRLS 2016. A hypothesis is that the paper-based scores will be more closely related to the subject grade and overall grades, partly because paper-based reading was more prominent in schools at this point in time. Drawing from Clinton’s meta-analysis (2019) suggesting that paper-based reading is associated with deeper comprehension and with metacognitive processes (accuracy in prediction of achievement), we hypothesize that students who are relatively stronger in paper-based reading test to have higher marks in schoolyear 6 than students who scored better in digital reading. Against this background, two specific research questions are posed:

1. What is the relationship between paper-based and digital reading achievement in grade 4 and core subject marks and GPA in grade 6?
2. Is there a stronger relationship between grades in Grade 6 and the reading results in PIRLS for students who are relatively stronger in paper-based reading?

As the education agenda of global agencies changed after 2015 to emphasise minimum standards of quality for all countries to be delivered by 2030, the OECD has sought to expand its most successful comparative instrument, the Programme for International Student Assessment (PISA), to include low- and middle-income countries (LMICs). In 2014, it introduced PISA for Development (PISA-D) as the means to establish PISA as a universal measure of learning, and in 2020, it declared PISA-D a success. The most innovative feature of PISA-D was that the assessment included out-of-school youth (OOSY); that task was sub-contracted to Educational Testing Service (ETS). The OOSY population is a geographically dispersed group which present considerable challenges to any researchers seeking to access them. Given this, we ask: who did the OECD assess? More specifically, how did the OECD define the target population of PISA-D out-of-school assessment, what was the sampling frame, and were they accurately represented in the PISA-D OOSY sample?

Much of the existing literature on the OECD influence is based on different theoretical positions. These differences in perspective have real consequences, however, often determining which legitimation dynamics researchers see and which they overlook. In this paper, we seek to adopt a holistic approach by drawing on Suchman’s (1995) framework for analysing the multiple sources of organisational legitimacy and the means by which it is promoted and repaired.

In applying Suchman’s framework, we argue that PISA-D was a macro level exercise designed to legitimate the OECD’s extension of PISA into LMICs and to establish its role in a new arena. The incorporation of OOSY in the assessment was a key micro level endeavour which would allow the OECD to achieve that end and, if not done properly, would challenge aspects of its legitimacy. For example, not assessing sufficient OOSY would debase the quality of the OECD’s products and services; this would also damage the OECD’s moral claims with regard to monitoring the attainment of the SDGs and promoting an inclusive approach. In parallel, at the cognitive level, this would challenge the whole logic of the novelty and value of PISA-D. Overall, the successful identification and assessment of OOSY was vital to ensuring its legitimacy. This would require the OECD to either address the considerable difficulties of accessing OOSY or find a tactical solution which obscured the many challenges to its legitimacy.

Suchman (1995) also analyses how organisations respond to challenges to their legitimacy and identifies three broad approaches: (a) offer normalising accounts; (b) restructure, and (c) don’t panic. He suggested that although legitimacy crises may coalesce around performance issues, most challenges ultimately rest on failures of meaning, where ‘audiences begin to suspect that putatively desirable outputs are hazards, that putatively efficacious procedures are tricks, or that putatively genuine structures are facades’ (1995, 597). Consequently, the initial task in mending a breach of legitimacy usually will be ‘to formulate a normalising account’ that separates the threatening revelation from larger assessments of the organisation as a whole. He identified ‘justifications’ and ‘explanations’ as the two main types of normalising accounts. Suchman also noted that organisations may also re-establish legitimacy through micro-level strategic restructuring, in the sense that ‘narrowly tailored changes that mesh with equally focused normalising accounts can serve as effective damage-containment techniques’ (ibid., 598).

PISA has become the “world’s premier yardstick” against which the “quality, equity and efficiency” of national education systems are evaluated (Gurría in OECD, 2018a, p. 2). PISA claims to be able to compare, on a single scale, the performance of education systems around the globe. These comparative measures have “contributed to the constituting of a global commensurate space of educational performance” (Rizvi & Lingard, 2010, p. 135), regardless of the varying political, economic, social, and cultural contexts of participating nations. Data from PISA are being used by countries to identify “gaps” in their education systems and to develop policies to “move up” on the league tables (Meyer & Benavot, 2013; Wiseman, 2013). A recent development in this space has been the OECD’s development of an internationally comparable measure of Global Competence.

The use of PISA measures as benchmarks for shaping global policies and governing education makes it important to examine the process of how “PISA knowledge” is arrived at. Developing a set of measures requires normative decisions about what the concept encompasses. The assessment of global competence provides a useful example of examining the development of this particular form of global knowledge. Given the multifaceted definitions and understandings of this term, this paper empirically examines the challenges such efforts at stabilising the definition faced, and the ways in which these were negotiated. Locating our study in the interdisciplinary field of Science and Technology Studies (STS), and deploying the concept of tinkering (Knorr Cetina, 1981), we attend to the practices that stabilised the assessment of Global Competence in PISA 2018.

To make globally acceptable knowledge, various epistemic, cultural and political perspectives are brought together in relations of mutual learning and construction, and through iterative processes of expert consultation, country feedback, committee endorsement, etc. These encounters, where diverse perspectives are brought together, have the potential to be hijacked by more outspoken or forceful participants. Moreover, these processes typically take several months during which a range of unexpected events may occur or challenges posed to the successful completion of the endeavour. Tinkering is the way in which actors and events are managed – through cajoling, placating, compromising, modifying, etc. to ensure that the project does not collapse.

Drawing on empirical data relating to the development of the assessment of global competencies, we provide examples tinkering in the development of PISA’s tests of global competency. We highlight three key tinkering moves by the OECD during the process of developing the assessment. In the first move, the OECD replaced the initial Global Competence Expert Group with another group of experts to placate the PISA Governing Board, which objected to the heave economic slant of the first expert presentation. In the second tinkering move, the OECD retrospectively aligned the PISA assessments of global competency with the global competence framework with the UN SDGs. This enabled the OECD to gather more (of the right) allies to support its efforts, and provided “a moral legitimacy the OECD has not enjoyed with the traditional PISA initiative and its narrow economic focus” (Auld & Morris 2019b. p.11). A third tinkering move was the push by the OECD to administer the assessment even when only a minority of the countries decided to participate, arguing that more nations might join subsequent rounds.

International large-scale assessments (ILSAs) are playing an increasingly important role in decision-making and reforms, both nationally and internationally (e.g. Grek, 2009; Lindblad et al., 2018). One of the most influential ILSAs is the Programme for International Student Assessment (PISA). Given the impact PISA has on educational debate and policy, it is crucial that results are trustworthy. Yet, parallel to an increase in the number of ILSAs, there has been growing validity concerns regarding for example the content being tested, the influence on national educational systems and potential bias due to lack of sample representativeness (Grek, 2009; Jerrim, 2021; Meyer & Benavot, 2013). Relatively few studies, however, have focused on whether students are motivated to do their best in ILSAs as compared with high-stakes tests.

A motive for our research is international evidence suggesting that tests of low stakes impact student motivation and effort (Finn, 2015; Wise & DeMars, 2005). Whereas the relation between high- and low-stakes testing has been studied previously, findings are inconsistent, and we know little about this relationship in Sweden. Therefore, the following study examines whether there is evidence for the hypothesis that the lack of personal consequences may bias PISA test scores downwards. Indeed, self-reports from Swedish students indicate that they do not do their best in PISA (Eklöf & Hopfenbeck, 2019). However, PISA test scores have not yet been compared to external criteria such as national test scores. The theoretical framework used to interpret the results of the present study is the expectancy-value theory (Eccles & Wigfield, 2002; Wigfield & Eccles, 2000), postulating that test motivation depends on the student's expectations of succeeding at a particular task, the value the student places on the task, and the interaction between the two (Eccles & Wigfield, 2002). The expectancy-value theory has successfully been used in previous studies to explain the test-taking motivation construct (e.g., Eklöf & Knekta, 2017).

Previous research on the relationship between national tests and PISA/TIMSS revealed moderate to high but imperfect correlations (Skolverket, 2022; Wiberg, 2019; Wiberg & Rolfsman, 2019). One possible explanation is that ILSAs have low stakes for students while national tests have high stakes. In order to test the assumption that motivation influences student achievement, we will examine whether test motivation moderates the relationship between PISA scores and the national test scores. Skolverket (2022) found a correlation of .61 between the two measures but our hypothesis is that the relationship is different for different levels of motivation to take the PISA test. With reference to the expectancy-value theory we assume that the average level of test motivation is higher for national tests since this is a test with higher stakes. For students that were particularly unmotivated to do the PISA test, the correlation with their national test scores could therefore be lower. Consequently, the study examines the following research questions: (1) What is the correlation between test motivation in PISA and PISA achievement? and (2) Is the relationship between low-stakes PISA test scores and high-stakes national test scores moderated by students’ test motivation in PISA?

The effectiveness of initiatives in an educational context is often interpreted in terms of their impact on learning outcomes for every unit of investment. Governments invest a significantly high amount of money and effort in large-scale assessments (LSAs) with the intent to provide data-driven evidence to policymakers and researchers. Such evidence indicates the quality parameters of the education system in terms of learning level, equity, sustainability, and other predefined dimensions. Validity of such information is paramount due to its crucial role in decision-making for inputs, functional strategies, and goal setting for intended outputs, which if implemented as intended are most likely to lead to improvement in learning outcomes. Therefore, it is not an exaggeration to say that the effectiveness of LSAs can also be interpreted in terms of gain in learning outcomes or other dimensions, like any other measure. However, measuring and demonstrating effectiveness of LSAs remains a challenge due to multiple reasons besides the complexities involved in efficacy and effectiveness research, like the notion that assessment data themselves offer solutions. Authors make a compelling argument that data including assessment data do not provide solutions, rather they assist the policy and research community by providing valid evidence and insights enabling informed policy formulation and implementation decisions. Rich data and information provided by large-scale assessments can further be used to analyze the impact of those policy decisions.

The paper consists of three major parts. The first part reviews existing initiatives and proposes a logic model based on reasoning to estimate the effectiveness through evidence and/or counterevidence. A logic model depicts how an initiative is expected to make a difference, using explicit statements of the activities that are likely to bring about the intermediate changes and the impact the initiative intends to make. The proposed model postulates that if evidence generated by LSAs at T1 point of time were utilized to make appropriate modifications in policy and interventions regarding inputs, processes, organizational functioning, governance, monitoring mechanism, and outputs can lead to lead to learning gains per unit of investment at T2 point of time.

The second part builds on policy research and secondary analyses of large-scale assessments conducted in India to generate insights into the policy and practice that emerged from large-scale assessments. While the study primarily uses the assessment data and information from the National Achievement Survey (NAS) and the Annual Status of Education Report (ASER), it makes an effort to corroborate the findings with International and national LSAs in a similar context.

The third part recommends a policy implementation framework consisting of a series of steps to design system-specific strategies and monitor efforts. These steps are organized into two main phases: i) a ‘diagnostic’ phase to identify priority areas or enabling outcomes; ii) an ‘action’ stage to devise, implement and evaluate concrete policy interventions. The diagnostic stage mainly consists of cost-effective action-oriented surveys with a tiered approach, while the action stage consists of evidence-driven developmental goals and action plans for various levels of the system, alignment between all actors involved, customized interventions at school level with continuous monitoring in the cycle of 'assess-act-assess'

Education systems around the world have emphasized the need to transform assessments to improve learning. Proposed framework and model may be vital in designing learning systems to improve learning outcomes through effective systemwide assessments. However, there cannot be a one-size-fits-all policy mix. Feasible policy choices depend on contexts, social preferences, and political constraints. A robust and independent institutional framework, stakeholder engagement, and credible communication strategies are vital to enhancing the effectiveness of LSAs and eliminating learning poverty to achieve sustainable development goals.

High schoolers are less inclined to attend and persist as members of non-formal education (NFE) organizations, despite the increase options at their disposal (Afterschool Alliance, 2020). This development stands in contradistinction to the growing public and academic interest in such frameworks throughout the world, owing to their potential contribution to the lives of young people. More specifically, youth organizations, grassroots associations, and youth councils, inter alia, provide opportunities for teens to connect to positive role models, form social ties, and broaden their repertoire of personal skills (Polson et al., 2013). Voluntary attendance in NFE programs indeed have a positive bearing on teenagers in an array of fields, especially when they take part on a regular and ongoing basis (Fulton, 2019).

However, despite the well-documented benefits of NFE participation for older youth, their participation wanes with age. A drop in attendance raises questions as to the relevance of NFE for older youth and their compatibility for adolescents' developmental needs (Deutsch & Jones, 2013). Correspondingly, data from Israel indicate that there are disparities in the regularity and persistence of participation between adolescent boys and girls. Surveys conducted by the Israel Ministry of Education (National Authority for Measurement and Evaluation, 2015) show that boys tend to dropout at earlier ages than girls in long-term civic-community programs.

Given the declining number of youth, particularly boys, the study's broader objective is to examine how widespread is the phenomenon among NFE frameworks, and how different organizations identify and respond to this demographic change. As such, the study's objectives are: (a) to collect up-to-date data on older youth (16-18) participation and persistence in the past 5 years; (b) to Inquire as to management-level's awareness to the demographic changes; (c) To locate possible explanations for the phenomenon; (d) acquaint ourselves with the institutional attitude towards working with older youth, from a gender-oriented perspective.

Out of all the various NFE frameworks and attendant goals, the present study concentrates on organizations that center around a vision of fostering leadership and active citizenship. These outlets provide youth an opportunity to experiment with decision-making processes, formulate policy, and embrace communal values within a democratic environment (Akiva et al., 2014; Checkoway, 2011). In light of the above, this project falls under the field of civic participation and social activism. Like other scholars, I am interested in the burning social questions that pertain to the downward trend of late-teens’ participation in civic enterprise. My point of departure is that NFE frameworks that allow for experimentation with decision-making, problem-solving, and policy-making on the communal level are bound to ratchet up participation in society down the road.

current proposal for presentation is written in respond to Q (a) collect up-to-date data on older youth (16-18) participation and persistence in the past 5 years; Q (b) Inquiring as to management-level's awareness to the demographic changes. Finishing interviews, I realize that almost all the Interviewees were not aware to the phenomenon of a decrease in the participation of boys and that the number of girls exceeds boys in an appreciable margin. From that, I chose to present in this article preliminary findings revolving around the field of organizational blindness.

In this paper, we propose to discuss the first results of an ongoing research project focused on the analysis of the post-pandemic changes introduced in the Portuguese schools of first cycle (the first four years of schooling – grades one to four) and the second cycle (the next two years – grades five and six) of basic education (ISCED 1).

At the national level, the several research works conducted on the shutdown of schools, with the imposition of confinement and of an emergency remote teaching (e.g., Alves & Cabral, 2020; Benavente et al., 2020; CNE, 2021; Fernandes et al, 2021; IAVE, 2021; Martins, 2020) highlighted as main effects on the education system: 1) the worsening of school inequalities translated, namely, in the differentiated access to working conditions in the family space, to technological equipment, to knowledge and to digital literacy; 2) the loss of learning; 3) the increased risk of dropping out; 4) limitations at the level of the development of emotional and social skills of the students.

Similarly, other international studies (e.g., Bannink & Dam, 2021; Cohen-Fraade & Donahu 2022; König et al., 2020; Mari et al., 2021; MacIntyre et al., 2020; OECD, 2021; Pirone, 2021; Zacanjo et al, 2022), highlighted: 1) the relationship of the pandemic to the unravelling of social and economic inequalities and the worsening of school inequalities, particularly in Southern and Eastern Europe; 2) the effects of "fractured ecologies" (Bannink & Van, 2021, p.2 ) resulting from the dissociation of sharing the same physical space between teachers and students, amplified by computer-mediated communication, in the organization of teaching, curriculum compliance and motivation for learning; 3) the gaps in the digital skills of the teaching staff; and 4) the working conditions of teachers generated by the pandemic, in a perspective of psychological well-being, especially among those with children and with greater difficulties in balancing professional duties and personal and family life.

In the review of the scientific literature published between 2020 and 2022, we found, however, that the research conducted focused either on the analysis of the effects of the pandemic on education, as we have explained, particularly in secondary and higher education, or on the type of changes generated by crisis management plans implemented in schools.

The proposal of this project arises, precisely, from the identification of this gap and intends to answer the following research questions, considering the three central axes of analysis highlighted by this literature review:
Q1. What post-pandemic changes were implemented to mitigate school inequalities?
Q2. What post-pandemic changes can we identify in the working conditions provided to teachers and in the promotion of their well-being and mental health?
Q3. What post-pandemic changes can we identify in supporting the recovery of students' learning, in monitoring and assessing their learning?

In the sociology of knowledge according to Berger and Luckmann, it is assumed that reality is socially constituted and must be renegotiated from one generation to the next (cf. Berger & Luckmann, 1969). The institution of school is not excluded from this process, which becomes apparent with the introduction of new curricula again and again. A few years ago, the curriculum 21 was introduced in the German-speaking cantons of Switzerland, irrespective of historical and confessional boundaries, which contributed to the harmonisation of learning content. Religious education in particular experienced an innovation, which was replaced by the subject "Ethics, Religions, Community" (ERG). Religious education in the singular is now a thing of the past and has been transformed into the teaching of religions. The authors of curriculum 21 take it for granted that children are surrounded by a heterogeneous environment and have to deal with many religious traditions and world views. It is still worthwhile to deal with the Christian traces in society, but a lesson that deals with religions cannot stop at other world views. Adolescents should be introduced to different religions and thus be made capable of tolerance and democracy (Lehrplan 21). In the curriculum of 2003, religious education still focused on the Bible, the knowledge of which seemed indispensable for general education and the children were supposed to get to know the Christian cultural heritage (Lehrplan 2003). Social realities with their institutional knowledge are subject to constant processes of shift (cf. Foucault, 1974, p. 13). These processes of change are particularly noticeable in textbooks and their contents. Drawing on the sociology of knowledge, this means that the content of textbooks can change over time. Whereas in 2000 just over 75 per cent of the Swiss still belonged to the Roman Catholic or Protestant Church, the balance of power has shifted drastically in the last 20 years. Today, just under 54 per cent of Swiss citizens still feel they belong to a traditional Christian church (Bundesamt für Statistik). In 2023, we will encounter a diverse religious landscape in Switzerland, and this presentation is based on the assumption that the diversity of Swiss social relations should also have an impact on the content of religious textbooks. Textbook contents are not random products, but rather sources that are constantly renegotiated by various actors and textbook developers in social-historical discourse (cf. Wiater, 2003).

Using the example of the religious figure called Mary, which is exposed to different interpretive sovereignties beyond interdenominational boundaries and therefore it can be assumed that the perception of this person can manifest itself in different ways in a certain social construct in religious textbooks, this contribution to textbook research aims to show how representations of Mary in image and word in the textbooks of the curriculum 2003 and the curriculum 21 come to light and do or do not do justice to a society of diversity. Thus, this paper is interested in the following questions:

How is the religious figure of Mary portrayed in the textbooks of curriculum 2003 and curriculum 21? Do they do justice to a heterogeneous Swiss society? Could there have been shifts in the representations of Mary in the period between 2003 and today? And if so, what social events might have contributed to this?

Since a pluralistic society changes the interrelationships among the various religious institutions and promotes ecumenical as well as interreligious exchange (cf. Berger, 2014, p. 48), it can be assumed that the representations of Mary must also be affected by this fact.

Globalisation, socio-political shifts and increasing diversification of religious beliefs and practices present challenges for schools around the world. This is a time of transition and school communities face challenges between traditional and new ways of understanding. Teachers are at the interface of this change, including how their personal identities fit within professional environments. This paper presents an investigation into nonreligious teachers' experiences in traditionally religious schools in the Republic of Ireland (RoI) and Northern Ireland. In the RoI and, until recently, in Northern Ireland (NI) schools with a religious ethos were exempt from employment equality legislation in relation to religion (NI Fair Employment and Treatment Order 1998; Irish Employment Equality Act 1998-2011 Section 37 (1)). Historically this has been justified on religious grounds and the right of religious schools to appoint teachers who share their beliefs. Over time, populations on both sides of the border have become more religiously diverse and there has been a significant rise in the number of people with no religious belief. Some schools have responded to this increasing plurality by changing how they describe their stated ethos, this has resulted in further uncertainty around what counts as a religious school and raises questions regarding the applicability of exemptions from equality legislation for all schools on the island. Furthermore, the continued use of exemptions from equality legislation in RoI would appear to be overly generous in comparison to other European states. The research question, therefore, was as follows:

To what extent is religion or belief a factor in the appointment or promotion of non-religious teachers in Post-Primary schools with a religious ethos on the island of Ireland?

The epistemology underpinning this research is social constructivism. Theories of identity and teacher agency, particularly ecological agency (Priestly et al. 2015), underpin the analysis of the findings.

The research methodology was qualitative and the researchers undertook semi-structured interviews with fifteen non-religious post-primary teachers. Thematic analysis supported by NVivo 10 computer software was used to analyse the data.

The key findings are that religion or belief was a factor in the appointments of all the teachers to varying degrees. In schools managed by Catholic authorities, candidates’ beliefs were explicitly taken into consideration. While in other schools, that hold religious values, implicit religious influences were at play in teacher appointments. It was also found that temporary contracts and probation periods meant teachers were subjected to a protracted assessment of their suitability for posts, including their ‘fit’ with a school's religious ethos. The majority of the participants felt a need to suppress their non-religious identity and conform to the schools’ religious culture, causing identity dissonance and personal ethical conflicts.

This paper is part of a mixed methods research project on the factors in and out of schools that shape young people’s expression on race and faith equality issues. Focusing here on policy discourse, the paper presents an analysis of two key questions: (1) how liberal political concepts including those of freedom of expression may be mobilised in education policy to facilitate wider right-wing political goals, and (2) how education policy in this context shapes young people’s ‘free’ public political expression - in particular on race and faith equality issues - at school. The international significance of this paper lies in its analysis of how education policy discourse is aligned with the revival of freedom of expression as a topic of right-wing political concern across the global north. This revival, it has been argued, seeks to undermine fragile race and faith equality progress through the translation of narrow ‘free speech’ and ‘cancel culture’ claims into policy and political discourse (Mondon and Winter 2020; Titley 2020).

Our focus is on education policy and politics in contemporary English education, where government figures have defined the anti-racist organising of movements such as Black Lives Matter as ‘cancelling’ freedom of expression in higher education, and as creating risks for impartiality on the teaching of equality in schools (Trilling 2020). Political figures in the US, France and Australia have made similar claims (Goldberg 2021), advancing an “antagonistic vision” of “who constitutes the public and what values should guide public discourse” (Titley 2020: 3). However, education policy texts are typically more politically measured, and freedom of expression is a far more complex phenomenon than binary notions of ‘free speech’ and ‘cancellation’ put forward in such political discourse allows. For example, in the context of curriculum-making, mundane processes of foreclosing what is not/cannot be taught, processes of editing, and the pursuit of efficiencies and profit all play a role in shaping what can be thought, said and felt in education contexts (Mondal 2018).

The paper analyses 80 education policy texts in the English and UK policy context with a view to unearthing not just how freedom of expression is directly defined in such texts, but to identifying the ways education policy contributes to the political, cultural and affective environment that makes certain kinds of expression possible for young people. Education policy has long been theorised in terms of discourse, i.e., a body of ideas, concepts and beliefs established as knowledge or truth, framing “what can be said, and thought, but also… who can speak, when, where, and with what authority” (Ball 1993, 14). The paper draws on this theoretical tradition to understand ‘freedom’ as existing in a complex, contextual relationship to power/constraint, rather than being its simple opposite. As notions of disciplinary power and subjectivation arising from Foucault (1975) and Butler (1990) indicate, a focus on discourse helps us see the performative, i.e., normalising power of discourse in shaping the possibilities of everyday youth expression (Youdell 2006), alongside more commonly understood juridical/legal forms of constraint on expression (e.g. hate speech).

As such, a key analytic goal in this paper is to identify what kinds of subject positions and thus, possibilities for expression, are made available to young people through education policy texts. But freedom of expression, and questions of race and faith equality involve political passions (Youdell 2011). Therefore, drawing on affect theories, we seek to analyse how the possibility of young people and their political expression on race and faith equality becoming a particular subject and object of feeling is also created/closed down through policy discourse (Ahmed 2004; Kitching et al. 2015).

Without doubt, everyday assessment and responses to student performance are central facets of school quality and an important field of innovations in school. The development of a “new” and formative performance resp. assessment culture, which is suitable for initiating and supporting the acquisition and development of both subject-specific and interdisciplinary/generic competences of students (e.g. self-regulated learning, social competences), seems to be a necessary condition for the development of teaching and learning in general (cf. MfSW NRW 2009; Beutel et al. 2017; Wiliam 2018). In this context, the term “culture” refers to a fundamental change of assessment practice that is not limited to a selective use of some additional or alternative diagnostic instruments by only a few teachers (cf. Jürgens & Diekmann 2006; Box 2019, 42/143). According to Sacher (2014, 264) the development of a new resp. formative performance and assessment culture in schools will only succeed, if it is based on a jointly formulated assessment concept in which the teaching staff fixes objectives, guiding principles and concrete agreements on assessment practice. Such an assessment concept – Sacher points out – must be implemented, regularly evaluated, discussed, and revised as an essential part of the school program. In which way, to what extent and how successful individual schools undertake efforts regarding the requested change and development of performance and assessment culture(s) (cf. e.g. Winter 2012) has hardly been empirically investigated (in Germany). Most of the findings on performance and assessment culture and associated innovations relate to schools that can be described as “extraordinary”. Extraordinary in the sense that they, for example, have been nominated for the (nationwide) German School Award (cf. Porsch et al. 2014, Beutel & Pant 2020) or have a special pedagogical profile (e.g. Montessori, cf. Diekmann 2018). There are almost no empirical findings that give a broader impression of focal points, achievements, or school form/grade specific characteristics of the change in performance/assessment culture at “ordinary” schools. One exception are the findings obtained in the context of an external evaluation (“Qualitätsanalyse”) of schools in North Rhine-Westphalia (federal state of Germany). During this evaluation, various methods were used to gain a comprehensive impression of the work and quality of schools. Among other things, classroom observations were conducted, and so-called school portfolios were reviewed. The school portfolios contained various documents specific to individual schools, such as school programs. A mandatory component of the school portfolios were the performance concepts developed by the individual schools. In summary the performance resp. assessment concepts schools had to submit during the evaluation are characterized as unsatisfying and in need of development (cf. MfSW NRW 2009, 34). Unfortunately, this conclusion is not really explained in detail. Regarding specific differences between school types and levels, only a few findings are reported. For example, it is pointed out that the performance/ assessment concepts at secondary schools are comparatively subject-specific (compared to the performance/ assessment concepts at primary schools). In contrast, performance/assessment concepts at primary schools apparently prove to be more elaborate about the formative use of individual diagnostics. (cf. MfSW NRW 2016, 30-32). The following questions arise from this:
Research question 1: How are performance/assessment concepts designed in terms of scope and content? Are there any school level/form-specific priorities or features?
Research question 2: Are performance/assessment concepts embedded in a whole school approach of school development?

Based on the policy enactment perspective (Ball et al., 2011), this study investigates how primary citizenship teachers do assessment policies in their practice and discusses its influencing factors. While the citizenship programme remains non-statutory at the primary level in many countries, such as England (Richardson, 2010), the programme is mandatory in China due to the emphasis on fostering socialist identity and moral cultivation. The programme standards mandate an assessment approach that focuses on students' 'values' and 'process performance' (Ministry of Education, 2022, p.49) rather than test scores. Advocating 'assessment for learning' (ibid., p. 50), the standards call for a greater emphasis on formative assessments. To ensure the full implementation of national education policies (Lu et al., 2018, p.113), China has established an internal agency system - the System of Pedagogical Research Officer. Primary citizenship programme pedagogical research officers are appointed by the district education authorities and have administrative powers. As intermediaries, they hold the authority and responsibility to interpret, translate, and organise citizenship assessment policies in practice.
Contrasting with the popular perception in China that relegates teachers to the role of policy implementers, scholars (Braun, et al., 2011; Ball, 2011) acknowledge teachers as policy enactors. As Ball (1994, p. 19) asserts, policies do not typically provide a set course of action, but rather create situations where the choices for what to do are limited or altered, or specific aims or outcomes are established. The majority of educational policies depend on their realisation through teaching, positioning teachers not merely as implementers, but as interpreters and 'translators' of policy (Perryman et al., 2017, p.745). This act of 'translation' suggests that while teachers adhere to policy, they also make adaptive modifications.
The study reveals that all schools employ standardised tests—developed by the district's education authorities—as summative assessments for students from Year 3 onwards, despite the absence of such tests for Year 1 and 2. Notwithstanding the stipulation in assessment policies that 'assessment results should be graded rather than scored' (Ministry of Education, 2022, p. 52), test results are ultimately rendered in the form of scores. Formative assessment post Year 3 is notably sparse, predominantly consisting of verbal feedback within classes. This is the case despite a unanimous acknowledgment that the curriculum standards advocate against basing judgments of students’ learning performance solely on test results.
Teachers perceive this disjunction—being aware of but not adhering to the assessment policies—as an outcome of the 'internal disintegration of the policies', a consequence of intermediary influences. In a manner akin to the role of medieval bishops interpreting the Bible, the pedagogical research officer wields unassailable authority within their community to interpret and translate official assessment policies. Their instructions and guidelines are considered the truly applicable policies, while the national assessment policies are often disregarded as overly 'idealistic' and 'abstract'. Furthermore, formative assessment is decried as a privilege available only to economically developed regions, which have the financial means to engage national and international experts for knowledge dissemination and practical guidance. Teachers also face considerable pressure from parents. As primary schools increasingly serve as childcare providers, teachers interact more directly and frequently with parents, many of whom express scepticism towards formative assessment due to its absence of tangible scores and rankings. The teachers identify the prevailing culture of competition or the 'rat race' endemic in nowadays China as the root of these challenges. As the country’s economy slows, societal pressure to compete escalates, underscoring the view that ‘excellence is not the sole goal, but more importantly, to be better than others'.

The use of scores from self-report measures are very common in several areas of research. Since those instruments provide researchers to measure some psychological constructs such as personality, attitudes, beliefs, emotions of too many respondents in a short time, they are preferred widely for data collection process (Alarcon & Lee, 2022; Curran, 2015; Ulitzsch et al., 2022). However, some important problems may occur when responders do not give their best effort to select the response correctly reflecting themselves (which is very common especially with the unmotivated responders) (Rios & Soland, 2021; Schroeders et. al., 2022). Individuals may respond to items without reading them, by misinterpreting them or be unmotivated to think about (Huang et al., 2012; Ward & Meade, 2022). This type of responding behaviour have been stated as random (Beach, 1989), careless (Meade & Craig, 2012), insufficient effort (Huang et al., 2012), disengaged responding (Soland et al., 2019) in the literature. In the context of this study, the term ‘careless responding’ with CR abbreviation is preferred. Careless responding (CR) behaviour is a major concern based on the data taken from self-report scales in any type of research (Meade & Craig, 2012). Even the amount is small, it may affect the data quality and results of the study severely. Careless responses may introduce a measurement error, weaken the relationship between variables and inflate the Type II error. It may also introduce a new source of construct-irrelevant variance to the process and end up with an undesirable effect on psychometric properties of the scale (item difficulty, average scores, test reliability, factor structure etc.). Briefly, CR has the potential to weaken the test scores’ validity in different ways (Beck et. al., 2019; Rios & Soland, 2021).

Considering the factors stated above, CR have become an important and interesting research topic for researchers with a growing interest. One of the most important aspects on CR research is the way how we can detect and cope with them to ensure the quality of survey data. Identifying careless responders and removing them from the dataset is one of the suggested ways to increase data quality. In the literature, there are several data screening methods mainly classified in two groups; priori and post-hoc. Priori methods are the ones that are planned and incorporated into data collection process before the administration of survey. On the contrary, post-hoc methods get involved in the process after data collection. They are implemented on the collected dataset and typically based on a statistical calculation.

While there are several studies focusing on the effect of careless responding on datasets and comparison of the efficacy of CR identification methods, there is still no clear answer about the detection accuracy of CR identification methods (Goldammer et al., 2020). Besides, this study will focus on prior methods that have been studied and focused less on previous studies.

The present study will handle three different ways of prior methods (instructed response items, reverse items and self-report items) which will be explained in method part in detail. In the context of this study, these three ways of CR identification will be used, examinees will be removed from dataset according to those methods separately and their effects on psychometric properties of data will be investigated. This study addresses the following research questions;

- How was the distribution of careless responders with respect to three different CR identification methods?

- How did psychometric properties of the data (scale mean, reliability, correlation between factors, factor structure etc.) change when careless respondents were removed from the data with respect to different CR identification methods?

How can we activate and create assessment systems that lead to a flourishing school where everyone is able to fulfil their potential and achieve both success and well-being? How might we shift assessment practices toward equity and justice/fairness? How do the assessment methods meet the diversity of the students? The research project SHIFT (Shaping and Inspiring a Fair Thinking in assessment) aims to investigate how a range of emerging trends within the international community can be used to answer these questions. These trends concern the literature on: (1) human capabilities (Sen, 1999) as a framework for ‘social justice’, (2) Assessment for Learning (Swaffield, 2011) as the horizon for understanding assessment, (3) Universal Design for Assessment (CAST, 2011) as the philosophy that attempts to go beyond the ‘model of adjustment’ and (4) such approaches as: fair and equitable assessment (Tierney, 2013; Montenegro & Jankowski, 2020), culturally-responsive assessment (Nortvedt et al., 2020), inclusive and universal assessment (Waterfield & West, 2006; Nieminen, 2022; Tai et al., 2021).

An increased focus on equity and justice emerges from the 2030 Agenda for Sustainable Development, where the commitment is to provide inclusive and equitable quality education at all levels, as well as from other European and international documents (OECD, 2012, 2005; UNESCO, 2015, 2022). The same concern is evident in empirical studies focused mainly in higher education contexts (Nieminen, 2022; Tai et al., 2021), and what has become evident as more and more assessment researchers and practitioners engage with the equity conversation is the desire for considering these issues also in school context. Moreover most assessment research is based on what can be described as a ‘technical perspective’, looking at whether assessment is efficient, reliable, valid, leaving less space for a “humanistic” perspective that highlights assessment to foster learning for human flourishing and for responsibility toward and within society (Swaffield, 2011; Fuller, 2012; Gergen & Gill, 2020; Hadji, 2021).

Dialogue into the paradigms of assessment is of paramount importance if assessment aims at embracing a focus on equity, ethics, and humanization and meeting the challenges of these times. The paradigm shift was initiated many years ago, moving from assessment of learning towards assessment for learning, giving greater attention on the role of learners (opening the way to participatory approaches connecting school and community), on a shift from product to process-focused assessment and on a view of learning as a lifelong process rather than something done to prepare for an exam. Although these changes have been partially incorporated into the debate about educational assessment, work remains to be done to ensure the necessary attention to the issue of diversity among learners. Such an approach would strengthen the value of the shift and enlarge the potential of the assessment process towards the promotion of all students’ learning and growth moving away from a model of adjustments, which makes specific reasonable accommodations for some students towards assessment models that allow all students to fully participate and learn in the most equitable way.

Coherently with the theoretical framework, the research design addresses the importance of engagement, participation and opportunities for access, choosing a community-based approach interconnected with the appreciative one, seeking to produce a new imaginary for approaches to assessment, with implications for both cultures and practices. The aim of the Programme is to connect assessment with justice and equity through a participatory process sustaining the shift towards a fair thinking in assessment.

Mathematics is perceived by many students as a particularly difficult subject, and many tend to experience higher levels of anxiety in relation to mathematics compared to other subjects (Goetz et. al., 2007). At the same time, mathematical competencies are fundamental to several aspects of contemporary society (OECD, 2013). Fostering motivation is therefore important for supporting students who experience difficulties in mathematics, especially since motivation is a driving force for learning mathematics over time (Wigfield et al., 2016). A number of studies have shown a positive relationship between motivation and achievement in mathematics, regardless of theoretical approach (e.g., Kriegbaum et. al., 2018; Prast et. al., 2018). Students who are motivated also tend to engage more in mathematical activities because they find them enjoyable and interesting (Eccles & Wigfield, 2004), and the development of motivation for mathematics during elementary school is related to the choice of mathematics-intensive careers (Musu-Gillette et. al., 2015).

One of the most important theories of motivation for mathematics is the Expectancy Value Theory of Motivation (EVM) proposed by Eccles, Wigfield, and colleagues. According to EVM, motivation is a function of a person's expectancy of success and the value they place on the task. Expectancy of success refers to a person's belief in his/her own ability to to successfully complete a task, and value refers to the importance or relevance of the task to the person's goals or interests. Students with a higher expectancy of success and a higher value placed on mathematics tend to have higher motivation and achievement in mathematics (Wigfield et. al., 2016).

Another theory relevant to motivation is Dweck's (1995) theory of implicit intelligence. The theory states that individuals can have implicit beliefs about the nature of intelligence that can be either fixed or malleable. People with a fixed mindset believe that intelligence is not changeable, whereas people with a growth mindset believe that intelligence can be developed through effort and learning. Students' implicit beliefs about intelligence are related to both academic achievement and motivation (e.g., Song et al., 2022), implying that students with a growth mindset tend to develop several adaptive academic behaviors, such as higher motivation and achievement, than those with a fixed mindset (Yeager and Dweck, 2012).

Wigfield et al. (2004) hypothesized that Dweck's theory of implicit intelligence is related to EVM in that individuals who believe their abilities cannot be improved through effort will not engage in activities they believe they are not very good at. However, few studies have examined how such motivational beliefs are formed in children. Eccles and Wigfield (2020) proposed in their situated expectancy-value theory (SEVM) that beliefs and values are also shaped by social context, such as family, peers, and culture. In a study of how parental beliefs about fixedness of ability affect interactions with their children, Muenks et al. (2015) found that parents with fixed mindsets engaged in more controlling and achievement-oriented behaviors and were less likely to engage in math-related activities with their children. Although few studies have examined how parents' mindset affects their children's motivation, a study by Song et al. (2022) showed that children reported having greater self-reported persistence when their parents had more growth mindset. Xie et al. (2022) also found that parents' mindset indirectly predicted math anxiety through their failure beliefs.

Thus, the present study aims to investigate the role of parents' beliefs about mathematical ability, i.e., their fixed or growth mindset, in fostering student motivation. Specifically, we focus on parents' beliefs of mathematical ability as innate or malleable, and whether and how parents' mindsets affect students' self-concepts about their ability, value, and achievement of mathematics.

Using data from the first two cycles of ICCS in 2009 and 2016, this paper analyses the relationship between expected political engagement and affiliation and engagement with religion as well as attitudes toward the influence of religion in society among lower-secondary students in 10 European countries. It reviews changes over time as well as of associations between indicators of religious attachment among young people with indicators of intended political engagement in the future. The databases provided by ICCS provide an excellent opportunity to investigate the links between religious affiliation and beliefs among young people as motivating factors driving expected individual engagement in society.

Religion has been identified as an important influence on civic participation and engagement (see Pancer, 2015; Putnam, & Campbell, 2010; Verba, Schlozman, & Brady, 1995) and research findings suggest that religious affiliation has an impact on political and social engagement among adults (see Ekström & Kvalem, 2013; Guo, Webb, Abzug, & Peck, 2013; Perks, & Haan, 2011; Verba et al., 1995). Similar observations have also been recently reported based on comparative international surveys across different countries (Pew Research Center, 2019a). It has been argued that religious organizations provide networks focused on political recruitment and motivation while participation in religion encourages adherents to consider features of society (a world view) that they see as desirable (Campbell, 2001; Jones-Correa & Leal, 2001; Putman & Campbell, 2010).

Pancer (2015) presented some evidence that schools and neighborhoods may contribute to both civic engagement and religious formation among adolescents. Vermeer (2010) viewed religious education at schools as a contributor to socializing young people in ways that had civic value while Francis et al. (2015) regarded church attendance and education about religion at school as factors that nurture tolerance in a religiously diverse society. In this sense engagement with religion could also be viewed as an important part of a broader civic engagement.

Research also suggested that, even after controlling for other variables, religious tradition and attendance of religious services tend to be related to indicators of civil participation (Smidt, 1999; Storm, 2015). However, other studies have also reported negative effects of religious affiliation on democratic citizenship as manifested in lower levels of political knowledge and lack of political efficacy among strongly religious people (Scheufele, Nisbet, & Brossard, 2003). Research among US adolescents (Porter, 2013) indicates that moral identity may be positively associated with voluntary service and expressive-political involvement but negatively related to traditional-political involvement. Findings from ICCS showed that lower-secondary students with higher levels of civic knowledge were less likely to endorse religious influence in society (Schulz & Ainley, 2017; Schulz et al., 2018). Results also showed that in most countries students who attended religious services held more positive attitudes towards the desirability of religious influence on society (Schulz et al., 2010 & 2018; Schulz & Ainley, 2017).

The relationship between religious attachment and civic engagement is a phenomenon, which has frequently been highlighted in other studies. This paper provides evidence about changes in religious affiliation and attitudes toward the importance of religion for society between 2009 and 2016. Further, the paper explores how these variables relate to expected participation in the future while considering also the context of the general status of religion in each participating country. Using data from an optional component of the ICCS student questionnaire, this paper investigates the extent to which lower-secondary students from 10 European countries in 2009 and 2016 were attached to a religion, endorsed its influence on society and the extent to which their engagement with religion was related to their expected future participation.

Scientific Inquiry (SI) is one of the overarching goals for science education all over the world (Abd-El-Khalick et al., 2004). An understanding of SI is fundamental to scientific literacy, and involves combining content knowledge, process skills and an understanding of the processes and methods scientists use to generate new knowledge (Lederman et al., 2014). This study focuses on the last mentioned, which is described as learning about scientific inquiry (Hodson, 1996), currently often disused in terms of learning about scientific practices (Osborne, 2014). However, this learning goal is often obscured due to the conflation between SI as a pedagogical strategy and as a content matter (Gyllenpalm & Wickman, 2011; Lunde, et al., 2015). Both teaching and research have generally focused on SI as either a pedagogical strategy to learn science, or on students´ abilities to conduct scientific investigations. One reason for this is the tacit assumption that students automatically learn about scientific inquiry simply by doing inquiry. Yet, this assumption has since long been challenged by a large body of research which demonstrates the need for explicit instruction about scientific inquiry as content knowledge (Lederman et al., 2019). Another problem has been the lack of valid instruments for meaningful assessment of students’ understanding about SI (Lederman et al., 2014). We address these issues by using the VASI-questionnaire (Views About Scientific Inquiry) developed for this purpose and present findings from Sweden in primary-, middle- and secondary school. The data is a subset of a larger international project (see e.g. Lederman et. al., 2019) but we focus the analysis on the progression of students’ knowledge over time in a cross-sectional study design.

In Sweden the science curriculum is specified for the school years 1-3, 4-6, 7-9 and 10-12, and divided into the subjects physics, chemistry and biology from year 1 . Students begin learning about scientific inquiry in all science subjects already from the first year. A progression in students´ knowledge is then expected as the central content related to SI successively becomes more advanced in later school years (The Swedish National Agency for Education, 2022). Despite this focus on learning about SI in the curriculum, explicit teaching about SI seems to be rare in Sweden. Yet, practical activities where students are engaged in some form of scientific inquiry has a long tradition, although these are often used as a pedagogical strategy for other educational goals (Högström et al., 2012, Lunde et al., 2015).

The purpose of this study is to contribute to an increased understanding of students’ views of SI and how this can develop over time in order to better understand how this important topic can be addressed by teachers, curriculum developers, national test designers and text book authors. In particular, the study examines the following question:

What are students’ views about scientific inquiry in Sweden in primary-, middle and upper secondary school?

Large-scale educational studies are an important resource to inform policymakers and the general public about the reach and effectiveness of diverse aspects of educational systems in several countries. Competence assessment in institutional settings (e.g. schools) has been an essential factor to collect valid measurements of cognitive abilities or motivations, for example. In order to conduct the assessment sessions, a significant number of test administrators (TAs) are necessary to supervise and coordinate test groups in the participating schools. The TAs undergo specific training and follow a strict protocol to ensure that competence assessment sessions are standardized and comparable so that student achievement data can be meaningfully collected. The TA characteristics can affect the quality of assessment scores and survey data. Differences in their behavior can originate interviewer effects, systematically impacting the validity and comparability of competence assessment tests. While there has been a recent effort to change competence assessment testing to computer-assisted modes of data collection, there is very little research aimed to uncover whether the training sessions and protocols are effectively delivering the goal of preventing TA effects in the first place.

In this paper, we explore the presence and magnitude of interviewer effects on paper-and-pencil competence assessments for mathematics abilities and survey questions in a German nationally representative longitudinal educational survey (National Educational Panel Study - NEPS). For this purpose, we will replicate the Lüdtke et al. (2007) paper, to date the only empirical investigation of TAs interviewer effects we are aware of. Multilevel analyses for cross-classified data are taken to effect to decompose the variance associated with differences between schools and the variance associated with TAs. The results are of use to improve competence assessment testing procedures, particularly by unveiling whether interviewer training and protocols should be improved and to assess the existence and magnitude of interviewer effects in test assessment sessions under paper and pencil-based modes of data collection.

Causal inference is a crucial topic in empirical education research, as well as in other social sciences (Murnane & Willet 2011). In particular, addressing endogeneity (selection bias caused by unobserved confounders) is arguably the most important issue. However, existing methods in applied research have significant limitations in terms of applicability and policy implications.

First, despite the development of causal inference methods such as panel fixed effects model, difference-in-differences, regression discontinuity design, and instrumental variable regression, the data available for their application are limited. Omitted variable bias is a common problem in observational data, and methods that address this issue have high data requirements. While large-scale survey data used in educational research, such as PISA, TALIS, can provide valuable information, the applicability of causal inference methods is limited or non-existent.

Second, even if these methods could be used, many applied studies are limited to those that assume a linear model or a dichotomous treatment variable. If the true relationship between the outcome variable and treatment variable is nonlinear, the policy implications of the analysis by existing methods are limited or misleading. This is especially relevant in the field of education, where there are many continuous or multi-value discrete treatment variables with nonlinear effects. Class size, school size, years of teacher experience, and teachers’ working hours are typical examples(Jerrim & Sims, 2021; Kraft & Papay, 2014). As the vast amount of past empirical research and accompanying discussion on the educational production function has shown, empirical findings on the nonlinear effects of class size and years of teacher experience will have direct implications for the financial resources available to implement educational policy.

The question is, how can we address challenges like these that we often face?
It is necessary to develop a realistic identification strategy that can address endogeneity and nonlinearity. In this paper, I extend a model-based approach that uses identification via conditional heteroskedasticity (Klein & Vella, 2010) to address the above limitations on causal inference in education research.

Methods using conditional heteroskedasticity are not commonly addressed in applied research, but have been discussed in theoretical literatures. This method models the structure of error terms of equations, and differs from those based on usual design-based identification strategies, but has the significant advantage of having relatively realistic side information requirements for identification. Additionally, this approach can be easily combined with various types of existing regression models, providing more options for empirical research using observational data. I extend the linear model with the novel identification strategy to a semiparametric model (partial linear model) within Bayesian framework and demonstrate the effectiveness of the proposed model using simulated and real data.