Conference Agenda

.

This workshop discusses the methodological framework of recent empirical studies (Marsh et al., 2009; Televantou et al., 2015) that have addressed the impact of correcting for measurement error in student-level measures (i.e., student achievement) on compositional effects’ estimates. A compositional effect is revealed when students’ outcomes are associated with the aggregated characteristics of their peers in the school or the classroom, after controlling for pre-existing differences at the student level. Research findings often support what is taken to be the conventional wisdom, suggesting a positive, but weak effect of class- or school-aggregated achievement on students’ academic outcomes. Thus, for example, they suggest a positive association between the peers’ average achievement and a student’s academic achievement. Still, there is remarkably little agreement on this matter.
The present workshop begins by explaining how failing to account for measurement error at level 1 distorts derived estimates of school/class compositional effects. On the basis of illustrative analysis using large-scale data from education, it demonstrates how the use of doubly latent models can help overcome this problem. The demonstration shows in an empirical way how measurement error bias systematically leads to a positive bias in the school-/class. composition effect estimates. This suggests that non-existent, or even negative, school composition effects may misleadingly be estimated as positive and statistically significant—an artifact of the inadequacy in the underlying statistical methodology.

Further, the workshop considers academic self-concept toward mathematics as an educational outcome (Dicke et al., 2018; Televantou et al., 2021; 2023), showing how doubly latent models can be used to investigate the so-called big-fish-little-pond effect (BFLPE). The BFLPE suggests that average achievement at the classroom level or the school negatively predicts academic self-concept, despite the positive effect of achievement on self-concept at the individual level, and it is a robust finding concerning controls for measurement in compositional analysis (Dicke et al., 2018; Televantou et al., 2021).
The workshop concludes with a discussion of related theoretical, substantive, and methodological issues and with some guidelines for future research.
The methodology proposed by Marsh et al. (2009) can be relevant for any researcher concerned about how individuals may be affected by their interaction with other individuals within similar settings and whenever the variables involved in the analysis are subject to unreliability (e.g., in econometrics and health, in organizational psychology, social psychology, etc.). Specifically, in relation to school (and teacher) effectiveness studies, these models can be applied whenever the substantive interest lies in the investigation of the impact that the characteristics of the fellow students in a school (or a class) have on an individual’s outcomes.

Externalizing (e.g., hyperactivity, inattention) and internalizing problems (e.g., anxiety) are two broad categories of behavior that – when present to an elevated degree – lead to functional impairment and represent the cardinal symptoms of common disorders with begin in school age (Ahmad & Hinshaw, 2017). For example, in neurodevelopmental disorders, such as attention-deficit/hyperactivity disorder (ADHD), neurodivergent participants have been found to face increased behavioral challenges compared to their neurotypical counterparts (e.g., risk-taking, disruptive behavior; Reinke et al., 2023). Groups of individuals who have been identified in research findings as accurate judges such as teachers or parents (Ferdinand et al., 2007) are commonly included in assessments of students’ behavior. Research suggests that assessments of various types of behavior problems through multiple informants can often provide crucial insights that help form a solid foundation for the development and evaluation of effective interventions in educational and clinical settings (De Los Reyes et al., 2015). Despite the advantages of including reports from several informants, disagreements to varying degrees between reports are frequently reported. However, previous studies rarely further investigated the lack of conformity between raters.

More recently, studies have indicated that discrepant perceptions, often referred to as biases, can reveal multifaceted information about how children's behavior is perceived and expressed in various contexts (Achenbach, 2020). Rater biases, such as those related to children's competencies (i.e., positive illusory bias), gender, and special educational needs (SEN), should therefore be deemed as a crucial source of information in assessing behavior (Zurbriggen et al., 2023). Therefore, detected biases should be analyzed to understand, what variables can explain differences in informants’ ratings.

This symposium reveals a common denominator in the three contributions – all gathered information from multiple sources regarding emerging behavior problems of school-aged children. Going beyond rater agreements, each contribution addressed unique aspects and possible biases leading to discrepancies among informants.

The first contribution of the symposium investigates the consistency between different perspectives (e.g. teachers, parents, students) on students' internalizing and externalizing problems. Students' gender and SEN status are analyzed as possible sources of biases. The second contribution investigates biases in teachers' perceptions regarding their students' behavior and provides a thorough examination of the ambiguity of the term “behavior problems“ as applied by teachers to students and the influences on labeling besides students’ actual behavior (i.e., teachers’ stereotypical beliefs, general sensitivity to disruption, work-related stress experiences). Finally, the third contribution focuses on the discrepancies in the assessments of self- and other-perceived (a) social skills and (b) behavior problems of children with ADHD and ASD compared to non-diagnosed children. Overall, the results presented at this symposium contribute to the expansion of knowledge in the field of perspective-dependent phenomena and biases in the assessment of children’s behavior.

1. Introduction Over the past decades, there has been increasing interest in the assessment of students’ behavior problems related to Attention-Deficit/Hyperactivity Disorder (ADHD; e.g., inattention) and other externalizing (e.g., conduct problems) and internalizing (e.g., anxiety) problems (e.g., Reinke et al., 2023). Empirical research suggests that the characterization of students’ behavioral problem phenomena requires multiple informants (e.g., teachers, parents, students themselves). In contrast to single-informant reports, this approach is expected to provide sufficient sensitivity and specificity; however, the vast majority of multi-informant assessments of ADHD symptoms and related problems rely on external sources (e.g.; Mulraney et al., 2022; Narad et al., 2015) and retrospective childhood ratings (e.g., Lundervold et al., 2020) leading to substantial underrepresentation of children’s self-perspectives in research. The current study aims to investigate the consistency between self-reports, parent reports, and teacher reports of students’ internalizing and externalizing problems. Further, it will be analyzed if students’ gender and diagnosis of special educational needs (SEN) can explain the specificity (i.e., method bias) in teacher and parent reports. 2. Method The present study uses data from a random Finnish community sample of 1446 students (male= 47.6%) aged 9-11 years. Students’ externalizing problems (i.e., hyperactivity/inattention (H/I), conduct problems (CP)) and internalizing problems (i.e., emotional symptoms (ES), peer problems (PP)) were measured from students', parents’, and teachers' perspectives using the Strengths and Difficulties Questionnaire (Goodman, 1997). The dichotomous classification of SEN status used was based on information obtained from special education teachers in the participating schools about students’ received support in Finland’s three-tiered system. To assess the consistency, we applied a correlated trait-correlated method minus one (CT-C[M-1] model (Eid et al., 2003). 3. Findings & Conclusions The initial CT-C(M–1) model indicate good model fit (χ2WLSMV (1283, N = 1378) = 2054.55, p < .001, CFI = .944, SRMR = .087, RMSEA = .021). Results show in general moderate to low consistency between student and teacher or parent reports. Thus, the method specificity for parent and teacher reports was moderate to high, confirming the importance of using different raters. As expected, gender and the status SEN could predict the specificity in other reports of students’ behavior to some extent, in particular for externalizing problems. Overall, the results highlighted the vital role of multi-informant approaches in the assessment of student's behavior problems.

Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38(5), 581–586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x Lundervold, A. J., Halmøy, A., Nordby, E. S., Haavik, J., & Meza, J. I. (2020). Current and Retrospective Childhood Ratings of Emotional Fluctuations in Adults With ADHD. Frontiers in Psychology, 11, 571101. https://doi.org/10.3389/fpsyg.2020.571101 Mulraney, M., Arrondo, G., Musullulu, H., Iturmendi-Sabater, I., Cortese, S., Westwood, S. J., Donno, F., Banaschewski, T., Simonoff, E., Zuddas, A., Döpfner, M., Hinshaw, S. P., & Coghill, D. (2022). Systematic Review and Meta-analysis: Screening Tools for Attention-Deficit/Hyperactivity Disorder in Children and Adolescents. Journal of the American Academy of Child & Adolescent Psychiatry, 61(8), 982–996. https://doi.org/10.1016/j.jaac.2021.11.031 Narad, M. E., Garner, A. A., Peugh, J. L., Tamm, L., Antonini, T. N., Kingery, K. M., Simon, J. O., & Epstein, J. N. (2015). Parent–teacher agreement on ADHD symptoms across development. Psychological Assessment, 27(1), 239–248. https://doi.org/10.1037/a0037864 Reinke, A. L., Stiles, K., & Lee, S. S. (2023). Childhood ADHD With and Without Co-occurring Internalizing/Externalizing Problems: Prospective Predictions of Change in Adolescent Academic and Social Functioning. Journal of Attention Disorders, 10870547231187146. https://doi.org/10.1177/10870547231187146

1. Theory Many teachers are concerned that students with behavior problems may strain teaching, classmates, or themselves (MacFarlane & Woolfson, 2013). Although these concerns seem intuitively understandable, research has shown that the term “behavior problems” refers to a perspective-dependent phenomenon of various forms and degrees (Beaman et al., 2007; Crawshaw, 2015) that is susceptible to perception biases (Eckstein, 2019). Therefore, it is highly unclear what teachers mean when they use this expression without further explanation. This gives rise to a research desideratum we address in this paper: studies should investigate the extent to which the teacher-assigned label “behavior problems” is substantiated by students’ actual behaviors and to what extent it is due to other, idiosyncratic conditions. 2. Methods 85 elementary school teachers and 1412 students (11.7 years) answered a survey. The teachers reported the degree to which they consider each student in their class to have behavior problems. As presumed predictors of these labeling tendencies, we investigated the frequency of students’ undisciplined behaviors (ω = .84), non-behavioral student characteristics (sex; learning ability [ω = .72]), teacher characteristics (general sensitivity to disturbances [ω = .71]; work-related stress experience [ω = .80]), and context factors (latent class means of students’ indiscipline and learning ability). A two-level structural equation model was set up and estimated in Mplus 8.10 (Marsh et al., 2009; Muthén & Muthén, 2017-2023). All effects were estimated while controlling for the others. 3. Findings The model fitted the data well (X2 = 139.468, df = 71, p < .001; RMSEA = .024; CFI = .991). At level 1, significant effects on the teachers’ labeling tendencies were found for the individual students’ indiscipline (Beta = .50), sex (Beta = -.25), and learning ability (Beta = .21). At level 2, teachers’ general sensitivity to disturbances (Beta = .35) and work-related stress experience (Beta = .35) were found to be significant conditions of their general labeling tendency across all students; no significant effects were found for the latent class means of indiscipline and learning ability. In sum, the findings indicate that the label “behavior problems” was well substantiated by the students’ actual behaviors – but it was also due to various other conditions that had little or nothing to do with their behavior, such as teachers’ stereotypical beliefs (Anderson et al., 2012). Reflecting on the study’s strengths and limitations, we will discuss the implications of these results for future research and teaching practice.

Anderson, D. L., Watt, S. E., & Noble, W. (2012). Knowledge of Attention Deficit Hyperactive Disorder (ADHD) and Attitudes Toward Teaching Children With ADHD: The Role of Teaching Experience. Psychology in the Schools, 49(6), 511-525. https://doi.org/doi.org/10.1002/pits.21617 Beaman, R., Wheldall, K., & Kemp, C. (2007). Recent research on troublesome classroom behaviour: A review. Australasian Journal of Special Education, 31(1), 45–60. https://doi.org/10.1080/10300110701189014 Crawshaw, M. (2015). Secondary school teachers’ perceptions of student misbehaviour. Australian Journal of Education, 59(3), 293–311. Eckstein, B. (2019). Production and Perception of Classroom Disturbances – A new approach to investigating the perspectives of teachers and students. Frontline Learning Research, 7(2), 1-22. https://doi.org/10.14786/flr.v7i2.411 MacFarlane, K., & Woolfson, L. M. (2013). Teacher attitudes and behavior toward the inclusion of children with social, emotional and behavioral difficulties in mainstream schools: An application of the theory of planned behavior. Teaching and Teacher Education, 29, 46-52. Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., & Nagengast, B. (2009). Doubly-latent models of school contextual effects: Integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate behavioral research, 44(6), 764-802. Muthén, B. O., & Muthén, L. K. (2017-2023). Mplus user’s guide (8th ed.). Muthén & Muthén.

Children with Attention Deficit/Hyperactivity Disorder (ADHD) and Autism Spectrum Disorder (ASD) frequently overestimate their own abilities in different contexts, reporting higher self-perceptions than the others’ external perceptions (Lau-Zhu et al., 2019). This tendency to overestimate one’s capabilities, compared to external evaluations, is called positive illusory bias (PIB, Owens et al., 2007). However, it is not clear whether the two clinical populations overestimate their own abilities in the same way and if this overestimation impacts multiple areas of functioning (Martin et al., 2019). The present study investigated the accuracy of self-perception of abilities of children with ADHD and ASD compared to non-diagnosed (ND) peers in different areas of functioning. Specifically, differences in the estimation of (a) social abilities and (b) behavioral problems in the three groups were analyzed. Two hundred and twenty Italian children (85% M) between 8 and 16 years (M=11.48, SD=2.28) were included in the study. 50 children with ADHD (84% M), 49 with ASD (79% M) without intellectual disability and 121 ND (86% M) participants were enrolled and matched for sex, age, and intelligence quotient (IQ). Two parallel forms of a specific questionnaire measuring social abilities and behavioral problems were filled out by the children and their parents to compare their perceptions. Two different estimation indices were computed based on the discrepancy between the child’s perception and the adult’s report on children’s social abilities and behavioral problems. Separate linear regressions were run for both estimation indices to investigate the association between the two estimation indices and different independent variables: control variables (i.e., age and IQ) and group (ADHD, ASD and ND). Our results showed a different pattern in the two estimation indices. The self-perception of social abilities, independently from the group, decreased with higher age and was significantly impaired only in the ADHD population, compared to both the ASD and ND groups. Conversely, both children with ADHD and ASD estimate their own behavioral problems in a similar way to that of their parents. Our findings confirm that the overestimation of one’s own abilities, compared to external estimations, regards mainly subjects with ADHD (Capodieci et al., 2019). Moreover, this overestimation of abilities is not always present (Owens & Hoza, 2003). Our results revealed the importance of paying attention to the interpretation of self-reports during the assessment of abilities in children and adolescents with ADHD and helped in differentiating specific difficulties of self-perception abilities between ADHD and ASD.

Capodieci, A., Crisci, G., & Mammarella, I. C. (2019). Does Positive Illusory Bias Affect Self-Concept and Loneliness in Children With Symptoms of ADHD? Journal of Attention Disorders, 23(11), 1274–1283. https://doi.org/10.1177/1087054718763735 Lau-Zhu, A., Fritz, A., & McLoughlin, G. (2019). Overlaps and distinctions between attention deficit/hyperactivity disorder and autism spectrum disorder in young adulthood: Systematic review and guiding framework for EEG-imaging research. Neuroscience & Biobehavioral Reviews, 96, 93–115. https://doi.org/10.1016/j.neubiorev.2018.10.009 Martin C. P., Peisch V., Shoulberg E. K., Kaiser N., Hoza B. (2019). Does a social self-perceptual bias mask internalizing symptoms in children with attention-deficit/hyperactivity disorder? Journal of Child Psychology and Psychiatry, 60(6), 630–637. https://doi.org/10.1111/jcpp.13024 Owens, J. S., Goldfine, M. E., Evangelista, N. M., Hoza, B., & Kaiser, N. M. (2007). A Critical Review of Self-perceptions and the Positive Illusory Bias in Children with ADHD. Clinical Child and Family Psychology Review, 10(4), 335–351. https://doi.org/10.1007/s10567-007-0027-3 Owens, J. S., & Hoza, B. (2003). The role of inattention and hyperactivity/impulsivity in the positive illusory bias. Journal of Consulting and Clinical Psychology, 71(4), 680–691. https://doi.org/10.1037/0022-006X.71.4.680

The role of immigrants has become increasingly important in the developed countries’ population structure. The integration of immigrants into the society requires among other things education of the younger generations, which creates possibilities for succeeding in future work life. However, students’ immigrant background has in research often been associated with weaker educational achievement, which poses problems also for educational equality. The PISA studies have revealed that in international comparison, the difference between students with immigrant background and the native students is among the largest in Finland. It has been speculated that immigrant students’ lacking skills in the language of teaching could be reflected on their performance also in other areas of assessment, but there is only little evidence supporting this claim. Schnepf (2007) compared the differences between native and immigrant students in 10 countries based on PISA, TIMMS and PIRLS data, showing that in the USA, continental Europe and the UK, the performance gaps were largely explained by lacking language skills. However, there is little previous research on the relationship between the language utilisation rate and performance in assessments. Many studies support the hypothesis that language utilisation rate at home and with peers is associated with better learning outcomes (Brenneman, Morris & Israelian 2007; Dronkers & van der Velden 2013; Hannover et al. 2013; Levels, Dronkers & Kraaykamp 2008), but some studies have not found a link between them (Agirdag, Jordens & van Houtte 2014; Agirdag & Vanlaar 2016).

The aim of this study was to explore the effect of Finnish language utilisation rate on mathematics performance for immigrant students in Finland. The research questions were:

1. Do native students and students with immigrant background differ from each other in the mathematical thinking and reading, and the time spent on tasks?

2. How does the Finnish language utilisation rate explain the mathematical thinking performance of students with immigrant background, when their reading skills, time spent on tasks and gender are taken into account?

The ability to spell is a gradually developing, long, and complex process. It is directly linked to language skills, phonological, grammatical, and semantic awareness, cognitive functions (intelligence, perception, and memory), and metacognitive skills (Diamanti et al., 2014). Spelling ability is documented as a composite skill that is rather laborious to acquire and which is an essential part of writing (Oakley & Fellowes, 2016)

Some children experience great difficulty as compared to their peers in learning to spell; these children typically have difficulty learning to read as well. It is important to have a good understanding of these complexities in order to comprehend how children learn to spell and why some children have severe difficulties with this process. Theories about how children learn to spell need to account for the full spectrum of patterns, as do theories about why some children have difficulty in spelling and how we can assist them(Treiman, 2017b, 2017a).

The correlation between intelligence, language problems, and spelling is well established in research (Smith et al., 2016) and demonstrates that general intelligence and phonological awareness contribute to the acquisition of reading and spelling skills in children (Siddaiah & Padakannaya, 2015; Zarić et al., 2021).

Research findings suggest that there may be other unknown environmental factors contributing to spelling, such as family environment, neighborhood school, print exposure, environmental toxins, nutrition, the number of siblings, experiences such as visits to the library, and the number of books in the home (Lewis et al., 2018).

This study explores the correlation between students’ performance on a spelling test and specific socio-demographic characteristics, such as gender, father’s and mother’s occupation, who helps them with homework, the degree of satisfaction with their school performance, their parents’ degree of satisfaction with their school performance, difficulties they face in school subjects (reading, writing, arithmetic), essays, tests (oral and written), and use of leisure time.

Research in Greece investigating the development of spelling ability in individuals with and without LD seems limited; however, in recent years in Greece, there has been interest in the linguistic factors related to the development of spelling ability as well as in the analysis of spelling errors of students with and without LD (Protopapas et al., 2013)

In particular, the research hypotheses were:

Η₁: The gender of students with spelling difficulties will correlate with their spelling performance.

Η₂: Parents’ occupation (father and mother) will correlate with spelling performance.

Η₃: The satisfaction of children with spelling difficulties with their school progress will correlate with their spelling performance.

Η₄: Perceived parents’ satisfaction with school progress will correlate with their spelling performance.

Η₅: Children’s perceived difficulties in school subjects (dictation, reading, writing, arithmetic), essay writing, and oral or written assessments (tests) will correlate with performance in spelling.

Η₆: The use of free time will correlate with performance.

Research shows that returns of people that fled can positively influence post-war recoveries on country (Wahba 2021). Education is argued to be of special relevance for post-war recoveries in general. Attendance of children early childhood education (ECE) can play an important role for individuals and society, as participation is in general associated with a positive language, cognitive, and social development supporting a more successful educational career (e.g. Melhuish et al., 2015) and specifically important for the well-being in conflict zones (e.g. Osmanli et al., 2021). Disparities in ECE attendance, according to Boudon's work (1974), can be understood as the result of an interplay of the situation of the family (as supportive factors or barriers) and rational educational decisions. Apart from location and availability (e.g. Sixt 2013), disparities in attendance in ECE are often found with regard to, economic and cultural capital of parents (e.g. Adema et al. 2016) and for immigration countries also the migrant status of families (e.g. Müller et al. 2014). For conflict contexts, the role remigration plays for attendance in ECE and further trajectories has not been well researched.

In this article we therefore analyze the role remigration and war-related international connectivity plays for attendance in ECE in the Kosovan context. For more than 30 years Kosovo is classified as a crisis region, with different phases of war and stability, causing at least 4 different big waves of dynamic war- and crisis-related migration and remigration movements ( Hajdari and Krasniqi 2021). Studying inequalities in ECE attendance is of particular interest as children affected by big migrations waves in the 1990ies are now parents and in Kosovo (like in many conflict regions), with the exception of the preschool-year (age 5-6), non-compulsory, highly privatized and regional differences in availability can be found (Gjelaj et al., 2018). As studies reports on the risk of remigration to Kosovo in terms of reintegration, unemployment, economic situation (Möllers et al. 2017) as well as mental health and the associated loss of quality of life (Lersner et al. 2008) negative primary origin effects can be expected. However, when migration or war-related international personal encounter positively influenced parental acquisition of cultural and social capital (Farrell, Mahon and Mcdonagh, 2012) positive influence, in terms of educational aspiration and insights into the value of ECE, positive secondary origin effects seem plausible.

The educational landscape in Europe faces a range of challenges, and this symposium proposal highlights affective issues in education such as students’ expectations, confidence, well-being, and student-reported instructional quality. The overall aim is to investigate how these factors vary across students and their relationship with student achievement. Four contributions from international scholars use data from the international assessments PISA (Programme for International Student Assessment) and TIMSS (Trends in International Mathematics and Science Study) to address these issues. We highlight the European perspective and draw on specific examples from more than 20 European and even more countries outside Europe.

While the majority of studies conducted with ILSA data focus on student achievement, there is also an affective component of school adjustment that should not be ignored. The first two papers take a comparative perspective focusing on student math confidence. Using the TIMSS 2019 data for 8th graders across 39 countries, the first contribution provides a cross-national analysis of math-specific affective aspects like confidence, enjoyment, and value in math education, focusing on gaps in gender, socioeconomic status (SES), and immigrant status. The second paper, focusing on the Nordic countries, provides another related student perspective on math confidence. This study, using TIMSS 2019 data for fourth graders, examines how students perceive the quality of their instruction and its relation to their mathematics confidence and achievement. It shows that clear and effective teaching are associated with students’ confidence in math.

The other two papers provide specific examples from North Macedonia and Slovenia. The contribution from Slovenia focuses on both cognitive and non-cognitive outcomes by exploring the role of students’ wellbeing in academic motivation and achievement. The PISA 2022 results showed that Slovenian students feel less well-being compared to their peers in other OECD countries, which corresponds with lower literacy skills. The research highlights the importance of positive relationships between teachers and students, emotional support, and a sense of belonging at school for academic success. These findings suggest that improving students’ overall well-being could greatly enhance their motivation and achievement in school. In North Macedonia, the PISA 2018 and 2022 results show many students struggling to reach basic levels in essential subjects. This problem is exacerbated by cuts in education funding, both in terms of GDP and government spending. The youth job market is particularly troubling, with high unemployment rates. This reflects a gap between what the education system teaches and what employers need. Despite these challenges, students in North Macedonia have high hopes for their education and careers, but there is a clear gap between these aspirations and their actual school performance. This contribution sheds light on the factors that can explain this misalignment between student achievement and career expectations.

The session consolidates research on a theme that often receives too little attention. Collectively, these studies show the complex relationship between education policies, student well-being, academic performance, and job market outcomes. In summary, tackling educational challenges requires a comprehensive approach that looks at both cognitive and non-cognitive factors. These factors are essential for preparing students to meet their goals and contribute positively to society. The session investigates these issues both comparatively and in relation to specific countries to provide lessons learned from the international assessments. It is divided into six parts: four presentations, a discussion by a renowned expert, and an open discussion.

Equity in education is defined as the guarantee that all students are provided with the opportunities to benefit from their educational system regardless of their gender, socioeconomic status (SES), and family background (OECD, 2014). In the last decades, investments have been made to identify and monitor educational gaps and to better understand the phenomenon of inequality across several groups, such as gender, SES, and immigrant background (e.g., Strello et al., 2023), and to identity the most urgent needs of intervention in diminishing educational inequality. However, these efforts have been predominantly based on achievement, although school adjustment is not defined only as achievement but instead as a child’s success in dealing with all struggles and tasks faced within the school environment (Ladd, 1989). Adding to the achievement components of schooling, there is also an affective component of school adjustment which, we argue, should not be ignored. Hence, the goal of this study is to give a cross-national overview of the affective gaps based on gender, SES, and immigrant status, by focusing on three indicators of math-specific affective adjustment – confidence, enjoyment, and value. We analyzed the TIMSS 2019 dataset for 8th-grade students in the math domain. Thirty-nine countries were considered, amounting to a total of 224.080 students. Using regression analysis, we estimated gaps throughout different groups – male versus female (i.e., gender), high SES vs. low SES, and native versus non-native (i.e., immigrant background), on three different math-related affective outcomes – confidence, enjoyment, and value, leading to a set of 9 regression analyses. Analyses were performed for each country considering student weights. Although not the focus of this investigation, achievement gaps were also assessed and controlled for. In what concerned gender, there seems to be a rather consistent affective gap benefitting boys, especially in their confidence towards mathematics, even when controlling for achievement. As for SES, results replicate those of achievement, in the sense that students with high SES score higher on math-related affective components of learning in the vast majority of countries – however, this gap diminishes significantly when controlling for achievement. Finally, when looking into immigrant status, results are rather mixed, especially for math confidence. As for enjoyment and math value, non-natives show a slight tendency for higher scores, and this tendency holds even while controlling for achievement. Detailed results, implications, limitations, and suggestions for future research are presented and discussed in light of existing research, policies, and strategies regarding inequalities in education.

Ladd, G. W. (1989). Children’s social competence and social supports: Precursors of early school adjustment? In B. H. Schneider, J. Nadel., & R. Weissberg (Eds.), Social competence in development perspective (pp. 271-291). Amsterdam: Klumer Academic Publishers. OECD (2014). Excellence through equity: Giving every student the chance to succeed. Results from PISA 2012. OECD Publishing. Retrieved from https://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-II.pdf Strello, A., Strietholt, R., & Steinmann, I. (2023). Mind the gap… but which gap? The distinctions between social inequalities in student achievement. Social Indicators Research, 170, 399-425. https://doi.org/10.1007/s11205-023-03196-5

Existing research recognizes the significant role of teaching quality in influencing students' academic (mathematics achievement) and affective outcomes (e.g., mathematics confidence) (Hattie, 2009). Teaching quality can both enhance or diminish the impact of student background characteristics on cognitive achievement (Fauth et al., 2014; Hattie, 2009). Observing, quantifying, and accurately measuring differences in teaching quality presents theoretical and methodological challenges, which could potentially introduce bias and affect study validity (Nilsen et al., 2016). This underscores the need for more empirical research on the relationships between teaching quality and learning outcomes, particularly among primary school students where such research is still limited. This study aims to provide empirical evidence by comparing the relations between student-perceived instructional quality and mathematics achievement and confidence, and examining differences between classrooms in four Nordic countries. The Nordic context is chosen due to the similarities in culture, school systems, and resources among these countries, making it a suitable setting for this comparative analysis (Kavli, 2018). Utilizing data from the 2019 Trends in Mathematics and Science Study (Mullis & Martin, 2017), the study involves 15,839 fourth graders from Denmark, Finland, Norway, and Sweden. It focuses on the relevance of student-perceived instructional quality (Kyriakides & Creemers, 2008) in relation to both cognitive and non-cognitive outcomes, as well as examining variations across classrooms. The concept of instructional quality in this research encompasses two main constructs: classroom management and instructional clarity. Classroom management involves teachers' structural-organizational activities to engage students in learning and establish a conducive learning environment, while instructional clarity pertains to the effectiveness of pedagogical techniques for clear instruction and support (Nilsen & Gustafsson, 2016). Employing Multilevel Confirmatory Factor Analysis (MCFA) and Multilevel Structural Equation Modeling (MSEM), the study examines the relationships between instructional quality and two outcome variables: mathematics confidence and mathematics achievement. Considering the cultural and educational similarities across the Nordic countries, alongside their varied results in international large-scale assessments, the study is guided by two research questions: 1. What are the relations between student-perceived instructional quality (classroom management and instructional clarity) and students’ mathematics confidence and achievement in the Nordic context? 2. What are the relations to student background factors? The results indicate a positive relationship between instructional clarity and mathematics confidence at the student level across all four countries. At the classroom level, mathematics confidence is positively related to instructional clarity. Student background factors demonstrate weaker correlations with mathematics confidence than with mathematics achievement.

Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning and instruction, 29, 1-9. https://doi.org/10.1016/j.learninstruc.2013.07.001 Hattie, J. (2009). Visible learning: a synthesis of over 800 meta-analyses relating to achievement. Routledge. Kavli, A.-B. (2018). TIMSS and PISA in the Nordic countries In N. C. o. Ministers (Ed.), Northern Lights on TIMSS and PISA 2018. Nordic Council of Ministers. https://www.norden.org/en/publication/northern-lights-timss-and-pisa-2018 Kyriakides, L., & Creemers, B. P. M. (2008). Using a multidimensional approach to measure the impact of classroom-level factors upon student achievement: a study testing the validity of the dynamic model. School Effectiveness and School Improvement, 19(2), 183-205. https://doi.org/10.1080/09243450802047873 Mullis, I. V. S., & Martin, M. O. E. (2017). TIMSS 2019 Assessment Frameworks Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2019/frameworks/ Nilsen, & Gustafsson (Eds.). (2016). Teacher Quality, instructional Quality and Student Outcomes: Relationships Across Countries, Cohorts and Time (Vol. 2). Springer Open. https://doi.org/10.1007/978-3-319-41252-8. Nilsen, T., Gustafsson, J.-E., & Blömeke, S. (2016). Conceptual Framework and Methodology of This Report. In T. Nilsen & J.-E. Gustafsson (Eds.), Teacher Quality, Instructional Quality and Student Outcomes: Relationships Across Countries, Cohorts and Time. Springer Open.

Recently, the discourse surrounding the role of students’ well-being and its effects on learning motivation and academic achievement has gained more and more attention in the national and international research community and on the stakeholders’ level. Quality teacher-student relationships, social-emotional support from teachers, a sense of belonging at school, and achievement-related anxiety are often highlighted as important aspects of students’ well-being and have been confirmed in various studies (e.g. Barosso et al., 2020; Harding et al., 2019; Kozina, 2020; Shriver & Buffett, 2015) as significant predictors of both academic motivation and achievement. The latest PISA 2022 results for Slovenia show that, compared to their OECD peers, Slovenian 15-year-olds reported significantly below-average levels of all mentioned aspects of well-being. Since Slovenia also witnessed a significant decline in all three literacy domains in PISA 2022, the article fills the research gap in investigating the role of different aspects of students’ well-being in explaining students’ academic motivation and achievement. For the data analysis, we used the data from the PISA 2022 survey, which in Slovenia includes a representative sample of 6.721 students aged 15. From the 2022 questionnaire, we used separate scales addressing students’ well-being: perceived quality of teacher-student relationships, teacher support in mathematics class, sense of belonging at school, mathematics-related anxiety, and mathematics effort and persistency scale as an indicator of student’s academic motivation. For academic achievement, we used plausible values for all three PISA literacy domains scales. The internal consistency parameters and the multicollinearity between the variables were checked in the Slovenian sample. We used the linear regression procedure to analyse the size effects of different predictors when explaining students’ academic motivation and achievement using the statistical program IEA IDB Analyzer (Version 5.0.23), which, due to two-stage sampling in the PISA study, allows the use of individual students and sample weights. The results show that all four aspects of students’ well-being were confirmed as significant predictors of students’ academic motivation, with the highest effect sizes for the quality of teacher-student relationships and math-related anxiety. The results also showed that the quality of teacher-student relationships is the most significant predictor of academic achievement on all three PISA literacy scales. Following these findings, it is thus crucial to establish a system for strengthening the social-emotional competencies of Slovenian teachers and students and shift an education strategy to a more holistic approach that supports the strengthening of different aspects of students’ and teachers’ well-being.

Barosso, C., Ganley, C. M., McGraw, A., Geer, E., Hart, S. A., & Daucourt, M. (2020). A meta- analysis of the relation between math anxiety and math achievement. Psychological Bulletin 147(2), 134–168. https://doi.org/10.1037/bul0000307 Harding, S. et al. (2019). Is teachers’ mental health and wellbeing associated with students’ mental health and wellbeing? Journal of Affective Disorders, 242, 180–187. https://doi.org/10.1016/j.jad.2018.08.080 Kozina, A. (Ed.) (2020). Social, emotional and intercultural competencies for inclusive school environments across Europe: Relationships matter. Hamburg: Dr. Kovač. Shriver, T., & Buffett, J. (2015). The uncommon core. In J. A, Durlak, C. E. Domitrovich, R. P. Weissberg, & T. P. Gullota (Eds.). Handbook of social and emotional learning: Research and practice (pp. 15–16). New York, London: The Guilford Press.

North Macedonia has one of the highest proportions of students failing to demonstrate basic proficiency (Level 2) in all three domains of science, mathematics and reading among PISA-participating countries (52.2% in PISA 2018 testing; 57.4% in PISA-2022). Young citizens of North Macedonia continue to leave education with among the lowest learning outcomes in Europe. On the other hand, between 2018 and 2023, North Macedonia’s public spending on education as a percentage of GDP fell from 2.80% to 2.72%. The share of total government expenditure allocated to education also declined. While poverty rates have fallen in recent decades, low educational performance is limiting the employment and life opportunities of many individuals and impeding national development. The activity of youth in the labor market of North Macedonia is relatively low (46.7% in the first three quarters of 2022), either compared to the adults or their peers from the EU countries. One in four people over 15 are unemployed, compared to less than one in ten across OECD countries. Low activity of youth illustrates generally low employment probabilities in the country, and the difficulty of school-to-work transition, that can be attributed to (i) unwillingness of employers to bear the costs of on-the-job training of inexperienced youth (ii) skills mismatch between employer’s needs and skills produced by the education system, as well as (iii) the increasing tendency of youth to stay longer in formal education. Students’ academic performance on the PISA 2022 testing is not aligned with their expectations for further education and career. They hold ambitious expectations of future education, 72% of students expect to complete tertiary degree (34% expect to finish doctoral studies, ISCED 8). Students (83%) reported that they have a clear idea of their future job, and they expect to work in high-skill occupations, such as software developers, medical doctors, managing directors and chief executives. This research aims to define the factors that can explain this misalignment between education and career expectations within students’ academic performance. More specifically, data show that there are statistically significant differences in students’ education and career expectations when we compare them based on student’s academic achievement in math, science and reading, Index of economic, social, and cultural status, gender, and language of instruction (Macedonian and Albanian). Data from the research is further discussed with students in focus group discussions. Recommendations from the research will be shared with the state representatives responsible for the reforms in secondary education.

No references.

The role of information and communication technology (ICT) has become increasingly integral in shaping how we work, learn, and connect with others. This is especially recognized by UNESCO, who describes ICT as a “social necessity” for ensuring education as a basic human right, particularly in times of crises (UNESCO, 2023).

International large-scale assessments (ILSAs) such as those conducted by the International Association for the Evaluation of Educational Achievement (IEA), provide a lens to monitor the evolving role of ICT in education and its connections to student outcomes. Furthermore, ILSAs provide extensive contextual data, enabling comprehensive analyses of various aspects of ICT, such as access to ICT resources, students’ attitudes towards ICT, or teacher preparedness for the use of ICT in the classroom.

The goal of this symposium is to demonstrate how different ILSAs can be used to address a wide range of research questions related to ICT in education and to inform research, policy and practice. We focus on two IEA studies: the International Computer and Information Literacy Study (ICILS) and the Progress in International Reading Literacy Study (PIRLS).

ICILS aims to respond to the question: how well are students prepared for study, work, and life in a digital world? (Fraillon & Rožman, 2023). It examines eighth-grade students’ computer and information literacy and, as an optional module, students’ computational thinking. The 2023 cycle of ICILS marked 10 years of the study. PIRLS measures fourth-grade students’ reading achievement. Its latest cycle, conducted in 2021, is the only ILSA that successfully collected data during the COVID-19 pandemic, providing a rich data source to inform about the impact of the pandemic on reading achievement (Mullis et al., 2023). Despite these two studies having different research questions and focusing on different content domains, both ICILS and PIRLS provide valuable information on the topic of ICT in education across a diverse range of educational systems.

Four symposium papers provide different perspectives on how ICILS and PIRLS can be used to study the role of ICT in education. The first paper gives an overview of IEA studies on ICT in education, to then look at their representation in the academic literature. The main goal is to map the evolution of publications and to describe the type of research that has been conducted.

The second paper uses ICILS 2018 data to explore digital applications usage among foreign language teachers. Specifically, it aims to identify different classes of digital application usage as well as factors related to the identified classes.

Using PIRLS 2021 data, the third paper examines the implementation of remote learning during the COVID-19 pandemic in the Dinaric region. In particularly, it examines the different response measures implemented as well as the preparedness for digital remote learning.

The fourth paper evaluates two question formats used to assess teaching beliefs in the field trial of ICILS 2023. The two formats are compared on multiple criteria of data quality, providing insights into the use of alternative question formats in digital context questionnaires.

The presenting authors will focus on the main findings of their studies, highlighting the different ways in which ICILS and PIRLS data can be used. The discussant of the symposium will offer remarks about the presentations, reflecting on the evolving role of ICT in education and how ILSAs can help us study this topic from different thematic and methodological perspectives.

Over the last decades, information and communication technology (ICT) has become an important part of our lives, including education. Already in 1989, the International Association for the Evaluation of Educational Achievement (IEA) was interested in this topic, when it launched its first study about the introduction and use of computers in education (Pelgrum & Plomp, 1993). With over 30 years of different studies about ICT in education, the IEA continues its investigations of how technologies are used in schools and in classrooms and how prepared are students for a digital world through the International Computer and Information Literacy Study (ICILS). While the international reports of ICILS and its predecessors offer a broad range of information, they only scratch the surface of what can be done with the available data. Additional, and highly relevant, insights come from external publications. Following the reviews of Hopfenbeck et al. (2018) and Lenkeit et al. (2015), the present study is a systematic review of English-language peer-reviewed articles related to three IEA studies about ICT in education: Computers in Education (Comped), the Second Information Technology in Education Study (SITES) and ICILS. The main goal of this review is to map the evolution of publications based on these studies and to describe the type of research that has been conducted, both in terms of research topics and methodological approaches. Through this, we aim not only to identify crucial literature to be used by any established or newcomer researcher in the field but also to provide guidance on topics for future research. An additional goal is to encourage the use of ICILS in secondary research. The studies that are included in the review were identified through an electronic search was conducted across five different channels including, for example, multiple electronic databases and target searches in journals focused on international large-scale assessments or on ICT in education. After screening procedures, a total of 91 publications were deemed as relevant for the review. Results map the frequency of publications through years, journals and countries. Further results summarize the major topics studied across within four types of publications identified: descriptive studies, effectiveness studies, critiques or scale evaluations, and case studies.

Hopfenbeck, T. N., Lenkeit, J., Masri, Y. E., Cantrell, K., Ryan, J., & Baird, J.-A. (2018). Lessons Learned from PISA: A Systematic Review of Peer-Reviewed Articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research, 62(3), 333–353. https://doi.org/10.1080/00313831.2016.1258726 Lenkeit, J., Chan, J., Hopfenbeck, T. N., & Baird, J.-A. (2015). A review of the representation of PIRLS related research in scientific journals. Educational Research Review, 16, 102–115. https://doi.org/10.1016/j.edurev.2015.10.002 Pelgrum, W. J., & Plomp, T. (1993). The IEA study of computers in education: Implementation of an innovation in 21 education systems. Pergamon.

International comparative school performance studies, such as the IEA-Study ICILS (Fraillon & Rožman, 2023), offer insights into educational practices across Europe and the world. The methodological design of the ICILS-Study enables sub-samples to be formed, allowing for the examination of specific groups and the generation of knowledge that could be used to improve school systems. This methodological possibility is used in this contribution to identify classes of digital applications usage by foreign language teachers and related determinants. Previous non-subject-specific studies like Graves and Bowers (2018) were able to identify four media patterns (evaders, assessors, presenters, dexterous). Additionally, factors influencing ICT use, such as teachers' self-efficacy are well studied across subjects (Gerick, Eickelmann & Bos, 2017). However, specific digital application usage classes and their determinants in foreign language teaching remain unexplored, despite possible subject-subcultural influences. This contribution aims to answer two research questions: 1. To what extent can different digital application usage classes be identified for foreign language teachers compared to non-foreign language teachers in Germany? 2. To what extent is there a connection between identified digital application usage classes and determinants of ICT use for both groups? The study employs ICILS 2018 teacher data from Germany (n=2328; Eickelmann et al., 2019), taking into account data weighting (Tieck & Meinck, 2020). To answer the first RQ, a latent class analysis is conducted using MPlus8. The class solution is based on statistical information criteria (e.g. smallest BIC; Eshima, 2022). The analysis identifies three usage classes for foreign language (a) and non-foreign language (b) teacher groups: avoiders (a: 80.4%; b: 78.1%), selective users (a: 17.4%; b: 19.7%), and multiple users (a: 2.2%; b: 2.2%) of digital applications. For the second RQ, a hierarchical regression analysis was conducted using the IDB Analyzer to explore the connections between usage classes and determinants. The analysis, grounded in theoretical considerations, employs five regression models. Results highlight significant correlations, including foreign language teachers' affiliation with multiple users being linked to positive attitudes towards ICT (Model V; ß=.27, adjusted R²=.15). Correlations vary across usage classes and teacher groups. The findings contribute to the understanding of the integration of digital applications in language teaching. This provides valuable insights for researchers and policymakers, particularly in Europe. Potential explanations, such as subject-subcultural influence on digital application usage, related determinants and alternative methodological approaches are discussed.

Eickelmann, B., Bos, W., Gerick, J., Goldhammer, F., Schaumburg, H., Schwippert, K. et al. (Hrsg.). (2019). ICILS 2018 #Deutschland. Computer- und informationsbezogene Kompetenzen von Schülerinnen und Schülern im zweiten internationalen Vergleich und Kompetenzen im Bereich Computational Thinking. Münster: Waxmann. Eshima, N. (2022). An Introduction to Latent Class Analysis. Singapore: Springer Singapore. https://doi.org/10.1007/978-981-19-0972-6 Fraillon, J. & Rožman, M. (2023). International Computer and Information Literacy Study 2023. Assessment Framework. Amsterdam: International Association for the Evaluation of Educational Achievement (IEA). https://www.iea.nl/sites/default/files/2023-12/20231221%20ICILS2023_Assessment_Framework__Final_0.pdf Gerick, J., Eickelmann, B. & Bos, W. (2017). School-level predictors for the use of ICT in schools and students’ CIL in international comparison. Large-scale Assessments in Education, 5(1), 1–13. DOI: 10.1186/s40536-017-0037-7 Graves, K. E. & Bowers, A. J. (2018). Toward a Typology of Technology-Using Teachers in the ”New Digital Divide”: A Latent Class Analysis of the NCES Fast Response Survey System Teachers’ Use of Educational Technology in U.S. Public Schools, 2009 (FRSS 95). Teachers College Record, (8), 1-42. http://www.tcrecord.org/Content.asp?contentid=22277 Tieck, S. & Meinck, S. (2020). Weights and variance estimation for ICILS 2018. In Mikheeva, E., Meyer, S. (Eds.). IEA International Computer and Information Literacy Study 2018 - User Guide for the International Database. Amsterdam: International Association for Educational Achievement (IEA).

The COVID-19 pandemic caused severe global challenges for education systems and schooling worldwide, with the Dinaric region being no exception. Although the demand for digital competence among teachers and using digital tools and devices in teaching and learning has been present in the region for over a decade, the existing practices could not fully meet the difficulties associated with the COVID-19 pandemic disruption. Centering on the Dinaric area (i.e., Albania, Croatia, Kosovo, Montenegro, North Macedonia, Serbia and Slovenia) concerning the COVID-19 disruption, the paper sheds light on the disruption and response measures in the region against the demand for remote instruction during COVID-19. It examines the diverse response measures and how they were conveyed to different stakeholders, coupled with prior established practices and ease of access to digital devices and their use in teaching and learning. Data collected during the PIRLS 2021 cycle from students, teachers, school principals and parents and analyses of the PIRLS 2021 Encyclopedia (Reynolds et al., 2022) are used as primary sources in the analyses. Both national reports and responses from school principals indicate that the level of disruption varied at different times of school operation, prompting different types of responses from the schools, often dependent on school location and overall country response to the pandemic. Results also show certain common patterns across the Dinaric region concerning the systems’ wide range of activities to answer the challenge. National Ministries of Education coordinated technical and overall resource support across the most Dinaric countries. Access to different digital resources and access provided to students and teachers somewhat varied. Internet-based resources dominated distant learning resources. Sharing devices within the class was the leading established practice. In some cases, the availability of smartphones outpowered the presence of one’s own or shared computer (tablet), according to the student reports. Teachers’ professional development across the board was focused more on instruction related to digital literacies than integrating technologies into reading instruction. Parents’ perceptions of whether their child’s learning progress has been adversely affected during the COVID-19 disruption varied between and within countries.

Reynolds, K.A., Wry, E., Mullis, I.V.S., & von Davier, M. (2022). PIRLS 2021 Encyclopedia: Education Policy and Curriculum in Reading. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: https://pirls2021.org/encyclopedia

International large-scale assessments (ILSA) administer context questionnaires to students, teachers, and principals to collect information about school, classroom and learning conditions. These questionnaires usually consist of a series of rating type items which often face issues such as social desirability, self-presentation, and acquiescence bias (e.g., Lelkes and Weiss, 2015; Schaeffer and Dykema, 2020). There are alternatives to rating scales, such as forced choice items, rankings, anchoring vignettes or situational judgement tasks. Alternative item types can address some issues found with rating item types. It was found that ranking reduces the response style, and it improves data quality (Krosnick & Alwin, 1988). Furthermore, computer-based surveys enable administering items or response scales that are difficult to implement on paper. They provide an opportunity to use functions such as sliders, drag-and-drop, or drop-down menus. In the field trial of the International Computer and Information Literacy Study (ICILS) 2023, Q-sort was introduced as an alternative question type to assess teaching beliefs of secondary school teachers. Q-sort is a technique that was initially developed for clinical interviews, requiring respondents to arrange and rank a series of cards according to their preference. In this paper, we investigate the feasibility of using the Q-sort (ranking) format when collecting data about teaching beliefs in an international survey and explore and compare the quality and usefulness of the data gathered by two question types, ranking and rating. We use teacher data from 28 countries participating in ICILS 2023 field trial to investigate the effect of the question format using multiple criteria of data quality. The two question types were randomly distributed across the participating teachers within countries. We compare the two versions by the amount of missing data, distribution of responses, item and scale means, and the correlations between the scale scores and teacher characteristics. For ranking higher proportion of missing values were observed because the cognitive load is higher for the parallel sorting of a total of 18 items than for the rating items that are answered individually. In addition, we observed more variance in the responses from the ranking than in the rating version. The ranking removes the possibility that respondents can agree equally with all statements and can thus reduce acquiescence bias. Although some advantages were found for the ranking format, we could not suggest the implementation of the current version for further data collection because of the high amount of missing data observed.

Krosnick, J. A., & Alwin, D. F. (1988). A test of the form-resistant correlation hypothesis: Ratings, rankings, and the measurement of values. Public Opinion Quarterly, 52 (4), 526–538. Lelkes, Y., & Weiss, R. (2015). Much ado about acquiescence: The relative validity and reliability of construct-specific and agree–disagree questions. Research & Politics, 2 (3), 053168015604173. Schaeffer, N. C., & Dykema, J. (2020). Advances in the science of asking questions. Annual Review of Sociology, 46 , 37–60.

The purpose of this symposium is to discuss the implications of results from the latest implementation of the International Civic and Citizenship Education Study (ICCS), a comparative survey that was conducted in 2022 collecting data from students, teachers, and schools from 24 education systems (see Schulz, Ainley, Fraillon, Losito, Agrusti, Damiani, & Friedman, 2024). The purpose of ICCS is to investigate how young people are prepared to undertake their roles as citizens in a range of countries. ICCS 2022 is the fifth international IEA study in this area and the third cycle of ICCS. It is explicitly linked through common questions to the previous ICCS cycles undertaken in 2009 (Schulz, Ainley, Fraillon, Kerr & Losito, 2010) and 2016 (Schulz, Ainley, Fraillon, Losito, Agrusti, & Friedman, 2018). In addition to providing an opportunity for an evidence-based discussion of the variation in practices and outcomes of civic and citizenship education the symposium will also provide a forum for discussion of methodological issues related to the cross-cultural study of civic and citizenship education.

The symposium will primarily focus on the following aspects related to civic and citizenship education:

• Student perceptions of political system and institutions
• Student engagement with digital technologies
• School and teacher perceptions of diversity at schools
• The impact of the COVID-19 pandemic on civic and citizenship education

Educational systems, school and teachers seek to prepare young people to understand the society they live in, to engage with its political and social issues and become actively involved as citizens in later adult life. There is a consensus that formal education influences the extent of adult engagement in society (Pancer, 2015). The third cycle of the IEA International Civic and Citizenship Education Study, ICCS 2022, provides an opportunity to study both the extent and variation of civic knowledge, attitudes, and engagement based on recent data, and it allows to review changes in civic learning outcomes compared to previous cycles. It also provides a basis for a systematic analysis of contextual factors, at different levels of educational systems that influence civic-related learning outcomes.

The symposium includes four papers. The first paper is concerned with an analysis of ICCS 2022 data reflecting lower-secondary students views of their political systems and institutions. The second paper focuses on data about how young people use or expected to use digital technologies for civic engagement. The third paper focuses on how schools and teachers deal with aspects of diversity as part of education. The fourth paper discusses the impact of the COVID-19 pandemic on civic and citizenship education outcomes.

Over the past decade there have been growing concerns regarding a worldwide “democratic recession” (Diamond, 2015, 2021). These have arisen in response to an increases in authoritarian government practices in some countries as well as new political movements that have undermined support for traditional political parties, and, in some cases have challenged the stability of democratic systems (Boogards, 2017; Mair, 2002). These recent developments raise the question to what extent tendencies toward alienation and an understanding of and preference for populist solutions to government are shared by young people (Gidron & Hall, 2019; Henn & Weinstein, 2006), and whether education has the potential of promoting democratic principles to counteract prospects of growing alienation (Estellés & Catellví, 2020; Sant, 2019). ICCS results from 2009 and 2016 showed considerable support among lower-secondary students for democratic government and equal opportunities across countries (Schulz et al., 2010, 2018). ICCS 2022 addressed additional aspects related to attitudes toward government and the political system as well as perceptions of potential threats to democracy. ICCS data have also shown that students with higher levels of knowledge have lower levels of trust in institutions in countries where systems are generally perceived as more corrupt and less transparent, while in others there is a positive association (Lauglo, 2012; Schulz et al., 2018, 2024). Based on data from 19 European countries that participated in ICCS 2022, this paper focuses on how students perceive their political systems. The analyses consist of a descriptive review of student perceptions across different countries and include comparisons with results from adult surveys (Eurobarometer, 2023) and considering information about perceived corruption (Transparency International, 2022) as an important context to explain cross-national variation, as well as of multivariate models explaining variation in student perceptions of the political system with student background variables, trust in civic institutions, as well as school-related variables (such as civic knowledge, civic engagement at school). ICCS 2022 results show that while majorities of students across countries considered democracy as the best form of government, satisfaction with and critical views of the political system varied considerably. While trust in institutions had consistently positive associations with positive appraisals of the political system, positive associations with civic knowledge tended were only observed in some Northern European countries. More knowledgeable students were also more critical of democratic representation than those with lower levels of civic knowledge in countries, where the democratic systems are generally seen less functional.

Boogards, M. (2017). Lessons from Brexit and Trump: populism is what happens when political parties lose control. Zeitschrift für Vergleichende Politikwissenschaft, 11(4), 513–518. Diamond, L. (2015). Facing up to democratic recession. Journal of Democracy, 26(1), 141–155. Diamond, L. (2021) Democratic regression in comparative perspective: scope, methods, and causes. Democratization, 28(1), 22-42. Estellés, M., & Castellví, J. (2020). The educational implications of populism, emotions and digital hate speech: A dialogue with scholars from Canada, Chile, Spain, the UK, and the US. Sustainability, 12(15), 6034. European Commission (2023). Democracy. Report – Eurobarometer 522. Retrieved at: file:///C:/Users/acerschulzw/Downloads/Democracy_fl_522_report_en.pdf Gidron N., & Hall, P. A. (2020). Populism as a Problem of Social Integration. Comparative Political Studies, 53(7), 1027-1059. Henn, M., & Weinstein, M. (2006). Young people and political (in)activism: Why don’t young people vote?. Policy & Politics, 34(3), 517-534. Lauglo, J. (2013). Do more knowledgeable adolescents have more rationally based civic attitudes? Analysis of 38 countries. Educational Psychology, 33(3), 262–282. Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., Damiani, V. Friedman, T. (2024). Education for Citizenship in Times of Global Change. ICCS 2022 International Report. Cham: Springer. Transparency International (2023). Corruption Perceptions Index 2022. Retrieved at: https://www.transparency.org/en/cpi/2022

Digital technologies have redefined the ways in which young people can engage in society. Social media and virtual communities are instrumental in connecting individuals and amplifying arguments. This has led to a new era of civic engagement with digital participation as a form of engagement for students, demonstrated by activities such as organizing of climate protests and raising awareness of the plight of a minority group (de Moor et al., 2020; Cho, Byrne, & Pelter, 2020). There is a perception that developments with technology should usher in an era of greater civic engagement (Dubow, Devaux, & Manville, 2017). The release of the IEA’s International Civic and Citizenship Education Study (ICCS) 2022 (Schulz et al., 2024) provides new data on students’ knowledge of and engagement in civic and citizenship-related topics from 24, predominantly European based, educational systems. Students completed a test of civic knowledge, followed by a questionnaire that included questions about their current and anticipated future level of engagement with technologies for civic engagement. Previous cycles of the ICCS study reported increased use of digital technologies that did not necessarily lead to an increase in civic engagement (Schulz et al., 2018; Schulz et al., 2010). Preliminary analyses with ICCS 2022 data revealed only a small proportion of students who frequently engage in more active forms of participation. These students were most likely to be interested in civic issues, but also demonstrated lower levels of civic knowledge (see Schulz et al., 2024). Building on these earlier results, this paper will use data from ICCS 2022 and earlier cycles to explore changes over time in how students use social media to engage in civic activities, their intentions for doing so in the future, their level of trust in social media and their exposure to learning about the reliability of online information. The paper will also examine the characteristics of students who are currently and more likely to participate in future civic engagement activities using digital technologies. Our preliminary analysis reveals that while digital technologies open new avenues for civic engagement for young people, there is a notable gap in how they effectively harness these tools. This gap underscores the need for integrating digital literacy with civic education to nurture future citizens to become engaged and knowledgeable as technologies become increasingly ingrained in our everyday lives.

Cho, A., Byrne, J., & Pelter, Z. (2020). Digital civic engagement by young people. UNICEF. https://www.unicef.org/globalinsight/media/706/file/UNICEF-Global-Insight-digital-civic-engagement-2020.pdf de Moor, J., Uba, K., Wahlström, M., Wennerhag, M., & De Vydt, M. (Eds.). (2020). Protest for a future II: Composition, mobilization and motives of the participants in Fridays For Future climate protests on 20-27 September, 2019, in 19 cities around the world. Södertörn University. https://urn.kb.se/resolve?urn=urn:nbn:se:sh:diva-40271 Dubow, T., Devaux, A., & Manville, C. (2017). Civic Engagement: How Can Digital Technology Encourage Greater Engagement in Civil Society? RAND Corporation. Retrieved from http://www.jstor.com/stable/resrep17637 Schulz, W., Ainley, J., Fraillon, J., Kerr, D., & Losito, B. (2010). ICCS 2009 International Report. Civic knowledge, attitudes and engagement among lower secondary school students in thirty-eight countries. Amsterdam: IEA. Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., & Friedman, T. (2018). Becoming citizens in a changing world. IEA International Civic and Citizenship Education Study 2016 International Report. Cham: Springer. Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., Damiani, V. Friedman, T. (2024). Education for Citizenship in Times of Global Change. ICCS 2022 International Report. Cham: Springer.

The growing diversity of student populations at the global level has increasingly prompted schools to develop institutional and instructional practices for building multicultural and inclusive learning contexts (Griffith et al., 2016; Banks, 2020), allowing students and school communities to foster positive attitudes toward diversity (Solhaug, 2018). The concept of diversity embraces a wide range of socially ascribed or perceived differences, such as by sex, age, ethnic/social origin, language, religion, nationality, economic condition, or special learning needs (Daniels & Garner, 1999; Council of Europe, 2008). In this scenario, civic and citizenship education plays a key role for the promotion of knowledge and respect for other cultures and the inclusion of diverse groups into society (Schachner et al., 2019). ICCS 2022 included diversity as one of its focus areas. The study assessed a wide range of issues related to diversity, that concern the affective-behavioural area (e.g. students’ attitudes toward gender equality and equal rights for immigrants) and the contexts of school and classrooms (Schulz et al., 2023). The paper will present ICCS 2022 results related to how learning environments acknowledge and deal with diversity. After a brief overview of the relevance of the topic of diversity and inclusion within learning objectives, it will analyse data from teacher and school questionnaires concerning teachers’ self-reported preparedness to teach diversity and inclusiveness, their participation in training programs, schools and classroom activities dealing with diversity, and teachers’ opinions regarding the influence of cultural and ethnic differences and of socioeconomic differences on teaching activities. Findings showed a positive picture of how schools and teacher deal with diversity, however, there were considerable variations across countries. At the school level, activities to promote teaching to young people from diverse backgrounds, to foster tolerance toward diversity, and to support students with special learning needs were reported widely in most ICCS 2022 countries. Majorities among teachers reported to have conducted activities to address diversity in their classrooms and considered diversity as an important resource for education. More than half of them also reported attendance of pre- or in-service training courses on diversity and inclusiveness (Schulz et al., 2024). Based on these results, the final section of this contribution considers the interplay between democracy and intercultural dialogue (intended in its broader sense, see Council of Europe, 2018) as well as the implications at the school and classroom level for the promoting a democratic and intercultural learning environment for civic and citizenship education.

Banks, J. A. (2020). Diversity, transformative knowledge, and civic education. Routledge. https://www. routledge.com/Diversity-Transformative-Knowledge-and-Civic-Education-Selected-Essays/ Banks/p/book/9780367863197 Council of Europe (2018). Reference Framework of Competences for Democratic Culture. Council of Europe. https://www.coe.int/en/web/campaign-free-to-speak-safe-to-learn/referenceframework-of-competences-for-democratic-culture Daniels, H. and Garner, P. (Eds) (1999). Inclusive Education, World Yearbook of Education. Routledge. Griffith, R. L., Wolfeld, L., Armon, B. K., Rios, J. & Liu, O. L. (2016). Assessing intercultural competence in higher education: Existing research and future directions. ETS Research Report Series, 2016(2), 16-25. https://doi.org/10.1002/ets2.12112 Schachner, M. K. (2019). From equality and inclusion to cultural pluralism – Evolution and effects of cultural diversity perspectives in schools. European Journal of Developmental Psychology, 16(1), 1–17. https://doi.org/ 10.1080/17405629.2017.1326378 Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., Damiani, V. Friedman, T. (2024). Education for Citizenship in Times of Global Change. ICCS 2022 International Report. Cham: Springer. Schulz, W., Fraillon, J., Losito, B., Agrusti, G., Ainley, J., Damiani, V., & Friedman, T. (2023). IEA International Civic and Citizenship Education Study 2022 Assessment Framework. Cham: Springer. Solhaug, T. (2018). Democratic Schools – Analytical Perspectives. JSSE, 17 (1), 2-12. DOI 10.4119/UNIBI/jsse-v17-i1-1791

This research examines the effects of COVID-19 containment policies, particularly school closures and lockdowns, on the civic outcomes of eighth-grade students. The effects of school closures on student performance have been studies in an increasing number of studies but to data few studies have studied the effects on the performance in other domains and on socio-economic outcomes (Betthäuser et al., 2023; Di Pietro, 2023). By analysing trend data from the International Civic and Citizenship Education Study (ICCS; Schulz et al., 2024) and the Oxford COVID-19 Government Response Tracker (OxCGRT), the study investigates how these educational disruptions during the pandemic influenced students' civic knowledge, attitudes, and engagement intentions. The study uses a longitudinal approach, analysing changes in civic outcomes from 2016 to 2022 using data from almost 100.000 students across 15 education systems worldwide. The ICCS data 2016 provides a baseline of students' civic knowledge and engagement intentions prior to the pandemic. In contrast, the OxCGRT data offers a detailed index of governmental responses to COVID-19, including metrics on school closure durations and lockdown strictness. Significant findings emerge from this analysis. There is a clear negative correlation between the length of school closures and students' civic knowledge scores. Extended periods of school closure correlate with notable declines in students' comprehension of civic concepts and trust in civic institutions. Furthermore, increased average lockdown stringency is associated with heightened intentions among students to participate in protest activities. These patterns indicate a shift in the landscape of civic engagement, potentially leading to more active forms of civic participation in the future. The research emphasizes the necessity of considering the wider impacts of the COVID-19 pandemic on civic education. It suggests that while prolonged school closures and strict lockdown measures might be essential for public health, they can inadvertently affect the civic development of young people. This situation calls for a reassessment of civic education strategies during crises to ensure the sustainability of high-quality civic learning experiences. Conclusively, the study adds valuable insights to the discourse on the educational consequences of the pandemic. By providing empirical evidence of the direct connection between COVID-19 containment policies and students' civic outcomes, it underlines the importance of sustaining civic education amidst global challenges

Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023). A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nature Human Behaviour, 7(3), 375–385. Di Pietro, G. (2023). The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis. Educational Research Review, 39, 100530. Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., Damiani, V. Friedman, T. (2024). Education for Citizenship in Times of Global Change. ICCS 2022 International Report. Cham: Springer

Digitalization has altered almost all areas of life and has increasingly been established in schools. The expected potentials are manifold with regard to affective variables such as increased motivation to learn, the optimization of learning processes and the improvement of students’ competencies (Voogt et al., 2018). However, research does not consistently show positive correlations between the use of digital media (e.g. different tools, purposes or frequencies) and increased outcomes. In addition, the phase of interruption of regular teaching during the COVID-19 pandemic is noteworthy, as digitalization has maintained teaching processes in many places during this time. Research on disruption of schooling has mainly shown a decline in competencies, and indicated that digitalization has not been able to help prevent a drop in competencies to the same extent in all countries (Di Pietro, 2023; Kennedy & Strietholt, 2023; König & Frey, 2022). Against this background, the papers of the symposium will examine the overarching question of what potential the use of digital media has in the classroom in an international comparison and which factors are relevant for its effectiveness.

The first presentation by Simon Skov Fougt and Katja Neubert will address the debates about digitization, especially in the Nordic countries after the latest results of PIRLS and PISA showing a decline in students’ competencies. In these rather digitized countries, the decline in the competencies measured could not be fully compensated for by digitization, which was the expectation on many sides. The paper discusses an explanatory approach based on PIRLS data with a special focus on Denmark amongst the Nordic countries to the question how digitalization can nevertheless be seen as a positively influencing factor on reading literacy in fourth grade during the pandemic.

The second contribution by Ramona Lorenz, Ulrich Ludewig and Nele McElvany will broaden the perspective to several European countries. Given inconsistent findings on the effects of digital media depending on the purpose of use and differences according to the socio-economic background of the students, the paper explores the question of how digital schoolwork is related to reading literacy in fourth grade and if differences between countries and between European regions can be found. With data form PIRLS 2021 a multi-group two level model with cross-level interaction shows that no systematic negative relations between digital reading and reading literacy can be found but some positive relations at specific levels or in specific countries appear. Regional differences will be discussed in depth.

The third contribution by Fazilat Siddiq and Ronny Scherer consistently provides insights in how digital media should be used in a purposeful way so that digitalization can have the desired effect on students` competencies. Even if education systems worldwide have integrated digitalization, there is still a huge need for teachers to gain a better understanding and professional knowledge for implementing digital media in a reasonable manner. A theory-driven and research-based teaching program was developed to support teachers in enhancing lower secondary students` 21st century skills. By means of thematic analysis and network analysis, learning experiences within this teaching program are examined. Important implications for digitalization in schools complete the contribution.

The symposium concludes with a comprehensive discussion and thorough appraisal of the three contributions by Rolf Strietholt.

PIRLS 2021 and PISA 2022 showed significant declines in Danish students’ reading competence (Fougt et al. 2023; Gissel 2023), as well as we internationally have seen the largest decline ever. One proven factor here is the covid 19-pandemic (Kennedy & Strietholt 2023); however the waste majority of the public debate in Denmark following PIRLS 2021 and PISA 2022 focuses on digitalization, also at governmental level. The Danish Minister for Culture argued to abandon digitalization in schools, our Prime Minister claimed that smartphones were the biggest threat towards our children, and The Minister for Education gave a public excuse for ‘the digital experiment’ in schools. Several opinions makers and some researchers argue for screen restrictions in education (eg. Ågård, 2021; Rashid et al. 2024). This paper challenges this standing point with the hypothesis that the waste digitalization of Danish schools and the habituation of teachers and students might have helped to prevent a larger decline in Danish students’ reading competence during the pandemic, as schools relatively easy could transfer to online teaching. The paper mainly draws on PIRLS data and focuses on the comparable Nordic Countries with the following research question: How can digitalization be seen as a positively influencing factor on students reading competence during the pandemic? In 2022, Denmark was world ranking no. 1 for the third successive time in the latest UN E-government survey on digitalization (UN 2022). PIRLS 2016-2021 data show that Danish schools have been more digitalized for a longer time compared to all other PIRLS participants, also the other Nordic countries. Both in PISA and PIRLS we see a minor decline in students’ reading competence in Denmark as compared to the other Nordic countries, and at the same time, Denmark was by far the most school-closed country within the Nordic countries during the pandemic. This paper discusses this explanatory approach, and it is also including development in SES and in spoken language at home as other possible explaining factors.

Fougt, S. S., Neubert, K., Kristensen, R. M., Gabrielsson, R., Molbæk, L., & Kjeldsen, C. C. (2023). Danske elevers læsekompetence i 4. klasse: Resultater af PIRLS-undersøgelsen 2021. Aarhus Universitetsforlag Gissel, S.T. (2023). PISA 2022 LÆSNING. Delrapport. VIVE https://www.uvm.dk/-/media/filer/uvm/int/231204-pisa-2022-laesning-pdf-ua.pdf Kennedy, A. I., & Strietholt, R. (2023). School Closure Policies and Student Reading Achievement: Evidence Across Countries. Rashid, I., Bro, K. B. & Brixtofte, M. (2024). Skærmsund. En fire-ugers guide til sundere skærmvaner. Gyldendal UN (2022). UNITED NATIONS E-GOVERNMENT SURVEY 2022. The Future of Digital Government. Department of Economic and Social Affairs https://publicadministration.un.org/egovkb/en-us/Reports/UN-E-Government-Survey-2022

Reading literacy is an important foundation for educational achievement, social participation, and professional life. Digitalization is expanding to a considerable extent the reading opportunities that students have in their everyday lives, but also in the school context. The use of digital media is considered to have a wide range of potential benefits for learning. Nevertheless, research does not consistently point out positive effects of digital tools and e.g. has shown that digital reading, at least during leisure time, does not necessarily add to reading comprehension (Altamura et al., 2023). Other studies indicated a negative relation of the amount of daily use of digital devices with reading comprehension that could be compensated by a supportive use by the teachers within digital reading projects (Salmerón et al., 2022). Overall, international large-scale assessments show that in some countries more digital reading time in school is associated with higher reading literacy. In some education systems, however, a negative correlation is found (Lorenz et al., 2023). Furthermore, there is an ongoing discussion about the use of digital media for reading instruction and how it`s use differs between students with different socio-economic backgrounds. This discussion is driven from findings of particularly large learning deficits among children from low socio-economic backgrounds while learning digitally during the COVID-19 pandemic and on country level a larger gap between middle-income countries relative to high-income countries (Betthäuser et al., 2022). This leads to the question on digitization for school purposes in an international comparison: What relation of digital schoolwork with reading literacy can be found in fourth grade across European countries? Results indicate an overall positive association of the amount of digital schoolwork (finding and reading information; preparing reports and presentations) and reading literacy at the country level for all considered European countries in PIRLS 2021. Regional differences are that Northern European countries have both a higher level of digital reading for schoolwork and reading literacy, whereas Western European countries have a lower level of digital reading along with lower reading literacy. A multi-group two level model with cross-level interaction revealed effects at the class level, primarily in Eastern and Southern European countries. However, no effects at any level remain statistically significant after controlling for socioeconomic background and spoken language at home (other than test language). Results show no evidence in support of a negative association between digital schoolwork and reading literacy. Inequality between European regions will be discussed.

Altamura, L., Vargas, C., & Salmerón L. (2023). Do new forms of reading pay off? A meta-analysis on the relationship between leisure digital reading habits and text comprehension. Review of Educational Research. https://doi.org/10.3102/00346543231216463 Betthäuser, B.A., Bach-Mortensen, A.M., & Engzell, P. (2022). A systematic review and meta-analysis of the evidence on learning during the covid-19 pandemic. Nature Human Behaviour, 7(3), 375–385. Lorenz, R., Goldhammer, F. & Glondys, M. (2023). Digitalisierung in der Grundschule [Digitalization in elementary school]. In N. McElvany, R. Lorenz, A. Frey, F. Goldhammer, A. Schilcher & T. C. Stubbe (Hrsg.), IGLU 2021 – Lesekompetenz von Grundschulkindern im internationalen Vergleich und im Trend über 20 Jahre [PIRLS 2021 - Reading literacy of primary school children in an international comparison and trend over 20 years] (S. 197–214). Münster: Waxmann. Salmerón, L., Vargas, C., Delgado, P., & Baron, N. (2022). Relation between digital tool practices in the language arts classroom and reading comprehension scores. Reading and Writing, 36, 175–194. https://doi.org/10.1007/s11145-022-10295-1

In recent years, several countries have undergone major curriculum revisions, which has resulted in the inclusion of interdisciplinary competence areas such as digital competence, computational thinking, critical thinking, problem solving and collaborative learning into the compulsory K-12 curricula (Erstad & Siddiq, 2023). Such overarching competence areas are oftentimes labelled 21st century skills (Voogt & Roblin, 2012). Although the intentions in the curriculum are positive, there is currently little research-based knowledge about how such competences can be taught and assessed, and teachers report lack of access to professional development and teaching materials (Erstad & Siddiq, 2022; Kravik et al., 2022). To meet some of these challenges, the TEACH21st-project (Teaching and transfer effects of 21st century skills – collaborative problem solving in digital environments) was initiated in 2019 with the aim to develop teaching materials and practices that are knowledge- (theory-driven) and research-based. More specifically, applying a teacher design team approach (Becuwe et al., 2016) teachers, teacher educators, student teachers and researchers worked together to develop a teaching resource aimed at developing lower secondary students’ computational thinking and collaborative problem-solving competences. This program has been developed, piloted and revised through several iterations. The final teaching program includes materials (games, charts, tasks etc.) for the teachers and students, and practices (e.g., use of analogue and computer programming in combination to teach computational thinking, how to teach collaborative learning and use it as a pedagogical approach). The program consists of four modules that are built on the principles of: relevance (target learning goals in the curriculum); inclusion (all students should be able to participate independent of their previous knowledge, and provide adaptive teaching); engagement and activity (include engaging and fun tasks, involving physical activity and hands-on assignments); collaborative learning (students need to learn to collaborate and the tasks require positive dependence); and progression (the tasks move towards more advanced levels). Finally, this program has been conducted in 32 classes by their teachers (N=16) after attending a one-day professional development workshop. In this study, we will examine how the 9th grade students (N = 460) experience learning within this teaching program. The data consist of the students' reflection notes conducted after each of the four modules and observations (N = 24 classes). The data has been analyzed through a combination of thematic analysis (Braun & Clark, 2021) and network analysis (Epskamp et al., 2018). Results and implications will be discussed.

Becuwe, H., Tondeur, J., Pareja, R. N., Thys, J., & Castelein, E. (2016). Teacher design teams as a strategy for professional development: The role of the facilitator. Educational Research and Evaluation, 22(3-4). Braun, V., & Clarke, V. (2021). Thematic Analysis: A Practical Guide. London: Sage Epskamp, S., Maris, G., Waldorp, L. J., & Borsboom, D. (2018). Network psychometrics. In The wiley handbook of psychometric testing (pp. 953–986). Wiley. https://doi.org/10.1002/9781118489772.ch30. Erstad, O., & Siddiq, F. (2023). Educational assessment of 21st century skills—novel initiatives, yet a lack of systemic transformation, Editor(s): Robert J Tierney, Fazal Rizvi, Kadriye Erkican, International Encyclopedia of Education (Fourth Edition), Elsevier, 2023, Pages 245-255, ISBN 9780128186299, https://doi.org/10.1016/B978-0-12-818630-5.09038-2. Kravik, R., Berg, T., & Siddiq, F. (2022). Teachers’ understanding of programming and computational thinking in primary education – A critical need for professional development. Acta Didactica Norden. https://doi.org/10.5617/adno.9194 Voogt, J., & Roblin, N.P., (2012). A comparative analysis of international frameworks for 21st century competences: implications for national curriculum policies. J. Curric. Stud. 44 (3), 299–321. https://doi.org/10.1080/00220272.2012.668938

Networks hold a meeting during ECER. All interested are welcome.

Previous research has identified that cognitive ability and socioeconomic status (SES) indicated by parental education, occupation, or income (Marks, 2013) are the strongest predictors of school outcomes. Cognitive ability is the strongest predictor of school achievement, with correlations around .60-.70 (Gustafsson & Balke, 1993), while SES typically correlates around .30-.40 with school achievement (Sirin, 2005). Longitudinal investigations of the strength of the associations concluded that the influence of SES is declining (Marks, 2013). However, in Sweden the strength of the association between SES and achievement has increased during the last decades (Gustafsson & Yang Hansen, 2018), suggesting that equity of schooling outcomes has deteriorated.

Another important factor influencing school outcomes is gender. Girls tend to outperform boys in terms of grades internationally (Dwyer & Johnson, 1997), and this is true for Swedish students as well. Even more concerningly, boys are more at risk of dropping out of school in Sweden (World Bank, 2024).

In Sweden, compulsory education ends in the school year 9, while in the optional upper secondary school, there are 18 regular national programs of three years to choose from, six of which are preparatory for higher education such as university, and twelve of which are vocational. While entrance requirements vary between programs, all of them demand students to have passing grades in Swedish/Swedish as a second language, English, an d mathematics from their final year of compulsory schooling.

The main question, which can be investigated for all birth cohorts between 1948 and 2004, is the relative importance of cognitive ability, social background, cultural background, and gender as determinants of school failure and general school achievement, and how this varies as a function of school characteristics and societal factors.

Previous research has identified two main individual-level determinants of school outcomes: cognitive ability and socioeconomic status (SES) indicated by parental education, occupation, or income (Marks, 2013). Cognitive ability is the strongest predictor of school achievement, with correlations around .60-.70 (Gustafsson & Balke, 1993), while SES typically correlates around .30-.40 with school achievement (Sirin, 2005).

However, there are substantial country differences in these relationships, and longitudinal investigations of the strength of the associations have also been observed, and Marks (2013) concluded that the influence of SES is declining. On the contrary, for Sweden an increase in the strength of the association between SES and achievement has been observed during the last decades (Gustafsson & Yang Hansen, 2018), suggesting that equity of schooling outcomes has deteriorated. Moreover, gender differences have been observed in Sweden in terms of grades in line with international trends (Dwyer & Johnson, 1997), and boys are more at risk of dropping out of school in Sweden (World Bank, 2024).

Recently, much attention has been devoted to personality characteristics as determinants of success and failure in school, such as conscientiousness (Almlund et al., 2011), grit (Duckworth et al., 2007), and growth mindset (Dweck, 2008).

Other individual characteristics too have been shown to contribute to school achievement. Prominent examples are self-efficacy (Bandura, 1997), self-concept(Bong & Skaalvik, 2003), and intrinsic and extrinsic motivation (Ryan & Deci, 2000).

In Sweden, compulsory education involves school years 1 to 9, while in the optional upper secondary school, there are eighteen regular national programs of three years to choose from, six of which are preparatory for higher education such as university, and twelve of which are vocational. While entrance requirements vary between programs, all of them demand students to have passing grades in Swedish/Swedish as a second language, English, and mathematics from their final year of compulsory schooling.

The main aim of this study is to investigate individual and social determinants in the development of school failure using a longitudinal approach for two birth cohorts that have followed the same curriculum, born in 1998 and 2004.

Nationally and internationally, grade retention is a highly controversial measure to homogenise students with different competence levels. In Germany, some federal states (e.g., Hamburg and Berlin) have already abolished grade retention. In Bavaria and Bremen, however, grade retention rates are above the national average (2.3%, see Statistisches Bundesamt, 2018). In an international comparison, the rate of German pupils who have been retained at least once in the course of their educational career is above the OECD-average (Germany: 19.6%, OECD: 12.2 %; ꭓ² = 29558.56, df=1, p<.001; own calculations). Empirical evidence on the effectiveness of grade retention is still insufficient. International studies (Goos et al., 2021) showed that there are short-term improvements in performance after being retained, but they decrease in the medium and long term. Especially in highly hierarchically structured education systems such as Germany, Belgium, the Netherlands or Switzerland, grade retention is least effective (ibid.). In Germany, only a few reliable studies allow concrete statements about the effectiveness of grade retention: Positive effects of grade retention on students’ performance development were not proven (Ehmke et al., 2017; Fabian, 2020). Beyond this, Fabian (2020) also showed that there was no significant improvement in school grades of retained students. Marsh et al. (2017), however, found an increase in retained students’ math performance after repetition. With regard to the achieved school-leaving qualification, Bellenberg (1999) showed that grade retention is very often associated with school dropout and/or downward change of school track, thus reducing the probability of achieving higher school-leaving qualifications for repeating students. Demski and Liegmann (2014) reported only minor differences between repeaters and promoted students. Given the theoretical assumptions of credentialism (Bills, 2003), DiPrete et al.'s (2017) findings from an international comparison are noteworthy: In Germany, school-leaving certificates are particularly important for future success on the training and labour market. At the same time, objective competencies and obtained certificates are often incongruent (Brändle & Pohlmann, 2021). Empirical findings showed that particularly students with low qualifications successfully enter the training market if they have good grades and high educational aspirations – regardless of their cognitive and social skills (Holtmann et al., 2017). Due to the strong correlation between school-leaving qualifications and success on the training and labour market, the present study investigates whether grade retention has advantages or disadvantages for retained students. Previous research findings suggest that being retained might lead to lower school-leaving qualifications.

Collaborative problem-solving (CPS) skills have become an inevitable part of workforce readiness in contemporary society (Graesser et al., 2018). Numerous studies have shown that CPS is a powerful learning tool that could lead to more creative, efficient and comprehensive solutions than other approaches (Fiore, 2008). Sometimes it is the only possible way to solve complex problems. That is not surprising that the Organisation for Economic Cooperation and Development (OECD, 2019) includes the development of collaboration skills in the education development agenda for 2030. A lot of attempts were made to introduce CPS in everyday educational practice. However, the benefits of the CPS often fail to be achieved (Le et al., 2018).

Collaborative problem-solving is usually defined as working together toward a common goal (Hesse et al., 2015). It includes interdependency between group members in joint activity and shared responsibility for the group results.

Despite many contributions, there is a lack of instruments for measuring student-specific versions of collaborative processes during group work (Wang et al., 2009). The focus is often on the effect of this type of learning assessed through achievement data (Jansen, 2010) while the quality of the collaborative process is beyond research aims. Usually, self-assessment tools were used for this purpose accompanied by methodological limitation of subjective assessments. In these attempts, students' perceptions and experience with CPS are not distinguished from the quality of collaboration present during group work. Also, collaboration is assessed as an individual skill separate from its nature as a joint activity. This study aims to construct an instrument for assessing the quality of collaboration between students while trying to solve a complex problem. This study is part of the larger project PEERSovers with a focus on designing an evidence-based training program for enhancing high-school students' collaborative skills. The theoretical background for constructing the instrument involves a qualitative systematic literature review of 160 articles published between 2021 and 2022 that investigated differences between productive and unproductive peer collaboration (Baucal et al., 2023). Four aspects of peer interaction were identified as a result of this analysis. The first covers cognitive exchange between group members. Research shows that productive CPS includes argumentative dialogue between team members and constructive evaluation of ideas. Also, the effort is made to move from the personal opinion toward a shared understanding of the problem. Well-known Mercer studies (for example, Mercer et al., 2019; Mercer & Dawes, 2014) pointed out that exploratory talk during group work enhances the co-construction of joint cognitive activity, fosters critical thinking skills and contributes to the overall learning experience in educational settings. The second aspect refers to the emotional aspect of group work manifested through group atmosphere, presence of conflicts and tension, group cohesion, members' sense of belonging, mutual tolerance and empathy. In unproductive groups, members are disrespected and prevented from fully participating. Often the inequality in power is present during group work. Some members dominate in the dialogue space and prevent others from contributing. The third and fourth aspects are dedicated to two domains of group regulation: task activity regulation (time management, coordination of the activity, planning group activity, task-focus approach) and relationship regulation (group norms, sharing responsibility, dividing the assignment, efficient conflict management etc.). An unproductive group is often characterised by lots of off-task behaviour. Usually, few or only one participant takes overall responsibility for group work. We tried to operationalize these four aspects as dimensions of the instrument used for evaluating a CPS.

Verbal feedback is the oral communication between teachers and students that aims to provide constructive guidance on students’ progress, strengths, and areas for improvement, according to numerous educational scholars (Black & Wiliam, 2009; Hattie & Timperley, 2007; Shute, 2008). In secondary school settings providing effective feedback is a key component of a good education. The effectiveness of feedback in education is a widely studied and acknowledged aspect of the learning process (Black & Wiliam, 1998; Hattie, 2009; Karaman, 2021; Wisniewski et al., 2020). As teachers continually work to improve the learning outcomes for their students, the role of feedback, especially verbal feedback that takes place in classrooms everyday, becomes increasingly important.

The “Feed Up, Feed Back, Feed Forward” model, introduced by John Hattie and Helen Timperley in their influential 2007 paper, “The Power of Feedback,” presents a cyclical approach comprising three essential stages of effective feedback. These stages encompass setting clear objectives or “feed up,” delivering feedback on present performance, and proposing strategies for enhancement or “feed forward.” This implies that teachers should provide constructive feedback that is descriptive and focused on providing specific, actionable information aimed at helping the recipient improve or enhance their performance, skills, or understanding.

Many studies on verbal feedback have been conducted in the field of foreign or second language learning, exploring different types and functions of corrective feedback and their effects on language proficiency (Lyster & Saito, 2010). These studies have shown that providing oral corrective feedback not only helps students improve their accuracy and fluency in speaking, but also enhances their overall language proficiency. Although teachers might have experience or undergo professional development courses, their formative assessment practices could not be always effective. According to certain research findings, teachers’ attitudes about the usage of various forms of oral corrective feedback in the classroom do not necessarily align with their actual practices (Kim & Mostafa, 2021). Further comprehensive research on corrective feedback is necessary to investigate the alignment between teachers’ actual practices and their underlying ideas about feedback (Karimi & Asadnia, 2015). Therefore, this study focuses on the following research question: “To what extent do secondary school teachers provide constructive verbal feedback in classroom?”

Assessment and feedback are fundamental aspects of student experience in higher education as a measure of progress and achievement, and a central tenant of learning and engagement. Assessment and feedback play a pivotal role in increasing student knowledge and understanding, developing key skills, and promoting motivation and academic advancement.

The ever-increasing number of international students in higher education, representing a significant percentage student body, necessitates focused consideration of their experience of assessment and feedback. The research-based paper provides evidence based on the investigation of international students, and presents their voices in relation to the lived experience of assessment and feedback. The research focussed on elements of student experiences with regard to assessment and feedback, addressing the following Research Questions:

RQ 1- What do students currently encounter in terms of assessment and feedback?

RQ 2- How can we evolve assessment and feedback strategies to enhance the experience for international students?

The research utilised an interpretivist paradigm, concentrating on the perspectives of the respondents to develop knowledge of their experience and interpretation of this. Unlike research paradigms which primarily aim to universalise results, interpretivist research focuses on understanding the viewpoints of participants within their specific settings. It acknowledges that these viewpoints and behaviours are dynamic, altering based on temporal and situational factors. This facilitates the contrasting of outcomes across different time frames or locales (Cohen et al., 2017).

Higher Education needs to be ever responsive to technological innovation, pedagogical shifts, and the increasing diversity of the student body. The role of assessment and feedback within higher education remains central, acting as both a measure and a driver of student learning and engagement. Dr Katherine Hack, principal adviser in teaching and learning at the Higher Education Academy (HEA), stated that assessments and feedback are two of the most influential tools teachers have to direct and support learning (Advance HE, 2022). Indeed, assessment and feedback are an inherent and significant part of a student’s experience, and the prominence of this is captured annually in surveys such as the National Student Survey (NSS) for undergraduates (NSS) and the Postgraduate Taught Experience Survey (PTES). As such, continually re-evaluating and refining assessment and feedback, to align with the changing educational environment, is essential to keep practices contemporary and responsive to the student body and experience.

International students account for a notable percentage of postgraduate students across Higher Education Institutions (HEIs) worldwide, and represent a wide array of cultural and educational backgrounds. The difficulties for students needing to navigate an unfamiliar culture are well documented (Haider, 2018; Xie et al, 2019). Simultaneously, international students must also navigate new assessment and feedback practices as part of the transitional journey to Higher Education Institutions (HEIs), often in a different country. This presents a unique set of challenges, academic and cultural, adding an additional layer of complexity to an already nuanced landscape.

HEIs are faced with the task of ensuring that their assessment and feedback practices are inclusive and equitable catering to a diverse student body. The Quality Assurance Agency (QAA) began embedding equality, diversity, and inclusion (EDI) in its subject benchmarks in 2021, in the UK. This was part of a wider commitment to promoting EDI across HEIs, and to ensure that all students have an equal opportunity to succeed. This requires a continued commitment to academic rigour while adapting to the evolving needs and expectations of a diverse student body. This requirement can be applied globally, as HEIs seek to ensure that EDI is integrated within assessment and feedback. The paper seeks to investigate how this can be achieved.

Teaching quality has been empirically shown to be a key predictor of student learning (Stronge, 2013). In studying teaching quality, teaching effectiveness researchers have for years focused on the opportunities provided to students for learning, as these are crafted through teacher-student and student-student interactions with the content. Yet, following Fend’s (1981) distinction between opportunity and use and more recent work in the German-speaking countries on this issue (cf. Vieluf et al., 2020), teaching effectiveness researchers worldwide have started to more increasingly attend to not only the opportunities created for student learning, but also to how students make use of these opportunities, on the grounds that the former without the latter can only partially explain student learning.

Of particular interest in this line of research are the opportunities provided for student cognitive activation, often identified as the potential for cognitive activation, and students’ use of these opportunities, often identified as cognitive activity (Groß-Mlynek et al., 2022; Rieser & Decristan, 2023). This heightened interest in cognitive activation is justified both because of empirical findings corroborating its role for students’ cognitive and affective learning (e.g., Lazarides & Buchholz, 2019), but also due to studies showing cognitively activating teaching to be highly needed worldwide (cf. OECD, 2020).

Despite this increased interest, our review of the literature showed that in most extant studies scholarly attention has mostly been directed to the potential for cognitive activation without also exploring students’ cognitive activity. Only three studies concurrently attended to and measured both the opportunity and use for cognitive activation in relation to student learning (Lipowsky et al., 2009; Merk et al., 2021; Rieser & Decristan, 2023). These studies, however, differ not only in reporting mixed findings, but also in their methodological design: whereas the former two employed expert classroom observers’ ratings to capture the potential for cognitive activation and student ratings to capture cognitive activity, the latter utilized student ratings to measure both. Concurrently attending to different sources of information (e.g., expert classroom observers and students) is, however, critical, given scholarly calls (e.g., Fauth et al., 2020) to more systematically examine how the source of information contributes to the predictive validity of the teaching quality measures employed.

The scarcity of studies that concurrently attend to the predictive validity of opportunity and use (in cognitive activation); the mixed findings of these studies; and the fact that none of them concurrently used different sources of information to capture the opportunity for cognitive activation—note that the use of opportunities is typically captured only through student ratings—raise two questions:

- How does the predictive validity of ratings on the opportunity for cognitive activation compare with that of ratings on the use of cognitive activation?

- Does this differ when different sources of information (expert classroom observers vs. students) are employed to capture the opportunity for cognitive activation?

Addressing these questions can have important methodological implications for measuring aspects of teaching quality in more optimal ways, but also practical implications for teachers’ formative evaluation. However, to more adequately answer these questions, and especially the second one, attention needs to be paid to ensuring that the measures of the different sources obtained are aligned in the sense of tapping into similar—and if possible identical—aspects of teaching quality. Doing so becomes particularly important, given that our review of the literature showed only a few studies comparing aligned measures of teaching quality from different sources (e.g., van der Scheer et al., 2018)—and even in those cases, not with respect to the issue of their predictive validity.

Teaching quality has been researched extensively in the past years with a high number of empirical studies in educational sciences and psychology. To better understand how learning develops in the classroom, scholars are concerned with the reliable and valid measurement of teaching quality. In doing so, Helmke (2012) considers classroom observation as the “gold standard” amongst other ways of capturing teaching quality (e.g., student ratings in large-scale assessment) because of its direct assessment of teaching practices. However, classroom observation also draws on resources and can be prone to many sources of measurement error. Therefore, when performing classroom observation for any purpose (i.e., research, practical, policy) it is important to consider how to allocate (limited) resources such that high score reliability and valid conclusions about teaching quality are ensured.

Studies suggests that changing the presentation order of lesson segments could particularly affect score reliability (e.g., Mashburn et al., 2014). For instance, using the generic CLASS-Secondary observation system (Pianta et al., 2008), Mashburn et al. (2014) found that 20-minute lesson segments presented in a random order to raters achieved the best combination of reliability and predictive validity. In the present study, we used a different, hybrid observation system (i.e., comprising both generic and subject-specific aspects of teaching quality, Charalambous & Praetorius, 2018) that was first developed to capture teaching quality in German secondary mathematics classrooms, and that draws on the Three Basic Dimensions of teaching quality (classroom management, student support, and (potential for) cognitive activation, e.g., Klieme et al., 2009). The three basic dimensions have been shown to positively relate to students’ achievement in mathematics classrooms across several studies and various operationalizations (e.g., Baumert et al., 2010; for an overview see Praetorius et al., 2018).

Classroom management refers to teachers’ procedures and strategies that enable efficient use of time (time on task), as well as behavioral management (Kounin, 1970). Student support draws on self-determination theory (Deci & Ryan, 1985) and aims at both motivational and emotional support, as well as individualization and differentiation. Cognitive activation, finally, addresses opportunities for "high-order thinking" from a socio-constructivist perspective on teaching and learning (e.g., problem solving, Mayer, 2004).

Empirical evidence suggests that generic and subject-specific measures of teaching quality generate moderately correlated, but still unique information about classrooms (Kane & Staiger, 2012). Evaluating this finding, Charalambous and Praetorius (2018) conclude that subject-specific and generic measures together could explain more variance in student learning in mathematics than generic measures alone. Since subject-specificity might be considered a continuum rather than a binary characteristic, they argue that it could be meaningful for scholars to develop hybrid frameworks of teaching quality, which take both perspectives into account (i.e., generic and subject-specific, see also Charalambous & Praetorius, 2018).

The purpose of the present study is twofold: First, we aim at investigating the effect of presentation order on score reliability in two subjects. Second, we explore an optimal design for the implementation of our observation system in terms of score reliability. Towards this end, we assigned four trained raters to rate videotaped Norwegian mathematics and science lessons either in sequential 20-minute segments, or two nonsequential 20-minute segments.

Lesson Study (LS), a collaborative and inquiry-based model of teacher professional development, has received increased attention in recent years. It involves teachers in small groups identifying an issue in their teaching practice and organising an inquiry to learn more about it. Specifically, teachers jointly plan, teach and reflect on lessons. A rich body of mostly descriptive and qualitative studies suggests that with LS experience teachers may develop pedagogical knowledge, and may be able to identify students’ misconceptions (e.g. Cheung & Wong, 2014; Vrikki, Warwick, Vermunt, Mercer & van Halem, 2017). However, more large-scale controlled studies are needed in order to systematically evaluate the effect of LS (Benedict et al., 2023). In addition, even less evidence exists of the impact of teachers’ participation in LS on their students’ achievements (e.g. Cheung & Wong, 2014; Kager, Mynnott & Vock, 2023).

In addition, variations of the LS model include the presence of an external expert (i.e., LS facilitator, knowledgeable other, moderator). The literature identifies many ways that this external expert can support teachers, including enhancing in-depth discussions about students’ thinking, shaping the quality of the teachers’ inquiry, fostering teachers’ discussions by posing questions, encouraging teachers to share their experiences and managing the LS process (e.g. Akiba et al., 2019; Schipper et al., 2017; Bjuland & Helgevold, 2018; De Vrie, Verhoed & Goei, 2016). These responsibilities are not only vaguely described, but their effects have not been studied either. At the same time, research on teacher and school improvement argues for the important role of an advisory and research team that can work closely with teachers and support their attempt to design, implement and evaluate their action plans (e.g., Creemers & Kyriakides, 2012; Scheerens, 2013).

This paper addresses both limitations in the literature described above. First, it aims to examine how secondary mathematics teachers’ participation in LS affects their students’ achievement in reasoning. Second, it aims to examine how different types of support offered to LS teacher groups can further enhance students’ achievement. Specifically, it examines the impact of the support of a LS facilitator, who guides teachers through the LS process, fosters their discussions and promotes teacher learning which is expected to affect student learning outcomes. It compares this to the impact on student learning outcomes of the support of an LS advisor who in addition to guiding teachers through the LS process, offers subject advice and his/her own ideas to the teachers. Although having these kinds of support is not uncommon, little is known about the effect of different types of support that teachers may have in implementing LS.

In the contemporary era, the important advancements in technology are closely connected with the paramount importance of achievements in mathematics and sciences disciplines. The current societal landscape, characterized by technological progress and the prevalence of a data-driven environment, underscores the increasing importance of mathematical and scientific knowledge.

Mathematics is acknowledged as the foundational language supporting all STEM (Science, Technology, Engineering, and Mathematics) disciplines. Numerous stakeholders have underscored the imperative for a nation to enhance the mathematical skills and proficiency of its students (Mujtaba et al., 2014).

In a study analyzing TIMSS 2019 data for Turkey, teacher practices such as relating to daily life and prior knowledge, responding to student needs and encouraging students to participate in the discussion predicted mathematics achievement and explained the one-fifth of the between-schools variance (Sezer & Cakan, 2022).

Also, activities such as asking students to complete challenging exercises, which required them to go beyond the instruction, was an important predictor of mathematics achievement and had a positive relationship in 8th-grade (Sezer & Cakan, 2022). In Sweden, an analysis of TIMSS 2019 data showed that teaching activities such as asking to memorize formulas and listening to the teacher were positive predictors of TIMSS 8th-grade mathematics achievement, whereas relating information to daily life was a negative predictor (Eriksson et al., 2019).

The present study aims to investigate the extent to which specific teacher actions rated by students are predicting the Romanian students’ achievement in TIMSS 2019. Results could identify specific actions that could influence student achievement and propose those actions for further research and improvement.

Therefore, the research questions guiding the present study are as follows:

RQ.1 – To what extent do teachers’ actions, as observed and experienced by students, predict 8th-grade Romanian students’ mathematics achievement in TIMSS 2019 after controlling for socio-economic status?

RQ.2 – To what extent do teachers’ actions, as observed and experienced by students, predict 8th-grade Romanian students’ physics achievement in TIMSS 2019 after controlling for socio-economic status?

RQ.3 – To what extent do teachers’ actions, as observed and experienced by students, predict 8th-grade Romanian students’ chemistry achievement in TIMSS 2019 after controlling for socio-economic status?

RQ.4 – To what extent do teachers’ actions, as observed and experienced by students, predict 8th-grade Romanian students’ biology achievement in TIMSS 2019 after controlling for socio-economic status?

RQ.5 – To what extent do teachers’ actions, as observed and experienced by students, predict 8th-grade Romanian students’ earth sciences achievement in TIMSS 2019 after controlling for socio-economic status?

It has long been acknowledged that science skills play a crucial role in fostering economic development (Hanushek & Woessmann, 2012) and technological innovation (Varsakelis, 2006). Therefore, governments around the world are searching for ways to effectively enhance science education. Teachers and their instructional quality play an important role in student achievement and learning (Harris & Sass, 2011). It is also emphasized that teachers are one of the essential factors to enhance student skills and knowledge improvement (Harris & Sass, 2011). However, among several factors associated with students, teaching strategies, school, and home, teacher quality has an important role in student achievement.
The teacher quality framework (Goe, 2007) suggests that a teacher’s college degrees, certificates, and their test scores, among a group of inputs, can indicate who might be a successful teacher inside a classroom. However, teaching quality is not only defined by teacher certification and training but is also explained by what teachers do inside a classroom and how they teach, i.e., their classroom practices. Blömeke, Olsen and Suhl (2016) evaluated the relationship between educational input and process properties of schooling, and students’ cognitive outcomes with TIMSS 2011 data. The study revealed that teacher quality was significantly related to instructional quality and student outcome, while instructional quality was not a good predictor of student outcome.
Teacher’s knowledge of subject-matter, teaching skills, personal characteristics, and professional development have been found to be among the most effective characteristics of teachers (e.g., Toraman, 2019). However, no consensus on the essential teacher qualifications that explain students’ academic performance has been reached (Scheerens & Blömeke, 2016; Lee & Lee, 2020). Moreover, despite the emphasis on improving teaching qualifications (Goe, 2007), students from socioeconomically disadvantaged households and ethnic minorities are less likely to receive instruction from qualified teachers, since less qualified teachers are concentrated in schools and classrooms teaching students with low socioeconomic status and academic achievement (Luschei & Jeong, 2018).
Additionally, educational equity and quality are considered central points of Swedish school policy (Kelly et al., 2020). The Swedish Education Act underscores the school system’s mission of offering equal education quality, learning opportunities, and support to all students regardless of their background characteristics, and the type of schools they are attending (Swedish Education Act, 2010).
Against this background, the main objective of this study is to investigate the relationship between teachers’ experience, education, and their instructional practice, while controlling for students’ socioeconomic background and classroom SES composition. The study also aims to examine the schools’ compensatory effect of the teacher -related factors for educational equity, which is measured as the influence of home educational resources on students’ science achievements. According to the national curriculum for the compulsory school in Sweden, science is separated into the subjects of biology, chemistry, and physics. Therefore, the following research questions will be scrutinized in each of the science subjects:
1- Are teachers’ instructional practice, teachers’ experience and education, and classroom SES composition significantly related to student science achievement, controlling for students’ family background?

2- How do teachers’ experience and education, and classroom SES composition relate to their instructional practice?

3- How do instructional practice, teacher experience and education, classroom average achievement level and classroom SES composition mitigate students’ family background impact on their achievement?

Effective teaching is a multifaceted endeavour influenced by various factors. It extends beyond the mere possession of subject knowledge and teaching experience and is intricately tied to teaching methods (Darling-Hammond, 1997; Hudson et al., 2021; Leino et al., 2022; Shulman, 1987; Wharton-McDonald et al., 1998). International large-scale assessments (ILSA) such as the Trends in Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) have been widely used to establish links between teachers and their students in mathematics (e.g., Toropova et al., 2019), science (e.g. Fauth et al., 2019) and in reading (e.g., Johansson et al., 2015; Myrberg et al. 2019). However, research on teacher effects has yielded conflicting and inconclusive findings (Blömeke & Olsen, 2019; Coenen et al., 2018; Goe, 2007), partly due to the diverse methodological approaches employed in various studies. A major contributing factor is the lack of comparability and precision in defining teacher competence and teaching quality indicators. Furthermore, significant variations exist among countries in terms of the length, structure, and content of teacher education and instruction, necessitating country-specific analyses with accurate information on these specific features.

Investigating teacher effects on student achievement through large-scale data, such as PIRLS assessment data, presents distinct advantages. Firstly, large-scale assessments provide large samples, where whole classes of students are sampled, allowing comprehensive analysis of teacher effects across diverse student populations. Secondly, these assessments offer multiple measures for evaluating teachers and their teaching quality. However, a major challenge with ILSAs when estimating the impact of teachers on student outcomes is that we cannot account for students’ prior achievement. This complicates the task of isolating the direct influence of teachers on student learning outcomes.

In the present study, we address this limitation by utilizing the Swedish PIRLS 2016 sample to which additional register information from earlier and later grades was added. This means that we are not only able to account for prior achievement but also investigate long-term teacher effects on student performance. More specifically we are investigating the relationships between teachers’ reading specializations and the short-term and long-term impact of teachers’ reading comprehension practices in grade 4 on student performance in the PIRLS assessment and students’ subject grade in Swedish in sixth grade. We make use of scores from PIRLS, students’ national test results in grade 3 as well as subject grade in Swedish in grade 6.

Our research questions are:

1. What are the relationships between teachers’ reading specialization and students’ PIRLS achievement and subject grade in Swedish in grade 6?
2. What are the relationships between teachers’ reading comprehension activities and students’ PIRLS achievement and subject grade in Swedish in grade 6?

The relationship between student socioeconomic status (SES) and achievement is apparent in almost every educational system across the world. In the Nordic educational systems, although it may be weaker than other countries, SES is yet one of the strongest predictors of academic achievement (Sirin, 2005). Few studies, such as Myrberg & Rosén (2009), explore the direct and indirect effects of various factors on the relationship between SES and student achievement. Further investigation into these mechanisms is necessary, as SES is mostly used to control for selection bias (Gustafsson, Nilsen, & Hansen, 2018).

Given the Nordic educational systems’ aim of increasing equity, an important apsect to investigate is to what degree teacher related factors influence the relationship between student SES and achievement. Previous studies have indicated that teachers account for a significant portion of variance in achievement between classrooms (Darling-Hammond, 2014). Indeed, there is a consensus that teachers are a crucial school factor, and their competence is the foundation of high-quality schools, instruction, and learning (Blömeke, Olsen & Suhl, 2016). Teacher competence (e.g., higher quality instruction, efficient classroom management etc.) positively affects student achievement (Kelcey et al., 2019) and overall may have a positive effect on reducing inequity in education (Wößmann, 2008).

Specifically, an essential characteristic for effective teachers lies in their ability to deliver quality instruction, by explaining the content clearly and assessing student understanding of the subject matter (Ferguson, 2012). While instruction quality is related with student motivation, it has been documented to be positively related with student achievement in mathematics (Bergem, Nilsen & Scherer, 2016). This is an important aspect, especially in Sweden, where challenges arise due to the unequal distribution of well-qualified teachers across schools, leading to a widening achievement gap between schools and student groups (Yang Hansen & Gustafsson, 2019). Further, not many studies have investigated the effect that instructional quality has on the relationship between student SES and achievement in Nordic educational systems. Also, a limited number of studies have utilized representative International Large-Scale Assessment (ILSA) data in this context.

Thus, the main objective of the present study is to investigate how instructional quality relate to equity in education. Specifically, the study focuses on how student perceptions of instructional quality may mediate or moderate the relationship between SES and eighth grade students’ math achievement in the Swedish educational system. This study is grounded in the Dynamic Model of Educational Effectiveness theory that explores factors influencing student outcomes across all school levels, which can be either equitably or inequitably distributed. The model acknowledges the nested structure of educational systems and the relationships among various factors at different levels. Specifically, the model refers to observable instructional behaviors of teachers in the classroom and includes eight instructional quality dimensions: orientation, structuring, questioning, teaching-modelling, application, time management, creating a learning environment, and classroom assessment (Creemers & Kyriakides, 2013). The research questions that guide the study are:

1. To what extent does student perceived instructional quality and math achievement relate?
2. Does classroom level student perceived instructional quality have an effect on the relationship between student SES and math achievement?
3. To what extend does student level student perceived instructional quality mediates the relationship between student SES and math achievement?

The impact of the COVID-19 pandemic on children's and adolescents' development is a topic that has been intensively studied in recent educational research. However, the focus is often on the consequences that school closures and class cancellations had for students (e.g., Betthäuser, Bach-Mortensen, & Engzell, 2023; Patrinos, Vegas, & Carter-Rau, 2022). The consequences that COVID-19 infections can have on affected children and adolescents have been described primarily in clinical studies. Although younger individuals are less likely to have symptomatic infections or severe infections, they may experience symptomatic consequences that are persistent even after recovery (Behnood et al., 2022; Lopez-Leon et al., 2022). The severity of persistent consequences has been linked to the severity of symptomatology during illness (e.g., Radtke, Ulyte, Puhan, & Kriemle, 2021). For adults, such associations have already been empirically demonstrated regarding cognitive impairments (Hampshire et al., 2021). Although many clinical studies examined the consequences of COVID-19 infections in children, there is a lack of studies presenting results that are representative of specific subpopulations and that allow comparisons of groups of children that have already recovered from COVID-19 with groups that have not yet been infected. The present study addresses these research desiderata and examines the question of whether primary school children who have recovered from COVID-19 show disadvantages in terms of their cognitive abilities.

The influence of the school environment on pupils’ educational outcomes has long been established (Kutsyuruba et al., 2015; Mullis et al., 2013). Having a safe, structured and encouraging learning environment is associated with higher achievement and improved wellbeing (Cohen et al., 2009; Mullis et al., 2019; Thapa et al., 2013). Therefore, research on the school environment is important as it can have practical implications for educational policy. International large-scale assessments such as the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS), which are based on nationally representative samples of pupils at the target grade at the time of the assessment, allow researchers to examine aspects of the school environment from different perspectives. Factors such as school climate and school safety and discipline can be examined in both studies. The study cycles that are of particular focus in this paper are TIMSS 2019 and PIRLS 2021. These cycles can be seen as bookending the 2019/20 and 2020/21 academic years, during which extended periods of nationwide school closures occurred in Ireland as a result of the COVID-19 pandemic. These closures resulted in disruption to in-person teaching and learning and a transition to remote learning, which could potentially have impacted the school environment in the longer term.

Due to the unprecedented disruption in education that occurred between the administrations of TIMSS 2019 and PIRLS 2021, these data, stemming from school principals, class teachers, pupils, and parents/guardians, present a key opportunity to examine whether school environments in Ireland differed substantially between these time points. While we cannot infer causation when comparing cross-sectional datasets such as these, the nationally representative findings may help us to better understand the school landscape in the wake of the nationwide closures.

Being able to read can be seen as the foundation of a functioning democracy enabling learning, equal participation in society, and a condition for a healthy and successful life (European Commission, 2023; EDUFI, 2021). Reading performance is closely linked with other areas of academic performance, and there is a strong association between student well-being in school and reading performance (European Commission, 2023). Moreover, the danger of failing to meet academic or social expectations or to complete school with a basic level of academic proficiency has been termed “at-risk” (e.g. Novosel et al., 2012).

In Finland, the trends in students’ academic well-being (e.g. Helenius & Kivimäki, 2023; Read et al., 2022) and learning performance (e.g. Mullis et al., 2023; OECD, 2023) have been descending in the last decade. For example, Grade 4 students’ performance in reading has decreased from 2011 to 2021 as evidenced by the Progress in International Reading Literacy Study (PIRLS) (Mullis et al., 2023). The performance in reading declined by two points from 2011 to 2016 and by 17 points from 2016 to 2021. When examining the international reading benchmarks, the percentage of advanced achievers has dropped from 18% to 14% during this period. Meanwhile, the percentage of students at the low or below benchmark has doubled from 8% to 16%. We define these students as “students at risk”. They are in danger of not achieving adequate reading proficiency which is crucial for their learning success or failure in subsequent school years.

As for student well-being, the latest School Health Promotion Study (Helenius & Kivimäki, 2023) shows that more than one third of girls and one in five boys felt that their health was average or poor in Finland. The study also reports that experiences of physical threats and bullying have recently increased. Furthermore, school burnout has increased for a long time, especially among girls (Read et al., 2022).

Student well-being in school can be considered as a condition that enables positive learning outcomes but also as an outcome of successful learning and students’ satisfaction with their school experiences (Morinaj & Hascher, 2022). Student well-being in school consists of positive attitudes to school, enjoyment in school, positive academic self-concept, the absence of worries, physical complaints, and social problems in school, which can be used as indicators of well-being (Hascher, 2003).

In PIRLS, student well-being is indirectly measured by several indicators – such as school belonging, academic self-concept, experience of bullying, and absenteeism (Reynolds et al., 2024). PIRLS 2021 data was collected during the COVID-19 pandemic, but then there were no school closures in Finland. However, one year before these students’ schooling was disrupted, and they spent eight weeks in distance learning. Lerkkanen et al. (2022) showed that the Finnish students’ development in reading was slower from Grade 2 to 4 in the COVID sample compared to the pre-COVID sample. Previous research has detected the association between student well-being and learning performance but also the need for further examining this relation and the role of other factors associated with reading achievement, e.g. socioeconomic background (e.g. Bücker et al., 2018; Nilsen et al., 2022). For example, Manu et al. (2023) focused on the role of gender and parental education, and Torppa et al. (2022) the effects of the home literacy environment on the development of Finnish children’s reading comprehension.

In this study, we ask the following research questions, using the PIRLS reading assessment data from 2011 to 2021:

1) How has students’ well-being in school changed, if any, from 2011 to 2021?

2) How do students’ socioeconomic background and well-being factors predict the risk of low academic achievement in reading?