Conference Agenda
| Session | ||
09 SES 07 A: Innovations, Challenges, and Insights from International Large-Scale Assessments (Part 4): Students' Attitudes and Achievement
Symposium | ||
| Presentations | ||
09. Assessment, Evaluation, Testing and Measurement
Symposium Innovations, Challenges, and Insights from International Large-Scale Assessments (Part 4): Students' attitudes and achievement International large-scale assessments (ILSAs) such as TIMSS, ICCS, PIRLS, ICILS, PISA, and TALIS provide critical insights into educational outcomes and inequalities across diverse national contexts. They are widely used to monitor trends, enable international comparisons, and inform educational policy and practice (Johansson, 2016). At the same time, ILSAs function as laboratories for methodological innovation, supporting research that links individual outcomes to institutional and systemic conditions (Strietholt & Scherer, 2018). This symposium brings together recent research drawing on multiple ILSAs and is organized into five interrelated thematic sessions addressing digital transformation, civic education, educational inequality, student engagement and well-being, and methodological challenges in international assessment research. The first thematic area focuses on digital competencies and emerging technologies in education. Drawing on ICILS, TALIS, and TIMSS, papers examine cross-national variation in technology and AI integration, the role of teacher characteristics and professional learning, and how digital provision and pedagogical use relate to achievement and inequality, including disparities linked to socioeconomic background and early tracking. The second thematic area addresses civic attitudes, values, and engagement among young people. Using ICCS data, contributions explore students’ intentions to participate politically, their attitudes toward minorities, immigrants, and gender equality, and how school experiences relate to tolerant dispositions and engagement with social and environmental issues. The third thematic area examines structural inequalities and contextual effects in education systems. Papers investigate how school composition, segregation, and socioeconomic context shape achievement and well-being across countries, including intersectional patterns by gender, immigrant background, and socioeconomic status. The session also addresses systemic resource inequalities through evidence on mathematics and science teacher shortages and their unequal distribution across schools. The fourth thematic area focuses on students’ attitudes, motivation, and psychosocial factors. Contributions examine long-term gender gaps in reading behavior and achievement, the relationship between teacher gender and student outcomes, and trends in test-taking motivation relevant for interpreting achievement changes. The fifth thematic area is explicitly methodological, with papers analyzing item-level features of achievement scales, participation and sampling standards, and the measurement of socioeconomic status, highlighting how technical choices shape conclusions about trends and inequality. Overall, the symposium fosters dialogue among researchers, policymakers, and practitioners and advances rigorous, policy-relevant use of ILSAs to support more equitable and effective education systems. References Johansson, S. (2016). International large-scale assessments: what uses, what consequences? Educational Research, 58(2), 139–148. doi:10.1080/00131881.2016.116555 Strietholt, R., & Scherer, R. (2018). The contribution of international large-scale assessments to educational research: Combining individual and institutional data sources. Scandinavian Journal of Educational Research, 62(3), 368-385. Presentations of the Symposium Students’ Test Motivation Trends and Achievement Associations in Finnish PISA 2015–2022
International large-scale assessments aim to measure student competencies reliably across countries and time. However, questions have arisen about how much students’ effort and test-taking motivation affect their performance. Previous research (Butler & Adams, 2007; Penk & Richter, 2017) has shown that in low-stakes assessments, test-taking motivation and effort can affect achievement as strongly as socioeconomic status, making it a critical factor for interpreting trends. The recent decline in PISA scores has raised questions about whether changes in motivation contribute to these patterns.
This study addressed two research questions:
(1) How has test motivation changed across PISA cycles and by background variables?
(2) How has its relationship with mathematics performance evolved?
We examined Finnish PISA samples with national questionnaire items from 2015 (N = 5,353), 2018 (N = 4,942), and 2022 (N = 7,216). A latent variable for test motivation was constructed using item response theory. Its relationship with mathematics performance was examined through student-weighted linear regression models, controlling for year, gender, and socioeconomic status. This approach enabled us to assess both temporal changes in motivation and its explanatory power for achievement.
Results indicate that test motivation has declined for both boys and girls since 2015. Furthermore, the strength of the association between motivation and mathematics performance has weakened over time. The regression coefficient linking motivation to achievement differed significantly between 2015 and 2022, suggesting that motivation explained a smaller share of performance in the most recent cycle. Finally, we compared predicted score differences across cycles in models with and without motivation included. Accounting for motivation substantially reduced observed declines: the mathematics gap between 2015 and 2018 narrowed from 6.0 to 1.7 points, and the gap between 2015 and 2022 decreased from 26.4 to 17.2 points.
These findings highlight two key trends: declining test motivation and a reduced linear dependency between motivation and achievement. By using national extensions to PISA questionnaires, this study offers a novel perspective on student engagement during large-scale assessments and its implications for interpreting performance trends. The results underline the need to consider motivational factors when interpreting international comparisons, and raise questions about how changes in students’ attitudes towards testing may affect the validity of trend estimates. Future research should explore why the link between motivation and achievement is weakening and whether similar patterns occur in other countries.
References:
Butler, J., & Adams, R. J. (2007). The impact of differential investment of student effort on the outcomes of international studies. Journal of Applied Measurement, 8(3), 279–304.
Penk, C., & Richter, D. (2017). Change in test-taking motivation and its relationship to test performance in low-stakes assessments. Educational Assessment, Evaluation and Accountability, 29(1), 55–79. https://doi.org/10.1007/s11092-016-9248-7
The Impact of Emotional and Physical Safety at School on Student Achievement
Background and Purpose: Recent OECD PISA results show that students in Latvian schools report higher levels of bullying, abuse and insecurity than many peers across OECD countries. These experiences can affect concentration, confidence and engagement, influencing how reliably students demonstrate their competencies in assessment situations. Bystanders are also affected by tense school climates, which may lower academic performance and well-being. As large-scale assessments highlight the importance of contextual factors, understanding how emotional and physical safety relate to measured learning outcomes has become increasingly important.
The purpose of this study is to analyse safety-related trends in Latvian schools over the last decade and examine how emotional and physical safety indicators are associated with student achievement.
Research Questions:
1. What characterises school environments with high versus low levels of bullying and perceived safety?
2. How are emotional and physical safety indicators associated with student achievement across PISA cycles and national examinations?
3. To what extent do school‑level antibullying programmes influence safety indicators and students performance?
Methods: The study employs a mixed-methods design based on secondary analysis of PISA 2015, 2018 and 2022 cognitive test results and student questionnaire data, supplemented by national basic education examination outcomes. Quantitative analyses include descriptive trends, multilevel modelling of student‑ and school‑level variance, and cluster analysis identifying school profiles combining safety indicators and achievement. Qualitative analysis of school regulations and internal safety procedures supports interpretation of structural factors shaping students’ perceived safety.
Expected Outcomes: The study is expected to identify long-term trends in bullying and perceived school safety, reveal associations between safety-related background indicators and student achievement, and provide insight into the potential effectiveness of antibullying programmes. The development of safety‑achievement school profiles will highlight patterns that differentiate safer from less safe learning environments and show how these differences manifest in assessment results. The findings will contribute to a deeper understanding of how school climate factors interact with cognitive performance in large-scale assessment contexts.
Relevance: The study contributes to international discussions on how emotional and physical safety shape students’ learning experiences and influence the reliability of measured achievement. By integrating large-scale assessment data with school-level contextual information, it offers evidence on how safety conditions affect student performance and highlights the importance of supportive learning environments for ensuring equitable and valid demonstrations of student competencies.
References:
Bolarinwa, E.,L. (2024). Effects of Bullying behaviour on Secondary School Students’ Performance in Jos South Local Government, Plateau State, Nigeria. Pieejams: https://ijrpr.com/uploads/V5ISSUE10/IJRPR34067.pdf
Cornell, D., & Mayer, M. J. (2010). Why do School Order and Safety Matter? Pieejams: https://www.scopus.com/pages/publications/77951463808?origin=resultslist
Hinduja, S., & Patchin, J., W. (2023) Cyberbullying: Identification, Prevention, and Response. Pieejams: https://cyberbullying.org/Cyberbullying-Identification-Prevention-Response-2023.pdf
Mayer, M. J., Nickerson, A. B., & Jimerson, S. R. (2021). Preventing school violence and promoting school safety: Contemporary scholarship advancing science, practice, and policy. School Psychology Review, 50(2–3), 131–142. https://doi.org/10.1080/2372966X.2021.1949933
Okeke, C., Akobi, T., & Maseli, M. (2024). The perceived impact of bullying on academic performance of high school learners. Webology, 21(1). https://webology.org/data-cms/articles/20240116111144amWEBOLOGY%2021%20(1)%20-%204.pdf
Ray, G., McDermott, C. D., & Nicho, M. (2024). Cyberbullying on social media: Definitions, prevalence, and impact challenges. Journal of Cybersecurity, 10(1), Article tyae026. https://doi.org/10.1093/cybsec/tyae026
Salamanca, D. V., & Rollo, C. R. (2024). Bullying encountered and their perceived impacts on students’ sense of self. European Journal of Education Studies, 11(3). https://oapub.org/edu/index.php/ejes/article/view/5231
Reading More, Achieving More? Long-term Gender Gaps in Reading Behavior and Reading Achievement
Gender gaps in reading achievement favoring girls are observed consistently across countries and over time, yet the mechanisms underlying changes in these gaps remain insufficiently understood. Psychological explanations often emphasize relatively stable gender differences in cognitive abilities or motivation, but such accounts struggle to explain why gender gaps vary substantially across countries and historical periods (Hyde, 2005). This study investigates whether changes in gender differences in reading behavior are associated with changes in gender differences in reading achievement at the country level, thereby providing evidence for environmentally driven explanations of gender gaps.
Using country-level data from six cycles of international large-scale assessments of reading conducted between 1970 and 2016 (Mullis et al., 2017), we examine long-term trends in gender gaps in both free-time reading behavior and reading achievement at the end of primary school. The data combine multiple cross-sectional samples of different student cohorts, allowing us to study within-country changes over time rather than cross-national differences at a single point. Gender gaps in reading behavior and achievement are operationalized as standardized differences between girls and boys.
We estimate longitudinal linear models with country fixed effects, thereby accounting for all time-invariant country characteristics such as language, educational traditions, and institutional structures. This modeling strategy isolates within-country variation over time and allows us to assess whether increases or decreases in gendered reading behavior coincide with corresponding changes in achievement gaps.
Across model specifications, we find a positive and statistically significant association between changes in reading behavior gaps and changes in reading achievement gaps. Substantively, the estimated effect implies that realistic within-country changes in reading behavior gaps—typically on the order of 0.25 standard deviations—are associated with meaningful reductions in achievement gaps, amounting to roughly 15–20 percent of the average gender gap in reading achievement.
These findings are consistent with the Gender Similarity Hypothesis (Hyde, 2005) emphasizing gendered socialization and learning environments. Importantly, while this study does not directly test psychological mechanisms, it demonstrates that gender gaps can change over relatively short historical periods—changes that are unlikely to be explained by genetic or innate psychological factors. The results highlight reading behavior as a malleable, policy-relevant factor and suggest that encouraging boys’ engagement in reading outside school may contribute to reducing gender inequalities in reading achievement, in line with international policy goals on gender equality in education (United Nations, 2015).
References:
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice Hall.
Eagly, A. H., & Wood, W. (2012). Social role theory. In P. A. M. Van Lange, A. W. Kruglanski, & E. T. Higgins (Eds.), Handbook of theories of social psychology (Vol. 2, pp. 458–476). Sage. https://doi.org/10.4135/9781446249222.n49
Hyde, J. S. (2005). The gender similarities hypothesis. American Psychologist, 60(6), 581–592. https://doi.org/10.1037/0003-066X.60.6.581
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2017). PIRLS 2016 international results in reading. TIMSS & PIRLS International Study Center, Boston College.
OECD. (2015). The ABC of gender equality in education: Aptitude, behaviour, confidence. OECD Publishing. https://doi.org/10.1787/9789264229945-en
United Nations. (2015). Transforming our world: The 2030 agenda for sustainable development. United Nations.
Teacher Gender And Student Outcomes: Evidence From TIMSS–PIRLS On Achievement and Self-Confidence
Gendered professional roles shape everyday institutional practices, yet little is known about how these roles influence learning through direct professional–client interactions. In education, where teaching remains a highly gender-segregated profession in many countries, the gender relationship between teachers and students represents a potentially important but understudied mechanism linking occupational segregation to educational outcomes. This study examines whether and how teacher gender is associated with students’ academic achievement and self-confidence.
Teaching occupies a distinctive position among gendered professions. While often framed as a caring occupation, it also entails authority, expertise, and the transmission of socially valued knowledge. Gender imbalances within the teaching workforce vary across education systems, grade levels, and subject domains, yet primary education remains predominantly female in many contexts. Because schools play a central role in the intergenerational transmission of norms and competencies, gendered expectations attached to teaching may shape instructional practices and students’ learning experiences. Learning outcomes emerge through daily teacher–student interactions, where teacher gender may interact with student gender to influence classroom dynamics, pedagogical choices, and expectations, with consequences for both cognitive and non-cognitive outcomes.
Empirical evidence on gendered teacher effects remains mixed and highly context-specific, with most studies focusing on single countries and academic achievement alone. Using TIMSS–PIRLS 2011 grade 4 data, this study adopts a comparative perspective to examine gendered teacher–student relationships across countries. It analyzes associations between teacher gender and achievement in reading and mathematics, as well as students’ self-confidence in both domains. To address student- and school-level confounding, the study employs a student fixed-effects approach that exploits within-student variation across domains, following strategies used by Jerrim et al. (2020) and Cattaneo et al. (2017). Analyses are conducted in 14 countries where students have different teachers for reading and mathematics. Six of these countries are European: Germany, Hungary, Italy, Norway, Slovakia, and Sweden.
Results indicate significant associations between teacher gender and outcomes in a small number of countries. Male teachers are associated with higher mathematics achievement in two countries, and with higher reading self-confidence in one. A second set of analyses examines whether having a teacher of the same gender as the student is associated with outcomes. Same-gender teacher-student pairings are linked to higher reading achievement in eight countries and higher reading self-confidence in ten countries, with an additional effect on mathematics self-confidence observed in one country.
References:
Cattaneo, M. A., Oggenfuss, C., & Wolter, S. C. (2017). The more, the better? The impact of instructional time on student performance. Education Economics, 25(5), 433–445. https://doi.org/10.1080/09645292.2017.1315055
Jerrim, J., López-Agudo, L. A., & Marcenaro-Gutiérrez, O. D. (2020). The association between homework and primary school children’s academic achievement: International evidence from PIRLS and TIMSS. European Journal of Education, 55(2), 248–260. https://doi.org/10.1111/ejed.12374
| ||