Conference Agenda

International large-scale assessments (ILSAs) such as TIMSS, ICCS, PIRLS, ICILS, PISA, and TALIS provide critical insights into educational outcomes and inequalities across diverse national contexts. They are widely used to monitor trends, enable international comparisons, and inform educational policy and practice (Johansson, 2016). At the same time, ILSAs function as laboratories for methodological innovation, supporting research that links individual outcomes to institutional and systemic conditions (Strietholt & Scherer, 2018). This symposium brings together recent research drawing on multiple ILSAs and is organized into five interrelated thematic sessions addressing digital transformation, civic education, educational inequality, student engagement and well-being, and methodological challenges in international assessment research.

The first thematic area focuses on digital competencies and emerging technologies in education. Drawing on ICILS, TALIS, and TIMSS, papers examine cross-national variation in technology and AI integration, the role of teacher characteristics and professional learning, and how digital provision and pedagogical use relate to achievement and inequality, including disparities linked to socioeconomic background and early tracking.

The second thematic area addresses civic attitudes, values, and engagement among young people. Using ICCS data, contributions explore students’ intentions to participate politically, their attitudes toward minorities, immigrants, and gender equality, and how school experiences relate to tolerant dispositions and engagement with social and environmental issues.

The third thematic area examines structural inequalities and contextual effects in education systems. Papers investigate how school composition, segregation, and socioeconomic context shape achievement and well-being across countries, including intersectional patterns by gender, immigrant background, and socioeconomic status. The session also addresses systemic resource inequalities through evidence on mathematics and science teacher shortages and their unequal distribution across schools.

The fourth thematic area focuses on students’ attitudes, motivation, and psychosocial factors. Contributions examine long-term gender gaps in reading behavior and achievement, the relationship between teacher gender and student outcomes, and trends in test-taking motivation relevant for interpreting achievement changes.

The fifth thematic area is explicitly methodological, with papers analyzing item-level features of achievement scales, participation and sampling standards, and the measurement of socioeconomic status, highlighting how technical choices shape conclusions about trends and inequality.

Socio-economic disparities in academic achievement have been repeatedly reported in the past (Coleman et al., 1966; Mullis et al., 2023). Accordingly, the continuous monitoring of these disparities is an important objective of international large-scale assessment studies. An established approach to efficiently measure students’ socio-economic background is to ask them about the number of books at home (Hanushek & Woessmann, 2011). Recently, however, doubts have been raised as to whether this variable is still a valid indicator of socio-economic status (Heppt et al., 2022; Schwippert, 2019). As printed books can be replaced by digital alternatives such as e-readers, audiobooks, or tablets, their absence no longer necessarily indicates lower levels of prosperity, education, and appreciation of reading. Conversely, owning books despite numerous digital substitutes could indicate a particular level of affluence and appreciation of reading. Against this background, the present contribution examined if the number of books at home changed in the period from 2001 to 2021. Furthermore, the development of the relationship between the book variable and parental education, as well as academic achievement was investigated. To examine these research questions, data from N = 429,292 students from 18 (predominantly Western) countries that participated continuously in the Progress in International Reading Literacy Study (PIRLS), conducted every five years from 2001 to 2021, were used. The samples from the participating countries were drawn in such a way as to be representative. In addition to the number of books, which was measured using an ordinal 5-point scale, the highest level of education attained by the parents and the students’ reading literacy were included in the study. The results showed that, across all countries, the proportion of households with more than 100 books declined between PIRLS 2001 and 2021. The difference between 2001 and 2021 was statistically significant. However, country-specific trend patterns emerged: More specifically, twelve countries saw a decrease of the proportion of households with more than 100 books, five countries showed a rather stable development, and one country saw an increase. Regardless of this, both, the correlations with parents’ educational attainment and with academic achievement showed a slight, statistically non-significant increase across all countries, with similar patterns emerging in the different countries, but also some notable differences in terms of the strength and variation of the relationships. Overall, the results suggest that the number of books remains a valid indicator of students’ socio-economic background. Limitations and implications for future research are discussed.

Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., & York, R. L. (1966). Equality of Educational Opportunity. US Government Printing Office. Hanushek, E. A., & Woessmann, L. (2011). The Economics of International Differences in Educational Achievement. In E. A. Hanushek, S. Machin, and L. Woessmann (Eds.), Handbooks in Economics, Vol. 3 (pp. 89-200). North-Holland. Heppt, B., Olczyk, M., & Volodina, A. (2022). Number of Books at Home as an Indicator of Socioeconomic Status: Examining its Extensions and their Incremental Validity for Academic Achievement. Social Psychology of Education, 25(4), 903-928. https://doi.org/10.1007/s11218-022-09704-8 Mullis, I. V. S., von Davier, M., Foy, P., Fishbein, B., Reynolds, K. A., & Wry, E. (2023). PIRLS 2021 International Results in Reading. Boston College, TIMSS & PIRLS International Study. https://doi.org/10.6017/lse.tpisc.tr2103.kb5342 Schwippert, K. (2019). Was wird aus den Büchern? Sozialer Hintergrund von Lernenden und Bildungsungleichheit aus Sicht der international vergleichenden Erziehungswissenschaft. Journal for educational research online, 11(1), 92-117. https://doi.org/10.25656/01:16789

This study examines the relationship between reading demand and performance in science items in the Programme for International Student Assessment (PISA) in Finland, focusing on changes between 2012 and 2022. Reading demand was defined as the combined number of words, scientific terms, symbols, and visual elements in each item, following Mullis, Martin, and Foy (2013). The analysis included 38 science items common to both assessment cycles. In addition to reading demand, item characteristics such as content area, response format, competence, and difficulty level were considered (OECD, 2017). Results show that Finnish students’ science performance declined significantly during the period, with the largest decreases observed in items with medium and high reading demand. Of the 38 tasks analyzed, 30 showed a statistically significant decrease in solution rates. While reading demand was associated with overall item solution rates, items with higher reading demand tended to have lower solution rates. Changes in performance were not clearly linked to reading demand alone. Among the examined other item characteristics, response format emerged as the most influential factor: open-ended tasks and complex multiple-choice items exhibited the largest declines, whereas simple multiple-choice items showed smaller changes. Performance differences were also examined by reading proficiency and immigrant background. Students with weaker reading skills experienced steeper declines in solution rates compared to those with stronger skills, although changes occurred across all reading demand categories. Immigrant students performed worse overall than their native peers, but their performance changes were smaller, likely because their initial solution rates were lower. The findings highlight the importance of reading literacy in science education and indicate that item characteristics, including reading demand, have a complex relationship with performance trends. However, the reading demand measure used here does not fully capture the unique linguistic features of Finnish, which may affect the actual cognitive demands of the items. These results have implications for assessment design and curriculum development, particularly in supporting students with lower reading proficiency and diverse linguistic backgrounds.

OECD. (2017). PISA 2015 Assessment and Analytical Framework. OECD Publishing, Paris. Mullis, I. V. S., Martin, M. O., & Foy, P. (2013). The impact of reading ability on TIMSS mathematics and science achievement at the fourth grade. Boston College.

In PISA 2022, mathematical reasoning was defined for the first time as one of the mathematical processes in addition to the three problem-solving processes assessed in earlier cycles. According to the PISA Mathematics Framework (OECD 2023, 27) “mathematical reasoning (both deductive and inductive) involves evaluating situations, selecting strategies, drawing logical conclusions, developing and describing solutions, and recognizing how those solutions can be applied”. In this presentation, we examine how expected student actions for mathematical reasoning have been operationalized in PISA mathematics items. About half of the reasoning items were new and half were used in earlier PISA cycles, where they were defined as measuring one of the three problem-solving processes. In PISA 2022, these so-called trend items were redefined as reasoning items. Therefore, we investigated if there were differences in how the new and the trend items measured reasoning. We also analysed Finnish students' performance in the reasoning items and how this performance is related to mathematics’ lessons activities in fostering reasoning. A detailed examination of the PISA 2022 framework shows that the definition of reasoning process category emphasizes drawing conclusions, justification, and critical analysis. We found that expected student actions described in the framework could be divided into three main categories: (1) argumentation (including producing justification, producing evaluation of justification, selecting justification), (2) selecting or producing the result of the reasoning (conclusion), (3) explaining the meaning. The new reasoning items developed for the PISA 2022 study emphasized argumentation, especially the production of justifications. In contrast, the trend items emphasized producing or selecting the result of reasoning. Overall, reasoning measured in PISA is implicit rather than explicit. Only in three of the trend items, students were asked to give justification. In two items, students needed to interpret the given situation and explain the meaning. The Finnish students' performance in items requiring justification was lower than in items involving selecting the result of reasoning. In addition, the performance aligned well with the proficiency levels of each trend item, but in the new items the students showed greater variation by proficiency level of the items. The Finnish students' experiences in activities that would foster reasoning in mathematics lessons were clearly fewer than the OECD average and they were not related to students’ performance in reasoning items. Based on the results, it appears that the measurement of reasoning and the activities that promote it should be further developed in the PISA study.

OECD. (2023). PISA 2022 mathematics framework. In PISA 2022 assessment and analytical framework. Paris, France: OECD Publishing. https://doi.org/10.1787/7ea9ee19-en

Sub-paper had to be withdrawn.

.