JavaScript is Disabled
Your browser's JavaScript functionality is disabled. It has to be enabled to use this function of ConfTool.
Here you can find information on how to enable JavaScript
If you have any problems, please contact the organisers at ecer2026@eera.eu.

Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

Please note that all times are shown in the time zone of the conference. The current conference time is: 19th Aug 2026, 20:17:15 EET

Daily Overview

Session

09 SES 15 B: Unraveling Mathematics and Science Reasoning Using TIMSS and PISA

Time:

Friday, 21/Aug/2026:

11:00 - 12:30

Session Chair: Isa Steinmann

Location: 5026 | Linna building | 5 Fl.

cap: 55

Paper Session

Presentations

09. Assessment, Evaluation, Testing and Measurement
Paper

Home and School Context in Fourth-Grade Mathematics and Science Reasoning in Türkiye: Evidence from TIMSS 2019 and 2023

Mustafa Çapar, Derya Kılıçoğlu

Eskişehir Osmangazi University, Turkey (Türkiye)

Presenting Author: Çapar, Mustafa

International large-scale assessments repeatedly show that students’ achievement is systematically patterned by both home-based resources and learning supports and school-based institutional conditions, with important implications for equity and opportunity (Schütz et al., 2008; Werfhorst & Mijs, 2010)(Schütz et al., 2008; Werfhorst & Mijs, 2010). TIMSS-based research similarly demonstrates that contextual factors relate not only to overall achievement but also to performance across content and cognitive domains, including the knowing–applying–reasoning structure (Balfaqeeh et al., 2022; Mullis & Martin, 2017; Zhang & Li, 2019). Yet less is known about how such home and school context patterns align with cross-cycle changes in cognitive-domain performance.

Against this background, this study investigates how the extraordinary performance increase observed in Türkiye between TIMSS 2019 and TIMSS 2023 co-occurs with patterns in home and school context indicators. The analytical focus is on reasoning achievement in mathematics and science. This focus is justified because reasoning represents higher-order cognitive processes within the TIMSS cognitive domains (Mullis & Martin, 2017), and domain-level outcomes may be sensitive to item characteristics and cognitive-domain features that warrant explicit consideration (Liou & Bulut, 2020; Nixon & Barth, 2014). Descriptive results indicate that the mean increase in mathematics reasoning is larger (Δ=+40.13) and that the relative share of reasoning within total mathematics achievement also increases (Δ=+2.27), while relative changes for knowing and applying are small and negative; a similar pattern is observed in science.

The study is structured around the following research questions:

In TIMSS 2019 and 2023, how are home context indicators—home educational resources, parental education, and the home learning environment—associated with fourth-grade students’ absolute reasoning achievement in mathematics and science ?
How are these home context indicators associated with the relative share of reasoning within total achievement in mathematics and science ?
To what extent do school context indicators (e.g., academic emphasis, disciplinary climate, readiness, and resource shortages) provide additional explanatory power for reasoning outcomes after controlling for home context?
How do the direction and magnitude of these associations change between TIMSS 2019 and TIMSS 2023?

The conceptual and theoretical framework adopts an opportunity-structure perspective, viewing achievement as the outcome of knowledge-acquisition processes that are shaped by both family-based resources and institutional conditions that structure learning opportunities (Baumert et al., 2009; Schütz et al., 2008; Werfhorst & Mijs, 2010). Accordingly, the incremental contribution of school context beyond home context is treated as central to equity-relevant interpretations. The European/international dimension is established through TIMSS’s standardised measurement framework and international database conventions, which enable comparable analyses across cycles and systems (Fishbein et al., 2021, 2025; Mullis & Martin, 2017; von Davier et al., 2024).

Methodology, Methods, Research Instruments or Sources Used
This study draws on Türkiye’s Grade 4 data from TIMSS 2019 and TIMSS 2023 conducted by the IEA. The data include mathematics and science achievement assessments and standardised indicators derived from student, home, and school questionnaires (Mullis & Martin, 2017; Fishbein et al., 2021, 2025). The analytical sample comprises students who participated in the mathematics and science assessments in the respective cycles. All analyses apply student weights (TOTWGT) to account for TIMSS’s stratified, multistage sampling design (Fishbein et al., 2021, 2025).
Data preparation and analysis followed TIMSS international database conventions. The IEA IDB Analyzer was used to merge the relevant files and generate an analysis-ready dataset, after which SPSS was used for variable preparation, descriptive statistics, and regression modelling; Excel supported data checks and validation (von Davier et al., 2024). Given the use of international large-scale assessment data, analytic decisions and reporting were guided by recommended practices for secondary analyses (Rutkowski et al., 2010).
Reasoning outcomes in mathematics and science were operationalised in two complementary ways. First, absolute reasoning achievement was measured using TIMSS plausible values and summarised as M_REAS (mathematics) and S_REAS (science), consistent with plausible-value methodology (Wu, 2005). Second, a relative reasoning share indicator captured the proportion of reasoning within overall cognitive-domain performance (knowing, applying, and reasoning) and was operationalised as M_PCT_REAS and S_PCT_REAS, aligned with TIMSS’s cognitive domain structure (Mullis & Martin, 2017).
Home context was represented by composite indices capturing home educational resources (HERI), parental education (PEI), and the home learning environment (HLEI). School context was represented by indicators such as academic emphasis, disciplinary climate, student readiness, and resource shortages (Fishbein et al., 2021, 2025). Analytically, the study estimated multiple and block (hierarchical) regression models separately for mathematics and science in each TIMSS cycle. Home context indicators were entered in the first block, and school context indicators were added in the second block to quantify schools’ incremental explanatory contribution (ΔR²) beyond home context and to compare the magnitude and pattern of associations between TIMSS 2019 and TIMSS 2023 (Rutkowski et al., 2010).
Conclusions, Expected Outcomes or Findings
Türkiye’s TIMSS Grade 4 results show a substantial increase in mathematics and science achievement between 2019 and 2023. Descriptive evidence suggests that the improvement is not confined to a single content domain: gains are observed across mathematics content areas (including Number, Geometry, and especially Data) and across science domains (including Physical Science and Earth Science), consistent with TIMSS reporting structures. The relative stability of standard deviation values across cycles is consistent with a broad upward shift in performance rather than change concentrated among a narrow high-achieving subgroup.
The change is also not cognitively neutral. Within the TIMSS cognitive-domain framework, the relative share of Reasoning increases while the shares of Knowing and Applying decline in both mathematics and science. In Türkiye, the reasoning share rises by +2.27 percentage points in mathematics and +1.65 in science, indicating that overall performance gains are accompanied by a relative strengthening in higher-order reasoning.
Regression results addressing the research questions indicate that home-based factors are consistently associated with absolute reasoning achievement in both subjects. In particular, parental education (PEI) and the home learning environment (HLEI) show robust positive associations with reasoning outcomes, whereas home indicators explain comparatively less variance in the relative reasoning share. School-based factors remain relevant after controlling for home context, providing an additional but more modest increment in explained variance for absolute reasoning outcomes. Overall, the findings suggest that Türkiye’s 2019–2023 achievement gains and the reasoning-oriented shift are more strongly patterned by home-based opportunity structures, while school context contributes incremental explanatory power beyond home, especially for absolute reasoning achievement—an equity-relevant pattern in international large-scale assessment contexts.
References
Balfaqeeh, A., Mansour, N., Forawi, S. (2022). Factors Influencing Students’ Achievements in the Content and Cognitive Domains in TIMSS 4th Grade Science and Mathematics in the United Arab Emirates. Education Sciences, 12(9). https://doi.org/10.3390/educsci12090618
Baumert, J., Lüdtke, O., Trautwein, U., & Brunner, M. (2009). Large-scale student assessment studies measure the results of processes of knowledge acquisition: Evidence in support of the distinction between intelligence and student achievement. Educational Research Review, 4(3), 165–176. https://doi.org/10.1016/j.edurev.2009.04.002
Fishbein, B., Foy, P., & Yin, L. (2021). TIMSS 2019 User Guide for the International Database (2nd ed.). (2021930389; 2nd Ed.). Boston College, TIMSS & PIRLS International Study Center. https://timssandpirls.bc.edu/timss2019/international-database/
Fishbein, B., Taneva, M., & Kowolik, K. (2025). TIMSS 2023 User Guide for International Database. Boston College, TIMSS & PIRLS International Study Center. https://timss2023.org/data
Liou, P.-Y., & Bulut, O. (2020). The Effects of Item Format and Cognitive Domain on Students’ Science Performance in TIMSS 2011. Research in Science Education, 50(1), 99–121. https://doi.org/10.1007/s11165-017-9682-7
Mullis, I. V. S., & Martin, M. O. (2017). TIMSS 2019 Assessment Frameworks. Boston College, TIMSS & PIRLS International Study Center. http://timssandpirls.bc.edu/timss2019/frameworks/
Nixon, R. S., & Barth, K. N. (2014). A Comparison of TIMSS Items Using Cognitive Domains. School Science and Mathematics, 114(2), 65–75. https://doi.org/10.1111/ssm.12054
Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International Large-Scale Assessment Data: Issues in Secondary Analysis and Reporting. Educational Researcher, 39(2), 142–151. https://doi.org/10.3102/0013189X10363170
Schütz, G., Ursprung, H. W., & Wößmann, L. (2008). Education Policy and Equality of Opportunity. Kyklos, 61(2), 279–308. https://doi.org/10.1111/j.1467-6435.2008.00402.x
von Davier, M., Fishbein, B., & Kennedy, A. (2024). TIMSS 2023 Technical Report (Methods and Procedures). Boston College, TIMSS & PIRLS International Study Center. https://timss2023.org/methods/
Werfhorst, H. G. V. de, & Mijs, J. J. B. (2010). Achievement Inequality and the Institutional Structure of Educational Systems: A Comparative Perspective. Annual Review of Sociology, 36(Volume 36, 2010), 407–428. https://doi.org/10.1146/annurev.soc.012809.102538
Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2), 114–128. https://doi.org/10.1016/j.stueduc.2005.05.005
Zhang, L., & Li, Z. (2019). How Does Inquiry-Based Scientific Investigation Relate to the Development of Students’ Science Knowledge, Knowing, Applying, and Reasoning? An Examination of TIMSS Data. Canadian Journal of Science, Mathematics and Technology Education, 19(3), 334–345. https://doi.org/10.1007/s42330-019-00055-9

09. Assessment, Evaluation, Testing and Measurement
Paper

Exploring the Impact of Cognitive Activation on Science Achievement: Quasi-experimental Evidence from TIMSS 2015-2023 in Sweden

Zahra Hasani Yourdshahi, Leah Natasha Glassow, Linda Borger

University of Gothenburg, Sweden

Presenting Author: Hasani Yourdshahi, Zahra

In recent decades, research on teaching quality has increasingly emphasized inquiry-based
instructional practices that engage students in higher-order thinking and reasoning processes
(e.g., Kelley & Knowles, 2016). Among these approaches, cognitive activation,
i.e., teaching practices that challenge students intellectually, stimulate reflection, and require
them to apply knowledge in novel situations, has gained particular attention (e.g., Förtsch et al., 2016; Klieme et al., 2006). Empirical evidence has consistently shown that cognitively activating instruction is positively associated with student achievement, motivation, and long-term learning outcomes (e.g., Baumert et al., 2010; Klieme et al., 2009).

A previous study by Atlay, Tieben, Hillmert, and Fauth (2019), found that while cognitively activating teaching, as reported by students, was generally associated with higher student achievement, its benefits were greatest for students from higher socioeconomic status (SES) backgrounds. In science education, the relevance of cognitive activation is particularly strong. Scientific inquiry requires students to reason with evidence, interpret data, draw conclusions, and develop systematic strategies to investigate scientific questions, which aligns closely with cognitively activating instruction (Lederman, 2019). However, evidence from previous TIMSS cycles suggests that simply measuring the frequency of inquiry-based activities may not effectively capture their impact, since the relationship between inquiry frequency and achievement does not appear to be linear (Mullis, Martin, & von Davier, 2021).

This study empirically examines these issues in the Swedish context, investigating
whether teachers’ generic and subject-specific cognitive activation practices contribute to
student science achievement and whether these effects differ by student socioeconomic
background. Using data from the Trends in International Mathematics and Science Study
(TIMSS), which links eighth-grade students directly to their science teachers, the study
employs a within-student between-subjects (WSBS) design. This design
enables robust estimation of how teaching practices predict student learning outcomes.

Based on previous findings suggesting that cognitively activating teaching may offer greater advantages to students from higher SES backgrounds, we hypothesize that the effects of cognitive activation on science achievement will differ by SES, potentially offering greater
advantages to high-SES students, thereby widening rather than reducing existing achievement
disparities. Accordingly, this study addresses the following research questions:

To what extent do teachers’ generic and subject-specific cognitive activation practices
impact student achievement in science?
Does the effectiveness of teachers’ generic and subject-specific cognitive activation
practices on student achievement vary depending on students’ socioeconomic
background?

This study adopts the Three Basic Dimensions (TBD) framework (Klieme et al., 2009)
to conceptualize teaching quality, which distinguishes three core dimensions: Cognitive
Activation, Supportive Climate, and Classroom Management. While these dimensions
jointly characterize teaching quality, the present study focuses specifically on the Cognitive
Activation dimension and examines its association with student achievement in science.

Despite strong theoretical support, empirical findings on inquiry-based teaching and
cognitively activating practices are mixed. Other studies show positive relations
when inquiry involves hands-on or application-oriented activities such as experimentation,
modeling, or drawing conclusions from investigations (Grabau & Ma, 2017). Although cognitively activating instruction is assumed to support all learners, evidence suggests that its benefits may not be distributed evenly. Using PISA 2012 data, Caro et al. (2016) found that cognitive activation was generally associated with higher achievement but yielded stronger benefits for students from higher socioeconomic backgrounds. By contrast, longitudinal studies, such as Baumert et al. (2010) found no evidence of SES-based differences in the effects of cognitive activation. Overall, the existing evidence on how cognitive activation interacts with student socioeconomic background remains inconclusive, indicating a need for more robust research designs capable of isolating within-student variation, such as the WSBS approach used in the present study.

Methodology, Methods, Research Instruments or Sources Used
The present study uses Swedish data from TIMSS 2015, 2019, and 2023. In addition to student achievement scores, contextual information is collected from students, teachers, and school principals. From the student questionnaire, the scale related to students’ socioeconomic background is selected. In addition, from the teacher questionnaire, variables related to teachers’ generic and subject-specific cognitive activation practices are selected. The choice of variables is justified by previous literature indicating their influence on student achievement.
Given the hierarchical sampling structure of TIMSS studies, clustering is accounted for in the present study by adjusting standard errors at the classroom level. An ideal way to study the effect of teaching practices on student achievement would be to randomly assign students to different instructional approaches. In reality, however, the allocation of students to teaching practices is not random (Bietenbeck, 2014). To overcome these issues, earlier research has relied on student fixed-effects models (e.g., Bietenbeck, Piopiunik, & Wiederhold, 2018). Estimating such models requires data with repeated observations of students across different subjects. TIMSS makes this possible by providing multiple science subject teachers (biology, chemistry, and physics) for each student within the same cycle. Building on this feature, a student fixed-effects approach is applied to estimate the impact of teachers’ generic and subject-specific cognitive activation practices on science achievement, while eliminating unobservable characteristics that remain constant across biology, chemistry, and physics teaching contexts.
TIMSS grade 8 sample is used, since the students are usually taught science by at least two subject-specialist teachers in biology, chemistry, and physics. This structure allows a focus on practices of both generic cognitive activation (GCA) and subject-specific cognitive activation (SCA) within science classrooms.The WSBS identification relies on within-student differences in exposure to cognitive activation reflecting differences between teachers rather than systematic differences between science subjects. We therefore assume that teachers’ subject-specific cognitive activation (SCA) is functionally comparable across biology, chemistry, and physics for a given student. This assumption is plausible because the SCA indicators used in TIMSS represent cross-disciplinary scientific practices that can be used in all three science subjects. While the content necessarily differs by domain, the items target the same underlying pedagogical dimension aiming to provide students with opportunities to engage in inquiry-oriented scientific activity. The within-country student weight is applied. Data preparation is conducted in SPSS, while RStudio and Stata are used for factor score estimation, sensitivity analyses, and model estimation.

Conclusions, Expected Outcomes or Findings
The effects of teachers’ generic and subject-specific cognitive activation practices are estimated using within-student, between-subjects analyses with student and subject fixed effects (biology, chemistry, and physics). Models are developed stepwise, estimating linear associations between GCA/SCA and achievement, adding interactions with SES, and extending to quadratic terms (GCA², SCA²) to test for non-linearity. Across all specifications, the estimated effects of GCA and SCA are not statistically significant, the SES interactions are small and statistically non-significant, and teacher qualifications included as controls are not statistically significant predictors of student achievement in science lessons. In the Swedish TIMSS science context examined here, cognitively activating practices neither widened nor reduced SES-based achievement differences, indicating that cognitive activation did not function as an equity-relevant mechanism in this setting. Sensitivity analysis and robustness tests support the reliability of these findings, and the absence of significant effects under this more robust specification suggests that previously reported positive associations may partly reflect residual confounding rather than the independent influence of cognitive activation.
One limitation is that only about 50% of students showed within-student variation in GCA and SCA, which reduces identifying variation in the WSBS design and may help explain the null effects. Another limitation is that the study relies on teacher-reported measures of instructional practice, which may not fully capture implementation quality of cognitively activating instruction.

References
Atlay, C., Tieben, N., Hillmert, S., & Fauth, B. (2019). Instructional quality and achievement
inequality: How effective is teaching in closing the social achievement gap?. Learning
and Instruction, 63, 101211.

Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., ... & Tsai, Y. M. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American educational research journal, 47(1), 133-180.

Bietenbeck, J. (2014). Teaching practices and cognitive skills. Labour Economics, 30, 143-
153.

Bietenbeck, J., Piopiunik, M., & Wiederhold, S. (2018). Africa’s skill tragedy: Does teachers’
lack of knowledge lead to low student performance?. Journal of human resources,
53(3), 553-578.

Caro, D. H., Lenkeit, J., & Kyriakides, L. (2016). Teaching strategies and differential effectiveness across learning contexts: Evidence from PISA 2012. Studies in educational evaluation, 49, 30-41.

Förtsch, C., Werner, S., von Kotzebue, L., & Neuhaus, B. J. (2016). Effects of biology teachers’ professional knowledge and cognitive activation on students’ achievement. International Journal of Science Education, 38(17), 2642-2666.

Grabau, L. J., & Ma, X. (2017). Science engagement and science achievement in the context
of science instruction: A multilevel analysis of us students and schools. International
Journal of Science Education, 39(8), 1045-1068.

Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM
education. International Journal of STEM education, 3(1), 11.

Klieme, E., Lipowsky, F., Rakoczy, K., & Ratzka, N. (2006). Qualit.tsdimensionen und Wirksamkeit von Mathematikunterricht [Quality dimensions and effectiveness of mathematics teaching]. Untersuchungen zur Bildungsqualität von Schule, 127-146.

Klieme, E., Pauli, C., & Reusser, K. (2009). The pythagoras study: Investigating effects of
teaching and learning in Swiss and German mathematics classrooms. In T. Janik & T.
Seidel (Eds.), The power of video studies in investigating teaching and learning in the
classroom (pp. 137–160). Waxmann Publicing Co.

Lederman, N. G. (2019). Contextualizing the relationship between nature of scientific
knowledge and scientific inquiry: Implications for curriculum and classroom practice.
Science & Education, 28(3), 249-267.

Mullis, I. V., Martin, M. O., & von Davier, M. (2021). TIMSS 2023 Assessment Frameworks. International Association for the Evaluation of Educational Achievement.

09. Assessment, Evaluation, Testing and Measurement
Paper

Social-Emotional Skills and the Mathematics Gender Gap in PISA 2022: Curiosity as a Key Lever

Shuang Xie¹, Yixin Zhang², Shengjie Lin³, Haoyan Huang²

1: the Chinese University of Hong Kong, Hong Kong; 2: University of Helsinki, Finland; 3: University of New Mexico, USA

Presenting Author: Zhang, Yixin; Huang, Haoyan

Mathematics achievement serves as a critical gateway to higher education and future career success (Stieff & Uttal, 2015; Wai et al., 2010), and the gender gap in mathematics would contribute to unequal opportunities in them. Despite ongoing efforts to promote gender equity in education, persistent mathematics gender gaps--often favoring males--remain, especially during adolescence (Blakemore & Mills, 2014). Recent assessments indicate this disparity has even widened post-pandemic (Kuhfeld et al., 2025), threatening female students' education access and career development.

To bridge the gender gap in mathematics, extensive research has explored the roots of gender differences in mathematics, however, most studies have only focused on sociocultural factors like stereotypes and classroom environments. Considering the phenomenon that some female students still excel in mathematics achievement despite similar external challenges, individual traits--particularly social-emotional skills--should have considerable contributions (Gutman & Schoon, 2013; Soto et al., 2021). These skills, including goal-directed behavior, emotional regulation, and social relationship management, have great benefits to mathematics achievement and are largely malleable in educational practices (Guo et al., 2023; OECD, 2021). Therefore, examining the roles of these skills in enlarging/narrowing this mathematics gender gap is crucial to imply targeted education interventions, yet the related studies remain rare.

To address this gap, the present study investigates the contributions of social-emotional skills to gender differences in mathematics achievement (both skill-level disparities and gender-moderated effectiveness) among Macao adolescents using PISA 2022 data. Macao presents a particularly salient case, with one of the largest and fastest-growing mathematics gender gaps among PISA participating regions (OECD, 2023), providing timely insights for educators confronting emerging gender disparities.

1.1 The OECD Framework of Social-Emotional Skills

The OECD recently refined the Social-emotional Skills in PISA 2022, focusing on eight core social-emotional skills directly related to learning strategies, motivation, and self-efficacy (see Figure 1). This streamlined framework enhances measurement precision while maintaining strong theoretical and empirical relevance to academic achievement. Building on this foundation, the present study examines these eight skills' role in shaping the mathematics gender gap and explores their underlying mechanisms of influence. Many of these skills are tightly associated with mathematics learning motivation, engagement and achievement (Singh & Manjaly, 2022; Kwong, 2015; Duckworth et al., 2007; Guo et al., 2023)

1.2 Social-Emotional Skills in the Mathematics Gender Gap

Given their well-established links to mathematics achievement, social-emotional skills may influence the gender gap through two mechanisms: (1) skill-level disparities, where males and females differ in specific skill proficiency; and (2) gender-moderated effectiveness, where the same skill yields different academic benefits by gender. OECD surveys document notable gender differences in social-emotional skills, especially in Asian contexts (OECD, 2021, 2024b). Males often report higher emotional regulation and sociability, while females excel in empathy and cooperation. These patterns vary culturally, highlighting the need for context-specific research.

Beyond quantitative differences, evidence suggests social-emotional skills' effectiveness may be gender-dependent. For example, curiosity links more strongly to mathematics achievement among females, while persistence may benefit males more in STEM. Mathematics anxiety, disproportionately affecting females, underscores stress resistance's importance for their success (Else-Quest et al., 2010). However, limited by analytical methods, most existing studies have only captured part of this complexity, focusing on either skill-level disparities or gender-moderated effectiveness in one or two specific skills, without a comprehensive examination to identify the most influential skill for the mathematics gender gap and uncover its underlying dynamics.

Methodology, Methods, Research Instruments or Sources Used
Sample
This study draws on data from the 2022 Programme for International Student Assessment (PISA), a large-scale international survey of 15-year-old students’ academic proficiency and background characteristics, administered by the OECD. PISA employs a two-stage stratified sampling design: first, schools are randomly selected, then students within those schools are sampled. For this study, we focused on Macao, a region with high mathematics performance but a pronounced gender gap. The final analytic sample included 4,384 students (51.2% male, 48.8% female; mean age = 15.79). Males outperformed females in mathematics (M = 559 vs. 544), underscoring the relevance of this context.
Measures
Dependent Variables Mathematics achievement was measured using PISA’s plausible values (PV1-10), which estimate students’ proficiency across four content domains and account for measurement error.
Independent Variables Eight social-emotional skills were assessed via standardized indices: perseverance, curiosity, creative self-efficacy, cooperation, empathy, assertiveness, stress resistance, and emotional control. Each index was derived from 10-item self-report scales, with higher scores indicating greater competency relative to the OECD average.
Covariates Analyses controlled for age and Economic, Social, and Cultural Status (ESCS), a composite index based on parental education, occupation, and home resources.
Analytical Approach
We employed a multi-step analytical strategy. Descriptive statistics, correlations, and t-tests established baseline gender differences. Hierarchical linear modeling (HLM) examined associations between social-emotional skills and mathematics achievement, controlling for age and ESCS.
To decompose the gender gap, we used the Oaxaca-Blinder (O-B) method, which partitions the mean difference in mathematics scores into explained (due to observed characteristics) and unexplained (due to differential returns or unmeasured factors) components. To capture distributional effects, we applied unconditional quantile regression with the Recentered Influence Function (RIF), estimating gender gaps and their contributors at the 10th, 30th, 50th, 70th, and 90th percentiles.
All analyses accounted for PISA’s complex sampling design, plausible values, and missing data (handled via multiple imputation). Statistical procedures were implemented in R, with appropriate weighting and error estimation.
Conclusions, Expected Outcomes or Findings
Our decomposition analyses revealed curiosity as the most influential social-emotional skill affecting the mathematics gender gap. Despite lower reported levels, girls derived significantly stronger mathematics benefits from curiosity than boys. Boys' advantages in emotional control and greater returns from assertiveness contributed to widening the gap, while girls' creativity and their lower cooperation levels (which proved beneficial) helped narrow it. Notably, gender disparities were minimal among lower-achieving students but substantial at higher achievement levels, suggesting that high-performing girls face particular barriers in mathematics.

These findings suggest targeted educational approaches to address mathematics gender disparities, with curiosity development as the primary focus. Educators should implement inquiry-based learning and real-world problem-solving that specifically engages girls' interests and questions (Reinholz et al., 2022). To maximize curiosity's benefits, classroom environments should encourage creative exploration and intellectual risk-taking, as creativity also helps reduce the gender gap among high achievers (Repeykova et al., 2024). While fostering these exploratory skills, teachers should carefully balance collaborative activities with independent work, given cooperation's negative association with mathematics achievement. By prioritizing curiosity cultivation alongside strategic use of creativity and collaboration, schools can empower female students to realize their mathematical potential and increase their representation in STEM pathways.
References
Degol, J. L., Wang, M.-T., Zhang, Y., & Allerton, J. (2018). Do growth mindsets in math benefit females? Identifying pathways between gender, mindset, and motivation. Journal of Youth and Adolescence, 47(5), 976-990. https://doi.org/10.1007/s10964-017-0739-8
Duckworth, A., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087-1101. https://doi.org/10.1037/0022-3514.92.6.1087
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405-432. https://doi.org/10.1111/j.1467-8624.2010.01564.x
Else-Quest, N. M., Hyde, J. S., & Linn, M. C. (2010). Cross-national patterns of gender differences in mathematics: A meta-analysis. Psychological Bulletin, 136(1), 103-127. https://doi.org/10.1037/a0018053
Guo, J., Tang, X., Marsh, H. W., Parker, P., Basarkod, G., Sahdra, B., Ranta, M., & Salmela-Aro, K. (2023). The roles of social-emotional skills in students' academic and life success: A multi-informant and multicohort perspective. Journal of Personality and Social Psychology, 124(5), 1079-1110. https://doi.org/10.1037/pspp0000426
Gutman, L. M., & Schoon, I. (2013). The impact of non-cognitive skills on outcomes for young people: A literature review. Education Endowment Foundation.
Jaen, M., & Baccay, E. (2016). Curiosity, motivation, attitude, gender, and mathematics performance. The Normal Lights, 10(2). https://doi.org/10.56278/tnl.v10i2.255
Kuhfeld, M., Lewis, K., & Robinson, G. (2025). Boys regain the advantage in middle school STEM skills: Post-COVID trends in gender achievement gaps. NWEA Research.
Kwong, J. (2015). Open-mindedness as a critical virtue. Topoi, 35(2), 403-411. https://doi.org/10.1007/s11245-015-9317-4
Lee, H. J., Lee, J., Song, J., Kim, S., & Bong, M. (2022). Promoting children's math motivation by changing parents' gender stereotypes and expectations for math. Journal of Educational Psychology. Advance online publication. https://doi.org/10.1037/edu0000743
Majeed, S., & Rashid, W. (2022). Open-minded and its relationship to academic achievement among the students at the department of history in Diyala University. Res Militaris, 12(2).
OECD. (2021). Beyond academic learning: First results from the survey of social and emotional skills. OECD Publishing. https://doi.org/10.1787/92a11084-en
OECD. (2023). PISA 2022 results (Volume I): The state of learning and equity in education. OECD Publishing. https://doi.org/10.1787/53f23881-en
OECD. (2024a). PISA 2022 results (Volume V): Learning strategies and attitudes for life. OECD Publishing. https://doi.org/10.1787/c2e44201-en
OECD. (2024b). Social and emotional skills for better lives: Findings from the OECD survey on social and emotional skills 2023. OECD Publishing. https://doi.org/10.1787/35ca7b7c-en

09. Assessment, Evaluation, Testing and Measurement
Paper

Measuring STEM Teaching Conditions: coherence and profiles of teacher indicators in TIMSS 2019 and 2023

Daniela Avarvare, Lucian Ciolan

University of Bucharest, Romania

Presenting Author: Avarvare, Daniela

Large-scale international assessments increasingly shape how teaching quality, professional development, and teacher well-being are evaluated at the system level. International studies such as TIMSS provide standardized teacher questionnaire data that allow for cross-national comparison and monitoring over time.

Within TIMSS, teaching practices, professional development, and teacher well-being are operationalized through self-reported items and composite indicators, which function as indicator-based measures of complex professional phenomena. These indicators offer structured and comparable representations of teaching conditions, while at the same time raising important questions regarding construct alignment, internal coherence, and interpretive validity. Understanding whether such indicators form coherent measurement structures, and how they combine at the level of individual teachers, is therefore essential for the valid interpretation of findings derived from large-scale assessment data.

This study contributes to research in assessment, evaluation, and measurement by examining the coherence and interpretive structure of TIMSS teacher indicators related to STEM teaching. Using data from the TIMSS 2019 and TIMSS 2023 teacher context questionnaires, the study focuses on middle school mathematics and science teachers in Romania. Romania provides a relevant case for this analysis, as TIMSS indicators point to persistent issues related to teacher workload, participation in professional development, and workforce ageing—topics frequently referenced in system-level evaluation and policy discussions across European education systems.

Previous research using TIMSS teacher data has examined relationships among teacher-reported constructs such as job satisfaction, working conditions, and instructional context, providing evidence that these indicators capture related but distinct dimensions of teachers’ professional experiences. For example, Toropova et al. (2021) found that teacher job satisfaction is systematically associated with perceived working conditions, while not reducible to a single underlying construct, highlighting partial coherence among measured indicators. Similarly, Eriksson et al. (2018) questioned the extent to which TIMSS instructional practice items can be interpreted as stable measures, emphasizing the need for caution when using such indicators across cycles.

Other studies have adopted person-centred approaches to TIMSS data in order to examine heterogeneity underlying aggregate measures. Cheng and Hsu (2017) used latent cluster analysis to identify distinct profiles of instructional practices, demonstrating that teachers with similar average scores may nonetheless belong to qualitatively different patterns of reported practice. Likewise, Teig and Nilsen (2022) applied latent class approaches to show that instructional quality indicators cluster into distinct profiles, underscoring the limitations of system-level averages for representing teaching conditions.

Building on this body of research, the present study integrates indicator coherence analysis and profile-based analysis within a single measurement-oriented framework. The overarching objective is to examine what TIMSS teacher indicators can—and cannot—tell us about STEM teaching conditions over time, and to reflect on the implications of using such indicators for system-level interpretation and evaluation

Methodology, Methods, Research Instruments or Sources Used
The study employs a quantitative, descriptive, and comparative research design, using data from the TIMSS 2019 and TIMSS 2023 teacher context questionnaires. The analytical sample includes eighth-grade mathematics and science teachers in Romania. All analyses apply TIMSS sampling weights and account for the complex sample design to ensure nationally representative estimates and appropriate measures of uncertainty.
The analysis focuses on teacher-reported indicators related to job satisfaction and sense of professional purpose, perceived workload and time constraints, participation in professional development activities, and selected instructional practice indicators.
The study is guided by the following research questions:
RQ1. To what extent do TIMSS teacher indicators related to STEM teaching practices, professional development, and well-being form a coherent measurement structure?
RQ2. How do key TIMSS teacher indicators combine to form distinct measurement-based profiles of STEM teachers?
RQ3. What are the implications of indicator coherence and profile heterogeneity for interpreting and using TIMSS teacher data in system-level evaluation of STEM teaching conditions?
Indicator coherence is examined using weighted correlations and group comparisons to explore relationships among key teacher indicators. This analysis assesses whether indicators behave as parts of a coherent measurement structure, providing insight into construct alignment and potential tensions between measured dimensions.
Teacher profile analysis is conducted to illustrate how multiple indicators combine at the individual level. Teachers are grouped based on selected indicators (e.g., job satisfaction and workload) to identify distinct measurement-based profiles. The distribution and characteristics of these profiles are examined descriptively, with particular attention to how high levels of professional satisfaction may coexist with differing levels of workload and professional development participation.
Comparisons between the 2019 and 2023 cycles are used to examine the stability of indicator relationships and profile distributions over time. Given the repeated cross-sectional design of TIMSS, the analysis does not aim to establish causal relationships or individual change, but rather to assess the interpretability of measured patterns and differences at the system level.
Conclusions, Expected Outcomes or Findings
Preliminary analyses indicate that TIMSS teacher indicators related to STEM teaching conditions exhibit moderate internal coherence. For example, job satisfaction is consistently associated with perceived workload and participation in professional development, suggesting that these indicators capture related but distinct aspects of teachers’ professional experiences rather than a single unified construct.
The profile-based analysis further reveals heterogeneity among STEM teachers that is not visible in aggregate statistics. A substantial proportion of teachers report high professional satisfaction despite experiencing high workload and limited professional development participation, while other profiles combine lower satisfaction with high strain or more favorable conditions. These findings highlight the complexity of interpreting high average satisfaction scores and caution against simplistic evaluative conclusions based on single indicators.
Comparisons between TIMSS 2019 and 2023 suggest that both indicator relationships and profile distributions are largely stable over time, with only modest shifts. This stability supports the use of TIMSS indicators for system monitoring, while also underscoring the importance of cautious interpretation, particularly in light of contextual disruptions such as the COVID-19 pandemic.
From an assessment and evaluation perspective, the findings demonstrate that TIMSS teacher indicators are valuable for identifying broad patterns and tensions in STEM teaching conditions, but that their interpretive power depends on examining relationships between indicators and acknowledging underlying heterogeneity. The study underscores the need to treat large-scale teacher questionnaire data as partial and structured representations of professional reality, informing evaluation while explicitly recognizing measurement limits.

References
Cheng, Y. C., & Hsu, H. Y. (2017). Profiles of instructional practices in mathematics classrooms: A latent class analysis using TIMSS data. Educational Research and Evaluation, 23(3–4), 123–145.
Eriksson, K., Helenius, O., & Ryve, A. (2018). Using TIMSS items to study instructional practices: Measurement issues and implications. Studies in Educational Evaluation, 59, 104–114.
Teig, N., & Nilsen, T. (2022). Instructional quality profiles and student achievement: A latent class approach using TIMSS data. Learning and Instruction, 78, 101523.
Toropova, A., Myrberg, E., & Johansson, S. (2021). Teacher job satisfaction: The importance of school working conditions and teacher characteristics. Educational Review, 73(1), 71–97.
TIMSS. (2019). Encyclopedia: Education Policy and Curriculum in Mathematics and Science, Romania. https://timssandpirls.bc.edu/timss2019/encyclopedia/romania.html
TIMSS. (2019). Assessment Frameworks. https://timssandpirls.bc.edu/timss2019/frameworks/