Conference Agenda
| Session | ||
09 SES 03 B: Student Dispositions, Agency and Learning-Related Scale Validation
Paper Session | ||
| Presentations | ||
09. Assessment, Evaluation, Testing and Measurement
Paper Assessing Conditional Knowledge about Motivational Regulation Strategies in Grade 8 Students: A Cartoon-Based Adaptation of the SKT-SRL Humboldt-Universität zu Berlin, Germany Presenting Author:Self-regulated learning (SRL) is widely regarded as a key competence for successful learning, and its relevance has further increased with the expansion of digital and hybrid learning formats in secondary education. SRL refers to a form of learning in which learners proactively and autonomously regulate their learning process by applying a range of cognitive, metacognitive, and volitional strategies to monitor and guide their progress (Schiefele & Pekrun, 1996). In contemporary learning settings, students are expected to plan, monitor and adjust their learning in more autonomous ways, including the regulation of their motivation when difficulties arise (Wirth & Leutner, 2008). Despite the inclusion of motivational components in prominent SRL models, research on motivational regulation, particularly in relation to learners' selection of an appropriate motivational strategy in a given situation, remains comparatively underdeveloped (Grunschel et al., 2016; Zhang & Dong, 2022). One possible explanation for this may be the lack of adequate measurement instruments. Many established instruments in this area either record motivational orientations and beliefs or assess the general frequency with which strategies are applied (e.g. MSQL; Pintrich et al., 1991, SRQ-A; Ryan & Connell, 1989). However, these instruments do not provide insight into whether a specific strategy choice is situationally appropriate, that is to say, whether it "fits" the particular learning problem at hand. From a pedagogical-psychological perspective, this gap points to the importance of conditional strategy knowledge. Conditional knowledge refers to knowing when and why a strategy is useful in a specific context, and it is conceptually distinct from simply knowing what a strategy is (Paris et al., 1983). This distinction is particularly relevant for understanding why students sometimes fail to translate what they have learned about effective learning into actual adaptive regulation during learning (often discussed as a knowledge–action gap; Foerst et al., 2017). While process-near approaches such as SRL microanalysis can address this issue, they are costly and difficult to scale for larger samples (Perels & Dörrenbächer, 2020). In order to propose a suitable alternative, Perels and Dörrenbächer (2020) developed the Strategy Knowledge Test for Self-Regulated Learning (SKT-SRL), which operationalises conditional knowledge about scenario-based judgments. The theoretical basis of the test is Zimmerman's (2000) cyclical model of self-regulated learning (planning, performance, reflection). The instrument also draws on expectancy-value theory (Wigfield & Eccles, 2000) to assess motivational regulation strategies. Following the recommendations of Engelschalk et al. (2015) and Schwinger et al. (2007), motivational problems are differentiated according to whether they are primarily expectation-related (e.g. doubts about one's own competence) or value-related (e.g. low perceived relevance). Based on this measurement tool, which was originally designed for university students, an adaptation was made for the target group of secondary school students. Therefore, the case scenarios were transferred to typical situations of self-directed learning in a school context. To make the situational reference more vivid, reduce the amount of text and increase motivation to complete the test, the scenarios were converted into a visual cartoon format. Furthermore, a parallel version of the test was developed to facilitate longitudinal assessments and mitigate potential test-retest or practice effects. This article details the adaptation process and provides an empirical evaluation of the instrument’s construct and convergent validity. Methodology, Methods, Research Instruments or Sources Used The motivational scenarios from the SKT-SRL approach were rewritten to fit Grade 8 learning situations. As in the original, a motivational scenario was developed for each of the three SRL phases (planning, implementation, reflection) and parallel versions were created to support longitudinal use. To reduce reading load and to support engagement in a school-based assessment setting, scenarios were presented as short comic strips rather than text-only vignettes. The cartoon format was chosen pragmatically to keep the scenarios concise while still conveying situational cues that are critical for conditional judgments. Content validity was examined with an expert sample from educational psychology. Thirty-nine experts were contacted and asked to rate the usefulness of the proposed motivational regulation strategies for each scenario on a four-point scale and provided rankings of strategies. Expert agreement (Kendall’s W) and mean usefulness patterns were used to establish scoring standards and to identify potential deviations from the original instrument logic in the secondary-school adaptation. The instrument was implemented as part of the scientific evaluation of a pilot project in Berlin schools, in which two instructional models of hybrid learning are currently being piloted. 1315 Students completed the comic-based motivational SKT as well as additional self-report measures capturing SRL-related self-efficacy, motivation-related self-efficacy, learning enjoyment, and procrastination-related behaviour. Evidence for convergent validity is examined via correlations between conditional motivational knowledge scores and these theoretically adjacent constructs. Conclusions, Expected Outcomes or Findings The presentation will focus on the development and validation of instruments. The report will include expert-based evidence for content validity, including the degree to which experts converge in their usefulness judgments and whether the intended usefulness hierarchy is clearly reflected in ratings. It will also include empirical data from the pilot project implementation regarding the instrument's correlational validity. It is generally assumed that conditional knowledge about dealing with expectation-related motivational challenges has a stronger correlation with self-efficacy-related metrics. Conversely, knowledge about dealing with value-related motivational challenges is expected to show stronger correlations with enjoyment and indicators of the value of a task. In addition, it is assumed that higher conditional motivational knowledge is associated with a lower tendency to procrastinate. As part of an exploratory analysis, initial data is also reported on whether conditional motivational knowledge differs between the various hybrid learning models after the first year of testing the hybrid teaching models. References Engelschalk, T., Steuer, G., & Dresel, M. (2015). Wie spezifisch regulieren Studierende ihre Motivation bei unterschiedlichen Anlässen? Zeitschrift Für Entwicklungspsychologie Und Pädagogische Psychologie, 47(1), 14–23. https://doi.org/10.1026/0049-8637/a000120 Foerst, N. M., Klug, J., Jöstl, G., Spiel, C., & Schober, B. (2017). Knowledge vs. Action: Discrepancies in University Students’ Knowledge about and Self-Reported Use of Self-Regulated Learning Strategies. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01288 Grunschel, C., Schwinger, M., Steinmayr, R., & Fries, S. (2016). Effects of using motivational regulation strategies on students’ academic procrastination, academic performance, and well-being. Learning and Individual Differences, 49, 162–170. https://doi.org/10.1016/j.lindif.2016.06.008 Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8(3), 293–316. https://doi.org/10.1016/0361-476X(83)90018-8 Perels, F., & Dörrenbächer, L. (2020). Selbstreguliertes Lernen und (technologiebasierte) Bildungsmedien. In Handbuch Bildungstechnologie (pp. 81–92). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-662-54368-9_5 Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ryan, R. M., & Connell, J. P. (1989). Perceived locus of causality and internalization: Examining reasons for acting in two domains. Journal of Personality and Social Psychology, 57(5), 749–761. https://doi.org/10.1037/0022-3514.57.5.749 Schiefele, U., & Pekrun, R. (1996). Psychologische Modelle des selbstgesteuerten und fremdgesteuerten Lernens. In F. E. Weinert (Ed.), Enzyklopädie der Psychologie, Psychologie des Lernens und der Instruktion (pp. 249–278). Hogrefe. Schwinger, M., von der Laden, T., & Spinath, B. (2007). Strategien zur Motivationsregulation und ihre Erfassung. Zeitschrift Für Entwicklungspsychologie Und Pädagogische Psychologie, 39(2), 57–69. https://doi.org/10.1026/0049-8637.39.2.57 Wigfield, A., & Eccles, J. S. (2000). Expectancy–Value Theory of Achievement Motivation. Contemporary Educational Psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015 Wirth, J., & Leutner, D. (2008). Self-Regulated Learning as a Competence. Zeitschrift Für Psychologie / Journal of Psychology, 216(2), 102–110. https://doi.org/10.1027/0044-3409.216.2.102 Zhang, Y., & Dong, L. (2022). A study of the impacts of motivational regulation and self-regulated second-language writing strategies on college students’ proximal and distal writing enjoyment and anxiety. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.938346 Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–41). Academic Press. 09. Assessment, Evaluation, Testing and Measurement
Paper Learning Conceptions of First-Year Secondary School Students: A Validation Study 1: Radboud Universiteit, Netherlands, The; 2: Open University, Netherlands, The Presenting Author:Topic and background International large-scale assessments such as the Programme for International Student Assessment (PISA) report a sustained decline in reading and numeracy performance across many Western European countries (OECD, 2023). Educational policy responses have mainly focused on curriculum reform, instructional improvement, and teacher professional development. Although these approaches are important, meta-analytic evidence indicates that their average effects are often modest and highly dependent on contextual and learner-related factors (Hattie, 2009). Comparatively little attention has been paid to students’ own perspectives on learning, despite growing evidence that these perspectives are systematically related to learning processes and outcomes. Students’ ideas, beliefs, and mental representations about learning, referred to as learning conceptions, are associated with learning strategies, motivation, and achievement in core academic domains (Peterson et al., 2010; Robbers et al., 2015). Understanding of learning conceptions is therefore particularly relevant during educational transitions, which are a common feature across international education systems. Conceptual and theoretical framework Research on learning conceptions originates in phenomenographic studies that identified different ways in which learners understand learning (Säljö, 1979). Early distinctions focused on quantitative, content-related conceptions, emphasizing the acquisition and reproduction of knowledge, and qualitative, process-oriented conceptions, emphasizing understanding and meaning-making (Purdie & Hattie, 2002; Peterson et al., 2010). More recent theoretical developments have extended this framework by incorporating the social and motivational embeddedness of learning. Learning is conceptualized as a cooperative and socially situated process involving interaction and shared regulation with others (Tynjälä, 1997; Vermunt & Donche, 2017). In addition, learning conceptions include motivational aspects, such as beliefs about effort, control, obligation, personal growth, and attributions of success and failure (Pinto et al., 2018). When selecting or developing a questionnaire to measure learning conceptions, it is important to ensure that aspects of how students understand learning as identified in the literature are broadly represented, covering content-related, process-oriented, cooperative, and motivational elements, without assuming predefined dimensions or a fixed structure. Research gap and rationale Over the past two decades, several instruments have been developed to assess students’ learning conceptions in primary and secondary education (Klatter et al., 2001; Peterson et al., 2010). Of these instruments, only the questionnaire by Klatter et al. (2001), which is available in Dutch, is directly applicable in the Dutch context. However, both the practical and scientific utility of the questionnaire are limited by two key issues. Although reduced from 100 to 65 items, the questionnaire remains impractically long for 11- and 12-year-old students, increasing the risk of response fatigue. Methodologically, Klatter et al. (2001) divided the questionnaire into three correlated subfactors rather than conducting single factor analyses on the full item set and relied solely on the Kaiser criterion (eigenvalue > 1) to determine factors, which may have led to overfactoring (Goretzko, 2025). These choices raise concerns about the instrument’s validity. As a result, no concise and psychometrically robust Dutch instrument currently exists to assess learning conceptions in secondary education, creating a gap for both educators and researchers. Filling this gap is crucial for advancing educational practice and research. Research objective This study aims to psychometrically re-examinate the factorial structure of the original Learning Conceptions Questionnaire (LCQ; Klatter et al., 2001) to create a robust instrument assessing students’ learning conceptions in the first year of secondary education. Central research question To what extent can the original item pool of the Learning Conceptions Questionnaire be restructured into a valid, reliable, and robust multidimensional factor model for students in the first year of secondary education, and to what extent is the resulting factor structure theoretically interpretable in relation to established frameworks of learning conceptions in the educational literature? Methodology, Methods, Research Instruments or Sources Used The study employed the Learning Conceptions Questionnaire (LCQ) developed by Klatter et al. (2001), a 100-item instrument designed to assess students’ conceptions of learning. In contrast to the original validation study, conducted in the final year of primary education and based on multiple separate factor analyses, the present study re-examined the full item set within a single, integrated psychometric framework using data from 431 students in the first year of secondary education. Item-level distributions were inspected using descriptive statistics, including skewness and kurtosis, to evaluate distributional assumptions relevant for factor extraction. Sampling adequacy was subsequently assessed at the item level using a robust bootstrap-based procedure implemented via the RobustMSA.r script (Lorenzo-Seva & Ferrando, 2021) in R. This procedure identifies noisy, weakly discriminating, or content-redundant items that may bias factor recovery and inflate estimates of dimensionality. Items were removed when the lower bound of the 95% bootstrap confidence interval for the Measure of Sampling Adequacy fell below .50. Overall factorability of the refined item set was then evaluated using the Kaiser–Meyer–Olkin (KMO) statistic. Exploratory factor analyses were conducted using maximum likelihood estimation with oblique (Promax) rotation, consistent with the assumption of correlated latent constructs. Given the size of the initial item pool, EFAs were performed iteratively for solutions ranging from two to eight factors. Item retention was guided by established psychometric criteria, including adequate communalities (> .20; Yong & Pearce, 2016), sufficient primary factor loadings (> .30; Klatter et al., 2001), and the absence of substantial cross-loadings (difference ≥ .20 between primary and secondary loadings; Howard, 2016). The number of factors to retain was confirmed using Parallel Analysis and the Comparison Data Forest method (Goretzko, 2025), implemented in the EFAfactors package (Qin & Guo, 2025). Finally, the robustness of the selected factor structure was examined using factor analysis based on a covariance matrix estimated via the Minimum Covariance Determinant method (Bertrand & Zhang, 2013), followed by maximum likelihood extraction and Promax rotation. Conclusions, Expected Outcomes or Findings The present study identified a stable five-factor structure comprising 30 items, representing the following learning conceptions (including illustrative items): • Deep learning (“In mathematics, I always want to find out exactly how everything works”). • External regulation (“In Dutch, I prefer the teacher to tell me exactly what I have to do”). • Social learning orientation (“I like to get tips from other students about the best way to learn”). • Learning as duty (“If you work hard for a subject, you can always get good marks”). • Negative learning orientation (“I mainly go to school because I have to”). The internal consistency of the resulting scales ranged from acceptable to good, and robustness analyses confirmed the stability of the factor structure. The findings provide empirical support for the multidimensional nature of learning conceptions, in line with established frameworks (e.g., Säljö, 1979; Tynjälä, 1997; Purdie & Hattie, 2002). The refined instrument captures qualitative, process-oriented, social, and motivational dimensions of learning, indicating its potential utility for both educational research and practice. Future research could further explore cross-national differences in learning conceptions, particularly between higher- and lower-performing PISA countries and investigate associations with academic achievement and additional forms of validity, such as criterion-related and predictive validity. This instrument provides a promising basis for advancing both theoretical understanding and evidence-based educational practices. References Bertrand, F. & Zhang, Y. Y. (2025). robustfa: Object Oriented Solution for Robust Factor Analysis (R package version 1.2-0). https://CRAN.R-project.org/package=robustfa Goretzko, D. (2025). How many factors to retain in exploratory factor analysis? A critical overview of factor retention methods. Psychological Methods. Advance online publication. https://dx.doi.org/10.1037/met0000733 Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-ana lyses related to achievement. London: Routledge. Howard, M. C. (2016). A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve?. International journal of human-computer interaction, 32(1), 51-62. Klatter, E.B., Lodewijks, H.G.L.C., & Aarnoutse, C.A.J. (2001). Learning conceptions of young students in the final year of primary education. Learning and Instruction,11, 485 - 516. Lorenzo-Seva, U., & Ferrando, P. J. (2021). MSA: The forgotten index for identifying inappropriate items before computing exploratory item factor analysis. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 17(4), 296–306. OECD (2023), PISA 2022 Results (Volume I): The State of Learning and Equity in Education, PISA, OECD Publishing, Paris, https://doi.org/10.1787/53f23881-en. Peterson, E.R., Brown, G.T.L., & Earl Irving, S. (2010). Secondary school students’ conceptions of learning and their relationship to achievement. Learning and Individual Differences, 20 (3) 167 – 176. Pinto, G., Bigozzi, L., Vettori, G., & Vezzani, C. (2018). The relationship between conceptions of learning and academic outcomes in middle school students according to gender differences. Learning, culture and social interaction, 16, 45-54. Purdie, N., & Hattie. J. (2002). Assessing students’ conceptions of learning. Australian Journal of Educational Development Psychology, 2, 17 – 32. Qin H, Guo L (2025). EFAfactors: Determining the Number of Factors in Exploratory Factor Analysis. doi:10.32614/CRAN.package.EFAfactors, R package version 1.2.4, https://CRAN.R-project.org/package=EFAfactors. Robbers, E., Van Petegem, P., Donche, V., & De Maeyer, S. (2015). Predictive validity of the learning conception questionnaire in primary education. International Journal of Educational Research, 74, 61-69. Säljö, R. (1979). Learning about learning. Higher Education, 8, 443-451. Tynjälä, P. (1997). Developing education students’ conceptions of the learning process In different learning environments. Learning and Instruction, 3, 277-292. Vermunt, J. D., & Donche, V. (2017). A learning patterns perspective on student learning in higher education: state of the art and moving forward. Educational Psychology Review, 29(2), 269–299. https://doi.org/10.1007/s10648-017-9414-6 Yong, A. G., & Pearce, S. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in quantitative methods for psychology, 9(2), 79-94. 09. Assessment, Evaluation, Testing and Measurement
Paper Psychometric Validation of the Highly Sensitive Child Scale – Parent Report for Early Childhood in a Bilingual Context 1: Free University of Bozen-Bolzano, Italy; 2: University of Pavia, Italy Presenting Author:When addressing psychological constructs and their implications for professional understanding and practice in early childhood education, educational research relies on trustworthy findings and measurement instruments originating from psychological research. In line with the conference theme Knowing and Acting, this paper conceptualises psychometric validation not as a merely technical prerequisite, but as a core epistemic condition under which educational knowledge becomes empirically visible and, consequently, actionable. Tests and questionnaires actively shape knowledge claims about children’s characteristics in educational research and professional discourse; understanding how such instruments function under specific developmental, linguistic, and contextual conditions is therefore central to the generation of robust educational knowledge. Against this epistemological background, the study builds on the psychological conceptualisation of sensory processing sensitivity (SPS; Aron & Aron, 1997; Greven et al., 2025; Pluess, 2015), commonly discussed in the literature through the notion of highly sensitive children. SPS captures individual differences in responsiveness to physical, social and emotional stimuli and has been empirically linked to children’s socio-emotional functioning and well-being (Lionetti, 2019), central concerns in early childhood education. Research further suggests more pronounced vulnerability under adverse circumstances, alongside greater benefits in supportive environments (Li et al., 2022; Slagt et al., 2018), highlighting the importance of valid, context-sensitive measurement as a prerequisite for meaningful educational knowledge. As systematic studies of age-appropriate measurement instruments for SPS in preschool-aged children remain rare, empirically grounded knowledge on SPS is still limited. The Highly Sensitive Child Scale – Parent Report (HSC-PR; Pluess et al., 2018), one of the core measures of SPS, was originally designed for school-aged children and requires further systematic validation for children aged 3–6 years across languages and in large-scale samples; in particular, no validation is currently available for the German-language version. Moreover, empirical work has reported item-specific difficulties (Sperati et al., 2022), which may suggest that one original item formulation did not adequately reflect age-specific developmental characteristics of early childhood. Building on this gap, the present study aims to systematically examine the psychometric properties of the HSC-PR in preschool-aged children. The study draws on a large parent sample of children aged 3–6 years recruited through the vast majority of kindergartens in the Autonomous Province of Bozen/Bolzano (South Tyrol), a multilingual region in Northern Italy. By analysing the German and Italian language versions within a shared regional and cultural context and common institutional frameworks, the study enables a stringent test of cross-language comparability. In multilingual European education systems, comparable and transferable evidence depends on measurement equivalence rather than on translation alone (Putnick & Bornstein, 2016). The study is guided by three interconnected research questions: (1) To what extent does the HSC-PR demonstrate satisfactory psychometric functioning in preschool-aged children in the German and Italian language versions, with regard to item behaviour, reliability, and factorial structure, and which latent representation of SPS is most appropriate in early childhood? (2) To what extent is the measurement of SPS comparable across the two language versions and key demographic groups (gender and age), as examined through multi-group confirmatory factor analysis and measurement invariance testing within a shared regional and institutional context? (3) To what extent do development-informed item revisions improve the measurement of SPS and its underlying dimensions in preschool-aged children, and what does this imply for epistemic robustness in educational research? By approaching psychometric validation as an empirical examination of the conditions under which knowledge about children’s sensitivity becomes epistemically accessible and transferable, the paper contributes to international debates on situated educational knowledge, cross-language comparability, and the epistemic responsibilities associated with the use of standardised measurement instruments in early childhood research. Methodology, Methods, Research Instruments or Sources Used The study is based on a large parent sample of preschool-aged children (age 3–6 years, M = 53.4 months, SD = 10.8 months). The analytic sample comprised N = 3,598 respondents, including n = 3,054 parents completing the German-language version and n = 544 parents completing the Italian-language version of the online questionnaire. Participants were recruited across the vast majority of kindergartens in a multilingual region in Northern Italy. Sensory processing sensitivity was assessed using the Highly Sensitive Child Scale – Parent Report (HSC-PR). The instrument consists of 12 original items, complemented by three revised item formulations that were developmentally adapted for use in early childhood. To examine convergent and discriminant validity, the questionnaire additionally included the Children’s Behavior Questionnaire (CBQ), Very Short Form (36 items), as well as the Hyperactivity/Inattention subscale (5 items) of the Strengths and Difficulties Questionnaire (SDQ-25). All instruments used Likert-type response scales. Analyses are conducted separately for the German and Italian language versions, followed by systematic cross-language comparisons. Analyses include descriptive item-level statistics, internal consistency estimates (Cronbach’s α and McDonald’s ω), and confirmatory factor analyses testing alternative latent representations of sensory processing sensitivity (one-factor, three-factor, and bifactor models). Measurement invariance across language versions (German vs. Italian) and gender is examined using multi-group confirmatory factor analysis. Age-related measurement differences are explored with age treated as a continuous variable. Associations between HSC-PR scores and CBQ and SDQ subscale scores are examined to evaluate convergent and discriminant validity with respect to theoretically related and distinct constructs (e.g., temperament dimensions and hyperactivity/inattention). Conclusions, Expected Outcomes or Findings The study is expected to provide a differentiated psychometric evaluation of the Highly Sensitive Child Scale – Parent Report for use in early childhood, offering evidence on its reliability, factorial structure, and cross-language comparability in preschool-aged children. By analysing the German and Italian language versions within a shared context, the study clarifies the conditions under which measurements of sensory processing sensitivity can be meaningfully interpreted and compared in early childhood populations. With regard to factorial structure, the study is expected to contribute evidence on whether latent representations of sensory processing sensitivity established in studies with older children are supported in preschool-aged samples, or whether early childhood is characterised by different or less differentiated structural patterns. These findings will inform theoretical assumptions about the dimensionality of SPS in early developmental phases. At the item level, results are expected to show that a targeted, development-informed item revision demonstrates more appropriate psychometric functioning in early childhood than its original formulation, underscoring the necessity of empirical evaluation of item adaptations rather than assuming their validity. Beyond scale-specific results, the study contributes to methodological and epistemic discussions in educational research by framing psychometric validation as an examination of how knowledge about children’s sensitivity is produced and rendered transferable. The findings highlight that in early childhood education research, where standardised parent-report measures inform scientific discourse, professional training, and evidence-based frameworks, inadequate measurement entails risks of misinterpretation and misclassification of children’s sensitivity, which may in turn contribute to epistemically weak knowledge claims and inappropriate pedagogical responses. By establishing a valid early-childhood measure, the study delineates the empirical conditions under which knowledge about sensory processing sensitivity can be generated and meaningfully interpreted in early childhood education research, without turning sensitivity into a diagnostic label. References Aron, E. N., & Aron, A. (1997). Sensory-processing sensitivity and its relation to introversion and emotionality. Journal of Personality and Social Psychology, 73(2), 345–368. https://doi.org/10.1037/0022-3514.73.2.345 Belsky, J., & Pluess, M. (2009). Beyond diathesis stress: Differential susceptibility to environmental influences. Psychological Bulletin, 135(6), 885–908. https://doi.org/10.1037/a0017376 Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834 Greven, C. U., Lionetti, F., Booth, C., Aron, E. N., Fox, E., Schendan, H. E., Pluess, M., Bruining, H., Acevedo, B., & Bijttebier, P. (2019). Sensory processing sensitivity in the context of environmental sensitivity: A critical review and development of a research agenda. Neuroscience & Biobehavioral Reviews, 98, 287–305. https://doi.org/10.1016/j.neubiorev.2019.01.009 Greven, C. U., Trupp, M. D., Homberg, J. R., & Slagter, H. A. (2025). Sensory processing sensitivity: Theory, evidence, and directions. Trends in Cognitive Sciences. Advance online publication. https://doi.org/10.1016/j.tics.2025.10.007 Li, Z., Sturge-Apple, M. L., Jones-Gordils, H. R., & Davies, P. T. (2022). Sensory processing sensitivity behavior moderates the association between environmental harshness, unpredictability, and child socioemotional functioning. Development and Psychopathology, 34(2), 675–688. https://doi.org/10.1017/S0954579421001188 Lionetti, F., Aron, E. N., Aron, A., Klein, D. N., & Pluess, M. (2019). Observer-rated environmental sensitivity moderates children’s response to parenting quality in early childhood. Developmental Psychology, 55(11), 2389–2402. https://doi.org/10.1037/dev0000795 Pluess, M. (2015). Individual differences in environmental sensitivity. Child Development Perspectives, 9(3), 138–143. https://doi.org/10.1111/cdep.12120 Pluess, M., Assary, E., Lionetti, F., Lester, K. J., Krapohl, E., Aron, E. N., & Aron, A. (2018). Environmental sensitivity in children: Development of the Highly Sensitive Child Scale and identification of sensitivity groups. Developmental Psychology, 54(1), 51–70. https://doi.org/10.1037/dev0000406 Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004 Slagt, M., Dubas, J. S., van Aken, M. A. G., Ellis, B. J., & Deković, M. (2018). Sensory processing sensitivity as a marker of differential susceptibility to parenting. Developmental Psychology, 54(3), 543–558. https://doi.org/10.1037/dev0000431 Sperati, A., Spinelli, M., Fasolo, M., Pastore, M., Pluess, M., & Lionetti, F. (2022). Investigating sensitivity through the lens of parents: validation of the parent-report version of the Highly Sensitive Child scale. Development and Psychopathology, 36, 415–428. https://doi.org/10.1017/S0954579422001298 09. Assessment, Evaluation, Testing and Measurement
Paper A Questionnaire for Assessing Students’ Learning-related Agency 1: University of Tartu, Estonia; 2: University of Belgrade, Serbia Presenting Author:Students’ learning-related agency is receiving increasing attention in scientific literature and in policy debates (see, for example, The OECD Learning Compass 2030, OECD, 2019). In a broader sense, learning-related agency refers to learners’ decision-making in learning situations, including planning, implementing, and evaluating learning (Lipponen & Kumpulainen, 2011; Priestley et al., 2015). In the present study, we draw on the ecological model of agency (Leijen et al., 2024; Priestley et al., 2015), which helps to understand three main dimensions that influence learners’ decision-making related to learning. First, the iterational dimension indicates that decision-making is based on prior experiences. Over time, learners develop learning-related knowledge, attitudes, and skills that begin to influence their subsequent learning-related decisions (see, for example, Lange, 2010; Lauermann et al., 2017). Second, agency is oriented toward the future. The projective dimension reflects students’ short-term and long-term purposes, which guide decision-making toward their future plans. For example, when students have a clear purpose related to learning a particular subject, they are more likely to decide to engage actively in learning that content. By contrast, when students do not understand the relevance of learning certain content or skills, it becomes more difficult for them to engage in learning (see Eccles & Wigfield, 2020). Third, the practical-evaluative dimension indicates students’ capacity to make decisions in specific situations. This decision-making process is guided, on the one hand, by the iterational and projective dimensions and, on the other hand, by the cultural, structural, and material conditions of a particular learning situation. These various learning-related environmental conditions may be perceived as affordances, constraints, or resources, and may therefore either hinder or support learners’ agency. Learning environments that are responsive to students’ perspectives, offer alternatives, and support horizontal relationships and collaboration among students are considered supportive of learners’ agency, whereas rigid rules, hierarchical relationships, and an orientation toward competition are regarded as constraining agency (Nieminen et al., 2022; Priestley et al., 2015). In previous studies, learners’ agency in mathematics learning was operationalised using a three-dimensional questionnaire reflecting the dimensions described above (Leijen et al., 2024; Pedaste et al., 2023). Building on these studies, the present study aims to examine the psychometric properties of a revised agency questionnaire that conceptualises learners’ agency in learning-related decision-making situations beyond a specific school subject and includes additional items developed based on interviews with students from Grades 6 to 12. Using Confirmatory factor analysis (CFA), we examined whether the hypothesised three-factor measurement model of learners’ agency provides an adequate fit to the data when agency is operationalised in relation to students’ general learning experiences. Furthermore, using multi-group CFA, we investigated the extent to which the revised instrument demonstrates measurement invariance across students in Grades 6, 9, and 12. Accordingly, the study addressed the following research questions: (1) To what extent does a three-factor confirmatory factor model of learners’ agency fit the data when agency is operationalised in relation to general learning experiences?, and (2) To what extent does the revised learners’ agency questionnaire exhibit scalar measurement invariance across students in Grades 6, 9, and 12? Data was collected from 549 students (166 – 6th Grade, 133 – 9th Grade, 250 – 12th Grade) in Estonia. CFA showed that the initially developed questionnaire had a near-acceptable fit. Based on the empirical and theoretical considerations, we decided to remove some items; the final version of the questionnaire included 24 items. This version of the questionnaire had good model fit. A comparison of configural and scalar models suggested scalar invariance across grade levels. Therefore, the new questionnaire appears suitable and sensitive for monitoring and comparing student learning-related agency among 6th- to 12th-grade students. Methodology, Methods, Research Instruments or Sources Used A revised questionnaire was developed to measure students’ agency in the general learning context. This questionnaire is based on the ecological model of agency and operationalises student agency as a three-factor construct: iterational, projective, and practical-evaluative. Cognitive interviews were conducted with 15 students to refine item wording and ensure content validity. Data was collected electronically from 549 students (166 – 6th Grade, 133 – 9th Grade, 250 – 12th Grade) in Estonia. Students were asked to rate 37 items using a five-point Likert-type scale (1-Strongly disagree, 5- Strongly agree). The study had an ethics committee approval. The hypothesised three-factor measurement model was tested using CFA, treating items as ordered categorical variables and applying the WLSMV estimator. A multi-group CFA was subsequently conducted to examine the measurement invariance of the instrument across Grades 6, 9, and 12, with particular focus on scalar invariance following the establishment of configural invariance. All analyses were performed using Mplus (Version 8.11). CFA showed that the initially developed questionnaire with 37 items had a near-acceptable fit (χ² = 1463.181, df = 626, χ² / df = 2.33, RMSEA = 0.049, CFI = 0.926, TLI = 0.921, SRMR = 0.076). Based on the empirical evidence and theoretical considerations, several items were removed, resulting in a final 24-item version of the questionnaire. Conclusions, Expected Outcomes or Findings The 24-item version of the questionnaire had good model fit (χ² = 529.002, df = 249, χ² / df = 2.12, RMSEA = 0.045, CFI = 0.957, TLI = 0.952, SRMR = 0.053). Standardised factor loadings were above 0.50 for all items except one (λ = 0.49), and correlations between latent factors ranged from 0.54 to 0.61, indicating related but empirically distinguishable dimensions of learners’ agency. Measurement invariance across Grade 6, Grade 9, and Grade 12 students was examined using multi-group CFA, with a focus on configural and scalar invariance. The configural invariance model demonstrated good fit (χ² = 1096.31, df = 747, χ²/df = 1.47, RMSEA = 0.051, CFI = 0.955, TLI = 0.950, SRMR = 0.070). The scalar invariance model also showed good fit (χ² = 1255.01, df = 927, χ²/df = 1.35, RMSEA = 0.044, CFI = 0.958, TLI = 0.962, SRMR = 0.070). Although the χ² difference test between the configural and scalar models was statistically significant (Δχ² = 223.83, Δdf = 180, p = .015), comparisons of approximate fit indices supported the conclusion that scalar invariance was established across grade levels. The results of the study show that the developed questionnaire is suitable for studying student learning-related agency among 6th-, 9th-, and 12th-grade students in Estonia, as we confirmed the three factors and scalar invariance across the three grades. It means we can use the questionnaire to measure and compare students’ agency at different time points in their studies. Therefore, the questionnaire could be used in longitudinal studies to monitor changes in students' learning-related agency, as well as in cross-sectional studies to compare learning-related agency across grade levels. References Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology, 61, 101859. Lange, T. (2010). Tell them that we like to decide for ourselves: Children’s agency in mathematics education. In V. Durand-Guerrier, S. Soury-Lavergne, & F. Arzarello (Eds.), Proceedings of the Sixth congress of the European society for research in mathematics education (pp. 2587–2596). Lyon, France: European Society for Research in Mathematics Education. Lauermann, F., Eccles, J. S., & Pekrun, R. (2017). Why do children worry about their academic achievement? An expectancy-value perspective on elementary students’ worries about their mathematics and reading performance. ZDM Mathematics Education, 49, 339–354. Leijen, Ä., Baucal, A., Pikk, K., Uibu, K., Pajula, L., & Sõrmus, M. (2024). Opportunities to develop student’s math-related agency in primary education: the role of teacher beliefs. European Journal of Psychology of Education, 39(2), 1637-1659. Lipponen, L., & Kumpulainen, K. (2011). Acting as accountable authors: Creating interactional spaces for agency work in teacher education. Teaching and Teacher Education, 27(5), 812–819. Nieminen, J. H., Chan, M. C. E., & Clarke, D. (2022). What affordances do open-ended real-life tasks offer for sharing student agency in collaborative problem-solving? Educational Studies in Mathematics, 109(1), 115–136. OECD (2019). Learning Compass 2030. https://www.oecd.org/education/2030-project/teaching-and-learning/learning/learning-compass-2030/. Visited 31 Jan 2026. Pedaste, M., Raave, D. K., & Baucal, A. (2023). Digitaalsete õppematerjalide kasutamise efekt õpilaste õpitulemustele DigiEfekti projekti lõppraport. Tartu Ülikool. The Effect of Using Digital Learning Materials on Students’ Learning Outcomes: Final Report of the DigiEffect Project. University of Tartu. Priestley, M., Biesta, G. J. J., & Robinson, S. (2015). Teacher agency: An ecological approach. Bloomsbury Academic. | ||