Conference Agenda

With a sixfold increase in the number of publications between 2010 and 2020 alone (Salmela-Aro et al., 2021), the construct of student engagement continues to attract international research interest. Defined as a meta-construct encompassing students’ behavioural, cognitive and emotional reactions to school (Fredricks et al., 2004), it is widely regarded as a robust predictor of academic performance (Wong et al., 2024) as well as dropout (Archambault et al., 2022), and an important mediator of various contextual factors that influence children's adjustment to school (e.g., Reschly & Christenson, 2022; Skinner & Raine, 2022). Nonetheless, the preponderance of this body of work is conducted in American schools, and/or employs measures that have been developed in and for the use in this context (Martins et al., 2022; Salmela-Aro et al., 2021; Wong et al., 2024). The present study introduces the Student Engagement with Schoolwork scale, currently available in English and German. This scale seeks to address the aforementioned gap by providing efficient and flexible means of measuring the construct at a domain-general level, from late primary to early secondary school.

The selection of engagement indicators was made in accordance with theoretically derived content dimensions (e.g., "participation", "attention") which were delineated a priori in theoretical blueprints (Kalkbrenner, 2021). A mixed-methods approach was employed to investigate validity of the new measure (McCullagh, 2026). Confirmatory factor analyses (N = 558) indicate that the scale consists of two correlated factors (affective engagement and cognitive-behavioral engagement), with an uncorrelated method-bifactor accounting for the impact of item valence. Partial measurement invariance was confirmed across gender, age, home language (German, other), and school track (Gymnasium/high-track, other). Supplementing these general quantitative findings, the presentation will concentrate on the qualitative evidence to further support the construct validity of the new measure. Results from cognitive interviews conducted with a sample of 15 students in grades 4 through 7 are presented. An evaluative content analysis was conducted (Kuckartz & Rädiker, 2022), pursuing two interlocking objectives. The initial objective was to ascertain whether the selected engagement indicators effectively measure the intended behaviors and emotions. For this purpose, the analysis drew upon an intricate coding framework (κ = .65) to assess respondents' accuracy with respect to the comprehension of item content. Secondly, the individual applicability of the quantitative measurement model was investigated by incorporating response consistency, defined as the tendency of interviewees to respond similarly to items intended to measure related content. Identification of instances in which students deviate from this expected pattern allows for a unique examination of the circumstances under which the current quantitative model may have limited applicability. Narrative and case-based summaries are employed to formulate hypotheses for the adjustment of theoretical and measurement models in future quantitative research.

The Strengths and Difficulties Questionnaire (SDQ) is a brief and comprehensive instrument used to evaluate the mental health of children and adolescents. It includes 25 items, with 15 reflecting negative attributes and 10 highlighting positive ones. These items are divided into five specific dimensions, each consisting of five items: emotional problems (EP), conduct problems (CP), hyperactivity-inattention (HA), peer problems (PP), and prosocial behavior (PB). Each subscale is rated on a 3-point Likert scale ("Not True," "Somewhat True," or "Certainly True"). It can be administered quickly, applied across children and adolescents, and incorporates information from multiple informants. Freely available in over 90 languages, the SDQ offers a cost- and time-efficient method for assessing child and adolescent mental health across diverse populations (SDQinfo.com). A review by Kankaanpaa and colleagues (2023) revealed that the SDQ has particular problems with reliability and factor structure. It was found that some items (“obedient”, “agreement with adults”, “persistent”) weakened construct validity, while reverse-coded items could lead to measurement error. Besides these quantitative studies,there are limited studies examining how the SDQ is interpreted by this age group (11-17) and whether the problems with these items stem from comprehensibility.

The conceptual framework of the study is based on the Cognitive Interview framework (Willis, 2005), which uses Tourangeau’s (1984) cognitive response model. According to this model, responding to a survey item is a complex four-stage process: 1-Comprehension, 2-Recall, 3- Judgment, and 4- Response Selection. This practical framework has been used to identify specific disconnects, particularly in the “comprehension” and “judgment” stages, such as whether students understand the terms or interpret the statements correctly.

The research question of this study is “How adolescents aged 11-17 cognitively interpret the SDQ items and to what extent these subjective understandings correspond with the intended theoretical meaning of the scale?” Rather than assuming that the scale is universally understood correctly, the study focuses on whether there is an error in the literature. In this context, as part of a project aimed at revising the SDQ, this study also aimed to identify how these items are understood by students, possible word errors, ambiguities, and cultural incompatibilities using cognitive interview methods and to present these findings.

Finally, considering that the SDQ is a widely used screening tool globally, addressing the psychometric inconsistencies reported in the literature is of great importance. It has been suggested that limitations such as low factor loadings or reliability issues may not only stem from statistical or translation errors but may also arise from a fundamental lack of comprehensibility and developmental appropriateness for adolescents. Such psychometric inconsistencies are thought to stem from participants’ inability to interpret the items as intended. Consequently, this study aims to demonstrate that identifying and addressing these ambiguities at the meaning level is a necessary prerequisite for enabling more reliable measurements and more accurate cross-cultural comparisons in the future.

Adolescence is a critical developmental period marked by heightened vulnerability to mental health problems, which affect approximately one in seven adolescents worldwide and contribute substantially to long-term individual and societal burden (WHO, 2024). Early identification of emotional, behavioral, and social difficulties is therefore essential, particularly in school and community settings. The Strengths and Difficulties Questionnaire (SDQ) is among the most widely used screening instruments for child and adolescent mental health, owing to its brevity, open-access availability, and multi-informant format. The SDQ comprises 25 items assessing five domains—emotional symptoms, conduct problems, hyperactivity/inattention, peer problems, and prosocial behavior—and has demonstrated utility comparable to longer instruments. Despite its widespread use and extensive cross-cultural application, persistent debate regarding the SDQ’s underlying factor structure raises important concerns about the validity and interpretability of its subscale scores, underscoring the need for continued psychometric evaluation.

Since its introduction in 2001, numerous studies across diverse cultural contexts have examined the SDQ’s factor structure, yielding mixed findings. While some investigations support the original five-factor model, many others report only moderate or poor fit, calling into question the construct validity of the subscale scores. Moreover, relatively few studies have focused specifically on adolescent samples. As noted by Goodman et al. (2010) the internal structure of the SDQ remains unresolved—a concern that persists more than a decade later.

In general, validation studies of the three-factor model have produced less favorable psychometric results than those of the five-factor solution (e.g., Karlsson et al., 2022). The bifactor model has emerged as another prominent alternative. Each SDQ item loads simultaneously on a general difficulties factor and on its respective domain-specific factor. This specification allows the general factor to capture shared variance across all items, while the specific factors retain residual variance unique to each domain. Accordingly, the bifactor model enables evaluation of whether the SDQ primarily reflects a broad, overarching difficulties construct and whether the subscales contribute distinct and meaningful variance beyond the general factor.

Across studies, bifactor models have frequently outperformed both five- and three-factor solutions in terms of global model fit (e.g., Murray et al., 2019). In many cases, the general difficulties factor accounted for the majority of item variance, whereas several specific factors—particularly peer problems and, in some studies, conduct problems—exhibited weak or unstable loadings. These findings suggest that the SDQ may function more reliably as a measure of overall psychosocial difficulties than as a multidimensional instrument capturing well-differentiated domains. However, results have not been uniform. Some studies have reported inadmissible or poorly fitting bifactor solutions (e.g., Duinhoven et al., 2019), often attributable to instability in weaker subfactors. Moreover, even when fit indices favor the bifactor model, concerns remain regarding the interpretability and reliability of specific factors with limited unique variance, as well as the practical utility of reporting subscale scores when most reliable variance is attributable to the general difficulties dimension.

The aim of the present study was to examine the underlying factor structure of the 25-item Turkish self-report SDQ for adolescents aged 11–17 years using CFA. Several theoretically grounded and empirically supported models were evaluated, including the original five-factor model, three-factor models, a second-order five-factor model, bifactor models specifying a general difficulties factor orthogonal to domain-specific factors, and models incorporating a method factor. Given the model misfit observed across CFA and ESEM solutions, Rasch measurement modeling was subsequently employed to examine item-level measurement properties. Rasch-based indices complemented the factor-analytic findings by providing a fine-grained evaluation of item functioning and identifying specific sources of misfit that may inform future scale refinement.