09. Assessment, Evaluation, Testing and Measurement
Paper
Does Social Well-Being Predict Academic Resilience? An Analysis of Swedish PISA 2018 Data
Deborah Elin Siebecke, Kajsa Yang Hansen, Maria Jarl
University of Gothenburg, Sweden
Presenting Author: Siebecke, Deborah Elin
Recent studies indicate that Sweden faces issues of decreasing educational equity (Siebecke & Jarl, 2022; Yang Hansen & Gustafsson, 2019), suggesting that the impact of socioeconomic background on achievement has increased. However, some students achieve high despite disadvantages in their socioeconomic background that place them at risk for low achievement. These students are often referred to as academically resilient and yield hope for a more equitable future. In general terms, resilience is grounded in the recognition that individuals’ responses to adversities differ (Rutter, 2012). While some struggle or fail in the face of adversity, others seem to adjust just fine. Those, who demonstrate positive adaptation despite being exposed to adversities, are usually considered resilient (e.g., Masten & Obradovic, 2006). The identification of supportive and risk factors can help socioeconomically disadvantaged individuals in becoming academically successful and, thus, improve educational equity.
Previous studies have indicated that individual and external resources, such as supportive adults and peers (Fergus & Zimmerman, 2005) and a student’s sense of belonging at school (Gonzalez & Padilla, 1997) can promote a student’s resilience. According to a framework by Borgonovi and Pál (2016), these indicators - that is a student’s sense of belonging at school and their relationship with their teachers, parents, and peers – also act as subdimensions of social well-being. This may imply a relationship between academic resilience and social well-being. Yet, research on the (social) well-being of academically resilient students is scarce, especially in Sweden. While the relationship between social well-being and academic resilience is underexplored, previous research does indicate a positive albeit small relationship between well-being and achievement (Bücker et al., 2018; Kaya & Erdem, 2021). However, this relationship is not straightforward and a multidimensional conceptualization of well-being is needed to assess which aspects are particularly important for achievement (Clarke, 2020). In general, well-being is hypothesized to be a multi-dimensional construct consisting of social, physical, and mental/psychological dimensions, which can further be structured in subdimensions (Colombo, 1984). The social dimension of well-being, for instance, can be measured by including subdimensions such as the students’ relationship with peers, parents and teachers and their sense of belonging at school (Borgonovi & Pál, 2016). These subdimensions have been found to be interrelated. For instance, a student’s sense of belonging is closely related to their relationship with peers and teachers (Govorova et al., 2020).
Thus, the main objective of the present study is to investigate whether and how students’ social well-being predicts their academic resilience. The present study focuses on social well-being, as one important dimension of student well-being, and attempts to capture its complexity by not only modeling its’ possible relationship to academic resilience but by also considering the interrelationship between subdimensions of social well-being. The study is anchored in Bronfenbrenner’s ecological theory and specifically focuses on the students’ microsystem, that is their close interaction with their immediate environments, as well as the mesosystem, which describes the interrelation among the environments in which the student participates (Bronfenbrenner, 1979).
Methodology, Methods, Research Instruments or Sources UsedMaking use of data from the Programme for International Student Assessment (PISA) from 2018, the study investigates the relationship between academic resilience and the social well-being of 15-year-old students in Sweden. After weighing the advantages and disadvantages of different approaches to operationalizing academic resilience, we decided to apply a definition-driven approach, which is said to reflect academic resilience "in its most literal sense: academic achievement despite adversity" (Rudd et al., 2021, p. 5). Thus, academically resilient students are defined as those who achieve at or above Level 3 in the PISA domains reading, mathematics, and science, despite falling in the bottom quartile of Sweden’s distribution of the Index of Economic, Social and Cultural Status (ESCS) (Agasisti et al., 2018). Level 3 corresponds to a median achievement level that is said to prepare students “for success later in life” (Agasisti et al., 2018, p. 8). This study only focuses on socioeconomically disadvantaged students, leading to a total sample size of 1337 students, 358 of whom were considered resilient. A dichotomous variable measuring academic resilience is used as a dependent variable in the present study.
The measure of social well-being is based on a well-being framework proposed by Borgonovi and Pál (2016) and adapted to the newer measures in PISA 2018 (for an overview, see Borgonovi, 2020). According to this framework, the social dimension of well-being can be measured using students’ self-reported data on the sense of belonging at school, exposure to bullying, teacher support, teacher feedback, and parental emotional support. Each of these subdimensions of social well-being was measured as a latent variable consisting of three to six indicators.
Data analyses were run in SPSS 29 and Mplus 8. First, confirmatory factor analysis was used to test whether the data fit the measurement models. Secondly, structural models based on an extensive literature review were built. The models reflect the interrelation of subdimensions of social well-being as well as their relation to academic resilience. Due to the nested data structure (i.e., the clustering of individual data in schools) but small intraclass correlations, a single-level model was used. Standard errors of the SEM parameters were adjusted by using the TYPE = COMPLEX command in Mplus, accompanied by the robust maximum likelihood estimator, cluster, and student weights. To evaluate model fit, local and global fit indices were consulted.
Conclusions, Expected Outcomes or FindingsPreliminary analyses resulted in well-fitting measurement models for all tested well-being subdimensions (i.e., sense of belonging at school, exposure to bullying, teacher support, perceived teacher feedback, and parental emotional support). A structural model linking these subdimensions with each other, as well as with the dichotomous endogenous measure of academic resilience resulted in an overall good global and local model fit. Model results confirm the interrelation of subdimensions of social well-being that was highlighted in previous research. For instance, parental support and students’ exposure to bullying significantly predict their sense of belonging at school. Yet, preliminary results suggest that only the students’ perceived support by their teachers significantly predicts their academic resilience while other subdimensions of well-being did not indicate any significant relationship with academic resilience.
The presentation of results includes a discussion of the study’s possible limitations due to cross-sectional data, reduced statistical power by cause of group sizes, and the necessary but rather artificial dichotomization of resilient vs. nonresilient students.
Even though the study focuses on academic resilience and well-being in Sweden, results can be of importance beyond the Swedish context. Issues of educational inequity and the importance of fostering student well-being are topical and prominent across Europe. For instance, in countries such as Austria, Belgium, France, and Germany, more than 15% of the variation in science performance can be explained by the student’s socioeconomic background alone (OECD, 2018). Research on the group of academically resilient students can shed light on the reasons why some students seem to defeat the odds and show positive adaptation despite adversity. Thus, more research on academic resilience and well-being is needed – in Sweden and beyond.
ReferencesAgasisti, T., Avvisati, F., Borgonovi, F., & Longobardi, S. (2018). Academic resilience: What schools and countries do to help disadvantaged students succeed in PISA. OECD Publishing.
Borgonovi, F. (2020). Well-being in international large-scale assessments. In T. Nilsen, A. Stancel-Piątak, & J.-E. Gustafsson (Eds.), International handbook of comparative large-scale studies in education: Perspectives, methods and findings (pp. 1–26). Springer International Publishing.
Borgonovi, F., & Pál, J. (2016). A framework for the analysis of student well-being in the PISA 2015 study: Being 15 in 2015. OECD Education Working Papers, 140. OECD Publishing.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard University Press.
Bücker, S., Nuraydin, S., Simonsmeier, B. A., Schneider, M., & Luhmann, M. (2018). Subjective well-being and academic achievement: A meta-analysis. Journal of Research in Personality, 74, 83–94.
Clarke, T. (2020). Children’s wellbeing and their academic achievement: The dangerous discourse of ‘trade-offs’ in education. Theory and Research in Education, 18(3), 263–294.
Colombo, S. A. (1984). General well-being in adolescents: Its nature and measurement. ProQuest Dissertations & Theses Global. https://search.proquest.com/dissertations-theses/general-well-being-adolescents-nature-measurement/docview/303323578/se-2?accountid=11162
Fergus, S., & Zimmerman, M. A. (2005). Adolescent resilience: A Framework for Understanding Healthy Development in the Face of Risk. 24.
Gonzalez, R., & Padilla, A. M. (1997). The Academic Resilience of Mexican American High School Students. Hispanic Journal of Behavioral Sciences, 19(3), 301–317.
Govorova, E., Benitez Baena, I., & Muñiz, J. (2020). Predicting Student Well-Being: Network Analysis Based on PISA 2018. International Journal of Environmental Research and Public Health, 17, 4014.
Kaya, M., & Erdem, C. (2021). Students’ Well-Being and Academic Achievement: A Meta-Analysis Study. Child Indicators Research, 14(5), 1743–1767.
Masten, A. S., & Obradovic, J. (2006). Competence and Resilience in Development. Annals of the New York Academy of Sciences, 1094(1), 13–27.
OECD. (2018). Equity in education breaking down barriers to social mobility. OECD Publishing.
Rudd, G., Meissel, K., & Meyer, F. (2021). Measuring academic resilience in quantitative research: A systematic review of the literature. Educational Research Review, 34, 100402.
Rutter, M. (2012). Resilience as a dynamic concept. Development and Psychopathology, 24(2), 335–344.
Siebecke, D. E., & Jarl, M. (2022). Does the material well-being at schools successfully compensate for socioeconomic disadvantages? Analysis of resilient schools in Sweden. Large-Scale Assessments in Education, 10(11), 11.
Yang Hansen, K., & Gustafsson, J.-E. (2019). Identifying the key source of deteriorating educational equity in Sweden between 1998 and 2014. International Journal of Educational Research, 93, 79–90.
09. Assessment, Evaluation, Testing and Measurement
Paper
Student Well-Being in School and Academic Achievement by TIMSS in Finland
Timo Salminen, Jonna Pulkkinen, Jenna Hiltunen, Jenni Kotila, Piia Lehtola, Juhani Rautopuro
University of Jyväskylä, Finland
Presenting Author: Salminen, Timo;
Pulkkinen, Jonna
Student well-being in school can be considered as a condition that enables positive learning outcomes but also as an outcome of successful learning and students’ satisfaction at school (Morinaj & Hascher, 2022). Students’ well-being in school refers to an emotional experience characterized by the prevalence of positive feelings and cognitions towards school, persons in school and the school context over the negative ones towards school life (Hascher, 2003). According to Hascher (2003), it consists of six dimensions, three positive, i.e., positive attitudes to school, enjoyment in school, and positive academic self-concept, and three negative, i.e., worries in school, physical complaints in school, and social problems in school, that can be used as indicators of well-being.
In Finland, the trends in students’ academic well-being (e.g. Helakorpi & Kivimäki, 2021; Salmela-Aro et al., 2018, 2021) and learning performance (e.g. OECD, 2019; Mullis et al., 2020) have been descending in the last decade. For example, grade 4 students’ performance in mathematics and science has decreased from 2011 to 2019 as evidenced by the Trends in International Mathematics and Science Study (TIMSS) (Mullis et al., 2020). The performance in mathematics declined by 10 points from 2011 to 2015 and by three points from 2015 to 2019. In science, the decrease from 2011 to 2019 was 15 points. When examining the international mathematics and science benchmarks (Mullis et al., 2020), these declines in learning outcomes mean that the percentage of high achievers has dropped from 49% to 42% in mathematics and from 65% to 56% in science during this period. Meanwhile, the percentage of the students below the low international benchmark has grown from 2% to 5% in mathematics and from 1% to 3% in science.
Previous research has detected the interrelation between student well-being and learning performance but also the need for examining this relation with possible associated factors in more detail (e.g. Bücker et al., 2018; Nilsen et al., 2022; Pietarinen et al., 2014). For example, the Programme for International Student Assessment (PISA) shows the relationship between students’ socio-economic status (SES), well-being and achievement (OECD, 2017). Further, the study using TIMSS data by Nilsen, Kaarstein and Lehre (2022) shows that a safe environment, as an aspect of school climate, and student self-concept, both indicating students’ well-being in school, declined from 2015 to 2019 and mediates the changes in mathematics achievement over time in Norway.
The above statements point out that both students’ well-being in school and academic achievements may have declined in the last decade in Finland. Thus, in this study, we ask the following research questions, using the TIMSS fourth grade assessment data:
1) How has students’ well-being in school changed, if any, from 2011 to 2019?
2) What is the relationship between students’ SES, well-being and achievement in mathematics and science?
Methodology, Methods, Research Instruments or Sources UsedThe present study is based on the three cycles of curriculum-based TIMSS assessment in Finland. The data includes the 4th graders who participated in TIMSS 2011 (N = 4,638), TIMSS 2015 (N = 5,015) and TIMSS 2019 (N = 4,730). In this study, we use school climate and safety, and students’ attitudes as indicators of well-being. School climate and safety include the scales of Students’ Sense of School Belonging (3 items) and Bullying (6 items). Students’ attitudes include the scales of Students Like Learning Mathematics (5 items) and Science (4 items), and Students Confident in Mathematics (7 items) and Science (6 items). These four-point scales are from TIMSS student questionnaires. From each scale, we selected those items that were the same in all three cycles of TIMSS assessment. As an indicator of students’ SES, we used Home Resources for Learning scale which is scored based on the number of books at home, the number of home study supports, and the parents’ educational level as well as the level of occupation. In TIMSS data, the Home Resources for Learning scale is divided into three categories. In this study, we recoded it into two categories: (1) students with many resources, and (2) students with some or few resources. In addition to the above-mentioned scales, the variables of our study include mathematics and science achievement scores.
The analysis was performed in three phases. First, we conducted a confirmatory factor analysis (CFA) to examine the validity of the variables that measure well-being. Second, to answer the first research question, we computed mean variables and studied the average changes in students’ well-being from 2011 to 2019 using these mean variables. The values of mean variables ranged from 1 to 4 (the highest value indicating the most positive view). Third, to answer the second research question, we investigated the relationship between students’ SES, well-being and achievement using the structural equation modelling (SEM) approach. This analysis was conducted for mathematics and science separately for each of the three TIMSS data sets. Five plausible values representing students’ proficiency in mathematics and science (see Martin et al., 2020) were used in the analyses. A two-stage sampling design used in the TIMSS assessment (Martin et al., 2020) was considered in the analyses.
Conclusions, Expected Outcomes or FindingsThe results of CFA confirmed the validity of the latent variables (i.e., sense of belonging, bullying, like learning and confidence) that are used to measure students’ well-being in this study. Overall, the students’ well-being was relatively good. Examination of the trends of means showed that there are some changes in students’ well-being from 2011 to 2019. After 2011, students’ sense of belonging increased and bullying decreased slightly. With respect to students’ attitudes, the trends between 2011 and 2019 were not so explicit. Between 2011 and 2015, students liking mathematics grew to some extent, whereas confidence in mathematics remained unchanged. Students liking science, instead, increased from 2011 to 2015 but decreased again from 2015 to 2019. In addition, students’ confidence in science declined between 2015 and 2019.
The preliminary results of SEM showed that students’ SES is related both to well-being and achievement. As expected, students with higher SES (i.e., students with many resources for learning) also feel better and achieve higher results in mathematics and science. Students’ SES seemed to be related to achievement not only directly but also indirectly via confidence. However, there was no indirect effect via other well-being variables than confidence. This study supports earlier research on the meaning of students’ well-being for learning.
In further studies, we will examine the relationship between student well-being and academic achievement also by PISA and PIRLS (Progress in International Reading Literacy Study) data collected not only before but also after the COVID-19 pandemic, which has affected, mostly negatively, students’ schooling, learning and well-being all over the world (e.g. OECD, 2021).
ReferencesBücker, S., Nuraydin, S., Simonsmeier, B. A., Schneider, M., & Luhmann, M. (2018). Subjective well-being and academic achievement: A meta-analysis. Journal of Research in Personality, 74, 83–94.
Hascher, T. (2003). Well-being in school – why students need social support. In P. Mayring & C. von Rhöneck (Eds.), Learning emotions – the influence of affective factors on classroom learning (pp. 127–142). Bern u.a Lang.
Helakorpi, S., & Kivimäki, H. (2021). Well-being of children and young people – School Health Promotion study 2021. Finnish Institute for Health and Welfare, Statistical Report 42/2021. https://urn.fi/URN:NBN:fi-fe2021112557144
Martin, M. O., von Davier, M., & Mullis, I. V. S. (2020). Methods and procedures: TIMSS 2019 technical report. International Association for the Evaluation of Educational Achievement (IEA).
Morinaj, J., & Hascher, T. (2022). On the relationship between student well-being and academic achievement: A longitudinal study among secondary school students in Switzerland. Zeitschrift für Psychologie, 230(3), 201–214.
Mullis, I. V. S., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 International Results in Mathematics and Science. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: https://timssandpirls.bc.edu/timss2019/international-results/
Nilsen, T., Kaarstein, H., & Lehre, A. C. (2022). Trend analyses of TIMSS 2015 and 2019: school factors related to declining performance in mathematics. Large-scale Assessments in Education, 10(1), 1–19.
OECD (2017). PISA 2015 Results (Volume III): Students’ Well-Being, PISA, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264273856-en
OECD (2019). PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://doi.org/10.1787/5f07c754-en.
OECD (2021). The State of Global Education: 18 Months into the Pandemic. https://doi.org/10.1787/1a23bb23-en
Pietarinen, J., Soini, T., & Pyhältö, K. (2014). Students’ emotional and cognitive engagement as the determinants of well-being and achievement in school. International Journal of Educational Research 67, 40–51.
Salmela-Aro, K., Read, S., Minkkinen, J., Kinnunen, J. M., & Rimpelä, A. (2018). Immigrant status, gender, and school burnout in Finnish lower secondary school students: A longitudinal study. International Journal of Behavioral Development, 42(2), 225–236.
Salmela-Aro, K., Upadyaya, K., Vinni-Laakso, J., & Hietajärvi, L. (2021). Adolescents’ longitudinal school engagement and burnout before and during COVID-19 – The role of socio-emotional skills. Journal of Research on Adolescence, 31(3), 796–807.
|
