09. Assessment, Evaluation, Testing and Measurement
Paper
Predicting school failure in Sweden: A longitudinal approach
Monica Rosén1, Erika Majoros2
1University of Gothenburg, Sweden; 2Umeå University
Presenting Author: Rosén, Monica;
Majoros, Erika
Previous research has identified that cognitive ability and socioeconomic status (SES) indicated by parental education, occupation, or income (Marks, 2013) are the strongest predictors of school outcomes. Cognitive ability is the strongest predictor of school achievement, with correlations around .60-.70 (Gustafsson & Balke, 1993), while SES typically correlates around .30-.40 with school achievement (Sirin, 2005). Longitudinal investigations of the strength of the associations concluded that the influence of SES is declining (Marks, 2013). However, in Sweden the strength of the association between SES and achievement has increased during the last decades (Gustafsson & Yang Hansen, 2018), suggesting that equity of schooling outcomes has deteriorated.
Another important factor influencing school outcomes is gender. Girls tend to outperform boys in terms of grades internationally (Dwyer & Johnson, 1997), and this is true for Swedish students as well. Even more concerningly, boys are more at risk of dropping out of school in Sweden (World Bank, 2024).
In Sweden, compulsory education ends in the school year 9, while in the optional upper secondary school, there are 18 regular national programs of three years to choose from, six of which are preparatory for higher education such as university, and twelve of which are vocational. While entrance requirements vary between programs, all of them demand students to have passing grades in Swedish/Swedish as a second language, English, an d mathematics from their final year of compulsory schooling.
The main question, which can be investigated for all birth cohorts between 1948 and 2004, is the relative importance of cognitive ability, social background, cultural background, and gender as determinants of school failure and general school achievement, and how this varies as a function of school characteristics and societal factors.
Methodology, Methods, Research Instruments or Sources UsedWe define four levels of school failure: premature failure, i.e., no grades or low grades in year 6; early failure, i.e., no grades in year 9; midway failure, i.e., not eligible for upper secondary school, and late failure, i.e., no final grades/exam within three years of finishing upper secondary school.
Starting with a basic model including grade point average (GPA) from compulsory school, along with cognitive abilities from grade 6 and background variables, predicting school failure. The differentiation of students into different programs will be dealt with through a dummy variable approach and/or through fitting separate models for different programs or groups of programs. As for the compulsory school model, explanatory variables will be added in the next step, using the same sources of information. Longitudinal data from two sources are used; the GOLD and the UGU databases which partially overlap in that the UGU participants in the seven birth cohorts 1972, 1977, 1982, 1987, 1992, 1998, and 2004 also are included in GOLD.
The data allow a large number of cohort comparisons, focusing on curricular and organizational aspects, and on societal changes such as increasing economic inequity and school segregation. Both comprehensive school and upper secondary school will be investigated.
Conclusions, Expected Outcomes or FindingsThe empirical results will be discussed in light of the educational research and political discourse that preceded the reforms, in which both gender and cognitive ability were considered to be of key importance. Along with descriptions of the changes in the school organization and school curricula, this study will contribute to an understanding of the three levels of curriculum (the intended, the implemented, and the achieved curriculum which in interplay with social and home background factors determine children’s opportunity to learn (McDonnell, 1995); and to the changes in the school system that followed with school reforms.
ReferencesDwyer, C. A., & Johnson, L. M. (1997). Grades, accomplishments, and correlates. In Gender and fair assessment (pp. 127–156). Lawrence Erlbaum Associates Publishers.
Gustafsson, J.-E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28(4), 407–434. https://doi.org/10.1207/s15327906mbr2804_2
Gustafsson, J.-E., & Yang Hansen, K. (2018). Changes in the impact of family education on student educational achievement in Sweden 1988-2014. Scandinavian Journal of Educational Research, 62(5), 719–736. https://doi.org/10.1080/00313831.2017.1306799
Marks, G. N. (2013). Education, social background and cognitive ability: The decline of the social. Routledge. https://www.routledge.com/Education-Social-Background-and-Cognitive-Ability-The-decline-of-the-social/Marks/p/book/9781138923225
McDonnell, L. M. (1995). Opportunity to learn as a research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17(3), 305–322. https://doi.org/10.3102/01623737017003305
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453.
World Bank. (2024). Education statistics—All indicators. https://databank.worldbank.org/source/education-statistics-%5e-all-indicators
09. Assessment, Evaluation, Testing and Measurement
Paper
Determinants of School Failure in Sweden
Monica Rosén1, Erika Majoros2
1University of Gothenburg, Sweden; 2Umeå University, Sweden
Presenting Author: Rosén, Monica;
Majoros, Erika
Previous research has identified two main individual-level determinants of school outcomes: cognitive ability and socioeconomic status (SES) indicated by parental education, occupation, or income (Marks, 2013). Cognitive ability is the strongest predictor of school achievement, with correlations around .60-.70 (Gustafsson & Balke, 1993), while SES typically correlates around .30-.40 with school achievement (Sirin, 2005).
However, there are substantial country differences in these relationships, and longitudinal investigations of the strength of the associations have also been observed, and Marks (2013) concluded that the influence of SES is declining. On the contrary, for Sweden an increase in the strength of the association between SES and achievement has been observed during the last decades (Gustafsson & Yang Hansen, 2018), suggesting that equity of schooling outcomes has deteriorated. Moreover, gender differences have been observed in Sweden in terms of grades in line with international trends (Dwyer & Johnson, 1997), and boys are more at risk of dropping out of school in Sweden (World Bank, 2024).
Recently, much attention has been devoted to personality characteristics as determinants of success and failure in school, such as conscientiousness (Almlund et al., 2011), grit (Duckworth et al., 2007), and growth mindset (Dweck, 2008).
Other individual characteristics too have been shown to contribute to school achievement. Prominent examples are self-efficacy (Bandura, 1997), self-concept(Bong & Skaalvik, 2003), and intrinsic and extrinsic motivation (Ryan & Deci, 2000).
In Sweden, compulsory education involves school years 1 to 9, while in the optional upper secondary school, there are eighteen regular national programs of three years to choose from, six of which are preparatory for higher education such as university, and twelve of which are vocational. While entrance requirements vary between programs, all of them demand students to have passing grades in Swedish/Swedish as a second language, English, and mathematics from their final year of compulsory schooling.
The main aim of this study is to investigate individual and social determinants in the development of school failure using a longitudinal approach for two birth cohorts that have followed the same curriculum, born in 1998 and 2004.
Methodology, Methods, Research Instruments or Sources UsedWe define four levels of school failure: premature failure, i.e., no grades or low grades in year 6; early failure, i.e., no grades in year 9; midway failure, i.e., not eligible for upper secondary school, and late failure, i.e., no final grades/exam within three years of finishing upper secondary school.
Information about gender and SES will be used to explain individual variation in cognitive abilities in school year 6. This model will be extended with results on national tests and will be used to predict achievement and school failure in school year 9. Next, explanatory variables derived from the student questionnaires (e.g., self-concept/self-efficacy, achievement goal preferences, motivation, coping, self-reported mental health) and registers (e.g., school relocations, participation in special needs education, mental health problems) will be added to the model to investigate to what extent they affect the risk for school failure.
For upper secondary school, a similar approach will be taken, starting with a basic model including grade point average (GPA) from compulsory school, along with cognitive abilities from school year 6 and background variables, predicting school failure in the form of dropout or low grades. The differentiation of students into different programs will be dealt with through a dummy variable approach and/or through fitting separate models for different programs or groups of programs.
Conclusions, Expected Outcomes or FindingsIn combination, the results from the models for compulsory and upper secondary school will provide good coverage of individual and social determinants of school failure.
ReferencesAlmlund, M., Duckworth, A. L., Heckman, J., & Kautz, T. (2011). Personality psychology and economics. In Handbook of the economics of education: Vol. 4 (pp. 1–181). Elsevier. https://econpapers.repec.org/bookchap/eeeeduchp/4-1.htm
Bandura, A. (1997). Self-efficacy: The exercise of control. W.H. Freeman.
Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15(1), 1–40. https://doi.org/10.1023/A:1021302408382
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087–1101. https://doi.org/10.1037/0022-3514.92.6.1087
Dweck, C. S. (2008). Mindset: The new psychology of success (Ballantine Books trade pbk. ed.). Ballantine Books.
Dwyer, C. A., & Johnson, L. M. (1997). Grades, accomplishments, and correlates. In Gender and fair assessment (pp. 127–156). Lawrence Erlbaum Associates Publishers.
Gustafsson, J.-E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28(4), 407–434. https://doi.org/10.1207/s15327906mbr2804_2
Gustafsson, J.-E., & Yang Hansen, K. (2018). Changes in the impact of family education on student educational achievement in Sweden 1988-2014. Scandinavian Journal of Educational Research, 62(5), 719–736. https://doi.org/10.1080/00313831.2017.1306799
Marks, G. N. (2013). Education, social background and cognitive ability: The decline of the social. Routledge. https://www.routledge.com/Education-Social-Background-and-Cognitive-Ability-The-decline-of-the-social/Marks/p/book/9781138923225
Ryan & Deci. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453.
World Bank. (2024). Education statistics—All indicators. https://databank.worldbank.org/source/education-statistics-%5e-all-indicators
09. Assessment, Evaluation, Testing and Measurement
Paper
Do They Achieve What They Aimed For? Trajectories and Achieved School-leaving Certificates of Retained Students.
Paul Fabian1, Katja Scharenberg1, Alyssa Grecu2
1Ludwig Maximilian University Munich; 2TU Dortmund / IFS
Presenting Author: Fabian, Paul
Nationally and internationally, grade retention is a highly controversial measure to homogenise students with different competence levels. In Germany, some federal states (e.g., Hamburg and Berlin) have already abolished grade retention. In Bavaria and Bremen, however, grade retention rates are above the national average (2.3%, see Statistisches Bundesamt, 2018). In an international comparison, the rate of German pupils who have been retained at least once in the course of their educational career is above the OECD-average (Germany: 19.6%, OECD: 12.2 %; ꭓ² = 29558.56, df=1, p<.001; own calculations). Empirical evidence on the effectiveness of grade retention is still insufficient. International studies (Goos et al., 2021) showed that there are short-term improvements in performance after being retained, but they decrease in the medium and long term. Especially in highly hierarchically structured education systems such as Germany, Belgium, the Netherlands or Switzerland, grade retention is least effective (ibid.). In Germany, only a few reliable studies allow concrete statements about the effectiveness of grade retention: Positive effects of grade retention on students’ performance development were not proven (Ehmke et al., 2017; Fabian, 2020). Beyond this, Fabian (2020) also showed that there was no significant improvement in school grades of retained students. Marsh et al. (2017), however, found an increase in retained students’ math performance after repetition. With regard to the achieved school-leaving qualification, Bellenberg (1999) showed that grade retention is very often associated with school dropout and/or downward change of school track, thus reducing the probability of achieving higher school-leaving qualifications for repeating students. Demski and Liegmann (2014) reported only minor differences between repeaters and promoted students. Given the theoretical assumptions of credentialism (Bills, 2003), DiPrete et al.'s (2017) findings from an international comparison are noteworthy: In Germany, school-leaving certificates are particularly important for future success on the training and labour market. At the same time, objective competencies and obtained certificates are often incongruent (Brändle & Pohlmann, 2021). Empirical findings showed that particularly students with low qualifications successfully enter the training market if they have good grades and high educational aspirations – regardless of their cognitive and social skills (Holtmann et al., 2017). Due to the strong correlation between school-leaving qualifications and success on the training and labour market, the present study investigates whether grade retention has advantages or disadvantages for retained students. Previous research findings suggest that being retained might lead to lower school-leaving qualifications.
Methodology, Methods, Research Instruments or Sources UsedData basis for the present analyses was the German National Educational Panel Study (NEPS, starting cohort 3; Blossfeld et al., 2011). The sample was initially representative of Grade 5 students in Germany and consisted of n = 6,491 students. For the present analyses, we excluded students in school tracks where different school-leaving certificates can be obtained (i.e., students from tracks with several educational programmes, comprehensive tracks). Students for whom no information on the attended school track was available were also excluded from analysis. This resulted in an analysis sample of n = 4,371 students, 118 of whom were retained in Grade 7 (2.5%). We treated missing values using multiple imputation (m = 55 ) in R 4.2.2 (R Core Team, 2023) via the package mice (van Buuren & Groothuis-Oudshoorn, 2011), accounting for the clustered data structure and the frequently non-normal data distribution. Based on this analysis sample, we calculated propensity scores using the Rubin Causal Model (Rubin, 1974) and conducted propensity score matching (Rosenbaum & Rubin, 1983) based on objective competencies in reading and math before grade retention as well as information based on key background characteristics provided by students and parents. Grade repeaters were then matched with non-repeaters (full matching, caliper = .10). This allowed us to compare retained students with similar students regarding key background characteristics, but who were regularly promoted. The highest achieved school-leaving certificate, operationalised by CASMIN (König et al, 1988), was then analysed visually. In addition, we created dichotomous dummy variables for (a) a qualification lower than the usual qualification in the respective school type, (b) a qualification appropriate to the school type (i.e., Certificate of Secondary Education at lower tracks [Hauptschule], General Certificate of Secondary Education [GCSE] at intermediate tracks [Realschule], A-level [Abitur] at academic tracks [Gymnasium]) and (c) a qualification higher than the usual qualification at the respective track (i.e., GCSE at lower tracks or A-level after being retained in intermediate tracks). We conducted logistic regressions to analyse the effect of grade repetition on the adequacy of the school-leaving certificate. Results show that a grade retention seems to reduce the chance of achieving a track-equivalent qualification (OR_fit = .92, p = .089). Grade retention had no influence on the chance of achieving a higher qualification than usual in the respective school track. However, grade repetition increased the chance of obtaining a lower qualification (OR_lower = 1.12, p < .001).
Conclusions, Expected Outcomes or FindingsIn summary, we found that students who were retained in Grade 7 were less likely to achieve a school-leaving certificate that is appropriate to their attended school track. Also – in line with findings from Bellenberg (1999) – the risk of achieving a lower school-leaving certificate increased when students were retained. Since there is some evidence that grade retentions do not lead to better grades (Fabian, 2020) – which is one of the most important goals of grade retention – the findings of Holtmann et al. (2017) become even more important: Even with low qualifications, but good grades and high educational aspirations, young adolescents’ successful transition to the training and labour market is more likely. In that regard, as can be assumed based on the findings of the present study, grade retention fails its goal to help students strengthen their academic outcomes and their chances of obtaining a track-adequate school-leaving certificate. Thus, further investigation is needed to analyse whether students with lower school-leaving qualifications than appropriate to the respective attended school track reach this lower qualification at least with better grades.
The reason why previous studies (e.g., Demski & Liegmann, 2014) did not find differences in educational attainment for repeaters and non-repeaters might be that in the past decades, “irregular” – i.e., non-linear – trajectories became more common. Thus, the achievement gap between repeaters and non-repeaters after compulsory education might be narrowed by further training. In their study, Demski and Liegmann (2014) used retrospective information of participants, so the effect of further training could be accounted for. However, we could not address these effects with the data of the present study, yet it will hopefully be possible with the ongoing studies of the NEPS in the future.
ReferencesBellenberg, G. (1999). Individuelle Schullaufbahnen: eine empirische Untersuchung über Bildungsverläufe von der Einschulung bis zum Abschluss. Weinheim: Juventa.
Bills, D. B. (2003). Credentials, signals, and screens: Explaining the relationship between schooling and job assignment. Review of Educational Research, 73(4), 441-469.
Blossfeld, H. P., & Von Maurice, J. (2019). Education as a lifelong process (pp. 17-33). Wiesbaden: Springer Fachmedien.
Brändle, T., & Pohlmann, B. (2021). Alles nur eine Frage der Kompetenz? Leistungs- und Chancengerechtigkeit bei der Vergabe von Schulabschlüssen und Abschlussnoten. Zeitschrift für Soziologie, 50(1), 58-77.
Demski, D., & Liegmann, A. B. (2014). Klassenwiederholungen im Kontext von Schul- und Berufsbiographien. In: A. B. Liegmann, I. Mammes & K. Racherbäumer (eds.). (2014). Facetten von Übergängen im Bildungssystem. Nationale und internationale Ergebnisse empirischer Forschung (pp. 173-189). Münster: Waxmann (2014)
DiPrete, T. A., Eller, C. C., Bol, T., & Van de Werfhorst, H. G. (2017). School-to-work linkages in the United States, Germany, and France. American Journal of Sociology, 122(6), 1869-1938.
Ehmke, T., Sälzer, C., Pietsch, M., Drechsel, B., & Müller, K. (2017). Kompetenzentwicklung im Schuljahr nach PISA 2012: Effekte von Klassenwiederholungen. Zeitschrift für Erziehungswissenschaft, 2(20), 99-124.
Fabian, P. (2020). Leistungskonsolidierung, Leistungssteigerung-oder etwas ganz anderes? Die Effekte einer Klassenwiederholung auf die Leistungsentwicklung. Münster: Waxmann.
Goos, M., Pipa, J., & Peixoto, F. (2021). Effectiveness of grade retention: A systematic review and meta-analysis. Educational Research Review, 34, 100401.
Holtmann, A. C., Menze, L., & Solga, H. (2017). Persistent disadvantages or new opportunities? The role of agency and structural constraints for low-achieving adolescents’ school-to-work transitions. Journal of Youth and Adolescence, 46, 2091-2113.
König, W., Lüttinger, P., & Müller, W. (1988). A comparative analysis of the development and structure of educational systems: Methodological foundations and the construction of a comparative educational scale. Mannheim: Universität Mannheim, Institut für Sozialwissenschaften.
Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Lichtenfeld, S. (2017). Long-term positive effects of repeating a year in school: Six-year longitudinal study of self-beliefs, anxiety, social relations, school grades, and test scores. Journal of Educational Psychology, 109(3), 425-438.
R Core Team (2023). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688-701.
|
