09. Assessment, Evaluation, Testing and Measurement
Paper
The Lagom Effect: School Composition and Inequality of Opportunities in Sweden
Victoria Rolfe
University of Gothenburg, Sweden
Presenting Author: Rolfe, Victoria
Sweden’s self-image as a leader in education was rocked in the early 2010s by the so-called PISA-shock, in which this formerly high-flying education system saw its performance in international assessments dramatically decline. This period of decline dominated the public and political discourse around education and reform in Sweden (e.g. Lundahl & Serder, 2020) for the rest of the decade. Over time, Sweden’s performance in Mathematics has recovered, as evidenced in numerous international assessments, including most recently TIMSS 2019 and PISA 2018 (e.g. Mullis et al., 2020; OECD, 2019a). Nevertheless, the improvement in the overall achievement of Swedish youth somewhat masks a persistent achievement gap which has been observed within the Swedish school system since the early 2000s, with growing variation in performance between schools in student grades (Skolverket, 2005, 2020). The achievement gap noted in domestic data has also been recorded in international data, with the achievement gap widening (Chmielewski, 2019) and Sweden’s decline in socioeconomic equality of outcomes the most severe among peer nations (Hanushek et al., 2014).
Socioeconomic status is a well-established predictor of educational outcomes (e.g. Sirin, 2005), and previous research using TIMSS data has confirmed this relationship in relation to mathematics outcomes for Swedish youth over multiple cycles of TIMSS between 2003 and 2015 (Authors, 2021). A longstanding strand of scholarship suggests that in addition to predicting achievement, socioeconomic background indicates varied opportunity to learn (OTL) course material, which in turn predicts test performance (e.g. Eggen et al., 1987). While this pattern of relationships has been consistently evidenced in the English-speaking world (e.g. Authors, 2021; Schmidt et al.,2013), in the Swedish context inequalities of opportunities have been inconsistently observed, appearing only among the 2003 and 2015 TIMSS cohorts (Authors, 2021).
A possible explanation for the lack of observable social reproduction through the delivery of the curriculum lies in the nature of the Swedish school system. A distinctive feature of the Swedish education system is its retention of the comprehensive model in which students are offered equal learning opportunities in integrated school settings (Arnesen & Lundahl, 2006) with limited within-school streaming when compared with other highly developed economies (Chmielewski, 2014). Of much interest to policy makers and researchers, reforms to the Swedish education system enacted in the 1990s introduced school choice and created a market for education (see Björklund et al., 2005). While admissions guidance prohibits cream-skimming (Põder et al., 2017), the exercise of school choice is socially segregated (Teske & Schneider, 2001) and the subsequent composition of schools can be interpreted as reflecting segregation beyond an expected neighbourhood effect (Böhlmark et al., 2016). Despite the observed social segregation between schools, analysis of international data suggests that the comprehensive school system in Sweden is still intact, with students of varying abilities attending the same schools, and that variation in performance between school is low when compared to other economies (OECD, 2019b).
Against this background, the following research questions are considered:
- Are between school socioeconomic inequalities in mathematics outcomes and opportunity to learn mathematics observable among eighth graders in Sweden?
- Do the relationships between socioeconomic status, opportunity to learn, and achievement very between high-, neutral-, and low-SES schools?
Methodology, Methods, Research Instruments or Sources UsedThe study uses Swedish data from the grade 8 sample of the Trends in Mathematics and Science Study (TIMSS) 2019. The focus of the study is between school variation in the relation between inequalities in opportunity to learn (expressed as content coverage) and mathematics outcomes, and thus data from the student, teacher, and school questionnaire is used. Socioeconomic status and opportunity to learn are both conceived as unobserved phenomena and are thus modelled as latent factors. Socioeconomic status is indicated by the number of books in the home, the highest reported parental education level, and the sum of five home possession items. OTL is indicated by manifest variables in which teacher responses to items regarding when content is introduced is summed to create indicators of content coverage in each of the four sub-domains of mathematics (number, algebra, geometry, and data).
Structural equation modelling is used in the study to model the relations between SES, OTL, and achievement in mathematics. Complex survey data such as the TIMSS 2019 dataset favours a multilevel approach to modelling, as it allows the variance in the dependent variable, in this case achievement, to be split across individual and school levels and provides model estimates at both levels. A two-level model is specified with individual achievement regressed on SES at the student level, and a trio of relations – achievement is regressed on SES and OTL, and OTL is regressed on SES – are specified at the school level. As the focus of the study is between school differences, data from the student and teacher questionnaires is aggregated to school level to build the between level of the model. The modelling process features two stages to reflect the research questions. In the first stage, model one – the basic model – is run to identify whether socioeconomic inequalities in outcomes and opportunities can be identified for the sample as a whole. In the second stage, model two separates schools into three groups with each school classified as high-, neutral-, or low-SES, with the goal of establishing whether patterns of inequalities differ between different school profiles.
Conclusions, Expected Outcomes or FindingsPreliminary results suggest that for the Swedish TIMSS 2019 grade eight cohort as a whole, SES remains a strong predictor of achievement at the individual and school levels, in line with earlier research. However, evidence of socioeconomic inequalities in OTL were not observed. When the cohort was categorised as high-, neutral-, and low-SES schools, patterns of inequalities differed between groups, with the most notable results seen in the neutral-SES group. For this group, SES at the school level was a very strong predictor of achievement, and OTL was a significant predictor of achievement, which was not replicated in the other two groups.
In Swedish, the neutral-SES schools could be described as lagom, a concept which roughly translates to ‘not too much, not too little’ or ‘just the right amount’. It is therefore highly relevant to stakeholders in the educational project that it is in these schools with a balanced socioeconomic intake that the Swedish system goes beyond its’ comprehensive character and appears to act in a compensatory manner in terms of mathematics provision.
ReferencesArnesen, A. L., & Lundahl, L. (2006). Still social and democratic? Inclusive education policies in the Nordic welfare states. Scandinavian Journal of Educational Research, 50(3), 285-300.
Authors. (2021).
Björklund, A., Clark, M. A., Edin, P. A., Fredriksson, P., & Krueger, A. B. (2005). The market comes to education in Sweden: An evaluation of Sweden's surprising school reforms. Russell Sage Foundation.
Böhlmark, A., Holmlund, H., & Lindahl, M. (2016). Parental choice, neighbourhood segregation or cream skimming? An analysis of school segregation after a generalized choice reform. Journal of Population Economics, 29(4), 1155-1190.
Chmielewski, A. K. (2014). An international comparison of achievement inequality in within- and between-school tracking systems. American Journal of Education, 120(3), 293–324.
Chmielewski, A. K. (2019). The global increase in the socioeconomic achievement gap, 1964 to 2015. American Sociological Review, 84(3), 517-544.
Eggen, T. J. H. M., Pelgrum, W. J., & Plomp, T. (1987). The implemented and attained mathematics curriculum: Some results of the second international mathematics study in The Netherlands. Studies in Educational Evaluation, 13(1), 119-135
Hanushek, E. A., Piopiunik, M., & Wiederhold, S. (2014). The value of smarter teachers: International evidence on teacher cognitive skills and student performance. National Bureau of Economic Research.
Lundahl, C., & Serder, M. (2020). Is PISA more important to school reforms than educational research? The selective use of authoritative references in media and in parliamentary debates. Nordic Journal of Studies in Educational Policy, 6(3), 193-206.
Mullis, I. V. S., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 International Results in Mathematics and Science. https://timssandpirls.bc.edu/timss2019/international-results/
OECD. (2019a). PISA 2018 Results (Volume I): What students know and can do.
OECD. (2019b). PISA 2018 Results (Volume II): Where All Students Can Succeed.
Põder, K., Lauri, T., & Veski, A. (2017). Does school admission by zoning affect educational inequality? A study of family background effect in Estonia, Finland, and Sweden. Scandinavian Journal of Educational Research, 61(6), 668-688.
Schmidt, W. H., Zoido, P., & Cogan, L. (2013). Schooling Matters: Opportunity to Learn in PISA 2012. OECD Education Working Papers(95).
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417-453.
Skolverket. (2005). Skolverkets lägesbedömning 2005. https://www.skolverket.se/download/18.6bfaca41169863e6a655903/1553958898329/pdf1516.pdf
Skolverket. (2020). Skolverkets lägesbedömning 2020. https://www.skolverket.se/publikationer?id=6436
Teske, P., & Schneider, M. (2001). What research can tell policymakers about school choice. Journal of Policy Analysis and Management, 20, 609-631.
09. Assessment, Evaluation, Testing and Measurement
Paper
Does Tracking Increase School Segregation of Immigrants? A Difference-in-Differences Approach.
Janna Teltemann1, Max Brinkmann1, Nora Huth-Stöckle2
1Universität Hildesheim, Germany; 2Bergische Universität Wuppertal
Presenting Author: Teltemann, Janna
Social integration in the context of increasing immigration is a challenge faced by many industrialized countries. The institutional set-up of an education system is a natural candidate for scrutiny from a policy perspective, since education is at the forefront of social integration and institutional structures are malleable in nature. In this study, we examine immigrant social integration through school segregation and how it relates to the institutional structure of an educational system, in particular the existence of (early between-school) tracking. As a premise for integration, school segregation is crucial since it determines how much interaction between immigrants and non-immigrants occurs. Tracking on the other hand is as controversial as it is momentous since students are placed into different types of schools from an age as young as ten (e.g. Terrin & Triventi 2022, van de Werfhorst & Mijs 2010). This is in stark contrast to the practice of integrated education systems, in which students may be grouped by ability for certain topics or classes, but are only separated as they approach maturity.
We therefore examine whether tracked education systems show higher levels of school segregation of immigrants as education systems that delay between-school grouping. While sociologists have long documented the negative effects of tracking with regards to equality of opportunity (e.g. Terrin & Triventi 2022, van de Werfhorst & Mijs 2010), we argue that there may be counteracting mechanisms at work in tracked systems when it comes to ethnic segregation between schools.
We follow theories on educational inequality to understand school segregation and tracking. These theories relate differences in family resources to differences in educational attainment or achievement (i.e. Boudon 1974, Bourdieu 1987, Lareau 2011). Resources in this context can comprise economic capital, strategic knowledge, social contacts and familiarity with modes of behavior in the education system. In this context, we expect that immigrant students are disadvantaged, as many of these resources cannot easily be translated from the home country to the receiving country. We can therefore expect that they will show, on average, lower achievement at the end of primary school (i.e. primary effects; Boudon 1974). These finding has been shown by numerous studies (e.g. Heath et al. 2008).
Since observed achievement is a major indicator of track placement, primary effects of ethnic and social origin increase the likelihood for immigrant students to be sorted into lower secondary school tracks. However, parental decision making (i.e. secondary effects; Boudon 1974) is another determinant of track placement and it is well-known that immigrant parents tend to choose more ambitious educational pathways (Esser 2016; Gresch et al. 2012) which could compensate for low track placement based on ability. Lastly, school segregation likely exists in non-tracked systems as well. First, because home-to-school-distances are a main factor in selecting a school, residential segregation, which is a common phenomenon in many countries, is reflected in school segregation. Further, school choice behavior of non-immigrant families may contribute to ethnic school segregation, as particularly high status families tend to avoid schools with larger numbers of immigrants (“white flight”; Amor 1980). They do so, because they use immigrant concentration as a proxy for (lower) school quality. This tendency might be lower in tracked-systems, as track level is an accessible indicator of school quality. Non-immigrant families therefore do not need to avoid schools with larger numbers of immigrants (c.p. Meier & Schütz 2007).
In sum, there may be counteracting mechanisms with regard to school segregation and the age of first tracking. We therefore argue that it remains an empirical question to determine which mechanism outweighs the other.
Methodology, Methods, Research Instruments or Sources UsedPrevious research on the effects of tracking on ethnic segregation point towards mixed effects of tracking (e.g. Kruse 2019). However, most previous findings look at data from single countries or cities. Moreover, they face the challenge of cross-sectional analyses that might be biased by unobserved heterogeneity.
We therefore aim at generating more generalizable findings on the impact of tracking on segregation by combining all data from PISA, TIMSS and PIRLS cycles between 1995 and 2018 for a total of 45 countries. In order to combine the data, we harmonized the relevant information, most importantly information on immigrant background. We define immigrant background by the place of birth of the student (abroad). Based on this information, we calculated measures of segregation (index of dissimilarity D, Duncan & Duncan 1955) for each study-year and each country.
Crucial for our analytical approach is the fact that some of the studies are implemented in primary school - when no education system is tracked - and others are administered in secondary school (in grade 8 or at age 15), i.e. after tracking has been exercised. According to our definition (tracking takes place before grade 8) this is the case for nine countries in our sample.
Our analysis is based on a difference-in-differences approach that compares the difference in ethnic segregation between primary and secondary school and between tracked and untracked countries. This approach enables us to account for all other time-stable differences between countries. We still included control variables that can change over time: the gross domestic product and the population density and the privatization of the education system. Such decisions (e.g. including control variables or excluding probable outliers) however might have substantial impact on the obtained estimates. We therefore do not conduct a single analysis, instead we follow the approach of multiverse analyses (Simonsohn et al. 2020).
The term "multiverse analysis" refers to a type of analysis that accounts for the problem of multiple “forking paths” (Gelman & Loken 2013), because a research design has to be operationalized with variables, samples and estimation techniques. By systematically varying these decisions across all possible paths, we will “expand” a multiverse that incorporates all possible paths. In other words, it is a systematic way of doing robustness checks.
Conclusions, Expected Outcomes or FindingsOur preliminary results suggest that the presence of early between-school grouping (as compared to late between school grouping) has no discernible impact on immigrant school segregation. While segregation increases in both types of education systems there are heterogenous effects across model specifications with respect to the effect of tracking. By varying the choice of fixed effects, control variables (GDP, private school density and population density) and sample restrictions (different GDP cut-offs and different cut-offs for minimum or maximum share of immigrant students in a country) we obtain about 4000 model specifications of which 60% show a small negative (but overwhelmingly insignificant) effect and 40% show a small positive (but overwhelmingly insignificant) effect on school segregation. In our next steps, we will examine the effects of selectivity on segregation. We expect that higher selectivity will limit the ambitious school choices of immigrant families and therefore lead to higher levels of school segregation.
ReferencesArmor, D. J. (1980). White flight and the future of school desegregation. School desegregation: Past, present, and future, 187-226.
Bourdieu, P. (1987). Die feinen Unterschiede. Suhrkamp.
Duncan, O. D., & Duncan, B. (1955). A Methodological Analysis of Segregation Indexes. American Sociological Review, 20(2), 210–217. Retrieved from http://www.jstor.org/stable/2088328
Esser, H. (2016). Bildungssysteme und ethnische Bildungsungleichheiten. Ethnische Ungleichheiten im Bildungsverlauf: Mechanismen, Befunde, Debatten, 331-396.
[English: “Education systems and ethnic educational inequalities” in “Ethnic inequality along the educational pathway: Mechanisms, Results, Debates”]
Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University, 348, 1-17.
Gresch, C., Maaz, K., Becker, M., & McElvany, N. (2012). Zur hohen Bildungsaspiration von Migranten beim Übergang von der Grundschule in die Sekundarstufe: Fakt oder Artefakt. Soziale Ungleichheit in der Einwanderungsgesellschaft. Kategorien, Konzepte, Einflussfaktoren, 56-67.
[English: “The case of high educational aspirations among migrants when transitioning from primary school to secondary school: fact or artifact?”]
Heath, A. F., Rothon, C., & Kilpi, E. (2008). The Second Generation in Western Europe: Education, Unemployment, and Occupational Attainment. Annual Review of Sociology, 34(1), 211–235. https://doi.org/10.1146/annurev.soc.34.040507.134728
Kruse, H. (2019). Between-school ability tracking and ethnic segregation in secondary schooling. Social Forces, 98(1), 119-146.
Lareau, A. (2011). Unequal Childhoods: Class, Race, and Family Life. Univ of California Press.
Meier, V., & Schütz, G. (2007). The economics of tracking and non-tracking (No. 50). Ifo working paper.
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour, 4(11), 1208-1214.
Terrin, E., & Triventi, M. (2022). The effect of school tracking on student achievement and inequality: A meta-analysis. Review of Educational Research, 00346543221100850.
Van de Werfhorst, H. G., & Mijs, J. J. (2010). Achievement inequality and the institutional structure of educational systems: A comparative perspective. Annual review of sociology, 36, 407-428.
09. Assessment, Evaluation, Testing and Measurement
Paper
The Importance of Relative Age for Academic Achievement and Socioemotional Competencies
Alli Klapp
University of Gothenburg, Sweden
Presenting Author: Klapp, Alli
This study examines the impact of the Relative Age Effect (RAE), measured by students´ birth month, the cognitive ability, school achievement and socioemotional competencies. A longitudinal approach is applied by using data from a Swedish cohort of students born in 1972, from 3rd Grade (age 10) until the end of upper secondary school (age 19).
When Swedish students begin their first school year in August every year (age 7) some children are almost one year older than some of their peers in the same year level. This is due to that the school entry cut-off date is 1st January.
Being among the oldest or youngest in a group of students has been shown to have effects on many outcomes throughout school trajectory. Being relatively young has been identified as a negative impact on cognitive and maturity issues (Rod Larsen & Solli, 2017). While being relatively older when starting school seems to affect achievement, working life and later outcomes positively. Outcomes such as higher achievements and reaching higher educational attainment (Crawford et al. 2014). You are also more likely to participate in high school leadership activities (Dhuey & Lipscombe, 2008) and being more successful in sports (Gibbs et al., 2012). Further, the literature shows the existence of physical advantages of being relatively old which gives advantages in individuals identification processes during their upbringing (McCarthy et al., 2016). While mixed results exist on the impact of relative age on earnings (Black et al., 2008; Fredriksson & Öckert, 2014).
However, the economic literature has found a reverse age effect (RAE), suggesting that being older when starting school is beneficial for earnings earlier in the working career while being younger is beneficial for earnings later in the working career (McCarthy et al. 2016).
The effect of relative age on noncognitive outcomes such as self-concept, self-confidence self-esteem, coping and resilience strategies is also evident (Duckworth et al. 2007; Dweck, 2006). Findings from several studies show that children and adolescents being relatively old in the school cohort become influenced in their self-confidence, self-beliefs, and social interactions in school positively (Crawford et al., 2014). Further, it has been shown that relatively old children have higher self-esteem (Thompson et al., 1999, 2004) and suffer less from psychological and behaviour problems (Muhlenweg et al., 2010) compared to relatively younger students.
Even though research has shown that many socioemotional competencies seem to be affected by RAE some may be more crucial for success in learning such as coping and resilience strategies (Duckworth et al., 2007; Dweck, 2006).
This study contributes to the research field by providing empirical support for long-term consequences of relative age in school on cognitive ability, school achievements and noncognitive competencies in terms of students´ academic self-concept, coping and resilience strategies.
Purposes
The main aim of this study is to examine the importance of the relative age effect, measured by birth month, for students´ cognitive and socioemotional outcomes by using a longitudinal approach. Following research questions will be investigated with longitudinal data from several time points:
How does relative age affect cognitive outcomes in terms of cognitive ability, GPA, and educational attainment?
How does relative age affect socioemotional outcomes in terms of perceived academic self-concept, coping and resilience strategies?
What are the long-term effects of relative age for cognitive and socioemotional outcomes and for subgroups of students related to gender and family socioeconomic status?
Methodology, Methods, Research Instruments or Sources UsedData from the Evaluation Through Follow-up (UGU) longitudinal infrastructure is used. The UGU database contain 10% national representative samples of students in 11 birth cohorts, born between 1948 to 2010. The cohort relevant to the present study were born in 1972 (N=9037). The participants were in grade 3 in the academic years 1987/88. The participants received a survey and a cognitive test in Grade 3 (age 10) and 6 (age 13) and a follow-up survey in Grade 10 (age 16 and without a cognitive test). The cognitive tests in Grades 3 and 6 were identical within the cohort and consisted of verbal, inductive and spatial battery of tasks.
The survey in Grade 10 (age 17) was sent to the participants home address by mail. Administrative and register data such as birth month, grades and educational attainment is available for all the participants through upper secondary education (age 19). The 1972 cohort is unique in the sense that the participants received cognitive ability tests at two time points in compulsory school. Another reason is that the participants finished upper secondary school in 1991.
Descriptive statistics and regression analyses were conducted, and outcomes were measured by cognitive ability in Grade 3 and 6, Grade Point Average in 9th Grade (age 16) and educational attainment in upper secondary school (age 19). All through the analyses gender and socio-economic status (SES) were included. Several multivariate multiple regression models have been estimated and logistic regressions are underway.
Confirmatory factor analyses (CFA) and structural equation models (SEM) will be estimated to investigate the importance of socioemotional competencies (about 20 items reflecting coping and resilient strategy factors) for the relative age effect. Analyses with a longitudinal growth modelling approach is ongoing.
Data management and preparation was conducted in the SPSS program, version 28. The analyses were conducted in the Mplus program, version 8.5 (Muthén & Muthén, 1998-2019).
Conclusions, Expected Outcomes or FindingsPreliminary results show that there is a significant negative age effect on cognitive ability in Grade 3 and 6 and on GPA in Grade 9. The negative age effect decreases over time, being strongest for the measure of cognitive ability in Grade 3 (age 10). The result show that there are no effects for the covariates on the main relations between birth month and the three outcome measures of cognitive ability in Grade 3 and 6, and GPA in Grade 9. Factors reflecting coping and resilient strategies are constructed by CFA and will be analysed in SEM. Growth model analyses including data from upper secondary school is ongoing.
ReferencesBlack, S., Devereux, P., Salvanes, K.G., 2011. Too young to leave the nest? The effects of school starting age. Rev. Econ. Stat. 93, 455–467.
Crawford, C., Dearden, L., Greaves, E., 2014. The drivers of month-of-birth differences in children's cognitive and non-cognitive skills. J. R. Stat. Soc. Ser. A (Stat. Soc.), 177, 829–860.
Dhuey, E., Lipscomb, S., 2008. What makes a leader? Relative age and high school leadership. Econ. Educ. Rev. 27, 173–183.
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087–1101.
Dweck, C. S. (2006). Mindset: The new psychology of success. New York, NY: random house.
Fredriksson, P., Öckert, B., 2014. Life-cycle effects of age at school start. Econ. J. 124, 977–1004.
Gibbs, B., Jarvis, J., Dufur, M., 2012. The rise of the underdog? The relative age effect reversal among Canadian-born NHL hockey players. Int. Rev. Sociol. Sport, 47, 644– 649.
McCarthy, N., Collins, D., & Court, D. (2016). Start hard, finish better: further evidence for the reversal of the RAE advantage. Journal of Sports Science, 34(15), 1461–1465.
Mühlenweg, A.M., Puhani, P.A., 2010. The evolution of the school-entry age effect in a school tracking system. J. Hum. Resour. 45, 407–438.
Rod Larsen, E., & Solli, I.F. (2017). Born to run? Persisting birth month effects on earnings. Labour Economics, 46, 200-2010.
Solli, I.F., 2017. Left Behind by Birth Month. Educ. Econ. http://dx.doi.org/10.1080// 09645292.2017.1287881.
Thompson, A.H., Barnsley, R.H., Battle, J., 2004. The relative age effect and the development of self-esteem. Educ. Res. 46, 313–320.
|
