Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

Please note that all times are shown in the time zone of the conference. The current conference time is: 17th May 2024, 03:52:17am GMT

 
 
Session Overview
Session
09 SES 03 A: Linking Education to Long-Term Outcomes
Time:
Tuesday, 22/Aug/2023:
5:15pm - 6:45pm

Session Chair: Alli Klapp
Location: Gilbert Scott, EQLT [Floor 2]

Capacity: 120 persons

Paper Session

Show help for 'Increase or decrease the abstract text size'
Presentations
09. Assessment, Evaluation, Testing and Measurement
Paper

General Matura Score as a Predictor of Personal Yearly Income more than 15 years later in Slovenia?

Gasper Cankar, Darko Zupanc

National Examinations Centre, Slovenia

Presenting Author: Cankar, Gasper; Zupanc, Darko

Every year, approximately 35% of Slovenian high school graduates who complete academically the most demanding upper secondary education take the General Matura (GM) examinations in Slovenia. The GM is comprised of five subject exams: Slovene language, Mathematics, First foreign language, and two subjects chosen by the student from a selection of over 30 subjects. The GM score in Slovenia is calculated as the sum of grades received in the five subject exams. The score can range from 10 (2+2+2+2+2), the lowest passing grade, to 34 (8+8+8+5+5), the highest possible score. Success on the GM is considered equivalent to completing upper secondary education, and in cases where university study programs have a limited number of applicants, GM scores are used as a selection criteria in the admissions process.

The use of GM scores for university admissions has been studied multiple times (Bucik, 2001; Cankar, 2000; Sočan, Krebl, Špeh & Kutin, 2016) with findings similar to research on external examinations in other countries (Kuncel, Hezlett, & Ones; 2001). However, there is a lack of research on the associations between GM scores (achieved at the age of 19) and various measures of personal success later in life and professional careers at ages 33-40 in national and international literature. In public discussions, you can often hear assertions that students’ school achievements and results on external exams have no relevance for later success on a labour market, income in their professional career, or other measures of success. Despite being a high-stakes examination, GM has not been systematically examined to determine its role and long-term value in the Slovenian educational system.

Our research aims to explore the predictive value of GM scores on socio-economic status (SES) and specifically on yearly income later in life for several cohorts of students. Our null hypothesis is that GM scores do not predict SES or yearly income of students in their professional careers.

While it is commonly assumed that success on the GM examinations at the end of upper secondary education is associated with success at university and to some extent later in professional careers, such claims are difficult to verify scientifically due to a lack of representative and valid data. This research aims to provide a deeper understanding of the associations between GM scores, socio-economic measures and personal success in professional careers.


Methodology, Methods, Research Instruments or Sources Used
We will utilize databases of National Examinations centre that will include whole cohorts of students taking General Matura between years 1995 and 2001, linking them to databases of Slovenian Statistical Office on yearly personal income for 2016 as reported in national tax database. We will also use other databases from National Statistical Office to create socio-economic status (SES) measure for each individual using also data on completed level of education, value of real estates owned and status of occupation.

As highest GM scores (30-34) are relatively rare, we will join cohorts together for the analysis. This will also increase statistical power. If we assume that GM graduates typically needed about five more years of university studies after GM before they entered labour market, then in the year 2016 they mostly had 10-16 years of professional career behind them. This should enable us to see some long-term effects on their income in the data.

We will explore regression models predicting yearly income or SES of graduates and use R as statistical environment for most analyses.

Conclusions, Expected Outcomes or Findings
General Matura examinations test students in many different ways and they include written and oral/internal parts, include multiple choice and open ended items, even essays and are both in form and content well aligned with curriculum. Students, who excel and achieve highest scores, most likely possess the combination of knowledge, skills, attitudes and perseverance that will enable them success in later stages of their life – at university or in the labour market.

Previous studies (Bucik, 2001; Cankar, 2000; Sočan, Krebl, Špeh & Kutin, 2016) suggest that the success at General Matura is associated with success at university. We therefore expect that it is also associated with income or SES as some of possible measures of that success later in person’s career.

Although the association should be there, expected predictive validity for selected measures of success of an individual will probably be low since individual aspirations, interests, career choices, motivation and many other factors not included in our model contribute and shape person’s career.
 
The size of associations or the lack of them will also provide insight for discussions about importance of school and school outcomes for life.  Regardless of someone’s position on engaging and often hot-tempered public discussion about meaning of GM scores or school outcomes in general this research will provide some new facts that can complement opinions and anecdotal arguments mostly present today.

With these findings in sight, the General Matura should not be seen as a goal in itself but as an indicator of person’s academic qualities that imply a success later in life.

References
Bucik, V. (2001). Napovedna veljavnost slovenske mature[Predictive validity of Slovenian Matura]. Psihološka obzorja, 10(3), 75-87.
Cankar, G. (2000). Napovedna veljavnost mature za študij psihologije [Predictive validity of Matura for Psychology Study course]. Psihološka obzorja, 9(1), 59.-68.
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the Graduate Record Examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127(1), 162–81. https://doi.org/10.1037/0033-2909.127.1.162
Sočan, G., Krebl, M., Špeh, A. & Kutin, A. (2016). Predictive validity of the Slovene Matura exam for academic achievement in humanities and social sciences. Horizons of Psychology, 25, 84-93.


09. Assessment, Evaluation, Testing and Measurement
Paper

Long-Term Effect of Academic Resilience on Salary Development, an Autoregressive Mediation Model

Cecilia Thorsen1, Kajsa Yang Hansen2, Stefan Johansson2

1University West, Sweden; 2University of Gothenburg, Sweden

Presenting Author: Thorsen, Cecilia

In research on educational equity, students from socioeconomically disadvantaged homes are typically depicted as low-performers and more likely to fail in school (Sirin, 2005). There are, however, students who, despite their disadvantaged backgrounds, manage to succeed in school. This capacity to overcome adversities in education and still reach successful achievements is referred to as Academic Resilience (Agasisti et al., 2018). Academic Resilience is built upon two critical conditions, namely, exposure to significant threats or severe adversity and achievement of positive adaptation despite major assaults on the developmental process (Kiswarday, 2012). Resilience is often captured by identifying protective and risk factors that predict the likelihood of achieving resilient outcomes. Risk factors are characteristics which heighten the risk of adverse outcomes, while protective factors are characteristics that function as a buffer against negative impacts and are associated with positive adaptation and outcomes (Masten, 2014). Resilient students are often characterized by high self-confidence, perseverance, willingness and capacity to plan, and lower anxiety (Martin & Marsh, 2006, 2009; OECD, 2011), strong engagement in class and academic activities (Borman & Overman, 2004). Thorsen et. al., (2021) also found that resilient students display both more perseverance and consistency of interest over time. Hence, both cognitive and non-cognitive skills are important for academic resilience.

Research on the economics of human development also highlight the value of skill formation for success in adulthood, particularly for disadvantaged children. Societal investments in strengthening both cognitive and non-cognitive skills for disadvantaged children give significant economic returns both at individual and societal levels (e.g. Heckman, 2006). More recent studies have particularly highlighted non-cognitive skill formation as an crucial enabler. Non-cognitive skills are associated with promoting both economic and social mobility, economic productivity and well-being in adulthood (e.g. Kautz et al., 2015; Soto, 2019). A wealth of studies on the labour market aligns with this reasoning, identifying positive associations with both cognitive and non-cognitive skills and labour market outcomes. Johannesson (2017) found that cognitive abilities and non-cognitive skills (academic self-concept and perseverance) predicted the risk of being unemployed via school grades. Further, personality, i.e. extraversion and conscientiousness, was demonstrated to lead to higher earnings (Fletcher, 2013). Edin et. al., (2022) found that one standard-deviation increase in cognitive skills is associated with a salary increase of 6.6 percent and an increase in non-cognitive skills is associated with a 7,9 percent salary increase after controlling for educational attainment.

Studies on academic resilience and skill formation are scarcer. Nevertheless, some studies have found that protective factors identified during childhood and youth such as self-control and ability to plan are predictive of a more successful transition into adulthood (see Burt & Paysnick, 2012 for a review). In a qualitative study following up on four resilient students a decade later Morales (2008) found that the students continued to perform at high educational levels. The participants adapted the protective factors identified at the start of the study (i.e. self-confidence and internal locus of control) and used them to meet new challenges.

Employing a resilience perspective, the present study aims to investigate the difference in salary development among individuals who have been identified as being academically resilient versus those who are not. We also want to explore if the salary development can be attributed to educational attainment (educational history) and work status as changing conditions, and cognitive and non-cognitive skills, such as, cognitive ability, perseverance and academic self-concept as time invariant prerequisites.


Methodology, Methods, Research Instruments or Sources Used
Data were retrieved from the Evaluation through Follow-up database (ETF), a longitudinal project built on 10% randomly selected national representative samples of ten birth cohorts in Sweden (Härnqvist, 2000). The sampled students were followed up in grades 3, 6, and 9 of compulsory school (the Swedish school system consists of 9 years of compulsory education from age 7), and in upper secondary school (non-compulsory). Participants are about 9000 individuals born in 1972 from the ETF database. Of these, about 2000 individuals were identified as having low socioeconomic status (i.e., student’s parents only completed compulsory or vocational upper secondary education) and of these about 700 individuals were identified as being resilient (scoring above the country mean on the national standardized test).  
Academic self-concept (ASC) in grade 6 was measured by three items (e.g. how do you feel about doing maths, reading, writing) answers were given on a three-point scale ranging from difficult to easy. In upper secondary school ASC was measured using three items (e.g. do you experience any problems in math, reading, writing) answers were given on a 4-point scale from completely without problems to very big problems. Perseverance in grade 6 and upper secondary school was measured by four items (e.g. do you give up if you get a difficult task) answers were given on a dichotomous scale. Continuous variables were created for both constructs using the factor scores generated by a principal component analysis. Cognitive ability was measured in grade 6 using tests of inductive ability, spatial ability and verbal ability (antonyms).
Information on salary was retrieved from population statistics. Information about the salary for these individuals is available between the years 1988 and 2010.
Method of analysis
To investigate the salary growth of resilient students multiple group growth model with time varying and time invariant covariates will be used. Growth modelling allows for investigating the development of salary over time for both resilient and non-resilient students, conditioned on the development of individual’s educational attainment and work status, and on their cognitive ability, and personality traits. Academic self-concept and perseverance will be used as time-invariant covariates and educational attainment as time-varying covariates.

Conclusions, Expected Outcomes or Findings
Our preliminary results revealed that the resilient group has a slower rate of change of their salary level after the upper secondary education. This may be due to the fact that majority of the individuals in this group did not enter directly into the labour market but continue to higher education. However, we observed a steeper trajectory of salary development for these individuals after completing their higher education. The individuals in the non-resilience group held a higher starting salary level but a slower growth in their salary level over time.  Additionally, we found that both cognitive and non-cognitive factors, i.e. perseverance and academic self-concept explained the salary growth for academically resilient students. The explanation power was much lower or non-significant for their counterparts. We expect even clearer difference in the salary development between resilient and non-resilient individual groups when we control for the time varying covariates such as their education level and their work status, as well as time invariant covariates such as, IQ.  
References
Agasisti, T. et al. (2018). Academic resilience: What schools and countries do to help disadvantaged students succeed in PISA. OECD Education Working Papers, No. 167, OECD Publishing, Paris.
Borman, G. D., & Overman, L. T. (2004). Academic Resilience in Mathematics among Poor and Minority Students. The Elementary School Journal, 104(3), 177-195.
Burt K.B., & Paysnick A.A. (2012). Resilience in the transition to adulthood. Development and Psychopathology, 24(2), 493-505. doi:10.1017/S0954579412000119
Edin, Per-Anders, Peter Fredriksson, Martin Nybom, and Björn Öckert. (2022). The Rising Return to Noncognitive Skill. American Economic Journal: Applied Economics, 14 (2): https://www.aeaweb.org/articles?id=10.1257/app.20190199
Heckman, J. J. (2006). Skill Formation and the Economics of Investing in Disadvantaged Children. American Association for the Advancement of Science, 312(5782), 1900-1002. https://doi.org/10.1126/science.1128898
Härnqvist, K. (2000). Evaluation through follow-up. A longitudinal program for studying education and career development. In C.-G. Janson (ed.), Seven Swedish longitudinal studies in behavioural science (p. 76-114). Stockholm: Forskningsrådsnämnden. Retrieved from: http://hdl.handle.net/2077/2697078-100.
Johannesson, E. (2017). The Dynamic Development of Cognitive and Socioemotional Traits and Their Effects on School Grades and Risk of Unemployment. A Test of the Investment Theory. Doctoral Thesis, University of Gothenburg: Acta Universitatis Gothoburgensis.
Kiswarday, V. (2012). Empowering Resilience within the School Context. Methodological Horizons, 7(14). https://doi.org/10.32728/mo.07.1.2012.07
Kautz, T., Heckman, J.J., Diris, R., ter Weel, B., & Borghans, L. (2014). Fostering and Measuring Skills: Improving Cognitive and Non-Cognitive Skills to Promote Lifetime Success. National Bureau of Economic Research Working Paper Series, No. 20749. http://www.nber.org/papers/w20749
Martin, A. J., & Marsh, H. W. (2006). Academic resilience and its psychological and educational correlates: A construct validity approach. Psychology in the Schools, 43(3), 267-281. https://psycnet.apa.org/doi/10.1002/pits.20149
Masten, A. S. (2014). Ordinary magic: Resilience in development. New York, NY: Guilford Press. https://doi.org/10.1002/imhj.21625
Morales, E. E. (2008). Academic Resilience in Retrospect: Following Up a Decade Later. Journal of Hispanic Higher Education, 7(3), 228–248. https://doi.org/10.1177/1538192708317119
OECD. (2011). Against the odds: Disadvantaged students who succeed in school. Retrieved from http://dx.doi.org/10.1787/9789264090873-en
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417-453.
Soto, C. J. (2019). How Replicable Are Links Between Personality Traits and Consequential Life Outcomes? The Life Outcomes of Personality Replication Project. Psychological Science, 30(5), 711-727. https://doi.org/10.1177/0956797619831612
Thorsen, C., Yang Hansen, K. and Johansson, S. (2021), The mechanisms of interest and perseverance in predicting achievement among academically resilient and non-resilient students: Evidence from Swedish longitudinal data. Br J Educ Psychol, 91: 1481-1497 e12431. https://doi.org/10.1111/bjep.12431


09. Assessment, Evaluation, Testing and Measurement
Paper

Relationship between Student Financial Aid and Degree Completion on Time in Portugal

Maria Eugenia Ferrao1,2

1Universidade da Beira Interior; 2CEMAPRE

Presenting Author: Ferrao, Maria Eugenia

Research studies on degree completion on due time are seldom in the higher education literature. Regarding the European Higher Education Area (EHEA), no study has been published so far addressing the topic of degree completion on due time (Yes/No) based on nationwide representative data. By considering an entire entrant cohort of first-time, full-time undergraduate students who attended their three-year program at the same institution, and that simultaneously considers students’ background, entrance scores and choices, eligibility for social scholarship, institutional organization characteristics and the area of study, this study explores the role of social scholarships/financial aid in overcoming the effects of students’ socioeconomic disadvantages. A previous study (Ferrão, 2023) analyzed the relationship of the aforementioned students’ characteristics on degree completion grade point average (GPA). Findings suggest that receiving (or not receiving) a social scholarship has no influence on students’ GPA, confirming recent institution research findings obtained with the Universidade of Minho data (Ferrão et al., 2021) for the 1st year GPA rating. Nevertheless, Ferrão et al. (2020) found a statistically significant fixed effect of scholarship on students’ persistence, at the level of significance of 10%. In addition, institutional research conducted at the Instituto Politécnico de Leiria points out that providing solutions for financial limitations may contribute to decrease the risk of dropping out. In fact, Carreira and Lopes (2021) report that the “main motives for dropping out referred were ‘financial difficulties’ (27% of the students) and ‘work-school incompatibility’ (20%), while ‘low academic performance’ (11%), ‘health reasons’ (8%) or migration (2%), for example, were less mentioned, confirming the importance of financial assistance to reduce dropout risk (for traditional students) […]” (p. 1353). Given that the financing of higher education in Portugal has progressively shifted from the state’s responsibility to that of the students' and their families, as in many other countries (Marginson, 2018; Tight, 2020), this calls for a more accountable evaluation of private/public funding and demands more effective social justice policies (Pitman et al., 2019). Thus, this study investigates the effect of receiving social scholarships/financial aid on degree completion on due time. Its specific objective is to estimate the fixed effect of receiving or not receiving a social scholarship on the probability of degree completion on due time. The study contributes to the themes of students’ success (degree completion), equity (financial aid), system evaluation and resource allocation. Since Portugal is one of the European countries where the costs of higher education are supported primarily by taxpayers, this topic of research matters not only for public policy regarding the increase of equity, but also for the efficiency of public resources allocation.


Methodology, Methods, Research Instruments or Sources Used
The population under study consists of students who entered undergraduate programs of 180 European Credit Transfer System (ECTS) by the national competition and who obtained (or not) their diploma three years later. The survey “Register of students enrolled in and graduated from higher education” (RAIDES) was used. The administrative RAIDES data (DGEEC - Direção-Geral de Estatísticas da Educação e Ciência, 2020), primarily collected for official statistics, offer great potential for secondary analyses such as quantitative based scientific research. The survey RAIDES is annually carried out in Portugal within the scope of the National Statistical System which is mandatory. Data were collected by higher education institutions and exported in XML format to the DGEEC twice a year (January and April; December 31 and March 31 as time reference, respectively), throughout the “Plataforma de Recolha de Informação do Ensino Superior” [Platform of Data Collection in Higher Education] (PRIES). Details on data collection, data processing and information about the agreement for data privacy protection may be found in previous studies such as Ferrão (2023) or Ferrão et al. (2022). For the purpose of this study, students’ data enrolled in the academic year 2013–14 and graduates’ data in the academic year 2016–17 were paired. Records of students who were not enrolled in their 1st year for the first time or whose access to higher education was different from the national competition are not considered.
Random coefficient models are well grounded in the literature on higher education and success measurement. In this study, multilevel logistic models are applied considering two hierarchical structures at two levels with dependent variable representing degree completion on due time (DC, Yes/No). Preliminary results were obtained with the statistical computing software MLwiN  (Rasbash et al., 2017), and the estimation procedure was the penalized quasi-likelihood of second order, PQL2 (Goldstein & Rasbash, 1996).

Conclusions, Expected Outcomes or Findings
Preliminary results show that, at the level of 5%, there is a statistically significant fixed effect of receiving a social scholarship on degree completion on time. The magnitude of the fixed effect depends on the set of controlling variables in the model. The odds ratio varies from 1.2 to 1.5. Other independent or controlling variables included in the linear predictor of the model are as follows: Entrance score, 1st choice programme-institution, gender, age at enrollment, parents’ education, area of study, non-local institution attended, type of institution, interaction between age and entrance score.
References
Carreira, P., & Lopes, A. S. (2021). Drivers of academic pathways in higher education: traditional vs. non-traditional students. Studies in Higher Education, 46(7), 1340–1355. https://doi.org/10.1080/03075079.2019.1675621
DGEEC - Direção-Geral de Estatísticas da Educação e Ciência. (2020). Documento técnico da plataforma de recolha de informaçao do Ensino Superior– RAIDES. DGEEC.
Ferrão, M. E. (2023). Differential effect of university entrance scores on graduates’ performance: the case of degree completion on time in Portugal. Assessment & Evaluation in Higher Education, 48(1), 95–106. https://doi.org/10.1080/02602938.2022.2052799
Ferrão, M. E., Almeida, L. S., & Ferreira, J. A. (2021). Higher Education equity in Portugal: On the relationship between student performance and student financial aid. World Education Research Association (WERA) 2020+1 Focal Meeting.
Ferrão, M. E., Prata, P., & Fazendeiro, P. (2022). Utility-driven assessment of anonymized data via clustering. Scientific Data, 9(456), 1–11. https://doi.org/10.1038/s41597-022-01561-6
Goldstein, H., & Rasbash, J. (1996). Improved approximations for multilevel models with binary responses. Journal of the Royal Statistical Society. Series A (Statistics in Society), 159(3), 505–513. https://doi.org/10.2307/2983328
Marginson, S. (2018). Global trends in higher education financing: The United Kingdom. International Journal of Educational Development, 58, 26–36. https://doi.org/10.1016/j.ijedudev.2017.03.008
Pitman, T., Roberts, L., Bennett, D., & Richardson, S. (2019). An Australian study of graduate outcomes for disadvantaged students. Journal of Further and Higher Education, 43(1), 45–57. https://doi.org/10.1080/0309877X.2017.1349895
Rasbash, J., Browne, W., Healy, M., Cameron, B., & Charlton, C. (2017). MLwiN 3.05. Centre for Multilevel Modelling, University of Bristol.
Tight, M. (2020). Student retention and engagement in higher education. Journal of Further and Higher Education, 44(5), 689–704. https://doi.org/10.1080/0309877X.2019.1576860


 
Contact and Legal Notice · Contact Address:
Privacy Statement · Conference: ECER 2023
Conference Software: ConfTool Pro 2.6.149+TC
© 2001–2024 by Dr. H. Weinreich, Hamburg, Germany