09. Assessment, Evaluation, Testing and Measurement
Paper
Developing a Standardized Eligibility Test for Tertiary Education in Sweden
Gudrun Erickson1, Jan-Eric Gustafsson1, Frank Bach2, Jörgen Tholin1
1University of Gothenburg, Sweden; Dept of Education and Special Education; 2University of Gothenburg, Sweden; Dept of Pedagogical, Curricular and Professional Studies
Presenting Author: Erickson, Gudrun
The paper focuses on the development of a basic eligibility test for admission into tertiary education in Sweden, a test aimed to provide opportunities for a wider group of applicants than today, thereby increasing inclusion and diversity in higher education. This is an emerging type of test, reflecting increasing national and international needs for documented competences required for access to higher education (e.g., the HiSET exam; ETS 2021).
To be allowed to apply for tertiary education in Sweden, basic entry requirements must be met. This corresponds to a leaving certificate from upper secondary level, comprising a combination of core subjects, in particular Swedish/Swedish as a second language, English and mathematics, but also a number of generic/key competences embedded in the national curricula, as well as in corresponding documents, e.g., in the European context (European Commission, 2018). Examples of such generic competences are problem solving, critical thinking and inferencing.
Traditionally, individual students who lack the formal requirements for tertiary education may have their competences evaluated by the university to which they apply. However, this is normally done in relation to a single course and is not generalizable, neither to other courses nor to other universities. These local evaluations are based on a central ordinance (UHRFS 2021:4) that defines the basic components required but not the extent or methods of the validation procedures. Hence, certain variability is self-evident.
To create a standardized alternative to local validation, a suggestion for a test of basic eligibility was made in a governmental investigation concerning entrance into higher education (SOU 2017:20). This resulted in a political decision to conduct a three-year trial round for a basic eligibility test (SFS 2018:1510). It was made clear that the test was intended for people from 24 years of age lacking formal, basic qualifications. In addition, it was decided that the result of the test should not be used for competition but for eligibility purposes only.
In 2020, the Swedish Council for Higher Education (UHR) formed a group of experts within different educational fields to discuss the development of a basic eligibility test. Three members of this group were given the task to develop a tentative framework for the activity. This work was conducted in close collaboration with the larger group, which lead to gradual revisions of the text. The final document was officially approved in January 2021 (UHR, dnr. 00012-2020). It consisted of some 40 pages and comprised subsections focusing on aims and background, including references to different international studies, e.g., PISA (OECD, 2018), test components, quality measures and control, as well as guidelines for use and future changes and developments.
The assignment to conduct the trial was given by the National Council/UHR to [university], which has a long tradition of large-scale educational assessment, regarding test development as well as analyses of results at national and international levels (Author 2 et al.; Author 2 et al.; Author 1 et al., Author 3 et al.) A three-year contract was signed in March 2021, after which the operative developmental phase started. The first, large-scale trial test was administered in October 2022, comprising a wide range of tasks: dichotomous, selected response items within different domains, English listening comprehension, integrated tasks also including graphs, tables etc., and two tasks targeting written production in Swedish and English.
The aim of the current presentation is to briefly
- describe and discuss the rationale and methodology of the test development process,
- present and reflect on some results of the process, and to
- look forward into, and discuss, possible future uses of the test being developed.
Methodology, Methods, Research Instruments or Sources UsedInitially, a decision was made to form a project steering group of four people, the intention being twofold: to broaden competences and to decrease vulnerability. It was also essential to establish the basic character of the envisaged product, namely a unified test, where only one standard was to be set, namely the pass level. In addition, the cut-off point was to be determined based on a holistic assessment, not on partial requirements for individual components of the test. It was also emphasized that the generic competences were seen as overarching concepts, however with working groups linked to different competence domains. These groups were directed towards Swedish L1/ L2, English, mathematics, natural and social sciences, and methodology. Closely linked to this structure were also so-called go-betweens (gb:s), i.e., people with competence in two or more of the domains.
There were basic requirements for each working group, namely three types of, often overlapping, competences and experiences: subject matter knowledge including pedagogical/teaching experience, experience of large-scale testing, and research competence. Furthermore, a specification matrix was developed for the different groups, on the basis of which they documented gradually information for the material developed, e.g., intended construct/competence, amount and type of input, format, estimated time, estimated/intended difficulty, etc.
The working groups followed internal plans when developing items and tasks. The members of the steering group, several of them with a domain-related background, kept in regular touch. Roughly once a month there were meetings for the whole group, approx. 25 people, in which different issues were discussed.
Piloting of material, item-based as well as productive writing tasks for Swedish and English, was conducted in a four-step, anonymous process: (1) mini trials per domain in smaller groups; (2) trials including material from two domains; (3) trials of a mix of tasks from different domains; and (4) pre-testing rounds aiming for a large number of participants, representing as wide a range as possible of individuals. In the latter case, tasks were always accompanied by a set of background questions and anchor items. In addition, extensive collection of test-taker feedback was made, using Likert-scales and open comments.
Standard setting is a self-evident aspect of test development, also in this trial project. This is always a challenge, not least for tests comprising both dichotomous item data and ratings of productive tasks. A short account of this will be given, including reactions to the first large-scale trial round.
Conclusions, Expected Outcomes or FindingsThe test development process focused upon in the current proposal is indeed work in progress, with continuous documentation undertaken. Hence, the conclusions drawn at this stage are tentative, and should partly be regarded as observations, which, however, will be combined into final conclusions at the end of the trial. Furthermore, they emanate from distinctly different data. One such observation is related to pre-testing and clearly shows the difficulty of finding the intended, large groups of test takers. Another aspect that needs constant attention is the nature of the test as unified, not consisting of independent parts. This is even more emphasized by the fact that two of the subtests, in particular, have a general character, including and integrating different competences in a way that makes them closely related to the overarching, generic purposes. Also, the rating of productive tasks is a multi-facetted activity that needs lots of consideration and re-consideration, especially when combined with other types of data and analyses. In addition, it can be concluded that anchor items are crucial for standard-setting purposes in making comparisons at different levels possible. Finally, it is obvious that test-taker feedback adds considerably to the quality of the process, as such, by giving the necessary perspective of the users, but also as a complement to analyses of performance.
We sometimes characterize the assignment to carry out the trial of this test as a task entailing building a boat while sailing it. This is obviously far from uncomplicated but still works quite well, much thanks to the distinct and positive collaboration that takes place in the process, between different actors: the national council and the university department, disciplines and groups within the project, and with potential users of what we hope will be a future national entrance test for tertiary education.
ReferencesAuthor 2 et al. (year 1)
Author 1 et al. (year 3)
Author 2 et al. (year 2)
Author 3 et al. (year 4)
Educational Testing Service (2021). 2020 Annual Statistical Report. https://hiset.org/s/pdf/HiSET_2021_Annual_Statistical_Report.pdf
European Commission (2018). Council Recommendation of 22 May 2018 on key competences for lifelong learning. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.C_.2018.189.01.0001.01.ENG&toc=OJ:C:2018:189
OECD (2018). PISA 2018 Assessment and Analytical Framework. https://doi.org/10.1787/b25efab8-en
Regeringskansliet (2017). SOU 2017:20. Tillträde för nybörjare – ett öppnare och enklare system för tillträde till högskoleutbildning [Admission for beginners – a more open and accessible system for entrance into tertiary education]. https://www.regeringen.se/rattsliga-dokument/statens-offentliga-utredningar/2017/03/sou-201720/
Sveriges Riksdag (2018). Förordning om försöksverksamhet med behörighetsprov för tillträde till högskoleutbildning [Ordinance on trial activities with eligibility tests for admission to higher education]. (SFS 2018:1510). https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/forordning-20181510-om-forsoksverksamhet-med_sfs-2018-1510
Universitets- och högskolerådets författningssamling (UHRFS 2021:4). Föreskrifter om ändring i Universitets- och högskolerådets föreskrifter (UHRFS 2013:1) om grundläggande behörighet och urval [Regulations regarding change in UHR’s regulations on basic eligibility and selection] . https://www.uhr.se/globalassets/_uhr.se/publikationer/lagar-och-regler2/uhrfs/2021/uhrfs-2021-4.pdf
Universitets- och Högskolerådet (2021). Ramverk för det nationella behörighetsprovet [Framework for the national eligibility test]. (UHR, dnr. 00012-2020)
09. Assessment, Evaluation, Testing and Measurement
Paper
A Controversially Received Reform: The 2018 Reform of Finnish Higher Education Student Admission
Sirkku Kupiainen, Risto Hotulainen, Irene Rämä, Laura Heiskala
University of Helsinki, Finland
Presenting Author: Kupiainen, Sirkku
Upper secondary exit exams are common in education systems worldwide, marking the passing of upper secondary education and acting as gatekeeper for higher education (Noah & Eckstein, 1992). This double role of the exam is especially salient in countries where the share of the age-cohort passing academic upper secondary education exceeds that of students accepted to higher education. If the exam plays a prominent role in admission, the high stakes of the exam are especially acute. This sets specific requirements to the comparability of the examination results across the exams of the different subjects if the examination is so constructed, and across years if delayed entering to higher education is common (cf. Beguin, 2000). Both clauses are relevant in Finland, the focus of the present study.
While an upper secondary exit exam is an integral part of the education systems of most European countries, both stratified and comprehensive, the form of the exam and the share of students sitting for it vary widely. Despite these differences, the academic track of upper secondary education usually comprises some form of an exit examination or final grades taken into account in tertiary student selection as a sole factor or in addition to an entrance examination, unless access is open to all or the selection happens later based on students’ study performance. In Finland, both academic and vocational upper secondary education provide a qualification for higher education (Orr et al., 2017), meaning that even if a 2018 reform decreed half of students to be accepted into higher education based solely on their matriculation examination results, Finland cannot fully abandon the entrance examinations, which earlier regarded all applicants.
Orr and colleagues (2017) classify the European Union member states according to upper secondary tracking and higher education institutions’ autonomy on student intake. In Finland, the state decides in collaboration with universities the number of students admitted to different programs and the outlines for admission policy, while universities decide the details for the latter. In 2018, a student admission reform in Finland mandated half of students to be accepted on matriculation examination results with universities deciding in collaboration how credit for the different subject-specific exams would be awarded. The main goal of the reform was to speed Finnish students’ slow transit from secondary to tertiary education, a problem that also the OECD has pointed out as one of the weak points of the Finnish education system. Due to a backlog of older matriculates vying for a place, two thirds of new matriculates are left yearly without a place in higher education. The reform was backed by research on the drawbacks of the earlier entrance examination-based student selection (Sarvimäki & Pekkarinen, 2016) and tied the credit to the number of courses covered by each subject-specific exam. Yet, the reform has raised vocal criticism. The second chance offered by an entrance exam has been dear to many, but the focus of criticism has been that due to its biggest course-load, advanced mathematics brings most credit even in fields where proficiency in it might appear of less value. An earlier reform of medical faculties’ student admission in 2014 increased the weight given to advanced mathematics, with the positive consequence of increasing the share of girls sitting for the exam. We expect the present reform to have a similar impact despite the current critique.
In this presentation, we explore the impact of the reform on upper secondary schools – on students’ course choices and attainment, on their plans for the exams to include in their matriculation examination, on student wellbeing and possible burnout, and on students’ and teachers’ views on the reform.
Methodology, Methods, Research Instruments or Sources UsedThe data is drawn from an ongoing (spring 2022 – spring 2023) study on the effect of the 2018 student admission reform on upper secondary schools and students, compiled to inform a re-writing of the respective admission criteria in 2023 due to the presumed negative impact of the reform on upper secondary students’ width and depth of studies (choice of subjects in the relatively free syllabus), and on their wellbeing. The data comprise questionnaires for students (n = 8,000), teachers, principals and guidance counsellors in sixteen upper secondary schools, register data on the sampled students‘ course choices and attainment, and additional focus-group interviews of students and teachers in five upper secondary schools. Furthermore, the data comprises national matriculation examination data of 2016–2022 to investigate possible changes in students’ exam choices across the implementation of reform.
Reflecting the cross-sectional survey data and the largely descriptive research questions, the results for the quantitative data will be mainly presented at the descriptive level, using ANOVA for variable-based profile analysis (e.g., math-oriented vs. humanistic-subjects- oriented students, high vs. low achievers, etc.) and group-level (e.g., gender, home background) comparisons. Due to the wide variability of students’ study paths within the relatively free upper secondary syllabus (of the 75 courses required for matriculation, only 45/52 are mandatory for basic/advanced mathematics) and students’ free choice for the order in which they study the different subjects (only advanced mathematics requires having a course in all of the five periods across the year), multi-level analysis is expected to be a valid option for only some specific questions. The interview data will be used at this point to just provide ‘real-life’ examples of how the students and teachers see and talk about the issues brought up by the quantitative data used as the bases for the focus-group discussions.
Conclusions, Expected Outcomes or FindingsThe preliminary data of 4,000 students suggest that, on average, students prefer the earlier practice of entrance examinations, still in use for half of new students. There were statistically significant but weak (p<.001, ƞ2 < .01) differences in this, related to students’ gender and choice of advanced vs. basic mathematics, with students of advance mathematics who perform, on average, better in all exams (Kupiainen et al., 2018) more in favor of the matriculation examination-based student admission. They were also more confident in being accepted to university on the basis of their examination results. Yet, most students predicted that they will prepare for the entrance examination once the matriculation exams are over (a necessity for many as the results from the matriculation examination-based selection come only just before the entrance examinations), a problem contrary to the goals of the reform and brought up also by the study on the impact of the reform from the universities’ point of view (Karhunen et al., 2022). As expected, the larger credit awarded for advanced mathematics was criticized especially by students of basic mathematics (5.46 vs. 4.43 on a seven-point Likert scale, p<.001, ƞ2=.068). While public discussion has blamed the reform for leading students to choose courses based on the credit awarded for the different exams in the admission process, according to the survey, students still see personal interest in the subject as a clearly stronger incentive for their choice of the exams they plan to sit for (mean 5.67 vs. 4.58 on a seven-point Likert scale). Even if the majority of students (58.6%) expected to be admitted to university based on their matriculation examination results, a much greater majority (82.5%) was ready to use the possibility allowed by the reform to take the exams anew if they were not.
ReferencesBéguin, A. A. (2000). Robustness of equating high-stakes tests. Thesis, University of Twente, Enschede, Netherlands.
Karhunen, H., Pekkarinen, T., Suhonen, T. & Virkola, T (2022). Opiskelijavalintauudistuksen seurantatutkimuksen loppuraportti (The final report of the follow-up study of the student selection reform). VATT Muistiot 67.
Kupiainen, S., Marjanen, J. & Ouakrim-Soivio, N. (2018) Ylioppilas valintojen pyörteessä (Students at the whirlwind of choices). Suomen ainedidaktisen tutkimusseuran julkaisuja. Ainedidaktisia tutkimuksia 14. https://helda.helsinki.fi/handle/10138/231687
Noah, H. & Eckstein, A. (1992). The two faces of examinations: A comparative and international perspective. In Noah and Eckstein (eds.), Examinations: Comparative and International Studies. Pergamon Press: pp.147-170.
Orr, D., Usher, D., Haj, C., Atherton, G., & Geanta, I. (2017). Study on the impact of admission systems on higher education outcomes. Executive summary. European Commission, Education and Training. Publication Office for the European Union.
Sarvimäki, M., & Pekkarinen, T. (2016). Parempi tapa valita korkeakouluopiskelijat (A better way to choose higher education students). VATT Policy Brief.
09. Assessment, Evaluation, Testing and Measurement
Paper
Impact on students' retention rates of a Progression Support System Program (SiAP) in a Post-Secondary Technical Education in Chile
Cristian Cardenas, Jose Cancino, Carolina Barrientos, Fernando Alvarez
INACAP, Chile
Presenting Author: Cardenas, Cristian;
Cancino, Jose
The International Standard Classification of Education (ISCED-11) has labeled post-secondary technical education as a short-cycle tertiary education program (5B). Chile, following OECD standards, has defined Tertiary Technical Education (from now on TTE) as oriented to give the necessary capacities and knowledge to perform as a professional in different areas of the labor market (Ley 21.091, 2018). Additionally, it has emphasized the opportunity to enhance successful trajectories, especially for the population that has been historically excluded from higher education and skilled jobs. In this sense, access to TTE is seen as an instrument of social mobility that seeks to reduce inequality (Brunner et al., 2022).
In turn, retention and dropouts, particularly for low-income students, have been a policy concern and a challenge to the technical educational system (Hällsten, 2017; Sarra et al., 2018; Brunner et al., 2022). The adverse effect of dropping out is dramatic and affects students and families in many ways, including greater marginalization and future lower labor market outcomes (Sosu & Pheunpha, 2019; Voelkle & Sander, 2008; O'Neill et al., 2011). Therefore, retention and dropout affect the goal of inclusion and equity that Chilean policymakers have tried to insert at the core of the TTE Chilean regulation since 2010 (Brunner et al., 2022).
As such, TTE has a disproportionate share of low-income students (Mountjoy, 2022; Sotomayor, 2018). In Chile, these institutions have more extensive participation of students from quintiles 1-3 of income. Namely, close to 50% of the enrollment of these institutions comes from the poorest 60% of the Chilean population (SIES, 2022). In this regard, a 2022 study of the Higher Education Information Service of Chile (SIES) indicates that the retention rate for first-year students of tertiary education is higher for universities (85%) compared to TTE institutions (70%) (SIES, 2022). The statistics are consistent with the study of the determinants of retention and dropout in TTE. When examining which factors influence the probability of student retention, evidence from across the globe indicates four main groups of variables: i) the sociodemographic background of the family, ii) the student's previous academic results, iii) accessibility or financing restrictions, and iv) institutional factors (Behr et al., 2020; Li & Carroll, 2017; Millea et al., 2018; among others).
The Professional Institute INACAP is one of the largest TTEs in Chile, with 15% (N=76781) of the total enrollment of the Chilean technical college system (SiES, 2022). Since 2014, INACAP has developed new and different mechanisms to support a successful trajectory of students through a program called the Progression Support System (SiAP). The program has at its core an inclusion and equity framework that has allowed a comprehensive set of initiatives articulated through a tutor that provides academic, psychosocial, and extracurricular support to students (particularly at risk) to help them successfully navigate their career pathways. The program's ultimate impact indicator is to increase first-year students' retention rates and avoid dropouts.
However, questions about the program's effectiveness have been raised, especially during the covid-19 pandemic. In this regard, we determined to measure the effect of the SiAP-program at INACAP on the first-year student retention rate for the cohorts 2017-2021. Additionally, the study sought to describe changes in the program policies and implementation during the pandemic (2020-2021) that might affect results. Finally, given the large enrollment of INACAP, findings will give insights into the institution and the Chilean tertiary education system about to what extent tutoring programming, as the core of a student system of academic progression support, facilitates retention in the educational system for vulnerable students.
Methodology, Methods, Research Instruments or Sources UsedThe INACAP SiAP-program is composed of different initiatives for all first-year students. The objective is to facilitate active and self-managed insertion into higher education. In this way, all students are assigned to an INACAP-SiAP tutor whose role is to support the development of academic skills and self-management of learning through academic monitoring and psychosocial accompaniment to identify support needs and activate internal and external networks promptly. Tutors must follow an order of priority for contact and accompaniment of students based on a student-risk predictor model.
The SiAP-program has reached an average coverage of 70% (n=37.523 students) and above of the total enrollment of first-year students for cohorts 2017-2021. As a first step to evaluate its effectiveness, we decided to develop its theory of change with the SiAP team (Weiss & Connell, 1995) to establish the causal relationship between the program's multiple actions and its expected impact (retention rate of first-year students).
From it, institutional data were gathered to analyze previous analyses' top results and limitations (2014-2016). Thus, selection biases generated by the multiple mechanisms through which students can be referred to each program’s components were identified. On one side, students who decide to participate in the different components of the SiAP voluntarily generate a self-selection bias. Besides, there is an endogeneity behind being referred to the program’s components, either due to having a low score in diagnostic evaluations (where academic performance would affect participation in the program) or by the tutor’s decision (where participation would be correlated with the error term).
From this, the method used was the quasi-experimental propensity score matching (PSM) (Caliendo & Kopeinig, 2008), representing the best impact evaluation tool to minimize biases and thus isolate the effect of treatment on the probability of retention.
The Kernel algorithm was used to take advantage of the information of all the observations located within the standard support to build a more precise counterfactual, applying a weighted average where greater weight was given to those observations that have a score closest to the treatment group and vice versa (Caliendo & Kopeinig, 2008). Additionally, policy and implementation changes in the program were tracked and analyzed to understand changes in the program better.
Conclusions, Expected Outcomes or FindingsThe effects of the INACAP-SiAP-program on the retention rate of first-year students are positive and statistically significant. Participation in the program increases the probability of retention for the 2017 cohort by an average of 6.3 percentage points (pp) and 10 pp for the 2018 and 2019 cohorts.
For the cohorts of 2020 and 2021, the magnitude of the effect increases dramatically to 40 pp and 58 pp, respectively. However, these 2020-21 estimates should be carefully analyzed due to external validity problems because of the pandemic, changes in the SiAP guidelines program during the Covid-19 pandemic, and the lower sample number of the control group due to the trend towards universal treatment.
Due to the above, heterogeneous analyzes in subgroups of interest was limited to the 2017-2019 cohorts. For these cohorts, the program's impact on retention rates is substantially more significant in students enrolled in evening programs hours, particularly for the 2018 and 2019 cohorts (6.8 vs. 16.6%; 7.5 vs. 17.6%. respectively). The impact is 135% higher than that obtained in daytime students for the 2019 cohort. This magnitude difference is similar to that reported in the 2018 cohort. Accordingly, the impact on working students (vs. non-working students) is higher by approximately 50% in the magnitude of the effect between the two groups for cohorts 2017-2019, which makes this a consistent result over time.
The study highlights the importance of student support systems (like the INACAP-SiAP) to help students stay on their career pathways. This effort is aligned with the equity and inclusion framework that educational policy in Chile has tried to enhance for students that see tertiary short-cycle education as an opportunity for professional jobs that allows them better opportunities in the labor market. Nonetheless, it is imperative to study the extent to which the program helps students graduate (on-time).
ReferencesBehr, A., Giese, M., Teguim K. & Theune, K. (2020). Dropping out from Higher Education in Germany an Empirical Evaluation of Determinants for Bachelor Students. Open Education Studies, 2(1), 126-148.
Brunner, J., Labrana, J., Alvarez, J. (2022). Educación superior técnico profesional en Chile: perspectivas comparadas. Santiago de Chile: Ediciones Universidad Diego Portales. https://vertebralchile.cl/wp-content/uploads/2022/07/Educacion-superior-tecnico-profesional-en-perspectiva-comparada.pdf
Caliendo, M. & Kopeinig, S. (2008). Some Practical Guidance for the Implementation of Propensity Score Matching. Journal of Economic Surveys, 22(1), 31-72. https://doi.org/10.1111/j.1467-6419.2007.00527.x
Faas, C., Benson, M. J., Kaestle, E. C., and Savla, J. (2018). Socioeconomic success and mental health profiles of young adults who drop out of college. J. Youth Stud. 21, 669–686. doi: 10.1080/13676261.2017.1406598
LEY 21091, 2018. Sobre educacion superior. 11 de mayo 2018 (Chile)
Li, W., & Carroll, D. (2017). Factors Influencing University Student Satisfaction, Dropout and Academic Performance. National Centre for Student Equity in Higher Education.
Millea, M., Wills, R., Elder, A. & Molina, D. (2018). What Matters in College Student Success? Determinants of College Retention and Graduation Rates. Education, 138(4), 309-322.
Mountjoy, J. (2022). Community Colleges and Upward Mobility (Working Paper No 29254). National bureau of economic research. https://www.nber.org/papers/w29254
O'Neill, L. D., Wallstedt, B., Eika, B., and Hartvigsen, J. (2011). Factors associated with dropout in medical education: a literature review. Med. Educ. 45, 440–454.
Ortiz, E. A., and Dehon, C. (2013). Roads to success in the Belgian French community's higher education system: predictors of dropout Bruxelles. Res. High. Educ. 54, 693–723.
Sarra, A., Fontanella, L., and Di Zio, S. (2018). Identifying students at risk of academic failure within the educational data mining framework. Soc. Ind. Res. 1–20.
Servicio de Información de Educación Superior (SIES) (2022). Ministerio de Educación. Matricula en Educación Superior en Chile. https://www.mifuturo.cl/wp-content/uploads/2022/10/Matricula_Educacion_Superior_2022_SIES_.pdf
Sosu EM and Pheunpha P (2019) Trajectory of University Dropout: Investigating the Cumulative Effect of Academic Vulnerability and Proximity to Family Support. Front. Educ.
Sotomayor, C.; Valenzuela, J. P. (2018). Rentabilidad de la educación superior técnica entregada por los Centros de Formación Técnica Estudios de Políticas Públicas (pp., 120-133.).
Voelkle, M. C., and Sander, N. (2008). A structural equation approach to discrete-time survival analysis. J. Individ. Dif. 29, 134–147.
Weiss, C.H. and Connell, J.P. (1995) Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families. In: New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, The Aspen Institute, 65-92.
|
