Conference Agenda

The School of Quantitative Research Methods in Education (QRM, https://www.gu.se/en/qrm) is a research school and an emerging network funded by the Swedish Research Council. QRM aims to contribute to rebuilding of the competence of quantitative methodological knowledge in educational research in Sweden and elsewhere. The proposed workshop contributes to this aim by training basic competence in using the statistical environment, R, which is a widely used, open source project. However, using R is still relatively new in educational sciences.

R is an increasingly used statistical environment maintained by an international team of developers. The proposed workshop is intended for analysts and researchers who do not have yet the knowledge and experience using R. Working knowledge of basic statistics is required.

The workshop will begin with a brief overview of the R environment based on Venables, et al. (2014). Then, based on Grolemund (2014), the workshop leader will assist the participants to download R as well as RStudio, a software application that makes R easier to use.

Afterward, basic data management tasks will be demonstrated and practiced, including importing and exporting data. Then basic statistical analyses will be demonstrated and practiced using several R packages, including plotting the data. Exporting outputs will be also practiced. Finally, based on the workshop participants’ interest and pace of work, advanced statistical analyses will be demonstrated and practiced.

The workshop will end with informing the participants about resources to further develop their knowledge in R and commonly used R packages for working with international large-scale assessment (ILSA) data. Participants should bring a laptop. The workshop leaders will hand out the presentation slides beforehand and the R scripts in case of inquiry afterward.

Draft agenda:

• Introduction
• Downloading and installing R and RStudio
• Data management
• Basic statistical analyses
• Creating plots
• Advanced statistical analyses (Optional)
• Summary and further avenues and resourses

The paper focuses on the development of a basic eligibility test for admission into tertiary education in Sweden, a test aimed to provide opportunities for a wider group of applicants than today, thereby increasing inclusion and diversity in higher education. This is an emerging type of test, reflecting increasing national and international needs for documented competences required for access to higher education (e.g., the HiSET exam; ETS 2021).

To be allowed to apply for tertiary education in Sweden, basic entry requirements must be met. This corresponds to a leaving certificate from upper secondary level, comprising a combination of core subjects, in particular Swedish/Swedish as a second language, English and mathematics, but also a number of generic/key competences embedded in the national curricula, as well as in corresponding documents, e.g., in the European context (European Commission, 2018). Examples of such generic competences are problem solving, critical thinking and inferencing.

Traditionally, individual students who lack the formal requirements for tertiary education may have their competences evaluated by the university to which they apply. However, this is normally done in relation to a single course and is not generalizable, neither to other courses nor to other universities. These local evaluations are based on a central ordinance (UHRFS 2021:4) that defines the basic components required but not the extent or methods of the validation procedures. Hence, certain variability is self-evident.

To create a standardized alternative to local validation, a suggestion for a test of basic eligibility was made in a governmental investigation concerning entrance into higher education (SOU 2017:20). This resulted in a political decision to conduct a three-year trial round for a basic eligibility test (SFS 2018:1510). It was made clear that the test was intended for people from 24 years of age lacking formal, basic qualifications. In addition, it was decided that the result of the test should not be used for competition but for eligibility purposes only.

In 2020, the Swedish Council for Higher Education (UHR) formed a group of experts within different educational fields to discuss the development of a basic eligibility test. Three members of this group were given the task to develop a tentative framework for the activity. This work was conducted in close collaboration with the larger group, which lead to gradual revisions of the text. The final document was officially approved in January 2021 (UHR, dnr. 00012-2020). It consisted of some 40 pages and comprised subsections focusing on aims and background, including references to different international studies, e.g., PISA (OECD, 2018), test components, quality measures and control, as well as guidelines for use and future changes and developments.

The assignment to conduct the trial was given by the National Council/UHR to [university], which has a long tradition of large-scale educational assessment, regarding test development as well as analyses of results at national and international levels (Author 2 et al.; Author 2 et al.; Author 1 et al., Author 3 et al.) A three-year contract was signed in March 2021, after which the operative developmental phase started. The first, large-scale trial test was administered in October 2022, comprising a wide range of tasks: dichotomous, selected response items within different domains, English listening comprehension, integrated tasks also including graphs, tables etc., and two tasks targeting written production in Swedish and English.

The aim of the current presentation is to briefly

• describe and discuss the rationale and methodology of the test development process,
• present and reflect on some results of the process, and to
• look forward into, and discuss, possible future uses of the test being developed.

Upper secondary exit exams are common in education systems worldwide, marking the passing of upper secondary education and acting as gatekeeper for higher education (Noah & Eckstein, 1992). This double role of the exam is especially salient in countries where the share of the age-cohort passing academic upper secondary education exceeds that of students accepted to higher education. If the exam plays a prominent role in admission, the high stakes of the exam are especially acute. This sets specific requirements to the comparability of the examination results across the exams of the different subjects if the examination is so constructed, and across years if delayed entering to higher education is common (cf. Beguin, 2000). Both clauses are relevant in Finland, the focus of the present study.

While an upper secondary exit exam is an integral part of the education systems of most European countries, both stratified and comprehensive, the form of the exam and the share of students sitting for it vary widely. Despite these differences, the academic track of upper secondary education usually comprises some form of an exit examination or final grades taken into account in tertiary student selection as a sole factor or in addition to an entrance examination, unless access is open to all or the selection happens later based on students’ study performance. In Finland, both academic and vocational upper secondary education provide a qualification for higher education (Orr et al., 2017), meaning that even if a 2018 reform decreed half of students to be accepted into higher education based solely on their matriculation examination results, Finland cannot fully abandon the entrance examinations, which earlier regarded all applicants.

Orr and colleagues (2017) classify the European Union member states according to upper secondary tracking and higher education institutions’ autonomy on student intake. In Finland, the state decides in collaboration with universities the number of students admitted to different programs and the outlines for admission policy, while universities decide the details for the latter. In 2018, a student admission reform in Finland mandated half of students to be accepted on matriculation examination results with universities deciding in collaboration how credit for the different subject-specific exams would be awarded. The main goal of the reform was to speed Finnish students’ slow transit from secondary to tertiary education, a problem that also the OECD has pointed out as one of the weak points of the Finnish education system. Due to a backlog of older matriculates vying for a place, two thirds of new matriculates are left yearly without a place in higher education. The reform was backed by research on the drawbacks of the earlier entrance examination-based student selection (Sarvimäki & Pekkarinen, 2016) and tied the credit to the number of courses covered by each subject-specific exam. Yet, the reform has raised vocal criticism. The second chance offered by an entrance exam has been dear to many, but the focus of criticism has been that due to its biggest course-load, advanced mathematics brings most credit even in fields where proficiency in it might appear of less value. An earlier reform of medical faculties’ student admission in 2014 increased the weight given to advanced mathematics, with the positive consequence of increasing the share of girls sitting for the exam. We expect the present reform to have a similar impact despite the current critique.

In this presentation, we explore the impact of the reform on upper secondary schools – on students’ course choices and attainment, on their plans for the exams to include in their matriculation examination, on student wellbeing and possible burnout, and on students’ and teachers’ views on the reform.

The International Standard Classification of Education (ISCED-11) has labeled post-secondary technical education as a short-cycle tertiary education program (5B). Chile, following OECD standards, has defined Tertiary Technical Education (from now on TTE) as oriented to give the necessary capacities and knowledge to perform as a professional in different areas of the labor market (Ley 21.091, 2018). Additionally, it has emphasized the opportunity to enhance successful trajectories, especially for the population that has been historically excluded from higher education and skilled jobs. In this sense, access to TTE is seen as an instrument of social mobility that seeks to reduce inequality (Brunner et al., 2022).

In turn, retention and dropouts, particularly for low-income students, have been a policy concern and a challenge to the technical educational system (Hällsten, 2017; Sarra et al., 2018; Brunner et al., 2022). The adverse effect of dropping out is dramatic and affects students and families in many ways, including greater marginalization and future lower labor market outcomes (Sosu & Pheunpha, 2019; Voelkle & Sander, 2008; O'Neill et al., 2011). Therefore, retention and dropout affect the goal of inclusion and equity that Chilean policymakers have tried to insert at the core of the TTE Chilean regulation since 2010 (Brunner et al., 2022).

As such, TTE has a disproportionate share of low-income students (Mountjoy, 2022; Sotomayor, 2018). In Chile, these institutions have more extensive participation of students from quintiles 1-3 of income. Namely, close to 50% of the enrollment of these institutions comes from the poorest 60% of the Chilean population (SIES, 2022). In this regard, a 2022 study of the Higher Education Information Service of Chile (SIES) indicates that the retention rate for first-year students of tertiary education is higher for universities (85%) compared to TTE institutions (70%) (SIES, 2022). The statistics are consistent with the study of the determinants of retention and dropout in TTE. When examining which factors influence the probability of student retention, evidence from across the globe indicates four main groups of variables: i) the sociodemographic background of the family, ii) the student's previous academic results, iii) accessibility or financing restrictions, and iv) institutional factors (Behr et al., 2020; Li & Carroll, 2017; Millea et al., 2018; among others).

The Professional Institute INACAP is one of the largest TTEs in Chile, with 15% (N=76781) of the total enrollment of the Chilean technical college system (SiES, 2022). Since 2014, INACAP has developed new and different mechanisms to support a successful trajectory of students through a program called the Progression Support System (SiAP). The program has at its core an inclusion and equity framework that has allowed a comprehensive set of initiatives articulated through a tutor that provides academic, psychosocial, and extracurricular support to students (particularly at risk) to help them successfully navigate their career pathways. The program's ultimate impact indicator is to increase first-year students' retention rates and avoid dropouts.

However, questions about the program's effectiveness have been raised, especially during the covid-19 pandemic. In this regard, we determined to measure the effect of the SiAP-program at INACAP on the first-year student retention rate for the cohorts 2017-2021. Additionally, the study sought to describe changes in the program policies and implementation during the pandemic (2020-2021) that might affect results. Finally, given the large enrollment of INACAP, findings will give insights into the institution and the Chilean tertiary education system about to what extent tutoring programming, as the core of a student system of academic progression support, facilitates retention in the educational system for vulnerable students.

Every year, approximately 35% of Slovenian high school graduates who complete academically the most demanding upper secondary education take the General Matura (GM) examinations in Slovenia. The GM is comprised of five subject exams: Slovene language, Mathematics, First foreign language, and two subjects chosen by the student from a selection of over 30 subjects. The GM score in Slovenia is calculated as the sum of grades received in the five subject exams. The score can range from 10 (2+2+2+2+2), the lowest passing grade, to 34 (8+8+8+5+5), the highest possible score. Success on the GM is considered equivalent to completing upper secondary education, and in cases where university study programs have a limited number of applicants, GM scores are used as a selection criteria in the admissions process.

The use of GM scores for university admissions has been studied multiple times (Bucik, 2001; Cankar, 2000; Sočan, Krebl, Špeh & Kutin, 2016) with findings similar to research on external examinations in other countries (Kuncel, Hezlett, & Ones; 2001). However, there is a lack of research on the associations between GM scores (achieved at the age of 19) and various measures of personal success later in life and professional careers at ages 33-40 in national and international literature. In public discussions, you can often hear assertions that students’ school achievements and results on external exams have no relevance for later success on a labour market, income in their professional career, or other measures of success. Despite being a high-stakes examination, GM has not been systematically examined to determine its role and long-term value in the Slovenian educational system.

Our research aims to explore the predictive value of GM scores on socio-economic status (SES) and specifically on yearly income later in life for several cohorts of students. Our null hypothesis is that GM scores do not predict SES or yearly income of students in their professional careers.

While it is commonly assumed that success on the GM examinations at the end of upper secondary education is associated with success at university and to some extent later in professional careers, such claims are difficult to verify scientifically due to a lack of representative and valid data. This research aims to provide a deeper understanding of the associations between GM scores, socio-economic measures and personal success in professional careers.

In research on educational equity, students from socioeconomically disadvantaged homes are typically depicted as low-performers and more likely to fail in school (Sirin, 2005). There are, however, students who, despite their disadvantaged backgrounds, manage to succeed in school. This capacity to overcome adversities in education and still reach successful achievements is referred to as Academic Resilience (Agasisti et al., 2018). Academic Resilience is built upon two critical conditions, namely, exposure to significant threats or severe adversity and achievement of positive adaptation despite major assaults on the developmental process (Kiswarday, 2012). Resilience is often captured by identifying protective and risk factors that predict the likelihood of achieving resilient outcomes. Risk factors are characteristics which heighten the risk of adverse outcomes, while protective factors are characteristics that function as a buffer against negative impacts and are associated with positive adaptation and outcomes (Masten, 2014). Resilient students are often characterized by high self-confidence, perseverance, willingness and capacity to plan, and lower anxiety (Martin & Marsh, 2006, 2009; OECD, 2011), strong engagement in class and academic activities (Borman & Overman, 2004). Thorsen et. al., (2021) also found that resilient students display both more perseverance and consistency of interest over time. Hence, both cognitive and non-cognitive skills are important for academic resilience.

Research on the economics of human development also highlight the value of skill formation for success in adulthood, particularly for disadvantaged children. Societal investments in strengthening both cognitive and non-cognitive skills for disadvantaged children give significant economic returns both at individual and societal levels (e.g. Heckman, 2006). More recent studies have particularly highlighted non-cognitive skill formation as an crucial enabler. Non-cognitive skills are associated with promoting both economic and social mobility, economic productivity and well-being in adulthood (e.g. Kautz et al., 2015; Soto, 2019). A wealth of studies on the labour market aligns with this reasoning, identifying positive associations with both cognitive and non-cognitive skills and labour market outcomes. Johannesson (2017) found that cognitive abilities and non-cognitive skills (academic self-concept and perseverance) predicted the risk of being unemployed via school grades. Further, personality, i.e. extraversion and conscientiousness, was demonstrated to lead to higher earnings (Fletcher, 2013). Edin et. al., (2022) found that one standard-deviation increase in cognitive skills is associated with a salary increase of 6.6 percent and an increase in non-cognitive skills is associated with a 7,9 percent salary increase after controlling for educational attainment.

Studies on academic resilience and skill formation are scarcer. Nevertheless, some studies have found that protective factors identified during childhood and youth such as self-control and ability to plan are predictive of a more successful transition into adulthood (see Burt & Paysnick, 2012 for a review). In a qualitative study following up on four resilient students a decade later Morales (2008) found that the students continued to perform at high educational levels. The participants adapted the protective factors identified at the start of the study (i.e. self-confidence and internal locus of control) and used them to meet new challenges.

Employing a resilience perspective, the present study aims to investigate the difference in salary development among individuals who have been identified as being academically resilient versus those who are not. We also want to explore if the salary development can be attributed to educational attainment (educational history) and work status as changing conditions, and cognitive and non-cognitive skills, such as, cognitive ability, perseverance and academic self-concept as time invariant prerequisites.

Research studies on degree completion on due time are seldom in the higher education literature. Regarding the European Higher Education Area (EHEA), no study has been published so far addressing the topic of degree completion on due time (Yes/No) based on nationwide representative data. By considering an entire entrant cohort of first-time, full-time undergraduate students who attended their three-year program at the same institution, and that simultaneously considers students’ background, entrance scores and choices, eligibility for social scholarship, institutional organization characteristics and the area of study, this study explores the role of social scholarships/financial aid in overcoming the effects of students’ socioeconomic disadvantages. A previous study (Ferrão, 2023) analyzed the relationship of the aforementioned students’ characteristics on degree completion grade point average (GPA). Findings suggest that receiving (or not receiving) a social scholarship has no influence on students’ GPA, confirming recent institution research findings obtained with the Universidade of Minho data (Ferrão et al., 2021) for the 1^st year GPA rating. Nevertheless, Ferrão et al. (2020) found a statistically significant fixed effect of scholarship on students’ persistence, at the level of significance of 10%. In addition, institutional research conducted at the Instituto Politécnico de Leiria points out that providing solutions for financial limitations may contribute to decrease the risk of dropping out. In fact, Carreira and Lopes (2021) report that the “main motives for dropping out referred were ‘financial difficulties’ (27% of the students) and ‘work-school incompatibility’ (20%), while ‘low academic performance’ (11%), ‘health reasons’ (8%) or migration (2%), for example, were less mentioned, confirming the importance of financial assistance to reduce dropout risk (for traditional students) […]” (p. 1353). Given that the financing of higher education in Portugal has progressively shifted from the state’s responsibility to that of the students' and their families, as in many other countries (Marginson, 2018; Tight, 2020), this calls for a more accountable evaluation of private/public funding and demands more effective social justice policies (Pitman et al., 2019). Thus, this study investigates the effect of receiving social scholarships/financial aid on degree completion on due time. Its specific objective is to estimate the fixed effect of receiving or not receiving a social scholarship on the probability of degree completion on due time. The study contributes to the themes of students’ success (degree completion), equity (financial aid), system evaluation and resource allocation. Since Portugal is one of the European countries where the costs of higher education are supported primarily by taxpayers, this topic of research matters not only for public policy regarding the increase of equity, but also for the efficiency of public resources allocation.

International large-scale assessments such as the Programme for International Student Assessment (PISA) or the Programme for the International Assessment of Adult Competencies (PIAAC) have shaped the political landscape of how educational systems are evaluated and have led to profound reforms of educational systems across the world. Besides the political impact, large and representative data sets previously unknown have become available to researchers to pursue research questions that allow for the extensive application of data science, learning analytics, and big data analytics. These data sets contain information on “core domain” skills such as mathematics and reading, a host of background data, but also data on “21^st century skills” such as digital learning, problem solving, and collaboration. In this presentation, I will provide an overview of the history of large-scale assessments and give an update of recent additions and innovations putting the spotlight on some of the most influential assessments and their policy and research impact. There will be a specific focus on the emerging role of 21^st century skills and data science both in the reporting and analyses of large-scale assessment.

Open Science principles require that data that are collected and analysed as part of research projects are made available to other researchers at the end the project (van der Zee & Reich, 2018). This allows not only for replication, validation and generalization of research findings (van der Zee & Reich, 2018; (Tedersoo et al., 2021), but also for secondary data analyses. In general, data sharing is crucial for efficiency of scientific knowledge generation (Allen & Mehler, 2019; Nosek et al., 2015). It is also highly valuable for the individual researcher as scientific articles for which the data are published are cited more than articles for which the data are not available (Colavizza et al., 2020; Drachen et al., 2016; Piwowar et al., 2007). While in educational research more and more data are available for secondary analyses, a considerable proportion are not shared.

The proposed symposium aims at shedding more light on the factors explaining the reluctance to make data available as well as giving an overview of what can be accessed and puts an emphasis on the legal requirements. It describes the educational data landscape across several European countries and elaborates on specific legal aspects researchers struggle with. More precisely, it highlights the potential of the rich data that exist but is not (yet) available to secondary users. Often, researchers are willing to share their data but are insecure about how to make the data sharable and how to properly comply with the legal aspects, e.g. consent and copyright. Moreover, researchers are not always aware of options of restricted access and different layers of protection (including consent, anonymisation/pseudonymisation, and restricted access). Therefore, in this symposium we elaborate on challenges and best practices for sharing research data and provide practical guidance.

The symposium is a joint effort of researchers from four institutions from different European countries working on various aspects of data reuse and access to facilitate high quality educational research. The involved institutions are the DIPF | Leibniz Institute for Research and Information in Education, the International Association for the Evaluation of Educational Achievement (IEA), the University of Applied Sciences, Graubünden, and the Swiss Centre of Expertise in the Social Sciences (FORS).

The symposium consists of three contributions. The first paper takes a comparative perspective on the availability of educational research data in five European countries, namely England, Norway, France, Sweden and Switzerland. These countries are compared along a number of relevant factors with regard to data access. Based on the analysis, implications for practice are derived. The second paper provides insight into the legal challenges of data sharing in an international research project. The third paper zooms in on the topic of the GDPR, it consists of two parts. First, the GDPR and its implications on research in education are described. Then, a description of how it got implemented in the context of TIMSS 2023.

With this symposium we aim at engaging in a debate with members from Network 12 “Open Science in Education”, as well as researchers from the other networks and we encourage emerging researchers to join the debate.

In recent years, there have been huge developments in the data landscape on education and learning. This is due to several reasons. First, the ways data can be collected and analysed. In addition, research data infrastructures at both the national and international level have taken a more prominent role and guide researchers throughout the research process with regard to questions relating to data management (Corti et al., 2019). Moreover, there has been a growing understanding that sound educational policy-making requires sound evidence based on high-quality and accessible data. Many research funders now require that data resulting from funded projects will be made available at the end of the project (Logan, Hart & Schatschneider, 2021). At the same time, the legal frameworks have evolved and several countries now allow for more easy data linkages. For example, linking data from the administrations to survey data has become more common (Harron et al., 2017). In this contribution, we draw a picture of the educational research data landscape in five European countries. The aim is threefold: We provide a description of the educational data landscapes across varying contexts, compare them and derive conclusions on what can be learnt from one context to the next. The countries included in the study are England, Norway, France, Sweden and Switzerland. These countries have been selected as contrasting cases in how educational data are administered, provided and how they can be accessed by researchers. The countries also have different legal bases for working with sensitive data and vary in the degree to which data infrastructures are centralized. In order to get a complete view on the educational data landscape in these countries we have carried out interviews with experts from each of the countries. The five countries are compared along the following characteristics: (1) Main data providers (e.g. universities, statistical offices, institutional repositories); (2) Types of available data (level of education, statistical data, learning systems, competencies etc.); (3) Possibilities for data linkage; (5) Laws and Regulations. The analysis shows that there is considerable variation in terms of what data are made available, the procedures, coverage and handling. At the same time, there are some commonalities across countries. The findings allow us to draw conclusions for future directions of the educational data landscape. We make suggestions on what can be learnt from other contexts and what would facilitate high-quality data-driven educational research that could inform educational policy and practice.

Corti, L., Van den Eynden, V., Bishop, L., & Woollard, M. (2019). Managing and sharing research data: A guide to good practice (2nd ed.). London: Sage. Harron, K., Dibben, C., Boyd, J., Hjern, A., Azimaee, M., Barreto, M. L., & Goldstein, H. (2017). Challenges in administrative data linkage for research. Big data & society, 4(2), 2053951717745678. Logan, J. A. R., Hart, S. A., & Schatschneider, C. (2021, 2021/01/01). Data Sharing in Education Science. AERA Open, 7, 23328584211006475. https://doi.org/10.1177/23328584211006475

Effective data sharing is a crucial principle in fostering transparency, reproducibility, and collaboration in scientific research, making it a vital topic in the context of Open Science. Additionally, the ability to find, access, interoperate and reuse data is crucial for advancing scientific discovery and innovation, making it relevant to the FAIR data principles (Wilkinson et al., 2016). This presentation highlights the legal challenges of sharing data from international research projects using the example of a qualitative research project in German and Australian schools. The research project was accompanied by a case study. The case study analysed issues such as data protection, intellectual property and compliance to gain a better understanding of how international data sharing can be done effectively and ethically. The study provides valuable insights for anyone involved in international research projects and helps uncover potential legal challenges. In the field of educational research, data collected is often highly personal and may include minors. However, researchers in education often have little legal knowledge, it is not part of their training. Especially, in an international context, different legal regulations make it difficult to comply with data collection and sharing. To investigate this issue, we selected an international qualitative research project for a case study (Argyrou 2017; Eisenhardt 2002; Yin 2018). The international project examined a digital student exchange between Australian and German students, using video recordings, interviews, student-created videos, chat communications, digital classroom materials, and student-drawn language biographies. This raised complex legal questions about the handling of students' personal data and the handling of personal data of third parties, such as their families. Copyright issues are also key, especially when copyrighted material is to be shared as part of Open Science and used in scientific publications. For the case study, we explored the challenges and issues that arose in collecting and sharing data of the international research project as well as the resources and opportunities to find solutions through interviews and documentation analysis. In our presentation, we summarize the legal challenges of collecting and sharing research data in the international project and show examples of the measures taken to overcome these challenges. Overall, the results of our case study provide valuable insights for those involved in international educational research projects and help identify and deal with potential legal challenges.

Argyrou, A. (2017). Making the case for case studies in empirical legal research. Utrecht Law Review.org, 13(3)2017, http://doi.org/10.18352/ulr.409 Wilkinson, M. D. et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3 (160018), doi: 10.1038/sdata.2016.18 (2016). Yin, R. K. (2018). Case study research and applications: design and methods. Los Angeles: Sage.K. Eisenhardt, K. (2002). Building Theories from Case Study Research. In M. Huberman & M. B. Miles. The Qualitative Researcher's Companion. Los Angeles: Sage.K.

The General Data Protection Regulation (GDPR), applicable in the European Economic Area (EEA), came into force in 2018 and impacted the day-to-day activities of anyone, who handles personal data, including IEA and the network of National Research Coordinators implementing IEA comparative studies and collecting data from students, parents/guardians, teachers, and school principals. This presentation takes the view that the GDPR provides some benefits for International Educational Research and documents how IEA interpreted and implemented some of the requirements of the GDPR and the benefits of these arrangements for participants in IEA Studies, IEA data and Open Science as a whole. The first aspect we look at is the obligation to provide information to the data subject, where personal information is collected from them. The data subject must be informed inter alia about the purpose of the processing, who will process their personal, who will have access to it, would it be transferred outside of the European Economic Area, for long the personal data will be processed and where it will be stored. In the field of educational research, this obligation imposed on the data controller, benefits the research participants by respecting the child’s evolving capacities. To this end, IEA has prepared a Data Protection Declaration template, which provides participating students, their parents/guardians and teachers with the necessary information. A step further is to prepare an additional child-friendly version of these documents, targeting 4 and 8 grade students. A second aspect to consider is some of GDPR’s core principles of processing of personal data and their positive impact for International Educational Research. Respecting principles such as lawfulness, fairness and transparency, purpose limitation, data minimization, integrity and confidentiality, as well as accountability strengthens the integrity and the ethics of the research and keep researchers accountable. Compliance with the GDPR also directly benefits IEA data itself, as it ensures that the collected data can be later analysed and used for research purposes. Using TIMSS 2023 as an example, this presentation seeks to show how the rules of data protection can be reconciled with the goals of Open Science to the benefit of study participants and international educational research. While IEA encourages Open Science by having IEA data publicly available, as part of the international report and the International Database (IDB), IEA also ensures that any personal data of participants is protected by adopting appropriate safeguards such as anonymization and pseudonymization techniques.

The full title of the GDPR: REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

Open Science principles require that data that are collected and analysed as part of research projects are made available to other researchers at the end the project (van der Zee & Reich, 2018). This allows not only for replication, validation and generalization of research findings (van der Zee & Reich, 2018; (Tedersoo et al., 2021), but also for secondary data analyses. In general, data sharing is crucial for efficiency of scientific knowledge generation (Allen & Mehler, 2019; Nosek et al., 2015). It is also highly valuable for the individual researcher as scientific articles for which the data are published are cited more than articles for which the data are not available (Colavizza et al., 2020; Drachen et al., 2016; Piwowar et al., 2007). While in educational research more and more data are available for secondary analyses, a considerable proportion are not shared.

The proposed symposium aims at shedding more light on the factors explaining the reluctance to make data available as well as giving an overview of what can be accessed and puts an emphasis on the legal requirements. It describes the educational data landscape across several European countries and elaborates on specific legal aspects researchers struggle with. More precisely, it highlights the potential of the rich data that exist but is not (yet) available to secondary users. Often, researchers are willing to share their data but are insecure about how to make the data sharable and how to properly comply with the legal aspects, e.g. consent and copyright. Moreover, researchers are not always aware of options of restricted access and different layers of protection (including consent, anonymisation/pseudonymisation, and restricted access). Therefore, in this symposium we elaborate on challenges and best practices for sharing research data and provide practical guidance.

The symposium is a joint effort of researchers from four institutions from different European countries working on various aspects of data reuse and access to facilitate high quality educational research. The involved institutions are the DIPF | Leibniz Institute for Research and Information in Education, the International Association for the Evaluation of Educational Achievement (IEA), the University of Applied Sciences, Graubünden, and the Swiss Centre of Expertise in the Social Sciences (FORS).

The symposium consists of three contributions. The first paper takes a comparative perspective on the availability of educational research data in five European countries, namely England, Norway, France, Sweden and Switzerland. These countries are compared along a number of relevant factors with regard to data access. Based on the analysis, implications for practice are derived. The second paper provides insight into the legal challenges of data sharing in an international research project. The third paper zooms in on the topic of the GDPR, it consists of two parts. First, the GDPR and its implications on research in education are described. Then, a description of how it got implemented in the context of TIMSS 2023.

With this symposium we aim at engaging in a debate with members from Network 12 “Open Science in Education”, as well as researchers from the other networks and we encourage emerging researchers to join the debate.

In recent years, there have been huge developments in the data landscape on education and learning. This is due to several reasons. First, the ways data can be collected and analysed. In addition, research data infrastructures at both the national and international level have taken a more prominent role and guide researchers throughout the research process with regard to questions relating to data management (Corti et al., 2019). Moreover, there has been a growing understanding that sound educational policy-making requires sound evidence based on high-quality and accessible data. Many research funders now require that data resulting from funded projects will be made available at the end of the project (Logan, Hart & Schatschneider, 2021). At the same time, the legal frameworks have evolved and several countries now allow for more easy data linkages. For example, linking data from the administrations to survey data has become more common (Harron et al., 2017). In this contribution, we draw a picture of the educational research data landscape in five European countries. The aim is threefold: We provide a description of the educational data landscapes across varying contexts, compare them and derive conclusions on what can be learnt from one context to the next. The countries included in the study are England, Norway, France, Sweden and Switzerland. These countries have been selected as contrasting cases in how educational data are administered, provided and how they can be accessed by researchers. The countries also have different legal bases for working with sensitive data and vary in the degree to which data infrastructures are centralized. In order to get a complete view on the educational data landscape in these countries we have carried out interviews with experts from each of the countries. The five countries are compared along the following characteristics: (1) Main data providers (e.g. universities, statistical offices, institutional repositories); (2) Types of available data (level of education, statistical data, learning systems, competencies etc.); (3) Possibilities for data linkage; (5) Laws and Regulations. The analysis shows that there is considerable variation in terms of what data are made available, the procedures, coverage and handling. At the same time, there are some commonalities across countries. The findings allow us to draw conclusions for future directions of the educational data landscape. We make suggestions on what can be learnt from other contexts and what would facilitate high-quality data-driven educational research that could inform educational policy and practice.

Corti, L., Van den Eynden, V., Bishop, L., & Woollard, M. (2019). Managing and sharing research data: A guide to good practice (2nd ed.). London: Sage. Harron, K., Dibben, C., Boyd, J., Hjern, A., Azimaee, M., Barreto, M. L., & Goldstein, H. (2017). Challenges in administrative data linkage for research. Big data & society, 4(2), 2053951717745678. Logan, J. A. R., Hart, S. A., & Schatschneider, C. (2021, 2021/01/01). Data Sharing in Education Science. AERA Open, 7, 23328584211006475. https://doi.org/10.1177/23328584211006475

Effective data sharing is a crucial principle in fostering transparency, reproducibility, and collaboration in scientific research, making it a vital topic in the context of Open Science. Additionally, the ability to find, access, interoperate and reuse data is crucial for advancing scientific discovery and innovation, making it relevant to the FAIR data principles (Wilkinson et al., 2016). This presentation highlights the legal challenges of sharing data from international research projects using the example of a qualitative research project in German and Australian schools. The research project was accompanied by a case study. The case study analysed issues such as data protection, intellectual property and compliance to gain a better understanding of how international data sharing can be done effectively and ethically. The study provides valuable insights for anyone involved in international research projects and helps uncover potential legal challenges. In the field of educational research, data collected is often highly personal and may include minors. However, researchers in education often have little legal knowledge, it is not part of their training. Especially, in an international context, different legal regulations make it difficult to comply with data collection and sharing. To investigate this issue, we selected an international qualitative research project for a case study (Argyrou 2017; Eisenhardt 2002; Yin 2018). The international project examined a digital student exchange between Australian and German students, using video recordings, interviews, student-created videos, chat communications, digital classroom materials, and student-drawn language biographies. This raised complex legal questions about the handling of students' personal data and the handling of personal data of third parties, such as their families. Copyright issues are also key, especially when copyrighted material is to be shared as part of Open Science and used in scientific publications. For the case study, we explored the challenges and issues that arose in collecting and sharing data of the international research project as well as the resources and opportunities to find solutions through interviews and documentation analysis. In our presentation, we summarize the legal challenges of collecting and sharing research data in the international project and show examples of the measures taken to overcome these challenges. Overall, the results of our case study provide valuable insights for those involved in international educational research projects and help identify and deal with potential legal challenges.

Argyrou, A. (2017). Making the case for case studies in empirical legal research. Utrecht Law Review.org, 13(3)2017, http://doi.org/10.18352/ulr.409 Wilkinson, M. D. et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3 (160018), doi: 10.1038/sdata.2016.18 (2016). Yin, R. K. (2018). Case study research and applications: design and methods. Los Angeles: Sage.K. Eisenhardt, K. (2002). Building Theories from Case Study Research. In M. Huberman & M. B. Miles. The Qualitative Researcher's Companion. Los Angeles: Sage.K.

The General Data Protection Regulation (GDPR), applicable in the European Economic Area (EEA), came into force in 2018 and impacted the day-to-day activities of anyone, who handles personal data, including IEA and the network of National Research Coordinators implementing IEA comparative studies and collecting data from students, parents/guardians, teachers, and school principals. This presentation takes the view that the GDPR provides some benefits for International Educational Research and documents how IEA interpreted and implemented some of the requirements of the GDPR and the benefits of these arrangements for participants in IEA Studies, IEA data and Open Science as a whole. The first aspect we look at is the obligation to provide information to the data subject, where personal information is collected from them. The data subject must be informed inter alia about the purpose of the processing, who will process their personal, who will have access to it, would it be transferred outside of the European Economic Area, for long the personal data will be processed and where it will be stored. In the field of educational research, this obligation imposed on the data controller, benefits the research participants by respecting the child’s evolving capacities. To this end, IEA has prepared a Data Protection Declaration template, which provides participating students, their parents/guardians and teachers with the necessary information. A step further is to prepare an additional child-friendly version of these documents, targeting 4 and 8 grade students. A second aspect to consider is some of GDPR’s core principles of processing of personal data and their positive impact for International Educational Research. Respecting principles such as lawfulness, fairness and transparency, purpose limitation, data minimization, integrity and confidentiality, as well as accountability strengthens the integrity and the ethics of the research and keep researchers accountable. Compliance with the GDPR also directly benefits IEA data itself, as it ensures that the collected data can be later analysed and used for research purposes. Using TIMSS 2023 as an example, this presentation seeks to show how the rules of data protection can be reconciled with the goals of Open Science to the benefit of study participants and international educational research. While IEA encourages Open Science by having IEA data publicly available, as part of the international report and the International Database (IDB), IEA also ensures that any personal data of participants is protected by adopting appropriate safeguards such as anonymization and pseudonymization techniques.

The full title of the GDPR: REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)

Market demands with their multitude of stimuli and information can be particularly overwhelming for children, whose cognitive abilities and skills are not yet fully developed and who lack market experience and knowledge (Mau et al., 2014). To better understand children’s purchase decisions, associated behaviors, and deficits, Mau et al. (2016) analyzed children’s behaviors in a simulated supermarket environment. They showed that children often behave differently at the point of sale than they intended and expected to when making their purchase decision. Slightly more than half of the children indicated that they would primarily look for low prices when shopping. Although the children in the subsequent observation of their shopping behavior had a limited budget and were tasked with buying the cheapest products, it was found that they clearly tended to select products more based on package design or brand. These findings indicate that children have difficulty implementing a basic requirement of goal-oriented consumer behavior, namely, taking the right actions to achieve a set goal (Bagozzi & Dholakia, 1999).

Regarding the question of how actions in purchasing processes could be implemented in a goal-oriented manner, we draw on basic theoretical frameworks of action regulation that include feedback loops, such as the cybernetic TOTE model (Powers, 1973). Carver and Scheier (1981) assume that the successful realization of a goal state (e.g., fulfillment of the shopping list) occurs by passing through loops in which the existing and target states (e.g., contents of the shopping cart vs. the shopping list) are repeatedly compared with each other and a deviation is successively reduced by operations until the loop is exited. Consequently, the execution of action is expressed as a sequence of corresponding operations and is always in the interplay between the goals of the agent and the situational requirements. According to Gillebaart (2018), setting standards or goals as well as monitoring deviations are aspects of self-regulation, while successful self-control comes into play within the feedback loop in the ‘operate’ phase. Self-control has been described by Baumeister et al. (2007) as the mental processes that enable people to control their thoughts, emotions, and behaviors to achieve higher-level goals. While operating, various aspects of self-control can be observed, such as suppressing the impulse to be tempted by alluring stimuli that are not in line with our long-term goals (e.g., completing the shopping list), avoiding situations that might lead one into temptation (e.g., forgo the candy shelf), or even delaying gratification with an immediate, smaller reward in order to obtain a larger, delayed reward. According to Inzlicht et al. (2014), as a result of repeated self-control efforts, there may be a change in the degree of self-control displayed. This is attributed to a change in task priorities, a shift in motivation away from so-called "have-to" to "want-to" goals that provide more pleasure and satisfaction. Therefore, the process of action regulation is always under the influence of changing motivations and the attendant changes in emotions and attention. Although self-control has been highlighted in its importance for the successful implementation of consumer goals (e.g., by avoiding impulsive purchases; Baumeister, 2002), there is still no study that specifically captures children's operations in the purchasing process and relates the extent of self-controlled behavior to the successful implementation of a purchase intention.

The increasing interest in collaboration as an educational competence important for successful schooling and a productive adult professional and civic life can be seen in the expanding literature and research evidence (e.g. Rychen & Salganik, 2003; National Research Council, 2011). Collaboration is marked as one of the social and emotional skills on the 2030 education development agenda, defined by the intergovernmental Organisation for Economic Co-operation and Development (OECD, 2019).

Collaborative problem solving (CPS) is an umbrella term for a variety of pedagogical models that enable students to learn by engaging in joint activities, relying on each other, integrating individual knowledge, skills, and efforts (Lai, 2011). With appropriate support and scaffolding, CPS could have a greater positive effect on student achievement, and peer social relationships than competitive and individual learning (e.g. Gillies, 2016; Johnson & Johnson, 2002).

The focus of this study is on students experience with CPS as simetric peer interaction during the regular school classes. These perceptions and experiences represented a base for examining how CPS is applying in context of secondary schools in Serbia. The research suggests that a productive collaboration requires both cognitive skills (e.g. Campbell, 2021; Shi et al, 2021), as well as social and emotional skills (e.g. Newman, 2016; Rogat & Adams-Wiggins, 2015). That is why we paid attention how students have reported about cognitive (e.g, argumentation, consideration and evaluation of various perspectives...) and social and emotional aspects (group cohesion, tolerance, atmosphere…) of collaborative work.

Analysing student responses to semi-structured interviews showed that, with certain inconsistencies and overlaps, two models of cooperation are clearly differentiated, presented here by key features.

Model 1 is oriented towards an efficient use of resources, including time, with a dominant utilitarian goal - getting the job done. It is characterized by a strict division of responsibilities, usually mechanical. The roles are defined, including the leader who can be self-proclaimed. The product is a collection of individual works: either loosely bound or bound by one group member. Solution/product quality is judged on the basis of external indicators. Cognitive aspect of CPS includes prior knowledge seen as a key success factor. Social and emotional aspect: a strict division of roles and the leader’s assumption of responsibility often excludes democratic patterns of behaviour such as negotiation and agreement; the atmosphere in the group depends on the degree of closeness of the members, any disagreement during group work can grow into a conflict.

This model can be termed parallel or utilitarian and quasi-cooperation, as the key cooperation determinants cannot be easily identified, except for work arrangements. According to our respondents’ experiences, this model dominates.

Model 2 is oriented primarily towards product quality; sometimes learning cooperation as a competence is cited as an explicit cooperation goal. Cooperation primarily has a cognitive goal, reflected in the usage of search strategies for task solving. There is a loose division of responsibilities and roles, usually according to participant competencies and interests; deadlines are on the back burner. The product is based on group consensus. Cognitive aspect includes awareness of the importance of argumentation and discussion Social and emotional aspect: an atmosphere of mutual trust and equality between team members remove barriers and allow freedom in presenting and considering different solutions and/or ways of solving tasks. There is mutual knowledge and respect. Cognitive and social and emotional aspects are interconnected, manifesting itself as solidarity with others and connecting as a form of strengthening personal capacities.

This model can be called collaborative or constructive due to its orientation towards the joint construction of knowledge. Unfortunately, according to student experiences, it is rarely represented in school practice.

When participating in assessments, it is assumed that examinees have invested effort to perform well; otherwise, scores will not reflect their true ability and will not be valid indicators of their proficiency (Baumert & Demmrich, 2001; Wise, 2015). However, lack of motivation and effort during the test-taking process, creates a threat to the validity of test outcomes, especially with International Large-Scale Assessments (Rutkowski & Wild, 2015).

Response-time data from computerized tests have enabled researchers to study test-taking effort, e.g. by identifying respondents who respond rapidly before a certain time point. However, there is a need to move beyond the examination of rapid responses through thresholds, since the time needed to respond to a test item is dependent on various factors, including ability and test-taking behaviors. Consequently, it is argued that to be interpreted appropriately, response times should be examined in relation to examinee’s performance at the item level.

The purpose of this study is to examine two novel response-based, indicators of test-taking behaviors that utilize a combination of examinee response and process (timing) data to better understand and describe test-taking effort in online assessments. These indicators, which have been named “Unsuccessful time management” and “Successful time management” will be empirically estimated with data from the fourth-grade e-TIMSS 2019 mathematics assessment. This study further aims to examine these variables in relation to achievement benchmarks, student background characteristics such as attitudes towards mathematics, confidence in mathematics, gender, as well as overall achievement. The ultimate goal of these analyses is to try to obtain further insights on examinees who participate in online assessments through the use of their timing data.

The availability of digital media devices such as smartphones, tablets, notebooks, etc. in households has become so commonplace today that we hardly think about our media use and its consequences. This also seems to be true for households with children. Even very young children come in contact with and use these devices from an early age on. For example, a study in the UK reported that children started using touchscreen media (smartphones, tablets) as early as six months of age (Cheung et al., 2017). As for the duration of media use, an Austrian study of parents with children up to the age of six years found that one-third of the children use digital media every day and around fifty percent use them several times a week (Institut für empirische Sozialforschung [IFES], 2020). The numbers are similar across Europe (e.g., Germany see Kieninger et al., 2020; UK see Bedford et al., 2016; France see Cristia & Seidl, 2015; Italy see Chindamo et al., 2019).

Parents face various educational challenges and must deliberate how to manage and regulate their children’s media use. For the investigation of parents’ educational strategies and behaviors, “parental mediation theory” has proven to be a valuable theoretical framework. Within this framework, the present study focused on children’s sleep and parents’ mediation of media use. The prevalence of digital media presents parents with several challenges, including the duration and frequency of media use but also concerns about possible harmful effects of media use itself. In this regard, parental mediation theory describes strategies parents use to minimize potentially harmful consequences of media use (Clark, 2011; Valkenburg et al., 1999).

One important area of research concerns the effects of media use on children’s development and health. In this context, sleep is an important variable. During the first years of life, sufficient and restful sleep is of particular importance, because sleep is essential for developmental processes (El-Sheik & Sadeh, 2015) such as neuronal and cognitive development processes (Ednick et al., 2009). Research suggests harmful effects of media use on children’s sleep (e.g., Hackl-Wimmer et al., 2021). Ensuring that children get sufficient and restful sleep is an important educational task for parents. Toddlers’ sleep may be influenced by a variety of factors, including digital media use (Hackl-Wimmer et al., 2021). There are diverse methods for quantifying sleep quantity and quality. However, research on children’s sleep suffers from methodological problems. Studies on young children’s sleep and media use mainly employ subjective data such as parent questionnaires to assess children’s sleep (e.g., Chindamo et al., 2019).

An approach that is often used in medicine is polysomnography (PSG). PSG comprises the recording of several physiological functions (e.g., heart rate and brain waves) in a sleep laboratory for clinical purposes. However, the methodology is not suitable for recording children’s sleep in daily situations at home. Therefore, we used and further developed a more practical approach, namely ECG (electrocardiogram) recordings with small portable devices. ECG is the recording of the electrical activity of the heart. ECG data allows the measurement of heart rate (HR) and the calculation of several parameters of heart rate variability (the variation in time between consecutive heartbeats). Due to technological progress, ECG data can be recorded using portable devices over a period of 24 hours or more. These devices are also suitable for studies with young children and allow recording of HR during sleep at home in the child’s familiar environment.

The aim of this study is to examine whether toddlers’ use of smartphones and audio media is related to their sleep quality (quantified as HR during restless sleep phases).

In most education systems around the world there is a strong case for increasing the mathematical skills of young people beyond the age of 16. Evidence from international student surveys such as PISA show that, for example, in the European Union about 23% of 15 year-olds in 2018 did not reach basic levels of skills in mathematics (OECD, 2019).

Incentivising young people to continue to study mathematics post-16 should not only help satisfy demands for mathematically and quantitatively skilled people in the labour market, but more generally help ensure that young people have the knowledge to succeed in an increasingly technological society (e.g., Mason et al. 2015; Smith, 2017; European Commission, 2022). Moreover, young people with good mathematical knowledge will benefit from the quantitative, analytical and problem-solving skills mathematics qualifications develop, which will support attainment in other disciplines, particularly those with a significant quantitative component.

In England, unlike other countries in Europe and the rest of the world, the study of mathematics post-16 is not compulsory for all students. A recent study comparing upper secondary mathematics participation in 24 countries (Hodgen & Pepper, 2019) showed that in England fewer than 20% of students persist with mathematics education in any form beyond the age of 16. In contrast, 18 countries have post-16 participation rates higher than 50%, with rates at more than 95% in eight of them, including Sweden, Finland, Japan and Korea.

One reason for low progression to post-16 mathematics in England could be a longstanding concern about how well the mathematics qualifications offered to students aged 14 to 16 (GCSE, General Certificate of Secondary Education) prepare students for advanced study in mathematics, with algebra frequently mentioned as the key problem (e.g., Wiliam et al., 1999; Hernandez-Martinez et al., 2011; Noyes & Sealey, 2011; Rigby 2017).

To increase uptake of mathematics and to improve students’ mathematical skills at all levels takes effort, funding and a range of interventions. In England, GCSE qualifications (in all subjects) were recently reformed “to ensure they are rigorous and robust, and give students access to high quality qualifications which match expectations in the highest performing jurisdictions”. For mathematics specifically, the new GCSE “focuses on ensuring that every student masters the fundamental mathematics that is required for further education and future careers”, and, in particular, aims to “be more demanding” and “provide greater challenge for the most able students” (Gove, 2013).

There were concerns that the new mathematics GCSE could deter students from post-16 mathematics (e.g., by reducing their confidence) and unintentionally reduce uptake (ALCAB, 2014; Lee et al., 2018). A decrease in post-16 mathematics entries in 2019 leant weight to these fears but, to date, there has been little published research on how the reform of GCSE mathematics has affected mathematics learning and progression to post-16 study. One of the few studies to consider this issue in detail was carried out by Howard and Khan (2019). Their qualitative research found that, in general, teachers were positive about the extent to which the reformed GCSE prepared students for post-16 mathematics. Their participants also reported that the reformed GCSE had positive implications beyond studying mathematics and that it would support students studying other subjects with mathematical content. Grima and Golding (2019) and Pearson Education (2019) reported similar findings from qualitative research in schools.

The current research aims to complement the qualitative analyses of existing research described above, by approaching the question of how the reform of GCSE mathematics has affected progression to and performance in post-16 mathematics and maths-related subjects via quantitative analysis of entries and performance data.

Assessment literacy has been a repetitious term in the assessment literature since it was popularized by Stiggins (1991) (Koh et al., 2018). Assessment literacy is mainly related to teachers’ assessment practices and skills in selecting, designing, or using assessments for various purposes (Stiggins, 1991). The term also defines the knowledge of principles behind selecting, adapting, or designing assessment tasks, judging students’ work and using obtained data to enhance their learning (Koh et al., 2018).

Mathematical thinking arises when students work on problem-like tasks (Jones & Pepin, 2016). However, traditional mathematics instruction and assessment mainly emphasize memorization instead of creative thinking or reasoning. Some research also supports this claim (see Jäder et al., 2015; Stein et al., 2009; Stevenson & Stigler, 1992; Vacc, 1993). On the other hand, such instruction and assessment fail to enhance students’ competencies in mathematics and lead them to follow rote learning (Hiebert, 2003). Hence, students must face challenging and unfamiliar problems that activate their higher-order thinking skills (HOTS).

HOTS require making explanations, interpretations, and decision-making. Students with HOTS can learn how to improve their success and reduce their weaknesses (Tanujaya, 2016). Hence, mathematics teachers should be knowledgeable about HOTS and how to enhance these skills to carry out quality mathematics instruction and assessment. For this reason, teacher education programs must support preservice mathematics teachers (PMTs) to understand the significance of engaging students with higher-level tasks.

Several categorizations are provided for HOTS in the field of education. Bloom’s taxonomy proposed that analysis, synthesis, and evaluation levels include HOTS (McDavitt, 1994). Stein et al. (1996) described a higher level of cognitive demand as doing mathematics or the use of procedures with connection to concepts, understanding, or meaning. A national categorization for mathematics competence levels was also provided in a Monitoring and Evaluating Academic Skills Study (MEASS) project. This framework comprises four categories, and the last two categories were devoted to students’ higher-order thinking skills (MoNE, 2015). The framework will be introduced during the presentation.

Although challenging tasks can promote students’ HOTS, research has shown that designing worthwhile mathematical tasks is not trivial (Leavy & Hourigan, 2020). Besides, preservice teachers (PT) cannot create such tasks (Silver et al., 1996). This is predictable since they have fewer opportunities to write tasks in their teacher education programs (Crespo & Sinclair, 2008). Significantly less is known about the PT’s ability to develop mathematical tasks (Crespo, 2003). Besides, there are fewer studies on how to help PTs to realize and discuss the quality of their mathematical tasks (Crespo & Sinclair, 2008). Thus, professional development (PD) studies must be conducted to increase PMTs’ capacity to develop tasks.

The study’s purpose is to improve the quality assessment task-writing skills of PMTs through feedback provided by different agencies such as researchers, peers, and students. It specifically aimed to answer the research question, “how does feedback provided by different agencies improve the quality of assessment tasks developed by PMTs.” This study also aimed to introduce the framework for mathematics competence levels (MoNE, 2015) to European society. Feedback was defined by Eggen and Kauchak (2004) as the information that teachers or students receive with regard to the accuracy or relevancy of their work through classroom practices. In this study, this term was used to refer to the information preservice mathematics teachers receive from researchers, students, and their peers about the quality and cognitive levels of their tasks.

On the backdrop of the recent educational data movement (Marsh et al., 2015; Schildkamp et al., 2019), teachers are expected to use different kind of data to inform their instructional decision-making. However, different studies have already demonstrated that teachers are reluctant to change their assessment practices (and conceptions), especially when new practices are framed within the rationale of institutional reforms (Boardman & Woodruf, 2004; Brown, 2004; Klieger, 2016; Remesal, 2007), or in new scenarios such as those that emerged during the COVID-19 pandemic. Despite the recognition of the importance of assessment, some studies (Hopfenbeck, 2015; Looney et al., 2018) have also identified the lack of modernisation and have indicated that assessment has not changed materially. Rcent studies on the use of assessment data for decision-making and teaching practice have showed that although teachers recognise the importance of using data gathered through assessment, sometimes, they are not able to manage several sources of information including data from LSAs (Farrell & Marsh, 2016; Mandinach & Gummer, 2016; Schildkamp et al., 2014).

While LSAs have been progressively recognised as relevant components of educational accountability systems, teachers’ negative attitudes towards LSA programmes and the lack of assessment literacy have been highlighted (Fullan et al., 2018; Klinger & Rogers, 2011). In this perspective, research evidence (Hopster-den-Ottera et al., 2017; Monterio et al., 2021; Schildkamp et al., 2019) suggests that the identification of practical assessment challenges for teachers, as well as the understanding of teachers’ conceptions of assessment are of paramount importance in order to ensure teacher assessment literacy, teacher professionalism, and effective school improvement.

The present paper, with a focus on the Italian school system, tries to offer new insights for this debate. Despite the increasing interest in researching teachers’ assessment conceptions and in understanding how these conceptions affect the assessment literacy development, in Italy these research topics, unfortunately, are still neglected. Therefore, given the current lack of empirical studies on teachers’ LSA conceptions an exploratory qualitative study has been realized (Creswell, 2014; Strauss & Corbin, 2007).

In Italy, between 2007 and the time of writing this study (2022), only one LSA programme was adopted. This programme is administrated by the Italian National Institute for School Evaluation and Assessment of Learning (INVALSI) which is subject to the Minister of Education. The LSA programme, aligned with the national curriculum, comprises a census-based administration of cognitive tests (2°, 5°, 8°, 10°, and 13° grades) in the subjects of Italian, Mathematics, and English. INVALSI reports examine the quality of the national school system and support the school improvement. Since its introduction, the national LSA programme, however, caused different problems: teachers have attacked and boycotted the LSA programme. Still nowadays, they continue to perceive the INVALSI programme as a means of control for schools, teachers, and students.

During the COVID-19 pandemic, schools and teachers shifted to remote instructional activities and experienced difficulties in navigating new (and old) mechanisms within the extant assessment practice (e.g., marking and grading student work on-line or sharing feedback on assignments). In the school year 2019-2020, the INVALSI programme was not administrated. The teachers’ (and students’) positive reactions at the cancellation of the yearly INVALSI programme contrast with the need for an assessment that should be embedded within the school system and aligned with the aim of improving the school quality (Wiliam, 2013).

The study sought to better understand how Italian teachers conceptualise the LSA and how they use its results, addressing the following research questions:

1. What do teachers think of the INVALSI programme?
2. How do teachers use the INVALSI results for their instructional practice and decision-making (at classroom and school level)?

Teacher education plays a significant role in developing assessment culture and practices as well as teachers' assessment literacy (e.g. Xu & Brown, 2016; Atjonen et al. 2019). Assessment literacy (AL) consists of e.g. individual’s and community’s awareness of the values and principles related to assessment, as well as information related to the purpose of assessment, assessment targets, strategies and assessment practices (e.g. Atjonen, 2017; Xu & Brown, 2016). It is based on an understanding of learning processes and environments, an understanding of the impact of the individual and the community on them, an understanding of the goals set for learning processes and the monitoring of their realization throughout the entire learning process (e.g. Atjonen, 2017). When considering assessment as support of learning and its importance for learning is recognized, there is a need for teachers to deepen their understanding of the purpose of assessment and implementation of it i.e. to be assessment literate (DeLuca & Braund, 2019).

AL can be viewed as both a teacher's and a learner's competence. Learners' AL is reflected, among other things, in the skills of receiving and utilizing feedback and in the learner's activity in developing their own competence (Atjonen et al., 2019). Assessment should therefore be an interactive process in which the teacher guides and supports not only learning but also the development of the learner's assessment skills, ie his or her learning skills. Teacher education can prepare teacher students for their future work by focusing on assessment that, on the one hand, develops their assessment skills as a student, such as self-assessment skills, and, on the other hand, provides the ability to plan, implement and develop assessment as a teacher. (Boud et al., 2013; Kearney & Perkins, 2014). Teacher education plays an important role in building the foundations for teacher’s AL and as a basis for continuous development throughout a teacher's career (DeLuca & Braund, 2019).

The shaping of teachers’ AL is comprehensively described using the TALiP framework (Teacher Assessment Literacy in Practice, Xu & Brown, 2016). According to the framework, a teacher's AL can be thought of as a complex identity work in which the teacher builds their assessment concepts on the basis of diverse assessment information and pedagogical content information, which in turn is influenced by e.g. curricula. Teachers’ perceptions of assessment are shaped by the influence of the teacher's perception of learning as well as the cognitive dimension and the affective dimension. Assessment concepts are structured as assessment practices and are influenced by, for example, macro-cultural socio-cultural conditions related to the school system and, at the micro-level, for example, the traditions and customs of the school or work community. According to the framework, teachers ’own assessment practices are ultimately the result of a number of compromises that the teacher has to make with his or her own knowledge, perceptions, beliefs, and external pressures. The teacher’s assessment identity is built through the teacher’s experiences, learning attitude and reflection, and inclusion and interaction. At its best, it develops throughout teacher’s professional career. (Xu & Brown, 2016; DeLuca & Braund, 2019)

In this study, we explore how assessment literacy (AL) of teacher students is constructed through self- and group-assessments and self- and group-reflections. The research questions are

RQ1: What kind of challenges and opportunities teacher students reason for when reflecting on their development of assessment literacy?

RQ2: What do teacher students consider most significant impact on in developing their assessment literacy?

Rapid Automatized Naming, (RAN, a measure of the ability to name aloud objects, colours, digits or letters) and phonological awareness are two reading-related measures that are shown to predict early reading ability strongly and reliably (e.g., Moll et al., 2014; Caravolas et al., 2019). However, the relevance and predictive power of each individual measure is still under debate. This study focuses the influence of RAN on early reading ability, problematizing to what extent RAN contributes to assessments aiming to predict reading ability. The main aim, however, is to problematize how this knowledge is best to put into practice. Thus, when is RAN relevant to use as an indicator of reading ability and when are other indicators more relevant?

Because of the efforts being made to identify children, at an early stage, who are at risk of reading difficulties children are in school screened for their reading ability. But screening children is both time-consuming and costly, and therefore it must be well thought out what is being screened for, how to interpret results, but also how are the results to be used and implemented in the school's operations? Thus, given that children's time in school is limited, any activity that focuses on reading achievement must in one way or another be based on knowledge established to improve reading. Hence, the time it takes to screen children needs to be balanced against the time it takes from teaching.

In previous research, RAN is claimed to be a measure of phonological processing time and reflects how fast representations can be retrieved from long-term memory (Bowey et al., 2005; Torgesen, et al., 1997). Further it has been debated whether RAN and phonological awareness each contribute unique information to early reading, or if the measures will be merely two ways of measuring one ability, phonological processing. For example, Chiappe et al. (2002) found that most of the variance contributed by RAN to reading ability is shared with phonological awareness. Further, and in line with the understanding of RAN and phonological awareness being two sides of the same coin, it is claimed by Ziegler et al. (2010) that RAN will only become the dominant predictor when phonological awareness tasks are not challenging enough.

However, another view is that RAN and phonological awareness are two distinct measures predicting early reading ability (Torppa et al., 2013). In a cross-sectional study de Groot et al., (2015) compared reading disabled children with more skilled readers and found that for the reading disabled children the combination of RAN and phonological awareness showed the highest predictive values. When comparing the effects of phonological awareness and RAN on reading ability phonological awareness appears to be the best predictor of reading disability whilst RAN is indicated to be the best predictor of above-average to excellent reading ability (de Groot et al., 2015).

Other longitudinal studies show that the predictive power of RAN and phonological awareness on reading appears to change by age (e.g. Kirby at al., 2003; Vaessen & Blomert, 2010) and hypothesized by some researchers to be connected to the reading strategy used (Vaessen & Blomert, 2010; Rodriguez et al., 2015). Reading development is assumed to shift from a slow sequential phonological decoding to an automatic orthographic processing (Ehri, 2005).

In order to investigate the relevance of RAN in reading assessment, the present study measured RAN repeatedly among a group of children who were followed from kindergarten through their time in elementary school.

An abundance of research has established that phonological awareness skills are important prerequisites for early reading acquisition (for a review, see Melby-Lervåg et al., 2012). Early develop­ment of phonological awareness implies that a child moves from implicit to explicit control of the sound structure of language, and this explicit control is critical when a child learns to understand and handle the alphabetic principle (e.g., Caravolas et al., 2013; Lundberg et al., 2010). Accordingly, there has been a long tradition of research on phonological training to prevent failure to acquire reading skills. Early examples of such studies are Bradley and Bryant (1983) and Lundberg, Frost and Petersen (1988), and results from later training studies have been summarized in several meta-analyses (e.g., Bus & Van Ijzendoorn, 1999, National Early Literacy Panel, 2008). However, even though the results of the training studies show positive effects, Torgesen (2000) found that around two to six percent of the participants in phonological interventions could be defined as “treatment resisters”. Thus, some children do not grasp the idea of phonemes as discrete entities, and they do not seem to enhance their phonological skills to an acceptable level by the training.

Most phonological interventions have been carried out in combination with, or just before, formal reading instruction starts, and studies have typically investigated development over short periods (Kjeldsen et al., 2019). In the present study phonological awareness training was carried out when children were four and five years old (school starts at age 7 in Sweden). The intention was to begin the study at this early stage when children’s explicit awareness of the structure of speech starts to emerge (Wolff & Gustafsson, 2015; Dodd & Gillon, 2001). The training addressed different aspects of phonological awareness, gradually moving from games and exercises with morphemes and syllables to phonemes. Explicit training of phoneme/grapheme mapping (National Reading Panel, 2000) was later introduced when children were six years old, one year before formal reading instruction started. This training was given to all children regardless of whether they belonged to the experimental or control group. Thus, since everyone received the six-year-old training, the potential effects in this study are derived from the early training at ages 4 and 5.

General fluid intelligence (Gf) is a core concept in the field of intelligence. It is interpreted as the capacity to solve novel, complex problems. Gf is highly correlated with phonological awareness in 4-year-old children (Wolff & Gustafsson, 2015), and both phonology and Gf have been found to relate to early reading ability. de Jong & van der Leij (1999) found that when Gf was controlled for, the relation between phonology and reading decreased, and the direct effect of Gf on reading decreased over time. These findings support the hypothesis that the influence of Gf on early reading skills is mediated through the development of phonological awareness. Thus, we may expect that children with high Gf typically will have a more favorable development of phonological awareness skills. One important question here is if the phonological training will decrease or increase this putative influence.

The research questions are: 1) Does structured phonological awareness training starting at the age of 4 affect reading related skills ten years later in grade 8? and 2) Are there differential effects of phonological awareness training as a function of children’s cognitive abilities? The present study thus aims at extending on the rich knowledge of effects of preventive phonological interventions preceding reading instruction. As to our knowledge there are very few previous studies which investigate long-term effects of phonological awareness training during a ten-year period.

A high percentage of students have considerable reading deficits, not only in Germany, but also in other European countries (Betthäuser et al. 2023; Wallner-Paschon et al. 2017). Therefore, formative diagnostics are essential to record the current learning levels of students so that specific interventions can be shaped at an early stage. This discourse has been intensified in Germany as a result of the significant increase in heterogeneity at primary schools and the disconcerting learning deficits of a large number of primary school children after the COVID-19 pandemic (Stanat et al. 2022). As a result, the German Standing Scientific Commission of the Standing Conference of the Ministers of Education and Cultural Affairs of the Federal States recommends the early intensification of nationwide diagnostics and the "provision of scientifically based, quality-assured diagnostic instruments and related support instruments" as formative diagnostics to ensure basic competencies (Köller et al. 2022, p. 74). Additionally, the Progress in International Reading Literacy Study (PIRLS), is not only conducted through a computer-based assessment since 2016, but also has a focus point in digital forms of reading (Mullis & Martin 2019).

In the collaborative BMBF-funded project DaF-L (digital everyday integrated supportive diagnostic - reading in inclusive education) an adaptive, digital and competence-oriented reading screening with adapted reading packages, consisting of reading texts and reading exercises, is being developed, tested, standardized, and subsequently made available as OER on the online learning application Levumi for the third grade of primary schools in Germany. For the project partners of four German universities an essential component in the development and improvement of digital applications is the cooperation between practice and research. Interviews with experts will be conducted in order to advance the reading packages as well as Levumi. The goal is to ensure a direct transfer of research to practice by examining the ecological validity and usability of the instrument as well as the professional development of the partners in the schools with regard to this unique form of the adaptive approach. Ensuing this, the objective is to enable low-threshold, data-based, and effective reading support, to identify conditions for the success of everyday support-based diagnostics and to improve the conditions for inclusive education in primary schools.

The reading packages for promoting reading comprehension in inclusive classes contain reading texts on three ability levels and reading tasks tailored to them. The reading packages are intended to promote the reading skill reading fluently and the reading abilities reading comprehension and strategies for reading comprehension. The ability to read fluently implies that the students can read “quietly, aloud, automatically, accurately, meaningfully, and quickly (KMK 2020, p. 16).” The students have to read texts and solve reading exercises; therefore, they read both repeatedly (Mayer 2018) and a lot (Kruse et al. 2015), thus promoting reading fluency. In the case of reading comprehension, the students read texts that correspond to their ability level and understand the meaning. Skills involve students identifying textual information at the local level, either explicit or gleaned through simple inferencing. In doing so, they also pay attention to linguistic means to ensure the context of the text as well as link text information, draw conclusions, and construct an overall understanding using their previous knowledge (KMK 2020). In the case of possessing strategies for reading comprehension or reading strategies, the students know to use basic cognitive and metacognitive reading strategies after reading. They work with after-reading strategies such as central text statements (KMK 2020).

The global question of how students can benefit from digital reading packages and how the usability of the application can be improved through collaboration between educators and researchers will be explored.

The educational inclusion of children with chronic disorders – such as developmental and physical disabilities – is a well-studied area, across Europe and globally. The siblings of these children, however, are less studied, particularly in relation to their educational experiences and outcomes. Siblings of children with chronic disorders have divergent, and often adverse, life experiences. Some siblings may experience positive outcomes, such as increases in prosocial functioning (Orm et al., 2022). Overall, however, siblings are at risk for negative psychological effects including emotional and behavioural problems (Havill et al., 2019; Vermaes et al., 2012). Reduced psychological well-being, in turn, can affect siblings’ educational experiences, functioning, and outcomes (Gan et al., 2017).

Studies of siblings of children with chronic disorders (herein “siblings”) tend to utilise mainly parent and self-ratings on measures of psychosocial well-being (Hayden et al., 2019). While these perspectives are important, they offer limited insight into the functioning of siblings within a school environment, particularly in the case of parent-rated measures. Given the centrality of school to children’s daily lives and the importance of education outcomes to later life outcomes, it is essential to consider the perspectives of teachers on sibling well-being. The present study aims to address this gap by examining:

1) Teacher ratings for siblings on the Strengths and Difficulties Questionnaire (SDQ)

2) Agreement between teacher and parent ratings on the SDQ, and

3) Factors that may explain disagreement between raters.

Previous studies of child psychosocial functioning – generally, rather than specific to siblings of children with chronic disorders – have found low to moderate levels of agreement between teacher and parent ratings on the SDQ (Murray et al., 2021). This pattern is also found across different measures of child psychosocial functioning. Across these studies, teachers tend to report fewer problems than parents, particularly in the case of internalising problems. Rather than signalling poor reliability of measures, however, the discrepancy between teacher and parent ratings suggests that children’s behaviour is context and rater-specific. Teacher and parent ratings may be seen as complimentary pieces of a larger picture and understanding differences can facilitate better targeted interventions (De Los Reyes et al., 2015). A number of factors may explain a lack of interrater agreement on the SDQ and other measures of child psychosocial functioning. Factors of the home environment, for example, can influence the level of agreement between raters; family stress has been found to be associated with less agreement between teacher and parent ratings while positive parent-child relationship is associated with more agreement (Cheng et al., 2018).

Overall, little is known about the educational experiences of siblings of children with chronic disorders. Furthermore, the SDQ is widely used across European countries and globally. A better understanding of the conditions under which teacher – parent agreement tends to be higher and lower will help researchers and practitioners to interpret SDQ ratings and target solutions and support accordingly.

All networks hold a meeting during ECER. All interested are welcome.

Sweden’s self-image as a leader in education was rocked in the early 2010s by the so-called PISA-shock, in which this formerly high-flying education system saw its performance in international assessments dramatically decline. This period of decline dominated the public and political discourse around education and reform in Sweden (e.g. Lundahl & Serder, 2020) for the rest of the decade. Over time, Sweden’s performance in Mathematics has recovered, as evidenced in numerous international assessments, including most recently TIMSS 2019 and PISA 2018 (e.g. Mullis et al., 2020; OECD, 2019a). Nevertheless, the improvement in the overall achievement of Swedish youth somewhat masks a persistent achievement gap which has been observed within the Swedish school system since the early 2000s, with growing variation in performance between schools in student grades (Skolverket, 2005, 2020). The achievement gap noted in domestic data has also been recorded in international data, with the achievement gap widening (Chmielewski, 2019) and Sweden’s decline in socioeconomic equality of outcomes the most severe among peer nations (Hanushek et al., 2014).

Socioeconomic status is a well-established predictor of educational outcomes (e.g. Sirin, 2005), and previous research using TIMSS data has confirmed this relationship in relation to mathematics outcomes for Swedish youth over multiple cycles of TIMSS between 2003 and 2015 (Authors, 2021). A longstanding strand of scholarship suggests that in addition to predicting achievement, socioeconomic background indicates varied opportunity to learn (OTL) course material, which in turn predicts test performance (e.g. Eggen et al., 1987). While this pattern of relationships has been consistently evidenced in the English-speaking world (e.g. Authors, 2021; Schmidt et al.,2013), in the Swedish context inequalities of opportunities have been inconsistently observed, appearing only among the 2003 and 2015 TIMSS cohorts (Authors, 2021).

A possible explanation for the lack of observable social reproduction through the delivery of the curriculum lies in the nature of the Swedish school system. A distinctive feature of the Swedish education system is its retention of the comprehensive model in which students are offered equal learning opportunities in integrated school settings (Arnesen & Lundahl, 2006) with limited within-school streaming when compared with other highly developed economies (Chmielewski, 2014). Of much interest to policy makers and researchers, reforms to the Swedish education system enacted in the 1990s introduced school choice and created a market for education (see Björklund et al., 2005). While admissions guidance prohibits cream-skimming (Põder et al., 2017), the exercise of school choice is socially segregated (Teske & Schneider, 2001) and the subsequent composition of schools can be interpreted as reflecting segregation beyond an expected neighbourhood effect (Böhlmark et al., 2016). Despite the observed social segregation between schools, analysis of international data suggests that the comprehensive school system in Sweden is still intact, with students of varying abilities attending the same schools, and that variation in performance between school is low when compared to other economies (OECD, 2019b).

Against this background, the following research questions are considered:

1. Are between school socioeconomic inequalities in mathematics outcomes and opportunity to learn mathematics observable among eighth graders in Sweden?
2. Do the relationships between socioeconomic status, opportunity to learn, and achievement very between high-, neutral-, and low-SES schools?

Social integration in the context of increasing immigration is a challenge faced by many industrialized countries. The institutional set-up of an education system is a natural candidate for scrutiny from a policy perspective, since education is at the forefront of social integration and institutional structures are malleable in nature. In this study, we examine immigrant social integration through school segregation and how it relates to the institutional structure of an educational system, in particular the existence of (early between-school) tracking. As a premise for integration, school segregation is crucial since it determines how much interaction between immigrants and non-immigrants occurs. Tracking on the other hand is as controversial as it is momentous since students are placed into different types of schools from an age as young as ten (e.g. Terrin & Triventi 2022, van de Werfhorst & Mijs 2010). This is in stark contrast to the practice of integrated education systems, in which students may be grouped by ability for certain topics or classes, but are only separated as they approach maturity.

We therefore examine whether tracked education systems show higher levels of school segregation of immigrants as education systems that delay between-school grouping. While sociologists have long documented the negative effects of tracking with regards to equality of opportunity (e.g. Terrin & Triventi 2022, van de Werfhorst & Mijs 2010), we argue that there may be counteracting mechanisms at work in tracked systems when it comes to ethnic segregation between schools.

We follow theories on educational inequality to understand school segregation and tracking. These theories relate differences in family resources to differences in educational attainment or achievement (i.e. Boudon 1974, Bourdieu 1987, Lareau 2011). Resources in this context can comprise economic capital, strategic knowledge, social contacts and familiarity with modes of behavior in the education system. In this context, we expect that immigrant students are disadvantaged, as many of these resources cannot easily be translated from the home country to the receiving country. We can therefore expect that they will show, on average, lower achievement at the end of primary school (i.e. primary effects; Boudon 1974). These finding has been shown by numerous studies (e.g. Heath et al. 2008).

Since observed achievement is a major indicator of track placement, primary effects of ethnic and social origin increase the likelihood for immigrant students to be sorted into lower secondary school tracks. However, parental decision making (i.e. secondary effects; Boudon 1974) is another determinant of track placement and it is well-known that immigrant parents tend to choose more ambitious educational pathways (Esser 2016; Gresch et al. 2012) which could compensate for low track placement based on ability. Lastly, school segregation likely exists in non-tracked systems as well. First, because home-to-school-distances are a main factor in selecting a school, residential segregation, which is a common phenomenon in many countries, is reflected in school segregation. Further, school choice behavior of non-immigrant families may contribute to ethnic school segregation, as particularly high status families tend to avoid schools with larger numbers of immigrants (“white flight”; Amor 1980). They do so, because they use immigrant concentration as a proxy for (lower) school quality. This tendency might be lower in tracked-systems, as track level is an accessible indicator of school quality. Non-immigrant families therefore do not need to avoid schools with larger numbers of immigrants (c.p. Meier & Schütz 2007).

In sum, there may be counteracting mechanisms with regard to school segregation and the age of first tracking. We therefore argue that it remains an empirical question to determine which mechanism outweighs the other.

This study examines the impact of the Relative Age Effect (RAE), measured by students´ birth month, the cognitive ability, school achievement and socioemotional competencies. A longitudinal approach is applied by using data from a Swedish cohort of students born in 1972, from 3^rd Grade (age 10) until the end of upper secondary school (age 19).

When Swedish students begin their first school year in August every year (age 7) some children are almost one year older than some of their peers in the same year level. This is due to that the school entry cut-off date is 1^st January.

Being among the oldest or youngest in a group of students has been shown to have effects on many outcomes throughout school trajectory. Being relatively young has been identified as a negative impact on cognitive and maturity issues (Rod Larsen & Solli, 2017). While being relatively older when starting school seems to affect achievement, working life and later outcomes positively. Outcomes such as higher achievements and reaching higher educational attainment (Crawford et al. 2014). You are also more likely to participate in high school leadership activities (Dhuey & Lipscombe, 2008) and being more successful in sports (Gibbs et al., 2012). Further, the literature shows the existence of physical advantages of being relatively old which gives advantages in individuals identification processes during their upbringing (McCarthy et al., 2016). While mixed results exist on the impact of relative age on earnings (Black et al., 2008; Fredriksson & Öckert, 2014).

However, the economic literature has found a reverse age effect (RAE), suggesting that being older when starting school is beneficial for earnings earlier in the working career while being younger is beneficial for earnings later in the working career (McCarthy et al. 2016).

The effect of relative age on noncognitive outcomes such as self-concept, self-confidence self-esteem, coping and resilience strategies is also evident (Duckworth et al. 2007; Dweck, 2006). Findings from several studies show that children and adolescents being relatively old in the school cohort become influenced in their self-confidence, self-beliefs, and social interactions in school positively (Crawford et al., 2014). Further, it has been shown that relatively old children have higher self-esteem (Thompson et al., 1999, 2004) and suffer less from psychological and behaviour problems (Muhlenweg et al., 2010) compared to relatively younger students.

Even though research has shown that many socioemotional competencies seem to be affected by RAE some may be more crucial for success in learning such as coping and resilience strategies (Duckworth et al., 2007; Dweck, 2006).

This study contributes to the research field by providing empirical support for long-term consequences of relative age in school on cognitive ability, school achievements and noncognitive competencies in terms of students´ academic self-concept, coping and resilience strategies.

Purposes

The main aim of this study is to examine the importance of the relative age effect, measured by birth month, for students´ cognitive and socioemotional outcomes by using a longitudinal approach. Following research questions will be investigated with longitudinal data from several time points:

How does relative age affect cognitive outcomes in terms of cognitive ability, GPA, and educational attainment?

How does relative age affect socioemotional outcomes in terms of perceived academic self-concept, coping and resilience strategies?

What are the long-term effects of relative age for cognitive and socioemotional outcomes and for subgroups of students related to gender and family socioeconomic status?

It is widely accepted that teachers are one of the most important school-level inputs for student academic success. Most educational research focuses on teacher effectiveness in terms of their contribution to student test scores, but there is a growing need to examine the teaching profession as an outcome in itself. A teacher workforce characterized by high turnover rates will not only negatively impact schools via administrative burdens as well as students and their educational futures, but also the teachers themselves via their working conditions and professional satisfaction.

There is a longstanding link between low-SES schools and teacher turnover, but this literature mostly comes out of the USA, with some exceptions. Disproportionate teacher turnover rates often affect lower-SES schools and classrooms in particular (Bacolod, 2007; Bonesrønning, Falch & Strøm, 2005; Hanushek et al., 2004; Feng, 2009; Glassow, 2023), impeding organizational and administrative school functioning, and potentially contributing to longer-term student behaviours such as college attendance and high school completion (Jackson, 2018). Moreover, high turnover rates may be symptomatic of worsening working conditions and professional satisfaction which have been documented in a number of education systems (Ball, 2016; Craig, 2017).

There is therefore a need to document the extent to which teachers mirror socioeconomic demographics of schools and concrete ways in which to democratize access to teacher competence in Sweden. This is a pertinent issue due to the demographic changes occurring in Sweden over the past several decades, the rising school inequality in the country (Karbownik, 2020; Yang Hansen & Gustafsson, 2016). The present study seeks to contribute to this gap in knowledge and examine whether changes in school composition (by family education level or language spoken by the students) results in changes in teacher turnover rates. Using teacher and student register data, the study first examines in a descriptive fashion whether there are growing differences between schools in terms of teacher turnover rates. Next, using a panel data model, the link between changes in student school composition and teacher turnover are explored. Whether or not causal conclusions can be made from such an approach will also be explored in the paper.

Allensworth, Ponisciak and Mazzea (2009) outline several main reasons teachers cite their dissatisfaction with certain schools: principal effectiveness, dysfunctional administration, challenging students, low salary, and limited autonomy which may be due to additional accountability practices. Vagi and Pivarova (2017) consolidate the literature employing theoretical frameworks for teacher mobility and offer person-environment fit theory (Dawis, 1992) as a theory which may encapsulate the myriad of environmental and personal factors which may be relevant for teacher mobility. While the focus of the study is on the role of socioeconomic composition of schools and classrooms in teacher mobility behaviours, person-environment fit theory allows for an accurate estimation of factors which may bias results unless they are under control, or unless proper methodologies are used which account for unobserved heterogeneity. Dawis (2004) highlights that job satisfaction or work stress are the result of successful or mismatched employees, respectively.

Against this background, the main research questions of the study are:

1) Are between school turnover rates growing in Sweden over the past several decades?

2) Do changes in socioeconomic and migration demographics of schools result in higher turnover rates? Specifically, do schools with a higher proportion of students with main languages other than Swedish exhibit a significantly higher proportion of teachers who leave?

3) Does this change depending on teacher qualifications? For example, are more experienced teachers more or less likely to leave as a result of these changes?

Whether or not teacher training equips teachers with the professional knowledge and competence they need to deliver high-quality teaching has been an important area of debate in recent decades (Darling-Hammond, 2016). Research has shown that teachers develop their knowledge, competence, and skills through teacher education and subject-specific specializations during teacher training (Coenen et al., 2018; Hill et al., 2019) as well as through years of teaching experience (Coenen et al., 2018) and professional development (Hill et al., 2019). There is evidence too that teachers play a key role in influencing student learning and achievement (e.g., Coenen et al.). However, despite the best of formal qualifications the conditions for the working environment and teachers’ job satisfaction can affect how teachers exercise their competence in classrooms (Collie et al., 2012). Indeed, teachers’ professional competence is recognized as a multi-dimensional construct consisting of a broad range of cognitive and affective aspects of teacher characteristics that interact with teacher work (Blömeke, 2017). For this reason, the importance of the working environment and working conditions cannot be overlooked as it has been shown in previous research that teachers’ workload affects teacher job satisfaction (Toropova et al., 2021). Teachers’ job satisfaction, in turn, is suggested to influence teacher instruction and the learning support offered to students (Klusmann et al., 2008). In addition, teachers’ working environment, in terms of greater classroom autonomy and fewer disciplinary problems (Nguyen et al., 2020) as well as the attractiveness of the teaching profession are suggested to be factors influencing whether teachers remain in the profession or not (Viac & Fraser, 2020). Furthermore, research also shows that school socio-economic status (SES) is associated with teachers’ working conditions and well-being. Teachers working in schools with a lower socio-economic status report not only higher mental workload but also poorer well-being (Virtanen et al., 2007). Considering that working conditions and job satisfaction are associated with important teacher and student outcomes more studies that examine relationships within this area are needed.

Many European countries struggle with changes in recruitment to the teaching profession, declining status of the teaching profession, and increasing teacher turnover (eg., Skaalvik & Skaalvik, 2011). In Sweden, these issues are perhaps particularly pertinent (Alatalo et al., 2021; Holmlund et al., 2020) since Sweden also faces increasing school segregation and increasing achievement gaps (Yang Hansen & Gustafsson, 2016). Against this background, the present study aims to investigate factors related to teachers’ workload and job satisfaction in Swedish compulsory schools. Our theoretical point of departure is based on Blömeke’s (2017) modelling of teachers’ professional competence. Teacher competence is modelled as a multi-dimensional construct where all teacher resources play together to deal with the demands and challenges of the classroom. We investigate the relationships between different teacher characteristics and working conditions and teachers’ sense of job satisfaction. We hypothesize that teachers with more experience and a subject-specific specialization in mathematics have higher job satisfaction. More time in the profession may have helped teachers to find coping strategies but not only that, more specialized teachers are likely to work with the subject and grade they are trained for. This in turn could reduce the workload and increase the sense of job satisfaction. Moreover, we hypothesize that schools’ socio-economic composition is associated with teachers’ sense of workload and job satisfaction. More specifically, our research questions are:

1.) To what extent are teachers’ characteristics related to their working conditions and job satisfaction?

2.) Do teachers’ sense of workload and job satisfaction vary depending on students’ socio-economic background?

Mathematics competence plays a crucial role in solving problems, developing analytical skills, and providing the essential foundation to build knowledge in understanding the content of other school subjects. However, international large-scale assessment (ILSA) studies have pointed to significant cross-country variation in students’ mathematics competency levels and their motivation to learn mathematics (Mullis et al., 2020).

Students’ motivation is seen as the driving force behind their learning of mathematics over time (Wigfield et al., 2016). This is coupled with more recent ideas on the need to support strong mathematics self-efficacy (Parker et al., 2014) and positive academic emotions. Expectancy-value theory points out that achievement-related choices are motivated by a combination of students’ expectations for success and task value in particular domains (Eccles & Wigfield, 2020). Control-value theory focuses on the emotions experienced while students are involved in an achievement activity, such as the succeeding or failing emotions that arise as an outcome of an achievement activity (Pekrun et al., 2017). Indeed, empirical evidence speaks in favour of a significant relationship between teachers holding beliefs about their students’ learning, teaching a particular subject or its nature, and student motivation and their academic achievement (Muis & Foy, 2010). Given the influence of teachers’ beliefs, research has found that students report gender-stereotyped teacher ability expectations, particularly in domains of mathematics (e.g., Dickhauser & Meyer, 2006; Lazarides & Watt, 2015). Meanwhile, motivation differs in specific-domain and genders, such as mathematics learning motivation (Eccles & Wigfield, 2020).

Notwithstanding, the positive relationship between motivation and achievement in mathematics has been confirmed (e.g. Garon-Carrier et al., 2016), yet different theoretical perspectives have led to diverse ways of capturing motivation, and thus different strengths and directions of the relationship (Pipa et al., 2017). Our review of the literature found that, while many studies have measured and incorporated motivation, the nature of the relationship between teacher beliefs and motivation, for example, whether gender mediates this relationship, remains unclear. This is particularly important in the context of motivation and its development, given motivation is also seen as an essential outcome of learning. The present study is designed to investigate the relationship between teachers’ beliefs of the nature of mathematics and different aspects of students’ motivation following the Expectancy-value (Eccles & Wigfield, 2020) and enjoyment of mathematics (Pekrun et al., 2017), focusing on gender differences in motivational patterns. Building upon the conceptual framework and research objective, we focus on the following research questions (a) Do teachers’ beliefs about the nature and learning of math affect students’ motivation and enjoyment, taking into account students’ math achievement and classroom composition? (b) Are these mechanisms different between boys and girls?

Standardized testing is a manifestation of the neoliberal agenda and human capital theory (Rizvi & Lingard, 2010). Testing is perceived as one of the instruments to hold teachers and schools accountable for students’ performance which can lead either to rewards or sanctions. Standardized testing is one of the means of implicit control and governance that allow policymakers and politicians to audit the education system (Graham & Neu, 2004). Critics of standardized testing argue that it widens the gap between different groups of the student population (Au, 2016), encourages teachers to teach to the test and ignore the unassessed curriculum content and other subjects (Lingard, 2011; Koretz, 2017; Bach, 2020), and facilitates the practice of gaming the system to illustrate the growth in student performance (Rezai-Rashti & Segeren, 2020; Heilig & Darling-Hammond, 2008). Despite the severe criticism of standardized testing, it still can be used as an effective tool to inform teaching and learning. Testing can help curriculum designers, test developers, teachers, and educators identify students’ needs and tailor instruction in relation to those needs (Hamilton et al., 2002; Brown, 2013; Singh et al., 2015). It can also allow policymakers to evaluate the success and efficacy of the education system and identify the potential issues that could be addressed (Campbell and Levin, 2009).

The purpose of the study is to construct and validate reading assessments that account for the local contextual factors such as curriculum standards and expectations and that could provide formative information to students and teachers. The current research study includes several stages: pre-pilot study, pilot study, and main studies. In this abstract, the results of the pilot study will be presented.

The study aims to answer the following research questions:

What are the students' perceptions of the proposed testing instrument?

What are the psychometric properties of the pilot test?

The theoretical framework that guides my research is entitled evidence-centered design by Mislevy and Riconscente (2006). ECD employs the concept of layers where each layer possesses its own characteristics and processes. The goal of domain analysis is to collect substantive information about the target domain and to determine the knowledge, skills, and abilities about which assessment claims will be made. Domain modelling organizes the results of domain analysis to articulate an assessment argument that links observations of student actions to inferences about what they know or can do. Design patterns in domain modelling are arguments that enable assessment specialists and domain experts to gather evidence about student knowledge (Mislevy & Haertel, 2006). The third layer – conceptual assessment framework - provides the internals and details of operational assessments. The structure of the conceptual assessment framework (CAF) is expressed as variables, task schemas, and scoring mechanisms. This layer generates a blueprint for the intended assessment and gives a concrete shape to it. Assessment implementation constructs and prepares all operational elements specified in CAF: authoring tasks, finalizing scoring rubrics, establishing parameters in measurement models, and the like. The assessment delivery layer is where students are engaged with the assessment tasks, their performances are evaluated and measured, and feedback and reports are produced. Thus, ECD provides an essential framework for approaching test design, scoring, and analysis. In my study, the ECD framework will act as guidance to ensure that each layer is constructed, and relevant evidence accumulated.

All model assumptions in the natural sciences are based on mathematical concepts, regularities or assumptions. For this reason, mathematical modelling is central to understanding the development and validation of models in the natural sciences. The ability to evaluate, change and apply models in the sense of gaining knowledge is understood as modelling competence. With the help of modelling cycles, the modelling process can be divided into individual steps. This enables an insight into the modelling process.

Blum & Leiß (2005) developed a framework for mathematical modelling. They distinguished between two main dimensions, "rest of the world" (which includes real-world problems, their structuring, mathematical description, and the interpretation and evaluation of mathematical results) and "mathematics". The translation between these dimensions is understood as mathematical modelling. Based on these dimensions, a seven-step modelling cycle was developed. Starting from a real situation/problem, the steps are to understand the situation (1), to simplify and structure it with a focus on the problem (2) followed by mathematisation (3), which results in the transition to the dimension of "mathematics". There, results are generated with mathematical methods (4) and translated back into the context and thus back into the dimension "rest of the world" with a focus on the problem (5). Now these results are validated in relation to the context (6) and an answer is given to the concrete problem (7).

Based on the cycle for mathematical modelling developed by Blum & Leiß (2005), various subject-specific modelling cycles were derived. Goldhausen & Di Fuccia (2014) derived a mathematical modelling cycle for the subject of chemistry. For this purpose, an additional dimension "chemistry" was added that is located between "rest of the world" and "mathematics". This is necessary because a real chemical situation (e.g. chemical experiment) must first be transferred into subject-specific models in order to be able to describe and interpret a situation.The steps of the mathematical modelling cycle were adapted to the specific requirements of a chemical contextualisation. In the first step, a problem/experiment is identified on a macroscopic level and a situation model is created (1). This is then translated into a chemical model (submicroscopic or symbolic level) ( Johnstone, 1991) (2). The chemical model is then mathematised (3), for which, according to Kimpel (2018), a deeper understanding of the model is necessary. With the developed mathematical model, mathematical results can be generated with the help of mathematical tools, similar to Blum & Leiß (2005) (4). These can then be translated back into the chemical model (5) and checked for their professional usefulness (6) so that they can finally be applied to the experiment/problem (7).

As diagnostic models, modelling cycles offer the possibility of gaining an insight into the complex cognitive processes of learners during modelling. In the field of mathematics didactics, modelling cycles have been used to develop a test instrument to measure mathematical modelling ability (Haines, Crouch & Davis., 2001; Brand, 2014; Hankeln, Adamek & Greefrath, 2019). In all cases, the steps of a modelling cycle were divided into empirically based categories. Items were constructed for these categories. Prior to testing, various models were postulated on the basis of empirical studies for the items. With the help of Rasch measurement, the data has been compared with the postulated models.

Since this type of test development has so far only been conducted for mathematical modelling in general, this study investigated if a questionnaire can be used to assess learners' mathematical modelling skills.

“Will we have it in our summative?” … is quite a familiar phrase for English Language and Literature, and not only, teachers of Diploma Program (DP) in International Baccalaureate, isn’t it? While we are proud of our students’ performance and diligence, we are painfully aware that their academic achievements might have been gained at the cost of both teaching (curriculum, teaching methods, delivery) and learning (curriculum, content, life skills). This influence is also evident from the literature, where Gates (1995) defines it as “a washback effect”, i.e., “the influence of testing on teaching and learning” (p.102). Prodromou (1995), while strictly differentiating between ‘testing’ and ‘teaching’, suggests a negative effect of backwash on teaching, which he believes, greatly complicates it.

Although external exams of English A and B courses (Paper 1, Paper 2 and Individual Oral) are barely considered to be tests, as they are full-fledged final written and spoken exams, we believe that the term “backwash effect” also carries some influence on language and literature learning and teaching. Teachers have to cover a huge amount of material within two years of DP, while students, along with other subjects, have to process this huge amount of material. So many exams and IB components, so little time. And no wonder, teachers are more and more leaning to the program to be optimised, which to our concerns, can lead to keeping only essentials.

Now, the question arises what the “essential” is and what is not. Hughes (1989) proposes the following to ensure positive backwash effect: test should develop certain skills; test content should be varied and cover wide-spectrum areas; it should have an effect of unpredictability; make both teachers and students understand the procedure of the test, and others. In the same vein, Bailey (2016) puts forward the following aspects: meeting language learning objectives and requirements, authenticity of the tests and samples, ensuring learner autonomy and self-assessment, providing feedback of the test results.

In our search for a balance between students’ academic achievements and their future non-academic life, there is a sinking feeling in our teaching practice that we might be overlooking the opportunities to develop their career/real life aptitudes and skills. Although there is a sufficient literature in negative effects of backwash of testing, such as IELTS, high stake tests, standardized testing (Paker, 2013; Watkins, Dahlin & Ekholm, 2005) and ways to turn it into positive one, we seek to explore the backwash effect of written and spoken exams on language teaching and learning. We believe that the assignments covered in Diploma Program are quite challenging and intensive. They require time-managements and analytical skills as leaners are asked to write a textual analysis on a given text type and a comparative essay based on at least two literary works they have studied in class (Language A). With regards to Language B, they have to produce the text type, which requires more than mere selection of the right answer from multiple choices. Despite such a difference in requirements of the final exams, we are still concerned about the fact that there are more exams in classroom than skills development, which is, we believe, a prevailing phenomenon in education.

With that in mind, the following research questions have been addressed:

1) What are both benefits and drawbacks of predominance of the exam preparation in English class?

2) How can the classroom be modified to prepare students for life (not just exams)?

Modern education is inextricably linked with three main concepts: learning, teaching and assessment. The teacher of the 21st century must equip students with solid knowledge that will help them in life. At present, the teacher directs and coordinates the work of students, and contributes to the development of students' skills of independence, self-criticism and the ability to find the necessary, reliable information in a huge flow of knowledge and information. Thus, the student must be able to find, analyze and apply the necessary knowledge from a numerous flow of information.

According to PISA research, “Kazakh schoolchildren have subject knowledge at the level of their reproduction or application in a familiar educational situation, but they have significant difficulties in applying knowledge in real life situations…” [1].

According to the results of the international PISA study for 2019, it was revealed that 9th grade students found it difficult to complete assignments for formulating scientific questions, which amounted to 11.6% and tasks for the scientific interpretation of evidence data, which amounted to 15.9%.

Taking into account the data of an international study, we conducted a comparative analysis of the curriculum in biology, and saw that from the 7th to the 9th grades, the number of hours for modeling various processes increases, according to certain topics and sections of the curriculum. So, for example, if in the 7th and 8th grades 5 hours are allotted for modeling, then in the 9th grade it is already 6 hours. Therefore, the teacher needs to develop the skills of working with models and apply this method of pedagogical approach to the organization of the educational process, starting from the 7th grade.

These results influenced the research and the search for solutions to the questions that arose.

The main research questions are:

-How to teach a student to master the relevant skills in life?

-How can subject knowledge be improved?

To answer questions, the teacher needs to think about teaching, think about new methods of assessing students' knowledge.

The updated criteria-based assessment model helps participants in the educational process to understand and define “At what stage of learning is the student?”, “What does the student need to do to achieve the expected results?”.[2]

The aim of the study: the use of modeling in biology lessons will help students improve the quality of knowledge and will contribute to the development of functional literacy and creative thinking skills.

The presented work is the result of many years of experience of the authors, which has been tested in the educational process and supplemented in accordance with modern requirements. We sincerely hope that the methodological recommendations in the work will help teachers to increase the effectiveness of the educational process in order to educate competitive students.

Like many other European countries Austria is currently implementing a nationwide Quality Management System for schools (QMS; https://www.qms.at/). Its aim is systematic and targeted school and teaching development based on a quality circle of plan–do–check–adjust (PDCA, ‘Shewhart Cycle’) and thus similar to other countries, especially some German Länder with which there is also a continuous exchange. The most important features are the introduction of a mandatory quality framework for all schools as well as an increased data or evidence orientation in school and teaching development in general. QMS-tools include the definition of a school’s pedagogical guiding principles, a school development plan, a balance and target agreement meeting between the principal and the regional school quality manager (formerly ‘school inspector’) and a quality handbook. To support the data orientation, an internet platform with several hundred instruments for internal evaluation was provided. Formerly different quality management programs for general and vocational schools (and thus different traditions and instruments) are being merged into QMS.

The implementation process and the diffusion of the QMS and its elements into the school system are formatively evaluated in an accompanying process (Rossi et al., 2019; Stockmann, 2011). The overall objective of this evaluation is the generation of knowledge for the optimization of the implementation process as well as its monitoring and documentation of progress.

The theoretical background of this research is based on Rogers’ (2003) ‘Diffusion of Innovations’ describing typical stages of immersion. Accordingly, knowledge of the innovation is the starting point, in concrete knowledge of the QMS model, which should subsequently lead to a positive attitude or acceptance (persuasion). The next stages are the informed decision of the actors to adopt the innovation (decision) and the actual implementation, which in the best case leads to reinforcement and confirmation. Coburn (2003) focuses attention on the depth of change, its sustainability and ownership in the medium and long term, although these are of little importance in the initial phase. A closer look at the context of implementation and the creation of necessary framework conditions follows the approach of implementation research (Petermann, 2014).

The design of the evaluation and the underlying theoretical assumptions lead to the following three guiding questions:

• How deeply has the nationwide Quality Management System already diffused into everyday school life?
• How can the implementation process be further promoted and supported?
• Are there different patterns in this respect in different school sectors, specifically between general education and vocational education?

According to the underlying model of Utilization-Focused Evaluation (Patton, 1997, 2000), the detailed questions are defined in the further course of the evaluation in close coordination with the persons responsible for QMS.

Past findings concerning whether and how feedback from external evaluations benefit the improvement of schools are inconsistent and sometimes even conflicting, which highlights the contextual nature of such evaluations and underscores the importance of exploring them in diverse contexts. Considering that broad international debate, we investigated the use and impact of feedback from external evaluations in compulsory schools in Iceland, particularly as perceived by principals and teachers in six such schools. The research questions guiding the study was “How and to what extent do schools use the feedback presented in external evaluation reports?” and “To what extent do schools sustain the changes made after using the feedback from external evaluations instrumentally?” The framework used for analysing the evaluation feedback use was based on Rossi et al. (2004) and Aderet-German and Ben-Peretz (2020) and distinguishes between instrumental, conceptual, strategic and reinforcement-oriented use.

To reach objective 4 and 17 adopted by the General Assembly in September 2015, go through working together, teachers and students need to form a community of knowledge seekers and builders as the UNESCO (2021) affirms, so one of the ways to achieve these two sustainable development goals is to form communities of professional practice.

Thus, improving education is significantly dependent on the ability of school leaders to connect everyone (teachers, families and local community) into a professional community of practice, which increases their professional capital. It has three dimensions; human capital is acquired and useful skills of all inhabitants or members of society (Smith, 1776); social capital, those characteristics of social organization, such as trust, standards and networks that can improve the efficiency of society by facilitating coordinated actions (Putnam, 1995) and decision capital, is what professionals acquire and accumulate through structured and unstructured experiences, practice and reflection , capital that allows them to make judgements.

In complex contexts, a new governance of the school is required, with horizontal leaders (leadership from the middle) that build projects, cultures and environments with a community vocation of commitment to educational improvement, while expanding the social capital of the Professional Practice Community.

COMMUNITY OF PROFESSIONAL PRACTICE

Interestingly, Domingo-Segovia et al. (2020) have used the term of “Professional Practice Community” as a broader term to “professional learning community” that includes the school and the local community context. Community that requires the emergence of fluid networks of interrelationship, communication and support for the learning of all and for all, with a shared and networked leadership, articulated from a broad perspective of “middle leadership” (Rincón, 2019).

Therefore, as an “extended” community, the stage and actors must be linked and expanded with the collective goal of educational improvement, expanding networks of influences and opportunities. Hence the importance of the “increase of professional capital” (Hargreaves & Fullan, 2014), the next point to be discussed.

PROFESSIONAL CAPITAL AND LEARNING ANALYTICS

The key to this term is the systemic development and integration of three types of capital – human, social and decision-making – into the teaching profession. Professional capital has to do with collective responsibility, not individual autonomy; with rigorous training, continuous learning, going beyond the evidence, being open to the needs and priorities of students and society “ (Hargreaves & Fullan, 2012).

In addition, the use of learning analytics defined as “the measurement, collection, analysis and presentation of data about students and their contexts, in order to understand and optimize learning and the environments in which it occurs” acquires a relevant value as it can be used as a means to extract the most effective methodologies, processes and tools in data measurement, collection, analysis and reporting of professional capital (Khalil & Ebner, 2016).

Thus, The rationale for this work was to understand the structure of how learning analytics can help the improvement and understanding of professional capital in the field of education by analyzing its scientific output. For this purpose, bibliometric maps offering a better understanding of the structure of a scientific domain through the graphical representation of the different units of analysis and their relationships (Small, 2006)

Thus, this research will answer the following questions:

RQ1: What are the key themes or knowledge grouped around the use of learning analytics for the improvement of professional capital?

RQ2: What is the research trend of the field under study?

RQ3: What are the research boundaries extracted from the network analysis?

The steps and tools used in this process are explained more specifically in the methodology.

There has been a large number of studies on the performance gap between immigrant and native students (Arikan, Van de Vijver, & Yagmur, 2017; Martin, Liem, Mok, & Xu, 2012; Pivovarova and Powers, 2019; Rodriguez, Valle, Gironelli, Guerrero, Regueiro, & Estévez, 2020). It is critical to identify variables that could have relation with the performance of immigrant students. Numerous studies have been conducted on the variables that are related to immigrant students’ performance. Some studies emphasize immigrants' resilience (Rodriguez et al, 2020), while others focus on exposure to bullying (Karakus, Courtney, & Aydin, 2022; Ponzo, 2013). It was found that native students had higher scores than immigrant students on three indicators of wellbeing such as positive affect, self-efficacy-resilience, and a sense of belonging to the school (Rodríguez et al., 2020). Investigating factors at the student- and country-level that predict immigrant students’ performance may assist policymakers in taking education related action.

Thus, this study focus on identifying student- and country-level variables that are associated with mathematics performance of immigrant students using PISA 2018 data. In this regard, student-level variables are chosen based on Walberg’s theory (Walberg, 2004). According to Walberg's theory of academic achievement, a student's success is impacted by their characteristics and their environment. The main psychological factors influencing academic achievement were categorized into three groups. Student ability, instruction, and psychological environment are the categories. Student aptitude refers to a student's capacity, growth, drive, or predisposition for extreme perseverance in academic work. Both the quantity and the quality of the instructional time are part of instruction. Psychological environments refer to the morale or students' views of their peers in the classroom and the home environment. The morale of students or their perceptions of their classmates in the classroom and at home constitute psychological settings (Walberg, 2004). On the other hand, country-level variables are chosen based on research. Some research suggests that migrant integration policy index (MIPEX) is associated with achievement (Arikan et al, 2017; He et al., 2017). In addition to this, some research claims that the human development index (HDI) was found associated with achievement (Arikan et al., 2020).

The following research questions of the current study are

RQ1: Which student-level (motivation to master tasks, resilience, cognitive flexibility/adaptivity, exposure to bullying, sense of belonging, discriminating school climate, students’ attitudes toward immigrants) and country-level (MIPEX and HDI) variables could predict mathematics performance of immigrant students and native students across European countries in PISA 2018?

RQ2: Is there a statistically significant difference between the mathematics performance of first-generation immigrant students, second-generation immigrant students and native students across European countries in PISA 2018?

RQ3: Is there a statistically significant difference between the mathematics performance of first-generation immigrant students, second-generation immigrant students and native students after controlling economic and social status (ESCS)?

Today, societies in many countries are multilingual. Multilingualism can contribute to success in school and later in working life. An individual's language development affects their reading development (Kirsch et al., 2002). Thus, in a learning context, such as in school, reading skills are effective tools for obtaining, organizing, and using information in various fields (Artelt, Schiefele & Schneider, 2001). Reading is a multi-component process (Grabe, 2009) and many students during middle school have difficulty moving from “learning to read” to “reading to learn”. Fluency, previous knowledge, experience, and word knowledge are important since the students are expected to read about, for them, unknown subjects in which words and linguistic structures are more complex (Wharton‑McDonald & Erickson, 2016). For a student who reads the information at school in a second language (L2), the reading process becomes even more complex. Grabe (2009) summarizes the major overall differences between reading in a first language (L1) and a second language (L2): “Linguistic and processing differences, developmental and educational differences and sociocultural and institutional differences” (Grabe, 2009, p.130). Research has also shown that it takes at least four to five years before an individual can use their second language (L2) as a school language (Cummins, 2017; Thomas & Collier, 2002). In PISA 2018 the students in Sweden performed at a higher level on the reading literacy test, than the average for OECD countries. However, the groups of students with a foreign background, both those born in Sweden and those born abroad, performed at a lower level than native Swedish students (National Agency for Education, 2019). Educators in several countries have expressed concern about how education for first- and second-generation immigrant students is designed (Cummins, 2011).

In this study, PISA data from 2018 was used to investigate the patterns of variation regarding the components in PISA defined as reading fluently (reading fluency) and the students' perception of the usefulness of reading strategies regarding memorizing and understanding texts(UNDREM) (awareness of the usefulness of reading strategies) effects on reading literacy performance for multilingual fifteen years old students in a Swedish context. The aim is to get a better understanding of similarities and differences in the students’ component skills reading profiles (CSRP), in this study defined as learners’ relative development of different reading subskills, between categories of students with different language backgrounds.

Two research questions were posed:

• What is the relative importance of reading fluency and the awareness of the usefulness of reading strategies in memorizing and understanding texts on an overall reading performance for native, second generation and first generation students?
• Are there similarities and differences between the three categories of students regarding the effects of reading fluency and the awareness of the usefulness of reading strategies on the processes locate information, understanding and evaluating and reflecting?

To analyse the results the theory component skills approach to reading was used. In the approach, the overall multicomponent reading process is divided into two processes, defined as lower-level processes and higher-level processes (Grabe, 2009; Koda, 2005). The approach can show if and how these processes interact and how much each of the processes contributes individually and collectively to reading comprehension for both L1 and L2 readers (Grabe, 2009; Koda, 2005). In this study reading fluency is assumed to be related to lower-level processes and awareness of the usefulness of reading strategies to higher–level processes. Both components are, in theories and research, described to be of importance in a reading process (Grabe, 2009; Koda, 2005). Knowledge about differences and similarities in the students’ component reading profiles can potentially be used in the future development of reading instruction for multilingual students.

In PISA 2018, girls outperformed boys in reading by almost 30 score points. However, the size of the gender gap did not seem to be related to the average performance. In 16 out of the 25 countries and economies whose mean score was above the OECD average, the difference in the reading performance between boys and girls was smaller than the average gender gap across OECD countries (OECD, 2019b). Among these high-performing countries, the difference between girls' and boys' performance ranged from 13 score points in B-S-J-Z (China) to 52 in Finland.

In societies where gender equality is enhanced, girls often perform better in reading and maths (Scheeren, van de Werfhorst and H., & BolStill, 2019). This paper examines the gender gap in learning in two well-performing neighbouring countries, Finland and Estonia. In both countries, gender equality is established and travels well across the sectors of society. In Finland, there has been a declining trend in students' PISA performance in all core assessment domains, reading, mathematics and science since 2009. At the same time, the gender gap in Finland has transformed to favour girls in mathematics and science (OECD, 2019a). Meanwhile, in Estonia, the country's average performance has increased in reading and mathematics and remained at its level in science. Also, in Estonia, the gender gap has narrowed in reading, is neutral in science and developed to favour boys in mathematics.

Even though gender differences are probably the most commonly examined education outcomes, it remains unclear what the underlying causes of the existing differences remain. Maccobly and Jacklin (1974) concluded their substantially extensive review that whilst some patterns persist, for example, female superiority in verbal skills and male superiority in mathematical skills, it is not easy to untangle the influence of stereotyping on individuals' perceptions of and behaviour towards, events and objects. According to them, it was also challenging to separate if, and to what extent, innate or learned behaviours underpin the development of behavioural or cognitive gender differences. The focus on masculinity in crisis is potentially fruitful, however, because it shifts the emphasis away from structural factors in post-industrial societies, which position boys as inevitable ‘losers’. Instead, it would be necessary to explore the characteristics of masculinity that inhibit boys as learners and citizens and how these might be challenged (Epstein et al., 1998).

There is a substantial variation in gender differences, but no equal starting point given the considerable differences between countries in their provision of preschool education, age of entry into formal schooling, age of school tracking, community resources such as libraries, training of teachers, general learning cultures for example (Topping et al., 2003). From this societal and educational structure point of view, Estonia and Finland are very similar. So, it is not easy to adduce which factors have the most significant influence and why. Previous research has shown that students' families' socioeconomic status has a somewhat differentiated effect on performance by gender (Van Hek, Buchmann, & Kraaykamp, 2019; Autor, 2019). Also, students' motivation and self-efficacy are among the most vital associates of their performance across PISA studies, specifically in Finland and Estonia (Lee & Stankov, 2018; Lau & Ho, 2022).

The following research questions were formulated to examine these topics: How do motivation and self-efficacy predict girls’ and boys’ proficiency in Finland and Estonia in PISA cycles from 2006 to 2018? Could the gender gap explain the differentiating trajectories of a country's educational outcomes?

Background: Young people’s economic, cultural and social status (ECSS) is one of the most prevalent constructs used for studying equity of educational outcomes. National, regional and international large-scale assessments have furthered the quantitative research concerning the relationship between economic, cultural, and social background indicators and educational outcomes (Broer et al., 2019; Lietz et al., 2017; OECD, 2018).

However, there are observed theoretical and analytical limitations in the use of existing ECSS indicators from large-scale assessments for the purpose of monitoring equity in education (Osses et al., Forthcoming). Theoretical limitations relate to inconsistencies in how the ECSS construct is defined and operationalised, which pose significant challenges for comparing results between large-scale assessments and limit the usability of findings in addressing policy issues concerning equity in education. For example, Osses et al. (2022), demonstrated that using alternative approaches for constructing an ECSS indicator leads to different judgements concerning education systems in terms of equity of learning achievement.

Analytical limitations relate to the validity and reliability of ECSS indicators used in large-scale assessments. Whilst studies often explore reliability, cross-national invariance and other psychometric properties of ECSS indicators, information about the performance of alternative indicators is not provided. In fact no studies were found that compare the performance of alternative ECSS indicators constructed by large-scale assessments; Oakes and Rossi (2003) is an example from health research.

Objective: This paper focuses on analysing the properties of two ECSS indicators constructed using alternative theoretical and analytical approaches, applied to the same student sample. Evidence on validity is provided to evaluate the relative merits and the comparability of the two indicators for monitoring equity in education.

Method: This study analyses the properties of students’ ECSS indicators constructed by PISA and TIMSS with the aim of providing evidence concerning the validity and comparability of these two indicators. The novelty of the methodological approach lies in estimating both indicators for the same sample of students – those in PISA 2018, and thus analysing the merits of each analytical approach.

Indicators are analysed in terms of its content – ie, evaluating alignment between the theoretical construct, the indicators and the items chosen for its operationalisation – and its internal consistency. Indicators’ internal structure is investigated using confirmatory factor analysis and item response modelling in relation to model fit and the precision with which the indicators measure the ECSS construct – that is, targeting and reliability. The use of plausible values as a strategy to reduce error in making inferences about the population of interest is also explored.

Preliminary results show that the TIMSS-like indicator constructed using PISA 2018 data may benefit from better defining the underlying construct and of theoretical support to provide evidence for evaluating the adequacy of indicators chosen in its operationalisation. In terms of internal consistency, results indicate that items in the TIMSS-like indicator are “too easy” for the PISA population of interest and, although response data show a reasonably fit to the measurement model, the chosen items provide an imprecise measurement of students’ ECSS.

Three key conclusions emerge from preliminary results. First, large-scale assessments should devote more time to clearly define and provide theoretical support for the construct of students’ ECSS. Second, items used in summary indicators of ECSS should be carefully inspected, not only in terms of their reliability but also in terms of the adequacy of response categories and fit to measurement model. Third, the use of plausible values should be considered in order to avoid bias and improve precision of population estimates. The PISA indicator is currently being analysed.

This session combines studies that examine different forms of missing data in international comparative large-scale studies. The overall aim is to investigate current challenges that have emerged in recent years, including issues around the sample representativeness and the validity of performance measures. Five contributions from international scholars use data from the international studies TIMSS (Trends in International Mathematics and Science Study), PIRLS (Progress in International Reading Literacy Study), and PISA (Programme for International Student Assessment) to investigate missing data related issues including the bias, scaling or reliability of these data. The contributions in this session evaluate issues in achievement tests as well as background surveys.

Missing values are a practical problem in virtually all empirical surveys. In particular, non- intentional missing values that are missing by design are a problem in empirical research because they can compromise the integrity of the data. Typical problems relate to the representativeness of data or the accurate measurement of constructs. In this session, we examine missing values in international comparative assessments, looking at missing values in both achievement tests and background surveys. Furthermore, the individual contributions examine both unit and item non- response. The different contributions study both the reasons for missing data and consequences for the integrity of the data. The papers address non-response and exclusions in background surveys and performance tests in the large-scale assessments.

The first two papers look at missing values in performance data. In the first paper, the authors look at students who were excluded from PISA in Sweden and draw on the results of national tests. The other paper, which uses performance data, compares what happens when missing values on individual test items are judged to be incorrect or not administered. This study is based on PIRLS data as well as a simulation study. The other three papers look at missing values in survey data. One of them investigates how exclusion rates have changed over a 20-year period using data from TIMSS, PIRLS, ICCS and ICILS, and the authors note an increase in some countries. The remaining paper the impact of the administration mode on the survey participation comparing paper- vs online parental surveys in TIMSS.

The session consolidates research on a theme that often receives too little attention, and that is non-response and exclusion in large-scale tests and surveys. The session investigates different methodological issues related to missing data in different international assessments. The session is divided into six parts, four presentations, a discussion by a renowned expert, and an open discussion.

The OECD claims that the PISA study allows for representative statements about student performance. In some countries, however, entire schools and individual students are excluded from taking the tests. In addition, an increasing number of students do not sit the test for various reasons and are therefore missing in the data. In PISA 2018 more than 10 percent of the Swedish students in the PISA sample were excluded from the test or did not participate for other reasons. While this is exclusions and non-response is considered a problem (Anders et al., 2020; Micklewright et al., 2012), it is difficult to quantify the bias because the performance levels of the excluded/non-responding schools or their students are generally unknown. To address this problem, we constructed a unique database combining Swedish PISA data with Swedish register data. This database includes national tests results, subject grades as well as information on parental education and migration background for all Swedish students tested in PISA 2018. Moreover, our database also comprises corresponding information for the full cohort of students that was eligible to sit the PISA test (100 000 students born in 2002). We compare the performances and background data of the PISA sample with the entire population of 15-year- olds to shed light on any bias in PISA. The results of the analyses reveal certain degree of bias and cast doubt on the representativeness of the 2018 PISA results in Sweden. Based on the results of the national tests available for all students from Sweden, we find that students who participated in PISA perform on average more than one standard deviation better than students who were excluded from PISA or did not participate for other reasons. The findings are discussed in relation to the general problem of missingness in survey data as well as in relation to the comparability of results over time in PISA.

Anders, J., Has, S., Jerrim, J., Shure, N., & Zieger, L. (2020). Is Canada really an education superpower? The impact of non-participation on results from PISA 2015. Educational Assessment, Evaluation and Accountability, 33, 1, 229-249. https://doi.org/10.1007/s11092-020- 09329-5 Micklewright, J., Schnepf, S. V., & Skinner, C. J. (2012). Non-response biases in surveys of school children: the case of the English PISA samples. Journal of the Royal Statistical Society. Series A (General), 175, 915–938.

In international large-scale assessments, students may not be compelled to answer every test item; hence, how these missing responses are treated may affect item calibration and ability estimation. Nevertheless, using a tree-based item response model or IRTree, it is possible to disentangle the probability of attempting to answer an item from the probability of a correct response (Debeer, Janssen, & De Boeck, 2017). In an IRTree, intermediate individual decisions are represented as intermediate nodes and observed responses as final nodes. Intermediate and end nodes are connected by branches that depict all possible outcomes of a cognitive subprocess. For each branch, it is possible to estimate a distinct probability (De Boeck, & Partchev, 2012). In the present study, we evaluate the usefulness of an IRTree model for skipped (omitted) responses, first with a simulation study and then on PIRLS data from 2006, 2011, and 2016. In the simulation study, we tested missing at random (MAR) and missing not at random (MNAR) scenarios. Moreover, we tested four missing response treatments, simulating the strategies of different large-scale assessments.The simulation study proved that the IRTree model maintained a higher accuracy than traditional imputation methods within a high proportion of omitted answers. In a second step, the IRTtree model for skipping responses was implemented for data of the last three cycles (2006, 2011, and 2016) of the Progress in International Reading Literacy Study (PIRLS). Correspondence between the official PIRLS results, a Rasch model, and the IRTree model was compared at three levels: items, students, and countries. We found some differences between the PIRLS and the Rasch model estimates at the item level; however, these do not significantly impact either the estimation of student ability or the country means and rankings. Moreover, the correlation between the scores estimated by the Rasch model and the IRTree model at the student level is high; however, it is not linear. In general, the results showed that while a change in the model may impact specific countries, it did not significantly impact the overall results or the country rankings. Nonetheless, when the information is disaggregated to compare a country's results over time, it is possible to observe how the increase or decrease in the proportion of skipped items can affect their overall results.

Debeer, D., Janssen, R., & De Boeck, P. (2017). Modeling Skipped and Not-Reached Items Using IRTrees. Journal of Educational Measurement, 54(3). 333-363. https://doi.org/10.1111/jedm.12147 De Boeck, P., & Partchev, I. (2012). IRTrees: Tree-Based Item Response Models of the GLMM Family. Journal of Statistical Software, 48, 1-28. https://doi.org/10.18637/jss.v048.c01

Cross-national comparisons of educational achievement rely upon each participating country collecting nationally representative data. When it comes to missing data, researchers would usually think of omitted answers, not reached questions, and perhaps more general non- response due to non-participation. However, in ILSAs, in most countries specific parts of the international defined target population are - due to various reasons - not considered as of interest or inaccessible, which results that right from the start a part of the population is disregarded. While obtaining high response rates are a key part of reaching this goal, other potentially important factors may also be at play. As noted by Anders et al (2021) and Jerrim (2021), response rates are only part of the story. Other factors – such as the precise definition of the target population and decision about how many schools/students to exclude – also have an impact as well. When taken together, this can result in the data collected having sub-optimal levels of population coverage, jeopardizing a key assumption underpinning cross-country comparisons - that the data for each nation is nationally representative. The paper focuses upon one such issue – exclusion rates – which has received relatively little attention in the academic literature. We elaborate on the causes of missing out a part of the target population, how these are calculated and reported, and how they changed over time. The data we analyze about such exclusion rates are drawn from all IEA studies conducted between 1999 and 2019. This incorporates six rounds of TIMSS, four rounds of PIRLS, two rounds of ICCS and two rounds of ICILS. All countries that took part in any of these studies/cycles are included. Using descriptive analyses (e.g. benchmarks, correlations, scatterplots) and OLS (ordinary least squares regression model) methods, we find there to be modest variation in exclusion rates across countries, and that there has been a relatively small increase in exclusion rates in some over time. We also demonstrate how exclusion rates tend to be higher in studies of primary school students than for studies of secondary school students. Finally, while there seems to be little relationship between exclusion rates and response rates, there is a modest association between the level of exclusions and test performance. Given the relatively high and rising level of exclusions in some countries, it is important that exclusion rates do not increase any further and – ideally – start to decline.

Anders, J., Has, S., Jerrim, J., Shure, N., & Zieger, L. (2020). Is Canada really an education superpower? The impact of non-participation on results from PISA 2015. Educational Assessment, Evaluation and Accountability, 33, 1, 229-249. https://doi.org/10.1007/s11092-020- 09329-5 Jerrim, J. (2021). PISA 2018 in England, Northern Ireland, Scotland and Wales: Is the data really representative of all four corners of the UK? Review of Education, 9(3). doi:10.1002/rev3.3270

International large-scale assessments (ILSA) administrate home questionnaires to parents or guardians to collect important information regarding students’ home context and early learning experience literacy that cannot be surveyed from students directly. Non-participation rates in these home surveys are often much higher than in student surveys and thus the representativeness of these surveys may be in jeopardy. Non-participation in these questionnaires may result in biased estimates. To make the home surveys more accessible, countries now have the option of administering them either via a paper-and-pencil or a computer-based format (e.g., DeLeeuw, 2018). In this study, we investigate two conditions related to parent’s non-participation. Firstly, we will investigate the non-participation bias accordingly to different levels of non-participation in home questionnaires in both TIMSS and PIRLS studies. Secondly, we investigate whether non-response rates can be attributed to an administration mode effect. To identify this mode effect, we take advantage of the international data from several TIMSS and PIRLS cycles and compare the participation rates on the home questionnaires before and after the switch from a paper-and-pencil format to an online survey system. Based on data from more than 70 countries, we examine the effect of changing the form of administration in fixed-effects analyses for countries and study cycles. In further analysis, we examine the interaction between respondents’ characteristics (e.g., education level) and mode effect on the participation rates. We find lower participation in almost all countries that switch to online surveys, with dramatic differences of up to about 20 percent. Furthermore, we find evidence that especially socially weak families participate less in the surveys.

DeLeeuw, E. D. (2018). Mixed-Mode: Past, Present, and Future. Survey Research Methods, 12(2), 75-89. https://doi.org/10.18148/srm/2018.v12i2.7402