Conference Agenda

Although the severity of the COVID-19 pandemic has somewhat subsided, its aftermath remains evident. While schools adapted quickly to changes, the progress of learning has also slowed down. Reading skills of German primary school students now show a deficit of up to one-third of a school year, which has been directly attributed to the pandemic experiences (Ludewig et al., 2022). Similar declines have been observed in other areas such as mathematics (Schult et al., 2022). These delays are critical, as children need to catch up on existing skills as well as continue acquiring new skills and knowledge.

The transition to a new school is often associated with anxiety due to new teachers, requirements, and social contacts (Tay & Hast, 2022). Following the transition, German students first become familiar with science as a distinct school subject, which builds on skills that have been critically delayed due to the pandemic, such as reading or numerical proficiency. Science anxiety, defined as “a debilitating combination of fearful negative emotion and cognition in the context of science learning” (Bryant et al., 2013, p. 432) hampers general participation in science lessons but also directly affects performance, success (Ucak & Say, 2019), and knowledge acquisition (Theobald et al., 2022). Preventing a domino effect starting with insufficient preparation thus gains importance. Science anxiety is comparatively underexplored but is distinct from test and generalised anxiety (Megreya et al., 2021).

To address the challenge of these three elements meeting – pandemic, transition and science anxiety – the current project aims to investigate science anxiety among German fifth-graders. The project’s goals include adapting an existing science anxiety rating scale (Megreya et al., 2021) for use with German students, In addition, training sessions using mindfulness have shown to alleviate test anxiety and positively impact knowledge acquisition (Theobald et al., 2022), reducing stress by improving attention and emotion regulation (Lam & Seiden, 2020). The project aims to additionally examine whether the implementation of a mindfulness program can successfully reduce science anxiety among German fifth-graders.

Survey data from International Large-Scale Assessments (ILSA) provide valuable information for governments, institutions, and the general public. High response rates are an important indicator of the reliability and quality of the survey, conversely low response rates in ILSAs can threaten the inferential value of the survey method. ILSA data are highly valued by the Ministries of Education of participating countries as a guide to inform policy-making.

An important ILSA is the Programme for International Student Assessment (PISA) which assesses the performance of 15-year-old students in reading, mathematics, and science. First administered in 2000, PISA has been implemented every three years since. Meeting the response rate thresholds specified by a low-stakes test such as PISA has often proven to be a challenge for many PISA participating countries (Ferrera et al. 2010). In the PISA 2022 cycle, an elevated number of countries were required to undertake Non-Response Bias Analysis (NRBA) due to low response rates (OECD, 2023). Ireland has participated in PISA since the first cycle in 2000 and had consistently met the response rate standards at both student and school level until 2022, when it failed to meet the student response rate. This leads us to the main research question ‘Why was there a change in the student response rate between PISA 2018 and 2022 in Ireland?’.

Two major differences were observed between 2018 and 2022, a move from spring to autumn testing and the COVID-19 pandemic. For Ireland, the PISA Main Study took place in the spring (March/April), this was followed by a Feasibility Study in the autumn (October/November). The purpose of the Feasibility Study was to evaluate the possibility of moving testing in Ireland to autumn and for the first time in PISA, testing took place in the autumn 2022. Secondly, while school restrictions were no longer in place in Ireland during testing in 2022, there was still a level of disruption associated with the COVID-pandemic in the school environment.

Various theories have been proposed to understand response rates and why some people participate in surveys and others do not. For example, the theory of cognitive dissonance which according to Festinger (cited in Miller, Clark, & Alayna, 2015) suggests that reducing the lack of agreement between people is an important factor in whether a person will respond or not to a survey. Alternatively, the theory of commitment or involvement suggests that the nature of the first request in the ‘foot in the door’ technique may have a significant effect upon participation (Freedman & Fraser, 1966). However, Self-Determination Theory (SDT) may provide a theoretical framework to facilitate the examination of the role motivation (extrinsic/intrinsic) may play in determining response rates. In SDT, three factors that assist motivation are competence, autonomy and relatedness, according to Deci and Ryan (1985). These three factors are seen as essential psychological needs that guide behaviour. Wenemark et. al. (2011) used SDT to redesign a health-related survey in an effort to improve response rates. In a similar vein, this study will use it to examine the change in the student response rate between PISA 2018 and 2022 in Ireland.

While the focus of this poster is on the changing response rates in PISA in Ireland, the implications of the findings will assist other countries participating in similar ILSAs. With the number of countries experiencing lower response rates in PISA 2022 at an unprecedented level, it is of urgent importance that countries begin to understand and address the complex reasons behind falling response rates in order to maintain the reliability and quality of these kinds of studies.