| Session | ||
06 SES 07 A: Open Learning in Higher Education and Teacher Education
Paper Session
| ||
| Presentations | ||
06. Open Learning: Media, Environments and Cultures
Paper The Mediating Role of Intrinsic Motivation on the Relationship Between Flexible Thinking and Learning Outcomes in MOOC vs Face-to-Face Environments 1Al-Qasemi college, Israel; 2The Hebrew university, Israel Presenting Author:Rapid changes occurring in our global world pose a challenge for higher education to advance online courses, such as Massive open online courses (MOOCs). MOOCs provide people from all over the world the opportunity to expand their education for free without any commitment or prior requirements (Colleagues & Author, 2016). Most MOOCs include short segments of video lectures arranged according to the course topics and the assessment method is based basically on closed-ended assignments. In this regard, research has focused on various aspects of learning via MOOC environments, such as: attrition and dropout rates (Ho et al., 2015), social engagement (Ferguson & Clow, 2015) and motivational patterns of MOOC enrollees (Kizilcec & Schneider, 2015). Most of these studies focused mainly on MOOC enrollees; however, little is known about cognitive and intrapersonal characteristics of MOOC completers, especially those who are registered university students (Colleagues & Author, 2019) and how these characteristics affect their learning outcomes (e.g., Author & Colleague, 2021). Using cognitive perspective (i.e. flexible thinking; Barak & Levenberg, 2016) and motivation theory (i.e. intrinsic motivation; Bandura, 2006), the current research sought to examine the effect of flexible thinking and intrinsic motivation on students' learning outcomes in a MOOC, taking into consideration a comparison with students who completed the course in a F2Fenvironment. More specifically, the current research examines relationships between flexible thinking and intrinsic motivation at the beginning of the course (Time1) and at the end of the course (Time 2) on learning outcomes after completing a MOOC and a F2Fcourse. Further, the study examines the mediating effect of intrinsic motivation between flexible thinking at Time 1 and learning outcomes at Time 2. Barak and Levenberg (2016, p.74) defined flexible thinking in education as “open-mindedness to others’ ideas—the ability to learn from others, manage teamwork, listen to multiple perspectives, and handle conflicts; 2. adapting to changes in learning situations—the ability to find multiple solutions, solve unfamiliar problems, and transfer knowledge to new situations; 3. accepting new or changing learning technologies—the ability to adjust to advanced technologies and effectively use them for meaningful learning” We argue that completers demonstrating flexible thinking at the beginning of the MOOC will have flexible thinking at the end of the MOOC (Hypothesis 1a) and F2F course (Hypothesis 1b). Intrinsic motivation refers to the inherent satisfaction to be engaged in activity for its own sake. Intrinsic motivation involves an inherent gratification prompted by the feeling that learning is interesting and enjoyable (Glynn et al., 2011). We argue that completers demonstrating intrinsic motivation at the beginning of the course will have intrinsic motivation at the end of the MOOC (Hypothesis 2a) and the F2F course (Hypothesis 2b). Flexible thinkers are open to new experiences, adapt to new situations, and easily generate new ideas (Barak & Levenberg, 2016a). They adjust to varying circumstances and work well in a climate of uncertainty (Bransford et al., 2000). Further, completing a MOOC is a great challenge as it involves the understanding of complex contents; MOOCs support diverse populations, as each population can contribute to the knowledge and experience of the others (Colleagues & Author, 2018). Thus, we argue that completers’ flexible thinking (Hypothesis 3a) and intrinsic motivation (Hypothesis 3b) at Time 2 will affect their learning outcomes at Time 2 more in MOOC environments than in F2F environments. Learning outcomes relate to students' achievement in the final course assignments. We argue that completers with flexible thinking at the beginning of the course may affect their learning outcomes at the end of the course through their intrinsic motivation (Time 2) only in MOOC environments (Hypothesis 4). Methodology, Methods, Research Instruments or Sources Used The study included a sample of two groups of undergraduate students (N=204) taking the same course “Teaching Thinking,” a MOOC course (N=141) and a F2F classroom course (N=63). The MOOC and the F2F course were taught by the same lecturer and all learning materials and assignments were the same. The students were recruited by sending a message through an online mailing list at the beginning and at the end of the course, inviting them to participate in this study. Participation was voluntary with no extra credit or compensation. Measures Flexible thinking: We used a questionnaire developed by Barak and Levenberg (2016b) (19 items), ranked on a 6-point Likert type scale 1(strongly disagree) to 6(strongly agree)), with three dimensions: learning technology acceptance, open-mindedness in learning, and adapting to new learning situations. A sample item for learning technology acceptance: "I adjust quickly to new learning technologies." Intrinsic motivation: We used a questionnaire developed by Glynn and colleagues (2011) 5 items; 1(strongly disagree) to 5(strongly agree). A sample item: "I enjoy learning ‘Teaching Thinking.” Learning outcome: We analyzed students’ grades in the final exam at the end of the courses. Control variables: Students' prior knowledge was controlled in the current research. Prior knowledge was examined by one question at the beginning of the course: "How familiar were you with the subject area of the course? a. I am mostly new to this subject area, b. I am somewhat familiar with the subject area, c. I am very familiar with this subject area, d. I am an expert in this subject area.” All research measures received Reliability Cronbach Alpha more than 0.7, and fit indices more than 0.9 for construct validity. The proposed model was examined using AMOS program. To examine mediation, a bootstrap analysis was conducted, and confidence intervals were calculated as recommended by Preacher et al. (2010). Conclusions, Expected Outcomes or Findings The MOOC environment model indicated a good fit between the model and the data (CFI = .99; NFI=.97; RMSEA = .049). A positive and significant relationship was found between flexible thinking in Time 1 and flexible thinking in Time 2 (β = .82; p <.001). The relationship between intrinsic motivation in Time 1 with intrinsic motivation in Time 2 was positive and significant (β = .21; p <.05), thus confirming hypotheses 1a, 2a. Fit indices were more than .90 between the data and the model in the F2F environment; however, RMSEA = .10, which should be lower than .10. A positive and significant relationship was found between flexible thinking in Time 1 and flexible thinking in Time 2 (β = .66; p <.001), thus confirming hypothesis 1b. The relationship between intrinsic motivation in Time 1 with intrinsic motivation in Time 2 was positive and significant (β = .37; p <.001), thus confirming hypothesis 2b. Regarding the MOOC model, intrinsic motivation was positively and significantly related to learning outcomes in Time 2 (β = .17; p <.05); however, flexible thinking was not related to learning outcomes in Time 2 (β = -.13). However, in F2F model, intrinsic motivation and flexible thinking were not significantly related to learning outcomes in Time 2 (β = .17; β = .06 respectively). Thus, hypothesis 3b was confirmed; hypothesis 3a was not confirmed. Finally, the indirect effects between skill flexibility (Time 1) to learning outcomes (time 2) through intrinsic motivation (time 2) in MOOC environments was found to be .03 (p < .01), with a 99.5% confidence interval ranging between .04 and .19. Mediation was not examined in F2F model because of lack of relations between flexible thinking and intrinsic motivation in Time 2 and learning outcomes in Time 2, thus, confirming hypothesis 4. References Bandura, A. (2006). Going global with social cognitive theory: From prospect to paydirt. In S. I. Donaldson, D. E. Berger & K. Pezdek (Eds.). The rise of applied psychology: New frontiers and rewarding careers (pp. 53–70). Mahwah, NJ: Erlbaum. Barak, M. (2014). Closing the gap between attitudes and perceptions about ICT-enhanced learning among pre-service STEM teachers. Journal of Science Education and Technology, 23(1), 1–14. Barak, M. (2018). Are digital natives open to change? Examining flexible thinking and resistance to change. Computers & Education, 121, 115-123. Barak, M., & Levenberg, A. (2016). A model of flexible thinking in contemporary education. Thinking Skills and Creativity, 22, 74-85. Bransford, J., Bransford, J.D., Brown, A.L. & Cocking, R.R., 1999. How people learn: Brain, mind, experience, and school. National Academies Press. Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. routledge. Cho, M.-H., & Heron, M. L. (2015). Self-regulated learning: the role of motivation, emotion, and use of learning strategies in students' learning experiences in a self-paced online mathematics course. Distance Education, 36(1), 80e99. Ferguson, R., & Clow, D. (2015). Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs). In The 5th International learning analytics and knowledge Conference (LAK15), 16e20 March 2015. Poughkeepsie, NY, USA: ACM Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48, 1159e1176. Green, G. C. (2004). The impact of cognitive complexity on project leadership performance. Information and Software Technology, 46, 165-172. Ho, A. D., Chuang, I., Reich, J., Coleman, C., Whitehill, J., Northcutt, C., et al. (2015). HarvardX and MITx: Two years of open online courses (HarvardX Working Paper No. 10). http://dx.doi.org/10.2139/ssrn.2586847. Kizilcec, R. F., & Schneider, E. (2015). Motivation as a lens to understand online learners: toward data-driven design with the OLEI scale. ACM Transactions on Computer-Human Interactions, 22(2). http://dx.doi.org./10.1145/2699735. Moore, R. L., & Wang, C. (2021). Influence of learner motivational dispositions on MOOC completion. Journal of Computing in Higher Education, 33(1), 121-134. Schunk, D. H., Pintrich, P. R., & Meece, J. L. (2008). Motivation in education (3rd ed.). Upper Saddle River, NJ: Pearson. Shroff, R. H., Vogel, D. R., & Coombes, J. (2008). Assessing individual-level factors supporting student intrinsic motivation in online discussions: a qualitative study. Journal of Information Systems Education, 19(1), 111e125. 06. Open Learning: Media, Environments and Cultures
Paper Why Are Online Learners Invisible? Self-presentation of International Students in Online learning 1Beijing Normal University, China, People's Republic of; 2Faculty of Psychology and Educational Science, Ludwig Maximilian University of Munich, Germany Presenting Author:Why Are Online Learners Invisible? Self-presentation of International Students in Online learning—A Case Study of Asian Students at German Universities Against the backdrop of international higher education, one characteristic of Asian students’ online learning is their invisibility. Culture is an important influence on this behaviour, and this study makes use of Goffman’s dramaturgical theory to analyse Asian students’ invisible online learning at German universities. A qualitative research method was applied to explore those students’ online learning experience. The manifestations of Asian students’ invisible online learning included the mystification of personal learning, weakened classroom interaction and dissociated classroom presence. The reasons that influence online learners’ invisibility include changes in the outside, interference from the back, the stage fright of actors, audience exit, and disbandment of the team. This study suggests that university teachers should improve their teaching abilities and help students build a diverse online learning community. This study borrows Goffman’s dramaturgical theory to analyse Asian students’ online learning interactions at German universities. Goffman is interested in making sense of human interaction from a sociological perspective, and with this in mind, the special human interaction – students’ invisible interaction, including their learning behaviour, learning willingness, and learning recognition – during online learning is framed here from the perspective of Goffman’s theory. Methodology, Methods, Research Instruments or Sources Used The investigation was conducted at the University of Tübingen, Germany, which is a well-known international university with many international students. Purposive sampling was adopted for sample selection, and the inclusion criteria were: (a) participants were college students at that moment; (b) they came from Asian countries before entering the university; (c) they had online learning experience at a German university for at least one semester; (d) both female and male students were included; and (e) their majors were diverse, including the social sciences (sociology, policy science, education science, economics, linguistics), natural sciences (astronomy, geography, archaeology), and engineering. Sampling continued until the interview data were saturated. A total of 17 Asian students participated in the interviews; there were eight Chinese students, three Korean students, three Malaysia students, two Japanese students, and one Indian student. Semi-structured interviews were used to collect data, and all interviews were conducted from February 2021 to February 2022. The main interview question asked participants to describe their perceptions and experience of online learning at a German university; each interview lasted around 90–120 minutes. This investigation occurred at the end of the COVID-19 period, so all interviews were conducted online with Zoom to ensure the health of all participants. To allow accurate and timely interactions that would be comparable to face-to-face interactions and to observe participants’ reactions, all students were asked to keep their cameras on throughout the online interview. Informed consent forms were sent to the participants in advance; these forms explained the research purpose and participants’ anonymity. All of the interviews were audio recorded; the recordings were sent to participants to confirm their meaning. All participant information has been kept confidential. Conclusions, Expected Outcomes or Findings To understand the manifestations of and reasons for Asian students’ tendency to seek invisibility in online learning in higher education, this study provided a social-cultural perspective to explore the characteristics of that invisibility and why they chose to become invisible online learners. A qualitative methodology was used to explore Asian students’ online learning experience at a German university. The findings indicate that manifestations of Asian students’ invisible online learning include the mystification of personal learning, weakened classroom interaction, and dissociated classroom presence. Within Goffman’s dramaturgical theoretical framework, the reasons that influence online learners’ invisibility include external changes, interference from backstage, stage fright, audience exit, and disbandment of the team. References Castro, M. D. B., & Tumibay, G. M. (2021). A literature review: Efficacy of online learning courses for higher education institution using meta-analysis. Education and Information Technologies, 26, 1367–1385. Ferri, F., Grifoni, P., & Guzzo, T. (2020). Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies, 10(4), 86. Goffman, E. (1955). On face-work: An analysis of ritual elements in social interaction. Psychiatry, 18(3), 213–231. Goffman, E. (2016). The presentation of self in everyday life. In W. Longhover & D. Winchester (Eds.), Social theory re-wired (pp. 482–493). Routledge. Gray, L. M., Wong-Wylie, G., Rempel, G. R., & Cook, K. (2020). Expanding qualitative research interviewing strategies: Zoom video communications. The Qualitative Report, 25(5), 1292–1301. Gilch, H., Beise, A. S., Krempkow, R., Müller, M., Stratmann, F., & Wannemacher, K. (2019). Digitalisierung der Hochschulen—Ergebnisse einer Schwerpunktstudie für die Expertenkommission Forschung und Innovation, Hanover, Germany. HIS-Institut für Hochschulentwicklung (HIS-HE). Hanh, N. T. (2020). Silence is gold?: A study on students’ silence in EFL classrooms. International Journal of Higher Education, 9(4), 153–160. Lemay, D. J., Bazelais, P., & Doleck, T. (2021). Transition to online learning during the COVID-19 pandemic. Computers in Human Behavior Reports, 4, 100130. Ma, J., Han, X., Yang, J., & Cheng, J. (2015). Examining the necessary condition for engagement in an online learning environment based on learning analytics approach: The role of the instructor. The Internet and Higher Education, 24, 26–34. Pan, W., Zhou, Y., & Zhang, Q. (2016). Does darker hide more knowledge? The relationship between Machiavellianism and knowledge hiding. International Journal of Security and Its Applications, 10(11), 281–292. Schmidt-Hertha, B., & Bernhardt, M. (2022). Pedagogical relationships in digitised adult education. Andragoška spoznanja, 28(1), 11–24. Singer, A. (2023). Exploring teachers’ public interactions and private conversations during the pandemic: A qualitative study using Goffman’s dramaturgical theory. [Master’s thesis, University of Manitoba]. Wut, T. M., & Xu, J. (2021). Person-to-person interactions in online classroom settings under the impact of COVID-19: A social presence theory perspective. Asia Pacific Education Review, 22(3), 371–383. Yan, L., Whitelock‐Wainwright, A., Guan, Q., Wen, G., Gašević, D., & Chen, G. (2021). Students’ experience of online learning during the COVID‐19 pandemic: A province‐wide survey study. British Journal of Educational Technology, 52(5), 2038–2057. 06. Open Learning: Media, Environments and Cultures
Paper Bridging Theory and Practice: A Case Study on the Implementation of Media Projects as an Integral Part of Teacher Education University of Education Weingarten, Germany Presenting Author:The TEgoDi concept (Teacher Education goes Digital) emerged from the need to enhance the digital media skills of teacher students, contributing an innovative dimension to teacher education (Müller et al., 2021). The integration of digital media-related competencies becomes paramount as educators need to navigate the potentials and limitations of digital media in pedagogical practices (McGarr & McDonagh, 2019). TALIS (Teaching and Learning International Survey) findings underscore the urgency, with only 53 percent of teachers regularly incorporating technology into teaching, and fewer expressing a desire for further education (Schleicher, 2020). Post-graduation, many students feel inadequately prepared for effective digital media use in teaching, emphasizing the need to consider subject-specific teaching–learning processes (Koehler & Mishra, 2009). Digital media-related competencies encompass a spectrum of knowledge, skills, and dispositions required by teachers to adeptly design teaching-learning processes (Falloon, 2020). In response to the need for a holistic and integrated approach, TEgoDi adopts a project-oriented strategy rooted in theories of situated learning (Lave & Wenger, 2008) and authentic learning (Herrington & Herrington, 2006). This approach is embedded within a teacher education programme, where prospective teachers engage in two projects: a media-based teaching project and a media development project (Müller et al., 2021). To facilitate students' competence development, various support structures are integrated, including regular feedback based on online learning analytics, self-assessments, and tutoring. A key factor of sustainability and success of the TEgoDi project is the curricular anchoring of the media projects. In this way, the media projects get the chance to be perceived not just as an add-on, but as integral part of teacher education. The TEgoDi approach has been implemented since 2023, and both media projects will be anchored in the study and examination regulations and new module handbooks across the curriculum from 2025. Although it is not yet compulsory, the integration of media projects into the courses is encouraged. To this end, a transitional statute has encouraged the facilitation of media projects in existing courses for trialling. The TEgoDi's media project implementation is currently in a pilot phase and is being evaluated ongoing basis. The formative evaluation follows the iterative development procedure (Allen & Sites, 2012), which encompasses three major development loops. Each loop is evaluated using feedback from students and lecturers and tutors from the TEgoDi project. Our presentation focuses on the final phase of the evaluation, in which early adopters integrate the described media projects into their courses and assess both the process and the outcomes. As we evaluate the TEgoDi's media project implementation in its pilot phase, our presentation addresses the following key research question providing insights into the effectiveness of the TEgoDi concept in teacher education: How do two pioneering media projects in teacher education influence the development of competencies among students, and to what extent do various supportive structures, including regular feedback based on online learning analytics, self-assessments, and tutoring, contribute to this process? In our presentation, we will address this crucial aspect of teacher education going digital. Through the exemplary good-practice examples, the challenges and their handling are illustrated, and outlook address the necessity of media projects and lessons learned. Achieving comprehensive integration involves embedding the projects in all study disciplines, which was accomplished through the adaptation of module handbooks and study examination regulations. Interdisciplinary workshops were pivotal in developing common minimum standards, ensuring the acquisition of media pedagogical competencies by prospective teachers across all study programs. Further details on the evaluation methodology are outlined in the subsequent section. Methodology, Methods, Research Instruments or Sources Used In addressing the research question, an experiential evaluation is conducted by drawing on insights and feedback from early adopters representing various disciplines and teaching formats. These early adopters play a crucial role in testing the media projects on both quantitative and qualitative levels, providing valuable perspectives that contribute to the fine-tuning of the TEgoDi concept. The evaluation follows a methodological approach that encompasses both quantitative and qualitative dimensions in a mixed methods design (Carter, Bryant-Lukosius, DiCenso, Blythe & Neville, 2014; Flick, 2018). In the context of the quantitative dimension, a questionnaire was distributed to students (N=69) participating in courses where the instructors, acting as early adopters, were testing the implementation of media projects. The questionnaire covered diverse aspects, such as attitudes toward digital media, attitudes regarding the integration of digital media in teaching, assessments of the usefulness of digital media in instruction, and self-evaluations of media pedagogical competencies. The questionnaires were descriptively analyzed using SPSS. Additionally, the research employs qualitative methods to provide a deeper understanding of the effects and challenges associated with the integration of media projects. Problem-centred interviews (Witzel & Reiter, 2012) (n=31) and focus groups (Stewart & Shamdasani, 2015) (n=5) as well as student feedback (Mandouit, 2018) with a total of 92 students was done to serve as essential tools to capture nuanced insights into the experiences of students during the implementation of the media projects. The analysis of the data utilized qualitative content analysis, following the methodological framework presented by Kuckartz and Rädiker (2023), with support from the Maxqda software (Rädiker & Kuckartz, 2020; Loxton, 2021). The qualitative dimensions complement the quantitative analysis, offering a comprehensive view of the multifaceted aspects influencing the successful implementation of media projects. It is noteworthy that the mandatory integration into all study disciplines is slated to commence in the summer semester of 2025. Before this period, the current phase serves as a trial period, utilizing experiences and insights to refine the media projects. This deliberate approach allows for a gradual implementation, accommodating the diversity of study disciplines and the varying requirements of teaching formats. Conclusions, Expected Outcomes or Findings The evaluation findings not only contribute to successfully integrating technological innovations in teacher education but also foster a pedagogical transformation crucial to meet evolving education sector demands. Continuous evaluation, particularly involving early adopters, enhances media project quality, ensuring a robust foundation for integration into all disciplines from 2025 onwards. Addressing the research question, examining two pioneering media projects provides valuable insights into their impact on student competency development. Findings emphasize the crucial role of supportive structures, including regular feedback, analytics, self-assessments, and tutoring. Outcomes align with the goal of enhancing media literacy and pedagogical understanding in teacher education: (1) Supportive structures in competency development: - Lessons highlight clear communication's importance, particularly in conveying intended outcomes like promoting media literacy. - Milestones and supportive structures offer crucial guidance, providing students orientation during their learning journey. - Enhanced monitoring, feedback, and tutoring commitment elevate overall project quality. - Balancing guidance and fostering independence is crucial, identified through lessons learned. (2) Shaping the role of educators: - Shifting educators' role to learning facilitators is pivotal for successful project implementation. - Considering educators' role change aligns with balancing guidance and fostering student independence. - Lessons emphasize challenges of prioritizing focus over breadth, with a recommendation to potentially reduce subject matter depth. - Focusing more on application and transfer could enhance project effectiveness. (3) Meeting the challenge of clear communication: - Clear criteria and literature sources are crucial for depth and academic challenge. - Lessons underscore the importance of explicit guidelines to avoid task over- or underestimation. In this list, focusing on depicted students, it's crucial to recognize the vital role of educators in implementing media projects within teacher education. Effective support hinges on educators perceiving it as added value, and interdisciplinary projects necessitate universally applicable standards. References Allen M. & Sites R. (2012). Leaving ADDIE for SAM. An agile model for developing the best learning experiences. Danvers: ASTD Press. Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J. & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncol Nurs Forum. 2014 Sep;41(5), 545-547. https://doi.org/10.1188/14.ONF.545-547. PMID: 25158659. Falloon G. (2020). From digital literacy to digital competence: the teacher digital competency (TDC) framework. Educ Technol Res Dev., 68, 2449-2472. https://doi.org/10.1007/s11423-020-09767-4. Flick, U. (2018). Triangulation in data collection. The SAGE handbook of qualitative data collection, 527-544. Herrington, A. & Herrington, J. (2006). What is an Authentic Learning Environment? In A. Herrington & J. Herrington (Eds.), Authentic learning environments in higher education (1-14). Hershey, PA: Information Science Pub. https://doi.org/10.4018/978-1-59140-594-8.ch001 Koehler M. & Mishra P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemp Issues Technol Teach Educ., 9(1), 60–70. Kuckartz, U. & Rädiker, S. (2023). Qualitative Content Analysis: Methods, Practice and Software. SAGE. Lave J. & Wenger, E. (2008). Situated learning. Legitimate peripheral participation. Cambridge: Cambridge Univ. Press. https://doi.org/10.1017/CBO9780511815355. Loxton, M. H. (2021). Analyzing focus groups with MAXQDA. MAXQDA Press. McGarr O, McDonagh A. Digital competence in teacher education (Output 1 of the Erasmus+ funded Developing Student Teachers’ Digital Competence (DICTE) project). University of Limerick. 2019. https://dicte.oslomet.no/ Mandouit, L. (2018). Using student feedback to improve teaching, Educational Action Research, 26:5, 755-769. https://doi.org/10.1080/09650792.2018.1426470. Müller, W., Grassinger, R., Schnebel, S., Stratmann, J., Weitzel, H., Aumann, A. et al. (2021). Integration of Digital Competences into a Teacher Education Program: A Sensitive Approach. Proceedings of the 13th International Conference on Computer Supported Education - Volume 1: CSEDU, 232–242. https://doi.org/10.5220/0010527202320242 Rädiker, S. & Kuckartz, U. (2020). Focused analysis of qualitative interviews with MAXQDA: Step by step. https://www.maxqda-press.com/wp-content/uploads/sites/4/978-3-948768072.pdf. Stewart, D. W. & Shamdasani, P. N. (2015). Focus groups: Theory and practice. Sage publications. Schleicher A. (2020). The impact of covid-19 on education insights from education at a glance 2020. https://www.oecd.org/educa tion/the-impact-of-covid-19-on-education-insights-education-at-a-glance-2020.pdf. Willis, G. B. (2020). Questionnaire design, development, evaluation, and testing: Where are we, and where are we headed? Advances in questionnaire design, development, evaluation and testing, 1-23. https://doi.org/10.1002/9781119263685.ch1. Witzel, A. & Reiter, H. (2012). The problem-centred interview. Sage. | ||