| Session | ||
22 SES 04 A: New Digital Challenges in HE
Paper Session
| ||
| Presentations | ||
22. Research in Higher Education
Paper Examining the challenges and opportunities of Generative AI use in Higher Education 1University of Melbourne, Australia; 2Deakin University Presenting Author:Generative AI (GenAI) is a specific type of Artificial Intelligence that can create new content in the form of text, images, music, code, and various other forms of digital media by using machine learning or ‘training’. ChatGPT is an example of a GenAI application that has been trained on a vast amount of publicly available data. It was made freely available in November 2022, albeit with usage limitations. The launch sparked intense public interest, with initial speculation about what the effects might be for education, jobs, and for society in general.
While ChatGPT was not the first model made available (e.g. GPT-2), it is an advanced model that allows for sophisticated interactions, with the remarkable capability of replicating human-like natural language processing. However, it cannot understand language. It is trained to detect complex patterns and assimilate that information into existing information (Bozkurt, 2023; UNESCO, 2023). As GenAI continues its rapid growth, evolving and improving outputs, there has been a wide range of perspectives, from those who have embraced the technology, those who oppose the technology, and some who are both enthusiastic and/or cautious. Gallant-Torres (2023) identifies the opposing extremes as ‘technophiles who defend it without regard to its risk and technophobes who reject it without considering its benefits.’
Research is beginning to emerge about the affordances and challenges of GenAI use in education. New skills are evolving with the use of GenAI such as ‘prompt engineering’, which is defined as the ‘art of designing, writing and fine-tuning prompts’ to elicit the most accurate and relevant responses from GenAI applications (Eager & Brunton, 2023). There have been significant opportunities that have already been identified as being embraced in higher education settings, such as integrating technology to promote learner-AI collaboration (Tan, Chen & Chua, 2023), personalised feedback and adaptive learning pathways (Eager & Brunton, 2023), automating processes, innovations in teaching and assessment and creating a more inclusive environment (Adiguzel, 2023; Moya & Eaton, 2023). However, the scope and the extent to which these practices have been adopted remain unclear. There are also complex issues emerging, as Farrelly and Baker (2023) highlight that ‘we are already seeing that minority and marginalised students are being accused of breaching academic integrity rules …perpetuating an existing inequitable pattern’. In 2021, UNESCO released ‘Recommendations on The Ethics of Artificial Intelligence’. The first statement highlights the importance of ‘the profound and dynamic positive and negative impacts of AI on societies, environment, ecosystems and human lives, including the human mind’ (UNESCO, 2021). The call for setting standards relating to AI technologies happened well before the launch of ChatGPT and the numerous other generative AI tools released since then. It is evident that GenAI has now been integrated into higher education settings (Ipek, 2023), amidst concerns about what the potential impacts could be on disciplinary knowledge and the assessment of key knowledge and skills. It is in this context that our research study aimed to examine the use of generative AI by academics and students in higher education, and their perceptions of the impact that the technology would have on teaching and learning. The key research question is:
Methodology, Methods, Research Instruments or Sources Used The data for this study was generated from April 24 to November 30, 2023, using a Qualtrics online survey. Students and academics answered questions that were tailored to each participant group, which were organised around four themes: 1) awareness of GenAI (e.g. ChatGPT); 2) current use and intention to use GenAI; 3) potential of GenAI to contribute to learning and assessment; 4) affordances and challenges related to the use of GenAI; and 5) support provided for using GenAI in higher education settings. The findings are based on survey responses from 243 students and academics, with two distinct data collection periods in semesters 1 and 2 to track how the use of GenAI changed during the first year it became available. The survey invited students who were enrolled in any course or degree program at a university, and academic staff in roles such as tutors and lecturers to participate. In the results, those who were enrolled in a course or program are referred to as ‘students’ and those with teaching roles are referred to as ‘academics’. The Qualtrics platform and Excel were used to analyse the quantitative responses to Likert questions. Questions that asked for a short text response were analysed using inductive thematic coding using NVivo. Axial coding was used to find relationships between first pass codes, and to iteratively assign categories that were derived from these relationships. The key categories that emerged from this process were able to be classified as either opportunities or challenges related to the use of GenAI in higher education. This was the first study in Australia to generate data on the use of GenAI and the perspectives of students and academics in higher education during a time when GenAI was gaining momentum and new products, such as models with the capability to generate information text-to-text, text-to-image, image-image and image-text, were rapidly being released to consumers. Conclusions, Expected Outcomes or Findings Students shared the purposes for which they used GenAI, including generating different outputs when assessment instructions and criteria were unclear, as a study partner, to generate revision materials and feedback on their work, to create plans, restructure writing, brainstorming, summarising literature, referencing, generating images, and students with English as a second language found the enhanced language support helpful. Academics also reported using GenAI to generate summaries and create information. Additionally, they used GenAI to develop teaching materials and exam questions, as a research tool, and to check what GenAI responses would be produced for set assessment tasks. One of the key challenges identified by respondents was the reliability of GenAI to produce accurate information and references. They found it difficult to fact check and had concerns about misinformation being reproduced. Other concerns related to the impact the techbology would have on learning and assessment, particularly in relations to people becoming reliant on technology rather than using ‘human thinking’. Ethical concerns about the difficulty detecting plagiarism were identified, as was equitable access and the possible impact on increasing the digital divide, especially for those who might not have access modern technology, tools and current information. As one academics stated, “There are so many ethical issues to work out in relation to AI, but we need to assist staff and students to understand appropriate boundaries, affordances and limitations of this technology. It will create an even bigger digital divide and inequality by placing limitations on what we want students to know and understand. It's important not to be left behind in this debate.” While there is potential for GenAI to enhance teaching and learning in higher education, critical issues remain on the impact of the technology on reliability, accessibility and ethical use in academia. References Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing Education with AI: Exploring the Transformative Potential of ChatGPT. Contemporary Educational Technology, 15(3). Bozkurt, A. (2023). Generative Artificial Intelligence (AI) Powered Conversational Educational Agents: The Inevitable Paradigm Shift. Asian Journal of Distance Education, 18(1), 198–204. Eager, B., & Brunton, R. (2023). Prompting Higher Education Towards AI-Augmented Teaching and Learning Practice. Journal of University Teaching & Learning Practice, 20(5), 1–19. https://doi.org/10.53761/1.20.5.02 Farrelly, T., & Baker, N. (2023). Generative Artificial Intelligence: Implications and Considerations for Higher Education Practice. Education Sciences, 13(11), 1109. https://doi.org/10.3390/educsci13111109 Gallent-Torres, C., Zapata-González, A., & Ortego-Hernando, J. L. (2023). The impact of Generative Artificial Intelligence in higher education: a focus on ethics and academic integrity. Electronic Journal of Educational Research, Assessment & Evaluation / Revista Electrónica de Investigación y Evaluación Educativa, 29(2), 1–19. Ipek, Z. H., Gözüm, A. I. C., Papadakis, S., & Kallogiannakis, M. (2023). Educational applications of the ChatGPT AI system: a systematic review research. Educational Process: International Journal, 12(3), 26–55. Moya, B. A., & Eaton, S. E. (2023). Examining Recommendations for Generative Artificial Intelligence Use with Integrity from a Scholarship of Teaching and Learning Lens. Electronic Journal of Educational Research, Assessment & Evaluation / Revista Electrónica de Investigación y Evaluación Educativa, 29(2), 1–21. https://doi.org/10.30827/relieve.v29i2.29295 https://doi.org/10.30827/relieve.v29i2.29134 UNESCO. (2023). Guidance for generative AI in education and research. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000386693_eng UNESCO. (2022). Recommendation on the Ethics of Artificial Intelligence. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000381137 22. Research in Higher Education
Paper ChatGPT in Higher Education: Exploring Challenges and Possibilities for Academic Literacy and Student Engagement Södertörn University, Sweden Presenting Author:Recent advancements in Artificial Intelligence (AI) technologies have sparked discussions within higher education(Kramm & McKenna, 2023; Peters et al., 2023; Popenici, 2023). Among these tools, ChatGPT stands out for its capacity to generate text, provide personalized recommendations, and potentially improve student learning outcomes. However, concerns have been raised about the impact of such AI tools on higher education teaching and academic integrity(Blackie, 2024; Rawas, 2023). This study aims to explore the intersection of artificial intelligence and education, with a specific focus on ChatGPT and its potential applications in higher education. More specifically, this paper seeks to investigate the possibilities of integrating ChatGPT into higher education courses with the goal of enhancing academic literacy and improving students' learning experiences. The study addresses two key objectives: (i) the integration of artificial intelligence tools, particularly ChatGPT, into higher education courses, and (ii) understanding the perceptions of students and their engagement with ChatGPT within the context of their academic activities. Drawing inspiration from situated/sociocultural perspectives in learning and Gee's (2008) concept of a "mediating device," we explore how learners’ understanding and capabilities are transformed when engaging with ChatGPT. As Gee(2008) suggests, what learners can understand and accomplish with a mediating device differs significantly from what they can do without it. When individuals engage with mediating devices, knowledge becomes distributed—some is manifested in their minds, some in their coordination with tools, and some in the tools themselves. This perspective informs our exploration of the impact of ChatGPT as a mediating device in enhancing students' learning experiences and academic literacy. Methodology, Methods, Research Instruments or Sources Used The study utilises a case study design, chosen for its suitability in exploring real-life activities within a specific context. The context, in this instance, was a pedagogy course on the philosophy of education at a Swedish higher education institution. The participants consisted of 8 first-year bachelor's students who were followed throughout the entire duration of the course, spanning one month. For the data analysis, a qualitative approach was employed for the examination of interview transcripts, writing assessments, and observational notes. The material combined students’ reflections and writing assessments with observational data from the course, offering a comprehensive understanding of ChatGPT's impact on students' experiences and academic outcomes. Coding and thematic analysis were applied to identify patterns and themes in the collected data. To uphold ethical standards, participant confidentiality was ensured, and voluntary participation was emphasised, with informed consent obtained from all participants before the study initiation. Conclusions, Expected Outcomes or Findings The preliminary findings highlight that the use of ChatGPT, when applied in a structured and informed manner, can positively influence both students' academic literacy and their overall engagement. Simultaneously, the findings underscore the significance of social aspects within courses, such as lectures and group work, in shaping the learning processes. The interplay between the integration of ChatGPT and the social dynamics of traditional teaching methods is crucial in understanding the possibilities of AI on students’ learning experiences. References Blackie, M. A. L. (2024). ChatGPT is a game changer: Detection and eradication is not the way forward. Teaching in Higher Education, 0(0), 1–8. https://doi.org/10.1080/13562517.2023.2300951 Gee, J. P. (2008). A Sociocultural Perspective on Opportunity to Learn. In D. C. Pullin, E. H. Haertel, J. P. Gee, L. J. Young, & P. A. Moss (Eds.), Assessment, Equity, and Opportunity to Learn (pp. 76–108). Cambridge University Press. https://doi.org/10.1017/CBO9780511802157.006 Kramm, N., & McKenna, S. (2023). AI amplifies the tough question: What is higher education really for? Teaching in Higher Education, 28(8), 2173–2178. https://doi.org/10.1080/13562517.2023.2263839 Peters, M. A., Jackson, L., Papastephanou, M., Jandrić, P., Lazaroiu, G., Evers, C. W., Cope, B., Kalantzis, M., Araya, D., Tesar, M., Mika, C., Chen, L., Wang, C., Sturm, S., Rider, S., & Fuller, S. (2023). AI and the future of humanity: ChatGPT-4, philosophy and education – Critical responses. Educational Philosophy and Theory, 0(0), 1–35. https://doi.org/10.1080/00131857.2023.2213437 Popenici, S. (2023). Artificial Intelligence and Learning Futures: Critical Narratives of Technology and Imagination in Higher Education (1st edition). Taylor & Francis Ltd. Rawas, S. (2023). ChatGPT: Empowering lifelong learning in the digital age of higher education. Education and Information Technologies. https://doi.org/10.1007/s10639-023-12114-8 22. Research in Higher Education
Paper Implications of ChatGPT for Education Policy and Global Citizenship: A Case Study in Initial Teacher Training University of Lisbon, Portugal Presenting Author:The availability to the public of the Generative Artificial Intelligence tool ChatGPT has led to several reactions in society at different levels. Regarding higher education several challenges have arisen, especially in terms of ethics and evaluation, and its integration into teaching and research practices. In this study, we intend to explore mainly the issues related to the integration and ways of using ChatGPT in higher education, especially in initial teacher training, and the implications of this use for education policies and global citizenship. With the rapid development and widespread accessibility of Generative Artificial Intelligence (Gen-AI), it is paramount to understand its implications in various areas of society, in terms of knowledge creation and its contribution to the Sustainable Development Goals (UNESCO, 2021), notwithstanding the necessary epistemological reflection on its use (Figueiredo, 2023). In higher education, Artificial Intelligence (AI) has the potential to completely transform teaching and learning (Rawas, 2023). The potential of ChatGPT shows remarkable benefits in teaching, research support, automated grading, administrative management, and human-computer interaction (Dempere et al., 2023). It can provide individualized recommendations to students, increase collaboration and communication, and further improve their learning outcomes (Rawas, 2023). However, have been identified ethical concerns and implementation issues about security in student assessment and plagiarism, misuse, and the possibility of misinformation, as well as wider social and economic impacts such as job displacement, the digital literacy gap, or decreased human interaction (Dempere et al., 2023; Rawas, 2023). ChatGPT, as a Gen-AI tool, can help conversationally with writing, learning, solving and assessment, as an assistant for instructors and a virtual tutor for students (Lo, 2023). A literature review highlights measures relating to assessment methods and the necessary institutional policies. Rethinking assessment tasks to reduce the risk of plagiarism by requiring students to demonstrate their skills in real-time and in person, for example. Course content, learning outcomes and assessment methods can also be modified to circumvent ChatGPT, by using it to generate lesson topics, test and exam questions, homework, or product ideas (De Winter, 2023). On the other hand, from a more constructive and training perspective, it will also be important to promote students' digital literacy in the use of Gen-AI tools. Teaching students about the risks of relying on AI-based technologies is important. These risks include hallucinations, which are false responses generated by AI, presented as facts, not explained by the training data (Dempere et al., 2023). For this reason, it is important to integrate these technologies responsibly, as a supplement to and not a replacement for human interaction (Fuchs, 2023), and there is a pressing need to regulate AI in Higher Education Institutions (HEIs). As far as initial teacher training (ITT) is concerned, this phenomenon is even more relevant, since these students, as future teachers, will soon be training pupils in education systems. It requires teachers and students develop digital competences and literacies, with a strong focus on critical thinking and fact-checking strategies (Kasneci et al., 2023). Methodology, Methods, Research Instruments or Sources Used A qualitative approach will be used with recourse to non-participant observation and narrative research methods through the analysis of experiences developed in the curricular unit Initiation to Professional Practice of a Master’s in Teaching. To this end, data was collected taking into account: i) what are the main difficulties and constraints in use; ii) what are the benefits in the planning and preparation of classes; iii) what are the adaptations to instructional methods, form of assessment, and pedagogical practices needed to use the ChatGPT in the teaching and learning process in an ethical and safe way. In addition to the data from the empirical study, supported by the literature review, two Gen-AI tools, ChatGPT and Elicit, were trialled and their outputs analysed. Given the recent availability of these Gen-AI tools to the public, quality scientific studies published in the Scopus and WoC databases on this subject are still scarce, and the quality of the articles mobilised was prioritised over quantity. The study's qualitative approach took a naturalistic and hermeneutic perspective, using content analysis of the field notes from non-participant observation and of student narratives carried out as a final assignment (Amado, & Freire, 204; Bardin, 2013). This methodology is often used in research in the social sciences and education, as the researcher is dealing with complex situations in which it is difficult to select variables. In this way, the researcher seeks to describe and analyse a phenomenon and its interactions and does not intend to quantify or generalise. The narrative research method provides in-depth knowledge of the respondents' experiences and is based on a constructivist and interpretive epistemology (Rabelo, 2011). It considers that a narrative can express the complexity of the experience, as well as the relationships and uniqueness of each action (Bolívar et al., 1998), allowing knowledge to be obtained through an account that captures the details of meanings beyond factual statements or abstract propositions. Finally, it should be noted that informed consent was obtained from the study participants, thirteen preservice teachers, and their identity and anonymity were safeguarded, in accordance with the institution's ethics charter and international benchmarks, as Ethical Guidelines for Educational Research (BERA, 2011). Conclusions, Expected Outcomes or Findings Generative AI literacy will be an indispensable asset, as it provides students with the skills to critically engage with AI systems, ensuring that they become active and discerning users. At the same time, prompt engineering makes it possible to improve the outputs generated in a more precise way and enables educators and students to maximize the usefulness of the educational resources created by AI (Bozkurt, 2023). This study corroborates that, for the development of AI literacy, it is important to acquire proficiency in understanding, interacting with and critically evaluating generative AI technologies, which is essential not only for the current digital age, but also for shaping the future of education. It is also important to understand the ethical considerations, prejudices and limitations inherent in such systems, as well as to promote critical thinking and digital citizenship among students, teachers and researchers. So, Gen-AI literacy can and should be integrated into the curriculum to cultivate a new generation of informed and responsible users, and teachers should adapt their teaching methods to incorporate AI, preparing students for a future where it is an integral part of their personal and professional lives. The impact of AI on education and higher education cannot be ignored, and it is essential to integrate it into teacher education as well (Moura, & Carvalho, 2024). Recommendations include emphasizing a humanistic approach, mobilizing interdisciplinary planning, empowering teachers, and enhancing trust and safety. It also concludes that it is essential to address and include issues relating to artificial intelligence in higher education and to reflect them in legislation and educational policy. References Amado, J., & Freire, I. (2014). Estudo de caso na Investigação em Educação [Case study in Education Research]. In Manual de investigação qualitativa em educação [Handbook of qualitative research in education], (pp.121–168). Imprensa da Universidade de Coimbra. Bardin, L. (2013). Análise de Conteúdo [Content Analysis]. Edições 70. Bolívar, A., Domingo, J., & Fernández, M. (1998). La investigación biográfico–narrativa en educación. Guía para indagar en el campo. [Biographical-narrative research in education. A guide to research in the field.]. Grupo FORCE, Universidad de Granada, Grupo Editorial Universitario. Bozkurt, A. (2023). Unleashing the Potential of Generative AI, Conversational Agents and Chatbots in Educational Praxis: A Systematic Review and Bibliometric Analysis of GenAI in Education. OpenPraxis, 15(4), 261–270. https://doi.org/10.55982/openpraxis.15.4.609 De Winter, J.C.F., Dodou, D., & Stienen, A.H.A. (2023). ChatGPT in Education: Empowering Educators through Methods for Recognition and Assessment. Informatics, 10, 87. https://doi.org/10.3390/ informatics10040087 Dempere, J., Modugu, K., Hesham, A., & Ramasamy, L.K. (2023). The impact of ChatGPT on higher education. Front. Educ., 8, 1206936. https://doi.org/10.3389/feduc.2023.1206936 Ethical Guidelines for Educational Research (BERA) (2011). Available online: https://eera-ecer.de/about-eera/ethical-guidelines (accessed on 9th January 2024). Figueiredo, A. D. (2023). Inteligência Artificial Generativa e Construção de Conhecimento (Generative Artificial Intelligence and Knowledge Building). Personal communication. In Processamento de Linguagem Natural: Tendências e Aplicações Práticas Conference. https://doi.org/ 10.13140/RG.2.2.25801.52328 Fuchs, K. (2023). Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse. Front. Educ, 8. https://doi.org/10.3389/feduc.2023.1166682 Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., et al. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274 Lo, C.K. (2023). What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Educ. Sci., 13, 410. https://doi.org/10.3390/educsci13040410 Moura, A., & Carvalho, A. A. (2024). Literacia de Prompts para Potenciar o Uso da Inteligência Artificial na Educação [Prompt Literacy to Enhance the use of Artificial Intelligence in Education]. RE@D - Revista de Educação a Distância e Elearning, 6(2), e202308. https://doi.org/10.34627/redvol6iss2e202308 Rawas, S. (2023). ChatGPT: Empowering lifelong learning in the digital age of higher education. Educ Inf Technol. https://doi.org/10.1007/s10639-023-12114-8 Rabelo, A. O. (2011). A importância da investigação narrativa na educação [The importance of narrative enquiry in education.]. Educação & Sociedade, 32(114), 171-188. https://doi.org/10.1590/S0101-73302011000100011 UNESCO (2021). AI and education: Guidance for policy-makers. UNESCO. https://doi.org/10.54675/PCSP7350 | ||