11. Educational Improvement and Quality Assurance
Paper
KNOWLO Project Promoting Knowledge Sharing Culture in Vocational and Higher Educational Institutions in Europe
Mudassir Arafat
International College of Cosmetology, Latvia
Presenting Author: Arafat, Mudassir
Undoubtedly the progress of any nation highly depends upon the proficiencies of its younger generation (Taleba & Hassanzadehb, 2015). Modern learning organizations and modern learning techniques have been continuously emphasized by the EU to harness the capabilities & exploit the potential of generation Z in the diverse areas of education in relation to ICT (Reding, 2003).
Previously, various researchers have focused on harnessing and exploiting new technologies that equip learners to learn with flexibility, comfort, and interaction. Nowadays learners are equipped with smart handheld or desktop devices to enable them to access digital learning materials with ease. Smart education and Modern Smart Learning organizations have become a concept that has acquired huge attention and is considered to be a necessity in this modern age. (Zhu, Yu, & Riezebos, 2016)
To overcome these changing trends and prepare the masses for a modern and technically advanced job market the European Union’s educational support program Erasmus+ has been working extensively on fostering many research, exchange, and development programs. KNOWLO project is one such program funded & supported by Erasmus+.
The aim of the KNOWLO Project is to develop a framework that helps traditional VET & HEIs transform into modern learning organizations.
This paper aims to offer crucial support to any traditional learning organization on the path of transforming into a modern learning organization. This paper can be treated as a case study towards identifying various aspects of consideration when undergoing transformation.
The KNOWLO project has 5 results to develop namely R1 to R5 where R1 is a Transformational Framework, R2 Self-Assessment tool for organizations, R3 Learning & sharing platform for VET & HEIs, R4 Resource Database, and R5 promotion of a Smart sharing culture.
This paper highlights crucial criteria developed under Result 1 for the transformational framework.
A SMART learning organization is a place where resources such as academics and intellect are used to their optimum (Vveinhardt & Minkute-Henrickson, 2015). One of the challenges is how to self-assess if an organization is traditional or smart to foster a knowledge-sharing culture.
The most common tool to assess is a survey that involves a crucial set of criteria necessary for the transformation.
A smart learning organization can possess various aspects of a traditional learning organization, however, the aspects that make it smart are features such as organizational self-awareness, communication, diversity, inclusion, emotional intelligence, and digitalization (Uskov et.al, 2019).
According to (Botella et.al, 2017) emphasis has mostly been given to student self-assessment in terms of their educational competencies and ability with digital learning and psychology. According to (J.W.Gikandi et.al, 2011) it is stated that online instruction, in general, is considered more beneficial compared to traditional mode. However, it is stressed that teaching and learning processes require to be centrally assessed so that they can provide learners with opportunities to demonstrate their abilities and capabilities of an organization to foster such developments.
The systematic review by Al-Kurdi et.al (2018) highlights the fact that there are limited contributions in the understanding of knowledge-sharing culture in HEIs in comparison to other sectors. According to (Taylor & James, 2001) Organizational transformations do not happen automatically, it needs proactive human mediation. Stakeholders’ participation often benefits such transformations in organizations. Lee (2018) claims that the impact of knowledge sharing on people has not been given due consideration. Hence, the KNOWLO project is committed to working in these areas, helping learning organizations with a framework that comprises all the essential criteria necessary for a Smart Learning organization.
Methodology, Methods, Research Instruments or Sources UsedThis article contributes to this research segment by presenting real-time results of an ongoing Erasmus+ funded educational project, where the "International College of Cosmetology (ICC) in Riga, Latvia", is the project coordinator, with project partners from Latvia, Slovakia, Italy, and the Czech Republic. The project is ongoing and is subject to be finalized by December 2023. As of current the data collected and analyzed is for Result 1, the development of smart framework criteria. The data gathered has been analyzed per different regions of the EU, enabling us to understand which EU regions are lagging behind technological transformations and other aspects of the transformation model.
The social-emotional intelligence model (Channell, 2021), the Technology Acceptance Model, and various other Models have been used to define the framework criteria. It substantiated that those various aspects of the criteria demand the incorporation of various paradigms.
Research involves 2 purposeful samples: in total 265 stakeholders as survey respondents from VET & HEIs from the partner countries. KNOWLO Consortium partners details: 1. International College of Cosmetology (Riga, Latvia), 2. Eurofortis It (Riga, Latvia), 3. Catholic University of Ruzomberok (Slovakia), 4. Harmony Academy (Tarnava, Slovakia), 5. Schola Empirica (Prague, Czech Republic), and 6. Euroreso (Italy).
Research question: Are the teachers exercising the use of Modern Technologies in teaching to its maximum?
Research methods: 1) data collection – stakeholders’ survey (closed-ended questions) for quantitative results and for qualitative results interviews. Five, structured Interviews of stakeholders’ from HEIs in Latvia, Slovakia, and the Czech Republic were done. The Interviewees were a professor from the Catholic University of Ruzomberok, Slovakia, a Language School manager from Harmony Academy Slovakia, and a Communication & Marketing Manager from the International School of Latvia, and the remaining two were adult learners from the International College of Cosmetology in Riga Latvia. Mixed methods of research were used. For qualitative data analysis, narrative analysis was used and for the quantitative data – descriptive statistics were used!
Research process: Stakeholders’ who participated in the KNOWLO survey were specifically targeted according to the project proposal and guidelines. Respondents answered questions that highlighted Organizational self-awareness, sustainable goals, Digitalization of learning practices in a Global Context, communication, cooperation, and people. The structured interviews helped understating, the stakeholders’ self-assessment patterns and tools exercised to evaluate results.
Research period: November 2021 – November 2022.
Conclusions, Expected Outcomes or FindingsAs stated above, this report only represents the development of Result 1, Transformational Framework. Key criteria established for the Stakeholders survey under R1 for the framework are 1. Organizational self-awareness, strategy, and development 2. Learning, communication, and cooperation 3. Organization and its people 4. Clients 5. Sustainability and Product Orientation 6. Digital transformation, global context, and value creation, and 7. Results & Benchmarking.
As each established criterion answered respective questions, criteria number 6. Digital transformation, in a global context, and value creation, are analyzed in this report in context with the research question (Are the teachers exercising the use of Modern Technologies in teaching to its maximum?).
The results analyzed highlighted that teachers from the Czech Republic were more likely to agree, that they use technology to its maximum whilst teaching than teachers from Italy. The Czech Republican teachers were not convinced that the assessment of students prior to training digitally can help in utilizing technology to its maximum.
Stakeholders from all the partner countries agreed on two things unanimously there is not enough technical equipment in the organizations and not enough government funding to support the use of technology to the fullest.
The Knowlo framework that will help traditional HEIs & VETs transform into modern learning organization starts with digital transformation, the qualitative results show that stakeholders agree that new methods/forms of learning and individual approach to learners is essential. Other aspects that the stakeholder's survey highlight 60% believe that organizational vision must be clear to all 60% that constructive feedbacks help an organization excel 67%.
KNOWLO project is ongoing and after the successful completion of the Project, an effective self-evaluation tool and framework will be made available for learning organizations seeking transformation from traditional to Modern learning organizations
ReferencesAl-Kurdi et.al, R. E.-H. (2018. gada 5. March). Knowledge sharing in higher education institutions: a systematic review. Journal of Enterprise Information Management, 31(2), 226-246. doi:10.1108/JEIM-09-2017-0129
Botella et.al, E. P. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74-98. doi:10.1016/j.edurev.2017.08.004
Channell, M. (2021. gada 13. October). Daniel Goleman’s Emotional Intelligence In Leadership: How To Improve Motivation In Your Team. Ielādēts no https://www.tsw.co.uk/blog/leadership-and-management/daniel-goleman-emotional-intelligence/#:~:text=Daniel%20Goleman's%20emotional%20intelligence%20theory,happier%20and%20healthier%20working%20culture
J.W.Gikandi et.al, D. N. (2011. gada December). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333-2351. doi:10.1016/j.compedu.2011.06.004
Lee, J. (2018 . gada 14 . May ). The Effects of Knowledge Sharing on Individual Creativity in Higher Education Institutions: Socio-Technical View. Division of Interdisciplinary Wellness Studies, Soonchunhyang University, 22 Soonchunhyang-ro, Asan, Chungnam 31538, Korea, 1-16. doi:https://doi.org/10.3390/admsci8020021
Reding, V. (2003). e-learning for Europe. European Council, Education & Culture. Brussels: Publications.eu.int. Ielādēts no http://europa.eu.int
Taleba, Z., & Hassanzadehb, &. F. (2015. gada 16th . January). Toward Smart School: A Comparison between Smart School and Traditional School for Mathematics Learning. Procedia - Social and Behavioral Sciences, 171 , 90-95. doi:https://doi.org/10.1016/j.sbspro.2015.01.093
Taylor & James, C. (2001). Fifth generation distance education. Instructional Science and Technology, 4(2), 1-14. Ielādēts no http://www.usq.edu.au/e-jist/
Uskov et.al, V. L. (2019. gada June). A Validation of “Smartness Features—Main Components” Matrix by Real-World Examples and Best Practices from Universities Worldwide. Smart Education and e-Learning 2019, 144, 3-17. doi: 978-981-13-8259-8
Vveinhardt, o., & Minkute-Henrickson, R. (2015). Transformation of a learning organization into a smart organization: expansion of human resource by intellectual capital. Proceedings of the EDULEARN15 Conference, 172-181. doi:978-84-606-8243-1
Zhu, Z.-T., Yu, M.-H., & Riezebos, &. P. (2016. gada 31. March). A research framework of smart education. Smart Learning Environments. doi:10.1186/s40561-016-0026-2
11. Educational Improvement and Quality Assurance
Paper
A Comprehensive Systematic Review of AI in NLP, EDM, and LA for Feedback in K-12 Education
Burcu Toptas1, Munevver Ilgun Dibek2
1Ankara University, Turkiye; 2TED University, Turkiye
Presenting Author: Toptas, Burcu;
Ilgun Dibek, Munevver
AI, or artificial intelligence, refers to computing systems that can perform tasks similar to those of humans, such as adapting, learning, and using data for complex processing (Popenici & Kerr, 2017). There are various branches and sub-branches of AI, but for feedback purposes, the most relevant ones are natural language processing (NLP), educational data mining (EDM), and learning analytics (LA) (Gardner et al., 2021). NLP is beneficial for feedback because it can analyze linguistic components of students’ written work and provide feedback on writing quality, syntactic complexity, and grammatical errors. EDM allows for data-supported feedback through data visualization and can also provide verbal feedback using NLP or manual input from instructors. LA uses student activity data to provide personalized feedback through an interactive dashboard. Feedback can be either semi-automatic or fully automatic, depending on the system used (Wongvorachan et al., 2022). AI has been incorporated into NLP, EDM, and LA, leading to the development of complex systems that can provide students with timely and individualized feedback. As a result, both their performance and learning process can be improved. It has been demonstrated that AI-based feedback systems are more effective and efficient than more conventional forms of feedback.
It has been demonstrated that incorporating AI into feedback improves student motivation and engagement, which results in higher learning outcomes (Alazmi & AlZoubi, 2020). Moreover, it has been revealed that AI-based feedback systems are economical and scale to large classrooms, making them appropriate for use in both traditional and online learning environments (Chang et al., 2020). Thus, the integration of AI in feedback is not only improving the learning experience of students but also transforming the traditional methods of feedback in education. According to Zawacki-Richter et al. (2019), the incorporation of AI in K-12 education has seen significant growth in recent years. Crompton and Song (2021) also note that the use of AI offers numerous possibilities for enhancing teaching and learning. One way AI is being utilized is in the automatic grading of essays, as reported by Yang et al. (2019). Additionally, AI can provide swift feedback to students, as stated by Benotti et al. (2018), and can adjust instruction to meet the unique needs of each student, as highlighted by Arnett (2016).
A systematic review of the application of AI and robotics in K-12 education was carried out by Hrastinski et al. (2019), with an emphasis on the relationship between teachers and students. However, the scope of the study was limited to papers from one international symposium and solely on robotics, rather than AI more broadly. Furthermore, it did not examine the potential of AI in enhancing feedback practices in K-12 education. Zafari et al.'s (2022) and Crompton et al.’s (2022) studies examined the current state of AI integration in K-12 education, with a focus on its general use, not just its use for feedback.
The aim of this study is to address the call by scholars (Banihashem et al., 2022) to investigate the role of NLP, EDM, and LA in enhancing feedback practices in K-12 education. This paper will provide researchers and educators with a deeper insight into the application of NLP, EDM, and LA for feedback purposes. In this regard, the present systematic review seeks to answer the following research questions:
1. What are the primary reasons behind the utilization of NLP, EDM, and LA in feedback studies in K-12 education?
2. What types of data are utilized in studies on NLP, EDM, and LA to provide feedback in K-12 education?
3. What NLP, EDM, and LA tools and techniques are employed by studies to facilitate feedback in K-12 education?
Methodology, Methods, Research Instruments or Sources UsedWe followed the PRISMA framework (identification, screening, eligibility and inclusion) to conduct the systematic review.
Search strategy
We searched the most relevant terms and synonyms which are overlapping the concepts that the present study focused on by identifying the prior systematics reviews (e.g., Banihashem et al., 2022; Zafari et al., 2022) on K-12 education, artificial intelligence or feedback practices. The search string included the following terms: ("learning analytic*" OR "educational data mining" OR "artificial intelligence" OR "natural language processing") AND (feedback OR "formative assessment" OR feedforward) AND ("K-12 student*" OR "K-12" OR "K-12 education" OR "primary school*" OR "primary education" OR “kindergarten*" OR “pre-primary” OR “middle school*" OR "secondary education" OR "secondary school*" OR “high school*” OR “1st grade” OR “2nd grade” OR “3rd grade” OR “4th grade” OR “5th grade” OR “6th grade” OR “7th grade” OR “8th grade” OR “9th grade” OR “10th grade” OR “11th grade” OR “12th grade” OR "grade 1" OR “grade 2" OR "grade 3" OR "grade 4" OR "grade 5" OR "grade 6" OR "grade 7" OR "grade 8" OR “grade 9" OR "grade 10" OR "grade 11" OR "grade 12"). Web of Science (WOS), ERIC, and IEEE databases were chosen considering their reputation and inclusion of numerous research studies on the topics that were addressed in the current study.
Criteria for inclusion
The following criteria were used to determine which articles were included: (a) journal articles published between 2013 and 2023; (b) articles written in English language, (c) peer-reviewed journals to ensure quality, and (d) empirical studies. However, conference proceedings were excluded.
Identification of relevant publication
During the initial screening phase in the selected databases (WOS [n=72], ERIC [n=55], and IEEE [n=443]), a total of 570 were identified. After eliminating duplicates (n=22) and non-peer-reviewed articles (n=10), a pool of publications (n=538) remained. In the second phase, the titles and abstracts were screened against our inclusion criteria, and 459 papers did not meet the criteria, 79 papers were further evaluated through full-text screening. The final pool of papers was then used for quality appraisal.
Quality appraisal
We adopted quality appraisal criteria from Theelen et al. (2019), based on the work of Savin-Baden and Major (2007) for evaluating qualitative studies and NICE (2012) for evaluating quantitative studies.
Conclusions, Expected Outcomes or FindingsPreliminary Findings and Expected Outcomes
The Rayyan program is utilized for reviewing research papers. It's a free online tool for scientists conducting systematic reviews and similar projects. Initially, we used it as a blind version for accuracy purposes. We independently evaluated the papers by marking "include" or writing reasons for “exclusion”. After both authors finished evaluating the articles, we switched to the unblinded version and resolved conflicts. We disagreed on five papers, with one author wanting to include while the other excluded. We ultimately included only three of these five studies. At least one of us used the word "maybe" in evaluating 34 papers, so we also evaluated each paper as a team. We used the label "maybe" because the abstract was not clear on which AI techniques used. After our team review, we chose 16 of them for further evaluation because the full text can aid in labeling the technique. Out of the articles where the technique was identifiable, 42 utilized LA, 16 used NLP, and 5 employed EDM. We employ Nvivo to conduct content analyses relevant to our research questions. We will also conclude our review by highlighting key challenges and opportunities for future research.
ReferencesAlazmi, B., & AlZoubi, A. (2020). The role of artificial intelligence in education. Journal of Education and e-Learning Research, 7(2), 19-30.
Arnett, T. (2016). Teaching in the machine age: How innovation can make bad teachers good and good teachers better. Christensen Institute.
Banihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. (2022). A systematic review of the role of learning analytics in enhancing feedback practices in higher education. Educational Research Review, 37.
Benotti, L., Martinez, M.C., & Schapachnik, F. (2018). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192.
Chang, K.E., Huang, Y.M., & Chen, W.H. (2020). A review of AI-based feedback systems for education. JETDE, 3(1), 1-14.
Crompton, H., Jones, M.V., & Burke, D. (2022). Affordances and challenges of artificial intelligence in K-12 education: a systematic review. JRTE, 1-21.
Crompton, H., & Song, D. (2021). The potential of artificial intelligence in higher education. Revista Virtual Universidad Católica Del Norte, 62, 1–4.
Gardner, J., O’Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: Breakthrough? Or buncombe and ballyhoo?. Journal of Computer Assisted Learning, 37(5), 1207–1216.
Hrastinski, S., Olofsson, A.D., Arkenback, C., Ekström, S., Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L-M., Fuentes, A., Gustafsson, U., Humble, N., Mozelius, P., Sundgren, M., & Utterberg, M. (2019). Critical imaginaries and reflections on artificial intelligence and robots in post digital K-12 education. Postdigital Science and Education, 1(2), 427–445.
Popenici, S.A.D. & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. RPTEL, 12(22).
Theelen, H., Van den Beemt, A., & den Brok, P. (2019). Classroom simulations in teacher education to support preservice teachers’ interpersonal competence: A systematic literature review. Computers & Education, 129, 14-26.
Wongvorachan,T., Lai, K.W, Bulut, O. Tsai, Y. & Chen, G. (2022). Artificial Intelligence: Transforming the Future of Feedback in Education. Journal of Applied Testing Technology, 23(1), 1-22.
Yang, Y., Xia, L., & Zhao, Q. (2019). An automated grader for Chinese essay combining shallow and deep semantic attributes. IEEE Access 7.
Zafari, M., Bazargani, J.S.,Sadeghi-Niaraki, A., & Choi, S.M. (2022). Artificial intelligence applications in K-12 education: A systematic literature review. IEEE Access, 10.
Zawacki-Richter, O., Marín, V.I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? IJETHE, 16(1), 1–27.
|
