Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

 
 
Session Overview
Session
24 SES 09 A: Integrating AI and Technology in Mathematics Education
Time:
Thursday, 29/Aug/2024:
9:30 - 11:00

Session Chair: Katarina Mićić
Location: Room LRC 019 in Library (Learning Resource Center "Stelios Ioannou" [LRC]) [Ground Floor]

Cap: 30

Paper Session

Show help for 'Increase or decrease the abstract text size'
Presentations
24. Mathematics Education Research
Paper

Killing Two Birds with the One GenAI Stone: Using GenAI in PD for Maths Teachers

Eleanor Byrne, Aibhin Bray, Brendan Tangney, Emma Berry

Trinity College Dublin, Ireland

Presenting Author: Byrne, Eleanor

Issues in mathematics education are wide-ranging, with the subject often perceived as hard, formulaic, and consisting of a series of unrelated abstract concepts, with a strong focus on assessment (Bray & Tangney, 2017). Historically an over-reliance on skills and procedures has led to a lack of mathematical fluency and conceptual understanding, with problem solving viewed as little more than worded calculations (Schoenfeld, 2016). The main drivers of reforms in mathematics curriculum internationally focus on efforts to address these issues by, at least in part, giving greater considerations to the student at the centre of the learning. Maths students should be supported to develop a positive disposition towards the subject, by highlighting connections between the different mathematical strands (NCCA, 2017) and teaching for robust understanding (Schoenfeld, 2017) to ensure maths becomes more relevant for school and society in general.

Recent iterations of maths education reforms continue to show that changing how we teach mathematics is difficult, teachers struggle to find time to engage with reform or create new resources and as a result tend to rely heavily on textbooks (O’Meara & Milinkovic, 2023). Although various technological advances have been heralded as a “silver bullet” that will solve the issues with student engagement with mathematics, take-up and implementation of such resources by teachers has often remained at the periphery (Bennison & Goos, 2010). Many reasons have been cited as barriers to teacher uptake of new technological developments, including systemic issues such as class-size, timetabling and cost (Bray & Tangney, 2017), as well as access and logistical problems. However, another pressing issue, is a need for professional development (PD) for teachers (OECD, 2015).

There is consistent evidence indicating that a sustained and experiential approach to PD is essential to support teacher change (Desimone, 2011). It is essential that practitioners are provided with opportunities to develop their own understanding of the value and relevance of any proposed change, as well as to recognise the impact that it might have on their practice and on student outcomes (Kärkkäinen, 2012).

The latest technological advancement that is predicted to have a significant impact on our societies and futures is generative AI (GenAI). Many questions have arisen about its potential impact on education, which are speculated to be both positive and negative (Giannini, 2023). While there are ethical concerns about the black box nature around the understanding of the AI processes, and the veracity of the information which it provides (Kaplan-Rakowski et al., 2023), there are also significant fears around cheating and plagiarism. However, when used appropriately, GenAI offers many opportunities, with UNESCO suggesting it can be used for activities ranging from idea generation to a reflection aid (Sabzalieva & Valentini, 2023). Of relevance to this work is the potential for GenAI to support teachers in the generation of, and reflection upon, lesson plans and resources that address the issues in mathematics education highlighted above. Hence the “two birds” reference in title – teachers are learning how to engage with GenAI while creating lessons which aim to meet the goals of curriculum reform. However, in order to support this, appropriate PD must be provided.

Constructivism and constructionism are the theoretical frameworks that underpins this research, acknowledging an approach in which both the technology and the user are constructing knowledge (Ackermann E., 2001). As part of a wider PD engagement with schools experiential GenAI workshops are being designed and delivered. The workshops support teachers through an immersive, iterative experience, to create and reflect upon lesson ideas, lesson plans and rich learning experiences, that give context and purpose to their lessons.


Methodology, Methods, Research Instruments or Sources Used
This research is following an action research methodology where an iterative series of workshops will be used to support maths teachers to engage with, and reflect on the potential of, GenAI to assist with the planning of lessons and materials which are contextualised to the needs and interests of their own cohort of students. Versions of the workshops are being rolled out in a number of different contexts including an PD intervention as part of the university’s outreach programme (Presenter, 2024) an Erasmus+ project spanning four countries – Ireland, Czech Republic, Austria, and Sweden – and in TCD teacher training courses.
The workshops will initially look at supporting teachers to engage in a collaborative dialogue with GenAI. The GenAI will be used to develop learning experiences by situating questions, tasks, and series of lessons within culturally significant contexts that are likely to interest students. The teachers will then be asked to reflect upon the generated materials to determine how useful they are perceived to be. Using Guskey’s five levels of effective PD evaluations as a framework (Guskey, 2002), participants will be asked to evaluate each workshop’s effectiveness, demonstrate their understanding of the material by beginning a dialogue with the GenAI and reflect on any materials with their colleagues to promote teacher efficacy. Hattie lists collective teacher efficacy as the greatest influence on student attainment (Donohoo et al., 2018), and it is hoped that teacher collaboration and reflection, supported by this intervention, will increase through the collective use of GenAI to develop materials for the classroom.
While determining direct student outcomes from this research will not be possible, we aim to generate qualitative and quantitative data to measure teacher’s perceptions of the effects on students, as well as their own self-efficacy in the use of GenAI to plan and create mathematics lessons that are relevant and engaging for their learners.

Conclusions, Expected Outcomes or Findings
This research is in its early stages and to date only one PD workshop supporting teachers to engage with GenAI has been delivered to a small number of participants (<20). Feedback from the group has been very positive, with a satisfaction rating of 8.5 out of 10, with an obvious appetite for further PD. The workshop materials have also been integrated into an undergraduate mathematics education module for prospective teachers and a postgraduate course in initial teacher education.  
GenAI has shown to be excellent at linking mathematics to real-life topics and giving multiple explanations in simple language. An example of this was the use of ChatGPT 4 to create questions that frame a series of maths lessons to give meaning and context. It was also used to convert questions into scenarios that might interest different groups of students, changing a question about party planning to one situated in the context of hurling (a popular Irish sport) or Fortnite (a popular computer game) in seconds. There are apparent gender biases evident already from using the technology, the GenAI creates baking and flower examples when asked for a female context, sport and computer games when asked for a male context. This will be highlighted going forward along with any other issues which arise. This is a rapidly changing field both in terms of capability and the range of platforms becoming available which will focus solely on education. Ongoing research will be needed to ensure education stays relevant.

References
Ackermann E. (2001). Piaget’s constructivism, Papert’s constructionism: What’s the difference. Future of learning group publication, 5(3), 438.
Bennison, A., & Goos, M. (2010). Learning to teach mathematics with technology: A survey of professional development needs, experiences and impacts. Mathematics Education Research Journal, 22(1), 31-56.
Bray, A., & Tangney, B. (2017). Technology usage in mathematics education research–A systematic review of recent trends. Computers & Education, 114, 255-273.
Desimone, L. M. (2011). A primer on effective professional development. Phi Delta Kappan, 92(6), 68-71.
Donohoo, J., Hattie, J., & Eells, R. (2018). The power of collective efficacy. Educational leadership, 75(6), 40-44.
Giannini, S. (2023). Generative AI and the future of education. ADG; UNESCO: Geneva, Switzerland, 2.
Guskey, T. R. (2002). Does it make a difference? Evaluating professional development. Educational leadership, 59(6), 45-51.
Kaplan-Rakowski, R., Grotewold, K., Hartwick, P., & Papin, K. (2023). Generative AI and teachers’ perspectives on its implementation in education. Journal of Interactive Learning Research, 34(2), 313-338.
Kärkkäinen, K. (2012). Bringing about curriculum innovations. In OECD Education Working Papers, No. 82. OECD Publishing (NJ1).
NCCA. (2017). Junior cycle mathematics syllabus. Dublin: Department of Education and Skills
O’Meara, N., & Milinkovic, J. (2023). Learning from the past: Case studies of past ‘local’curriculum reforms. In Mathematics Curriculum Reforms Around the World: The 24th ICMI Study (pp. 67-85). Springer International Publishing Cham.
OECD. (2015). Students, Computers and Learning. https://doi.org/doi:https://doi.org/10.1787/9789264239555-en
Presenter, E. B., Presenter, A. B., Presenter, B. T., Presenter, E. B. (2024, Aug 27-30). Expectancy-Value Theory in professional development for math teachers in areas of low SES European Conference on Educational Research, Nicosia, Cyprus.
Sabzalieva, E., & Valentini, A. (2023). ChatGPT and artificial intelligence in higher education: quick start guide.
Schoenfeld, A. H. (2016). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics (Reprint). Journal of education, 196(2), 1-38.


24. Mathematics Education Research
Paper

Use of Learning Analytics in K-12 Mathematics Education – Systematic Scoping Review of Impact on Teaching and Learning

Rebecka Rundquist, Kristina Holmberg, John Rack, Zeynab {Artemis} Mohseni, Italo Masiello

Linnaeus University, Sweden

Presenting Author: Rundquist, Rebecka

The generation and use of digital data and analyses in education comes with promises and opportunities, especially where digital materials allow use of Learning Analytics (LA) as a tool in Data-Based Decision-Making (DBDM). LA implies, analysing educational data to understand and optimise learning and learning environments (Siemens & Baker, 2012). In this paper we discuss LA as “a sophisticated form of data driven decision making” (Mandinach & Abrams, 2022, p. 196) as we explore how LA is used to support mathematics teaching and learning with digital materials in classroom practice. Data driven decision making or DBDM has been defined by Schildkamp and Kuiper (2010) as “systematically analyzing existing data sources within the school, applying outcomes of analyses to innovate teaching, curricula, and school performance, and, implementing (e.g., genuine improvement actions) and evaluating these innovations” (p. 482). DBDM is a key for the interpretation of LA, and can use any form of data, but in this review, the term DBDM is restricted to digital data. Using LA as a tool for DBDM could streamline data, making it more readily interpretable. However, questions remain about how usage can translate into practice (Mandinach & Abrams, 2022).

Quality of technology integration is not merely about technology use, but also about pedagogical use (Ottestad & Guðmundsdottir, 2018), about transformation and amplification of teaching as well as learning through use of technology (Consoli, Desiron & Cattaneo, 2023). LA within Digital Learning Material (DLM) can offer learners adaptive functions seamlessly embedded in DLMs or, provide learners (and teachers) compiled student assessments in relation to learning goals extracted from learning activities (Wise, Zhao & Hausknecht, 2014). The role of the teacher in student learning is clearly of central importance (Hattie & Yates, 2013; Yackel & Cobb, 1996), and teachers have a key responsibility to make digital technology a recourse in teaching to support student learning (Scherer, Siddiq & Tondeur, 2019).

This paper present findings from an exploratory systematic scoping review which was conducted regarding the use and impact of LA and DBDM in classroom practice to outline aspects related to Digital Learning Material (DLM), teacher usage, and student learning in the context of K-12 mathematics education.

A scoping review was deemed most appropriate since it can be performed even if there is limited number of published primary research (Gough, Oliver & Thomas, 2017), fitting new research areas such as LA, as it provides “a technique to ‘map’ relevant literature in the field of interest” (Arksey & O’Malley, 2005, p. 20), as well as combine different kinds of evidence (Gough, et al., 2017).


Methodology, Methods, Research Instruments or Sources Used
The methodology used the five-stage framework (Arksey & O’Malley, 2005), identifying the research question, identifying relevant studies, study selection, charting the data, collating, summarizing, and reporting the results. The databases ACM Digital Library, ERIC, PsycINFO, Scopus and Web of Science were chosen as they cover a wide range of topics within both technology and educational science to answer:
RQ1: How are analyses of digital data from DLM used in mathematics education?
RQ2: How do analyses of digital data from DLM impact teaching and learning?
The key elements of the research questions, Participants, Phenomena of Interest, Outcome, Context, Type of Source of Evidence (Arksey & O’Malley, 2005) were used to create the eligibility criteria. Publications that were included reported qualitative and/or quantitative data and were connected to the use of DLM and LA based on digital data involving students (between 6–19 years old) and teachers in mathematics K-12 education. The search was limited to papers published from 2000 up-to-date (March 2023) in English, Swedish or Norwegian. Exclusion criteria were developed to ensure consistency within the selection process.
Each record was screened by two reviewers and the relevance were coded according to the inclusion criteria. An independent researcher outside of the review group was consulted to design and validate the results of an inter-rater reliability test. The calculated inter-rater reliability score was 0.822, greater than 0.8, indicating a strong level of agreement (McHugh, 2012). After further screening 57 records were assessed to be eligible. At this stage the review pairs swapped batches and preformed data extraction showing, authors, year, title, location, aim, population, digital technology, method, intervention, outcomes, and key findings was performed for each record.
The final selection of 15 articles was made by group discussion and consensus. Discussions mainly centred around four components (use, analysis, learning and teaching). The heterogeneity in our sample demanded a configurative approach to the synthesis to combine different types of evidence (Gough et al., 2017). A thematic summary provided the analysis with a narrative approach to answer RQ1. To explore RQ2 more deeply, a thematic synthesis was performed (Gough et al., 2017). The analysis focused on LA-usage based on digital data for student learning, for teaching, and for teachers’ DBDM. PRISMA Extension for Scoping Reviews (PRISMAScR) (Tricco, Lillie, Zarin, O'Brien, Colquhoun, Levac et al., 2018) was used as guidelines for reporting the results.

Conclusions, Expected Outcomes or Findings
3653 records were identified whereof 15 studies were included. Results show that LA-research is an emerging field, where LA-applications is used across many contents and curricula standards of K-12 mathematics education. LA were mainly based on continuously collected individual student log data concerning student activity in relation to mathematical content. Eight of the studies included embedded analytics and all 15 studies included extracted analytics, but accessibility varied for students and teachers. Overall, extracted analytics were mainly mentioned as a function for teacher-usage, available as tools for formative assessment, where analytics need to be translated by teachers into some kind of pedagogical action (i.e., into teaching).
LA-usage supports a wide variety of teachers’ data use, and while mathematics teachers seemed to have a positive attitude towards LA-usage, some teachers were unsure of how to apply it into their practice. The thematic synthesis yielded two themes regarding teaching, which showed that teaching by DBDM focused on Supervision and Guidance. Results indicate extracted analytics is more commonly used for Supervision than Guidance.
Results regarding learning suggest that LA-usage have a positive effect on student learning, where high-performing students benefit most. The included studies examine students’ digital learning behaviour, by describing sequences of actions related to LA, learning outcomes and student feelings. Hereby, through the thematic synthesis, we capture parts of students’ studying-learning process and how it can be affected by LA usage. Finally, we suggest a definition of an additional class of LA, which we introduce as Guiding analytics for learners.
Going forward, research on using LA and DBDM is essential to support teachers and school leaders to meet today’s demands of utilising data, to be aware of possible unwanted consequences, and to use technology to enhance active learners and students’ ownership of learning.

References
Arksey, H., & O'Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32.
Consoli, T., Desiron, J., & Cattaneo, A. (2023). What is “technology integration” and how is it measured in K-12 education? A systematic review of survey instruments from 2010 to 2021. Computers & Education, 197, Article 104742.
Gough, D., Oliver, S., & Thomas, J. (red.) (2017). An introduction to systematic reviews. (2nd edition). Los Angeles, Ca.: SAGE.
Hattie, J., & Yates, G. (2013). Visible learning and the science of how we learn. Routledge.
Mandinach, E. B., & Abrams, L. M. (2022). Data Literacy and Learning Analytics. In Lang, C., Siemens, G., Wise, A. F., Gašević, D. & Merceron, A. (Eds.). Handbook of Learning Analytics (2nd. Ed., pp.196-204). SoLAR, Vancouver, BC.
McHugh M. L. (2012). Interrater reliability: the kappa statistic. Biochemia medica, 22(3), 276–282.
Ottestad, G., & Guðmundsdóttir, G. B. (2018). Information and communication technology policy in primary and secondary education in Europe. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Handbook of information technology in primary and secondary education (pp. 1–21). Springer.
Scherer, R., Siddiq, F., & Tondeur, J. (2019). The technology acceptance model (TAM): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education. Computers & Education 128, 13–35.
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education 26(3), 482–496.
Siemens, G., & Baker, R. S. J. d. (2012). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK '12). Association for Computing Machinery, New York, NY, USA, 252–254.
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA Extension for Scoping Reviews (PRISMAScR): Checklist and Explanation. Ann Intern Med, 169(7), 467–473.
Wise, A. F., Zhao, Y., & Hausknecht, S. N. (2014). Learning Analytics for Online Discussions: Embedded and Extracted Approaches. Journal of Learning Analytics, 1(2), 48‐71.
Yackel, E., & Cobb, P. (1996). Sociomathematical Norms, Argumentation, and Autonomy in Mathematics. Journal for Research in Mathematics Education, 27(4), 458–477.