16. ICT in Education and Training
Paper
Empirical Design of a Visual Learning Environment to Support Pupils’ Systems Thinking
Mina Mani Kashani, Måns Gezelius, Gunnar Höst, Marta Koc-Januchta, Jonas Löwgren, Konrad Schönborn
Linköping University, Sweden
Presenting Author: Mani Kashani, Mina
Climate change is a critical challenge facing society, and understanding earth systems, like the carbon cycle, has become an essential component of educational curricula around the world. The Swedish compulsory school curriculum emphasises learning about the carbon cycle and its connection with various biological and environmental issues. Understanding the carbon cycle is complex and requires recognising the carbon reservoirs, how carbon atoms circulate between reservoirs, and various dynamic relationships that exist within the system. These characteristics align with systems thinking skills, a crucial aspect of learning and teaching science. Assaraf and Orion (2005) have summarised eight hierarchical characteristics of systems thinking in the context of earth systems and have proposed the Systems Thinking Hierarchical (STH) model to describe how students learn about complex earth systems. The hierarchical levels of this framework include system thinking abilities that comprise analysis (e.g. identifying components), synthesis (e.g. relating components), and implementation (e.g. understanding hidden dimensions).
The carbon cycle is often taught through simplified and static diagrams in school textbooks, which can make learning about this abstract cycle and its interrelating components very challenging for high school students in grades 7-9 (e.g. Düsing, Asshoff, & Hammann, 2019). An example of a common difficulty in this context is to understand how carbon atoms move between various organisational levels. To address such challenges, carefully developed interactive visualizations that guide pupils through the components of the carbon cycle can help scaffold their systems thinking skills. Contemporary research in this area includes work on interactive learning environments in STEM contexts and adaptive feedback for supporting the learning of complex natural systems (Linn et al., 2014; Vitale, McBride, & Linn, 2016). Although these environments have proved promising, there remains a need to explicitly involve teachers in the design process, as well as connect established theoretical frameworks to learning goals of school science curricula. In this regard, not much effort has been directed to pedagogically-informed design and implementation of adaptive interactive learning environments for developing learners’ systems thinking. In fact, very little work has reported systematic design processes as an empirical contribution in the development of science education interventions (e.g. see Bopardikar, Bernstein, & McKenney, 2021).
In response, as part of a larger research program, the purpose of this work is to provide a theoretically and teacher-informed design process of an adaptive interactive visual learning environment that supports the development of grade 7-9 learners’ systems thinking skills in the context of the carbon cycle.
To respond to this aim we describe our iterative and theory-based design process by highlighting the main design activities and the rationale behind them, including: 1) content conceptualisation, 2) pedagogical (teacher) input, and 3) adaptive characteristics. The outcome of this process has resulted in an adaptive interactive visual learning environment with multiple learning tasks and quizzes organised in three modules. Each module is designed with coherent learning objectives aligned with a hierarchy of systems thinking skills and the Swedish school curriculum. Pupils interact with the learning tasks through three core mechanics including: A) dragging and dropping cards to complete a diagram, B) drawing arrows to complete the partial and global cycles, and C) clicking on the icons to reveal more information. Pupils’ interaction with this learning environment is supported through various forms of immediate (e.g. automatically correcting a misdrawn arrow) and delayed feedback (e.g. visual and textual verification of a correct response following a task response). Focusing on the carbon cycle, our work aims to provide a personalised learning experience for learners in grade 7-9 in scaffolding different levels of systems thinking.
Methodology, Methods, Research Instruments or Sources UsedWe employed an iterative explorative approach to design the target adaptive interactive visual learning environment for supporting systems thinking in the context of the carbon cycle. Emphasis was on defining requirements of the target by exploring alternative possibilities through multiple iterations (Floyd, 1984). Our design activities were structured in three clusters: 1) content conceptualisation, 2) pedagogical (teacher) input, and 3) adaptive characteristics.
The content of the learning environment was conceptualised according to the STH model (Assaraf & Orion, 2005) and Sweden's national curriculum through two interdependent processes. Firstly, we developed the domain ontology of the carbon cycle by aligning key learning objectives with the grade 7-9 curriculum and organizing them using the STH model. Defining coherent learning objectives such as identifying main carbon reservoirs and understanding the connection between them, provided the main structure of three learning modules. Secondly, in parallel, we designed interactive learning tasks and quizzes. The quiz questions aimed to enhance learning by building upon the interactive learning tasks by integrating the analysis, synthesis, and implementation systems thinking levels of the STH framework.
Pedagogical (teacher) input yielded from a panel of ten science teachers through three focus-group meetings and two sets of individual interviews was integrated with the design process for multiple purposes in several stages (e.g. Bopardikar et al., 2021). For mapping out the design space, the first focus-group meeting involved teachers reflecting on their pedagogical approaches and resources for teaching the carbon cycle. Through additional individual interviews, we asked for their feedback on the defined learning objectives and tasks with a modular structure and consequently integrated their feedback into the design of the environment. To verify our design approach for the three types of interaction mechanics, the main interaction patterns for four learning tasks were presented to teachers through individual interviews. These interviews resulted in adding quiz items to the learning modules to foster pupils’ systems thinking skills between STH levels. In the last step, we presented the panel a summary of the implemented learning tasks to validate our approach.
To implement an adaptive learning experience, we applied three adaptive difficulty levels to tasks and quiz questions. The difficulty level of the tasks and quiz questions was adjusted by implementing the mechanism of background logging of each pupil’s progress performance within the environment (Linn et al., 2014). Additionally, we designed and implemented various forms of immediate and delayed feedback to support pupils’ interaction and learning.
Conclusions, Expected Outcomes or FindingsApplying an iterative teacher-informed design-based approach resulted in Tracing carbon, an adaptive interactive visual learning environment for developing systems thinking skills in the context of the carbon cycle. Tracing Carbon entails twenty-one learning tasks and six quizzes embedded in three progressive learning modules for grade 7-9 aligned with the STH framework (Assaraf & Orion, 2005) and the Swedish school curriculum.
Pupils commence the learning experience by exploring how carbon circulates within a forest ecosystem in the first module. In the second module, students engage with global aspects of the carbon cycle, and in the third module they investigate the influence of human activities on the natural carbon cycle. As pupils progress through the learning modules, they actively interact with the visualisations and complete the visual based tasks while developing their systems thinking. This interaction is afforded through three core mechanics including: A) dragging and dropping cards to complete a diagram (e.g. components of the reservoirs), B) drawing arrows to complete the partial and global cycles, and C) clicking on the icons to reveal more information (e.g. about photosynthesis). Each learning module entails two quizzes that aim to support developing systems thinking skills in addition to reasoning and critical thinking.
Tracing carbon provides a personalised learning experience by adjusting the difficulty of the tasks and questions according to each pupil’s real-time performance. As pupils engage with Tracing Carbon, the environment tracks their progress, evaluates their performance, and adjusts the presented difficulty of the tasks and quiz questions. Various forms of immediate and delayed feedback validate pupils’ correct answers and supports them in addressing their errors during the tasks and quizzes.
Future work will explore pupils’ and teachers’ interaction with the environment and the impact of its adaptive characteristics on pupil’s learning of the carbon cycle.
ReferencesAssaraf, O. B.-Z., & Orion, N. (2005). Development of system thinking skills in the context of earth system education. Journal of Research in Science Teaching, 42(5), 518–560.
Bopardikar, A., Bernstein, D., & McKenney, S. (2021). Designer considerations and processes in developing school-based citizen-science curricula for environmental education. Journal of Biological Education, 1–26.
Düsing, K., Asshoff, R., & Hammann, M. (2019). Students’ conceptions of the carbon cycle: Identifying and interrelating components of the carbon cycle and tracing carbon atoms across the levels of biological organisation. Journal of Biological Education, 53(1), 110–125.
Floyd, C. (1984). A Systematic Look at Prototyping. In R. Budde, K. Kuhlenkamp, L. Mathiassen, & H. Züllighoven (Eds.), Approaches to Prototyping (pp. 1–18). Berlin, Heidelberg: Springer.
Linn, M. C., Gerard, L., Ryoo, K., McElhaney, K., Liu, O. L., & Rafferty, A. N. (2014). Computer-Guided Inquiry to Improve Science Learning. Science, 344(6180), 155–156.
Vitale, J. M., McBride, E., & Linn, M. C. (2016). Distinguishing complex ideas about climate change: knowledge integration vs. Specific guidance. International Journal of Science Education, 38(9), 1548–1569.
16. ICT in Education and Training
Paper
Interacting with a Visual Learning Environment of the Carbon Cycle: Pupils’ Use and Assessment
Marta Koc-Januchta, Gunnar Höst, Mina Mani, Måns Gezelius, Jonas Löwgren, Konrad Schönborn
Linköping University, Sweden
Presenting Author: Koc-Januchta, Marta
A fundamental prerequisite for developing environmental literacy for sustainability is understanding systems thinking (Kali et al., 2003). For example, developing the ability to interpret and understand the carbon cycle in terms of a system is necessary to grasp the monumental challenges posed by climate change (Shepardson et al., 2012). Although international school curricula, including countries like Sweden, promote the learning of the carbon cycle, science education research shows that understanding complex earth systems is challenging for pupils as it requires integrating knowledge from different levels of organisation and content areas (Düsing et al., 2019). Obstacles that pupils encounter include perceiving components of the system as separate “entities” rather than connecting them, or struggling to relate the system to everyday life (Assaraf & Orion, 2005). Systems thinking about earth systems requires mastering a range of skills, such as identifying the components of the cycle, through to thinking temporally about predictive implications of a system. Assaraf and Orion (2005) have articulated a framework of systems thinking abilities that consists of three hierarchical levels, namely Analysis (skills for identifying components of a system), Synthesis (skills for relating system components) and Implementation (skills for perceiving hidden system dimensions).
A large body of empirical evidence has confirmed the learning benefit of including pictorial elements in educational materials, and that careful design of multimedia resources that consider human cognitive processes has great influence on learning outcomes (Mayer, 2014). At the same time, Asshoff et al. (2010) claim that visually representing the complexity of natural processes such as the carbon cycle should provide more interactive and dynamic opportunities for learners. Therefore, it is rather surprising that the complexity of the carbon cycle is typically depicted and taught via static and often highly conventionalised diagrams. Little work has investigated how systems thinking can be supported through interactive, adaptive visualizations that also integrate aspects of canonical representations familiar to pupils and teachers.
This study forms part of a larger research program developing and testing an adaptive visual learning environment, termed Tracing Carbon, which supports pupils’ systems thinking skills in the context of the carbon cycle. Tracing Carbon comprises three modules, each integrated with interactive visual tasks and respective quiz questions aimed at probing abilities related to the three hierarchical levels (1-Analysis, 2-Synthesis, 3-Implementation). The current study purpose was to explore pupils’ interaction and performance with Tracing Carbon, guided by the following research questions. How do pupils:
- Interact with the Tracing Carbon learning environment when performing tasks?
- Perform on the quiz questions in terms of assigned hierarchy and difficulty levels?
- Assess the difficulty of quiz questions in terms of assigned hierarchy and difficulty levels?
Methodology, Methods, Research Instruments or Sources UsedA sample of 63 pupils aged 14-15 years from two Grade 8 classes engaged with the interactive visual learning environment about the carbon cycle as a part of a biology class. Tracing Carbon consists of interactive tasks and quizzes organized in three modules structured in chapters. In this study, the pupils had access to the first module (global aspects of the carbon cycle) and the first half of the second module (forest ecosystem), altogether comprising three chapters and three sets of quizzes. In each chapter, pupils first engaged with visual interactive tasks, followed by a quiz. After completing each quiz item, pupils also assessed the perceived difficulty of the item on a scale ranging from 0 (very easy) to 10 (very hard). Log file data automatically captured by the system provided information about the learning process, such as students’ mouse/pointer interaction with a particular graphical feature, or the number of mistakes pupils made while responding to the quizzes. Collectively, all pupils responded to quiz questions representing all three hierarchy levels. Additionally, quiz questions in each hierarchy level were assigned as “easy” or “hard”.
One type of visual interactive task in the system prompted pupils to draw arrows between components of the carbon cycle. Each arrow corresponds to a process that transfers carbon atoms between carbon reservoirs, such as when carbon atoms in carbon dioxide molecules are transferred to the biosphere through photosynthesis in plants. In a “simple” task, consisting of four reservoirs, the most common error (made by 39 pupils) was to draw arrows from Fossil fuel reserves to Land. In a “complex” task, consisting of 12 reservoirs, the most common mistake (made by 50 pupils) was to draw an incorrect-connection arrow from Decomposers to Plants.
Additionally, we performed GLM repeated measures analyses of variance with number of incorrect answers and difficulty assessment by pupils as dependent variables. We found for both variables significant main effects of assigned difficulty levels (easy vs. hard, F (1, 45) = 17.60; p < .001; η2 = .28 and F (1, 45) = 35.84; p < .001; η2 = .44, respectively). Questions assigned as hard resulted in a higher number of incorrect answers and a higher level of assessed difficulty by pupils. We also observed significant interaction effects for both dependent variables.
Conclusions, Expected Outcomes or FindingsAnalysis indicates that overall, the quiz items designated by the researchers as “easy” were associated with fewer mistakes and a lower perceived difficulty rating than quiz items designated as “hard”. This supports the validity of the quiz item design and integration in Tracing Carbon. However, quiz items designed to engage the second level (Synthesis) in the applied hierarchical systems thinking framework (Assaraf & Orion, 2005) deviate from this pattern. This calls for a deeper consideration of what makes a synthesis-level quiz item easy or hard. The required cognitive abilities might be expected to be more complex for quiz items designed to test for higher levels of the hierarchical systems thinking framework. Nevertheless, the findings do not indicate a corresponding consistent difference in the number of errors or perceived difficulty between quiz items related to the three levels. This result suggests that measurement of hierarchy level understanding is complex and cannot be simply reflected by number of errors alone. In addition, qualitative analysis could help shed light on what types of errors pupils made in the questions and if there is a link between type of mistakes made in interactive tasks and type of mistakes made in quiz questions.
Analysis of interaction data from log files reveals multiple errors related to both drawing erroneous arrows and in the wrong direction. However, the errors were not evenly distributed among the possible errors and could therefore be related to misunderstandings that are commonly found in the literature. For example, the very common incorrect connection made between decomposers and plants could be related to consistently reported erroneous conceptions where many learners believe that trees obtain their energy and building blocks from the soil, rather than from carbon dioxide and solar radiation (e.g. Wennerstam et al., 2020).
ReferencesAssaraf, O. B. Z., & Orion, N. (2005). Development of systems thinking skills in the context of earth system education. Journal of Research in Science Teaching, 42(5), 518-560.
Asshoff, R., Ried, S., & Leuzinger, S. (2010). Towards a better understanding of carbon flux. Journal of biological education, 44(4), 175-179.
Düsing, K., Asshoff, R., & Hammann, M. (2019). Students’ conceptions of the carbon cycle: Identifying and interrelating components of the carbon cycle and tracing carbon atoms across the levels of biological organisation. Journal of Biological Education, 53(1), 110-125.
Kali, Y., Orion, N., & Elon, B. (2003). The effect of knowledge integration activities on students’ perception of the earth’s crust as a cyclic system. Journal of Research in Science Teaching, 40, 545-565.
Mayer, R. E. (2014). The Cambridge handbook of multimedia learning (2nd ed.). Cambridge University Press
Shepardson, D. P., Niyogi, D., Roychoudhury, A., & Hirsch, A. (2012). Conceptualizing climate change in the context of a climate system. Environmental Education Research, 18(3), 323-352.
Wennersten, L., Wanselin, H., Wikman, S., & Lindahl, M. (2020). Interpreting students’ ideas on the availability of energy and matter in food webs. Journal of Biological Education, 1-21.
|
