Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

Please note that all times are shown in the time zone of the conference. The current conference time is: 17th May 2024, 07:46:32am GMT

 
 
Session Overview
Session
22 SES 06 C
Time:
Wednesday, 23/Aug/2023:
1:30pm - 3:00pm

Session Chair: Carolina Guzmán-Valenzuela
Location: Adam Smith, 717 [Floor 7]

Capacity: 35 persons

Paper Session

Show help for 'Increase or decrease the abstract text size'
Presentations
22. Research in Higher Education
Paper

Unexpected Outcomes in Personalized Assessments: Towards Digital Maturation of Teaching and Learning in Higher Education

Brian Denman, Meeri Hellstén, Helena Reierstam

Stockholm University, Sweden

Presenting Author: Denman, Brian; Hellstén, Meeri

The central theme of this paper focuses on the recent shift from standardization to differentiation in assessments through the advent of digitalization in higher education. Digitalisation has been borne as an artificial solution to academia’s issues with the contextual complexity of assessment. It addresses the issues surrounding a reluctance of some academics to mark and grade assignments for benchmarking, tracking, and scaffolding which have been swiftly overshadowed by digitalization, and particularly during the recent Covid-19 global pandemic. Elements of digital transition include Artificial Intelligence (AI) software applications, which have been able to inspire an entire new learning community in preparing graduates with new and desirable skills, knowledge, and ecumene with the aim of prospering in the world of digitalized futures.

This inquiry into digitalization has allowed us to explore tangible evidence of what is learnt in an equitable fashion, the breadth and depth of disciplinary knowledge, and how this knowledge is conceived, internalized, reflected on, expanded, and abstracted. Parenthetically, this combines linkages between theory / knowledge, disciplinary methodological approaches, and prospects for further application in the real world outside the university.

As a caveat, it is our understanding that technology continues to advance the workforce and, as such, so too should the advancement of university teaching and learning. Higher education institutions have responded by re-evaluating outdated teaching and learning methods and embracing new technologies in approaches ranging from establishing learning management systems to shifting from teacher-focused to student-centred learning. Unfortunately, much of the technology advancements to date and particularly as a function of the pandemic, have focused merely on online delivery---relying heavily on learning management systems---and much less on the technical utility and applicability of teaching and learning analytics.

Inspired by digitalization issues in the provision of the quality of teaching and learning in higher education, efforts to uncover an alignment between academic teaching and assessment have attracted our attention to trial what personalized assessments may accomplish across academic discipline and cultural and learning boundaries.

A digital Personalised Assessment Tool (PAT) was developed initially in Australia in response to Gonski et. al's white paper (2011) on the quality of education. The PAT is an attempt to provide a more equitable assessment solution in analyzing and recognizing learner difference among undergraduates and postgraduates from diverse learning backgrounds (urban, regional, rural, remote). Developed as an alternative to standard assessment tools and utilizing a theoretical modification of Seligman's 2002 strength-based assessment approach, the PAT was further enhanced for exploring how five higher order skills (creativity, communication, analytical thinking, problem solving, and reflective thinking) might be better benchmarked, tracked, and scaffolded using Fiske and Taylor's social cognition theory (2013). The PAT was subsequently trialed in China and Sweden to explore the applicability of teaching and learning analytics across cultures, institutions, and disciplinary boundaries.


Methodology, Methods, Research Instruments or Sources Used
In the current paper, we present on a comparative case study analysis that was undertaken to systematically describe cultural, institutional, and disciplinary contexts where the PAT was piloted (Australia, China, and Sweden).  The aim of the comparison was to determine the preparedness of academic staff to utilize the PAT, the identification of inferences drawn from using the PAT by various academic staff, and the utilization of the instructional data for individual and institutional gain.  The objective of the study was to determine if the PAT was viewed as equitable, technologically relevant, and whether it promoted 'quality' assessment.

The research questions explored:
1) Can alternative assessments like PAT be used to provide a more equitable approach to promote differentiation while, at the same time, adhere to core disciplinary requirements?

2) Is it possible to create a bottom-top approach towards implementing new technologies in academic assessments?

3) Could the benchmarking, tracking and scaffolding of salient higher order skills improve teaching and learning as well as feedback?

The current presentation reports on case studies that were collected from 2016-2023, which include over 150 undergraduates and postgraduates who volunteered to be assessed using the PAT along with academic staff approvals.  Ethical practices were adhered to, including GDPR policy guidelines, with emphasis placed on curriculum development by each respective higher education institution.

Conclusions, Expected Outcomes or Findings
While the comparative cases are exploratory in nature, the paper presents on empirical evidence in assessing each student (n=150) on five higher order skills and representing over 20 psychometric characteristics, in order to highlight how a systematic approach toward assessment invites a change in academic discourse toward quality assessment over time. The findings yield indications regardless of culture or institution.  Approaches toward the utility of PAT proved meaningful. These seemed to motivate a renewed interests in showcasing individual student's ability and learning capability. These revealed the function of a form of gatekeeping for admissions purposes on the one hand and enabling at-risk students to complete their studies in the other.  Presenting the data in the form of assessment visualisations further encouraged a non-apprehensive dialogue to take place between academic staff and students. Such dialogue subsequently improved assessment feedback practices and gave further carriage that allowed to delve deeper and more broadly into core curricular content. The paper presentation elaborates and illustrates the details of these findings and invites input and critical comments from the community of scholars attending the paper session.
References
Denman, Brian D. & Meeri Hellstén (in press).  Equity and Quality Education:  Retrospect and prospect.  International Handbook on Education Development in Asia Pacific.  Wing On Lee, Phillip Brown. A. Lin Goodwin and Andy Green, eds.  Springer Press.

Denman, Brian D., (2022).  "From Global Thinker to Innovative Mind".  In: International Handbook of Education Development in Asia Pacific.  Wing On Lee, Phillip Brown, A. Lin Goodwin, and Andy Green, eds.  Springer Press, https://link.springer.com/referenceworkentry/10.1007/978-981-16-2327-1_18-1

Fiske, Susan T., and Shelley E. Taylor (2013).  Social Cognition from Brains to Culture, 2nd Edition.  Sage, London.

Gonski, D., Boston, K., Greiner, K., Lawrence, C., Scales, B., & Tannock, P. (2011). Australian government review of funding for schooling: Final report December 2011. Department of Education EaWR (Ed). Canberra City, ACT: Department of Education, Employment and Workplace Relations.


22. Research in Higher Education
Paper

Seminars- Student Participation in the Digital Classroom.

Sunniva Hegvik, Anne Røisehagen

Inland Norway University, Norway

Presenting Author: Hegvik, Sunniva; Røisehagen, Anne

Today's society is characterized by changes that place greater demands on the competence development of professionals. The White Papers, Meld. St. 16 (2016-2017) and Meld. St. 14 (2019/2020)) points out that digitalization and flexibility are a key to making education accessible to adult students. During the corona-pandemic, mainly teaching in higher education was done through digital learning platforms. Bern, Lorentzen and Noranger (2021) have examined the student's participation in digital teaching. The survey indicates that visibility, belonging and community have an impact on the students' participation in the digital lessons, (Bern, Lorentzen, & Nordangen, 2021). Fossland and Tømte (2019) have looked into the tension between active participation and passive acquisition in knowledge development. Findings from their survey show that affiliation too group and the program appear to be decisive for participation and collaborative learning in online studies.

In this article we will look at how the use of online seminars can help to increase student participation in the digital classroom.

Our research question is:

  1. Which factors do students think are important for their own participation in online seminars?
  2. Which characteristics of the teacher do students think contribute to their own participation in online seminars?

Theoretic framework:

In the academic literature, it can be difficult to distinguish clearly between the classical seminar and the seminars as group teaching. It is therefore difficult to identify the criteria that characterize a good seminar in the academic literature. Roar Pettersen (2005) uses group teaching where one can recognize principles and working methods that we think are somewhat similar to seminars in our understanding. He describes group teaching as an arene for dialogue- and discussion-like teaching in small groups. What can be interpreted from the teaching activity seminar or group teaching is that several of these teaching methods are dependent on activity and participation. According to Illeris (2012), the most general form of interaction is participation. Participation can be characterized by the learner being part of a targeted activity that is common with someone else, a community of practice. That is, the more active one is, and the more one engages, the greater the chance that one learns something significant, and that one learns in such a way that one can remember it and use it in relevant contexts (Illeris, 2012 p.129).

Sfärd (1998) makes a division into two metaphors, acquisitions and participation, in a cognitive and cited/sociocultural learning perspective. Acquisitions refer to learning that occurs through concepts and cognitive structures. Participation points to a situated perspective on learning where interaction and activity are central. This metaphor has many similarities to Lave &Wenger's understanding of professional learning communities (Wenger & Lave, 1991). If the learning takes place in a community, participation is required, but participation will provide both opportunities and limitations. Vygotsky's sociocultural perspective on learning, human thinking and action is that one takes an interest in how individuals and groups acquire and exploit physical and cognitive resources (Wittek, 2004). From a sociocultural perspective, the interaction between collective and individual is in focus.

These different ways of understanding participation can be linked to the Norwegian philosopher Hans Skjervheim's understanding of how to make oneself a participant through engagement. According to Skjervheim, engagement can be about engaging with the other or what the other says, and thus contributing to subject-subject relationships (Skjervheim, 1996). In situations where one cannot, or will not, engage with the other directly, such as in a teacher-student relationship, it is nevertheless possible, according to Skjervheim, to meet in an equal subject-subject relationship, if one introduces a third paragraph in the relationship, where common directing towards a common matter constitutes a third paragraph (Skjervheim, 1996).


Methodology, Methods, Research Instruments or Sources Used
The growing demand for flexible online studies, which will make education more accessible, requires new knowledge in this field. Our research can show the potential that will lie in online seminars as a form of teaching and contribute to the further development of flexible online studies. This study will provide knowledge about how we as teachers can arrange for the students to experience themselves as part of a community.
Our data is collected from a part-time study for students who either take the course as part of a BA-course or as a single course in further education. We conducted a survey (N69). In practice, this included completing a digital questionnaire where there were structured questions with an opening for supplementary comments. For the questions with an opening for supplementary comments, we used thematic analysis. Through a thematic analysis, we have tried to track various factors that may affect the students' participation in the digital seminar, which can then be used to explain how the use of seminar can contribute to increased participation in the digital classroom. Thematic analysis, Clarke and Braun describe as a flexible, basic method of qualitative data analysis, attempting to identify and treat patterns or themes, through a fixed procedure of six steps; 1. become familiar with data; 2. Produce the first codes; 3. Looking for theme; 4. Review of topic; 5. Identify theme and name; 6. Writing a report (Braun & Clarke, 2006)

Conclusions, Expected Outcomes or Findings
Findings from the survey refer to various factors that seem to have an impact on the students' participation in online seminars. We have divided the factors that the students believed had an impact on their own participation in the webinar into three main areas; framework conditions, working methods and learning environment. (research question 1)
• Our findings related to working methods provide a basis for saying that work methods that facilitate discussion are important for student participation. (Working methods)
• Our findings clearly show that the structure of the seminar has a lot to say for the students' participation, and that clear and structured tasks are preferred over freer reflection. The content is also emphasized, where the relevance of the content in particular is of importance. (framework conditions)
• Our findings show that fellow students and teachers are a significant factor for participation in webinars.  According to our survey, the most significant factor for participation, associated with the learning environment, is the teacher. (learning environment)
Through the thematic analysis, we have, based on the students' free responses, identified various characteristics of the teacher that seem to contribute to increased participation. These are divided into two main groups, where we distinguish between the teacher's personal qualities and the teacher's practical didactic and professional skills. (research question 2)
• Our findings give us reason to believe that the teacher's ability to engage has an impact on student participation. However, the meaning of the term may be perceived as not very tangible.
• Our findings show that the teacher's ability to structure and organize has an impact on the students' participation.

References
Bern, L. T., Lorentzen, N. Ö., & Nordangen, M. (2021, 4). Fortellinger om tid og synlighet:en studie av studenters deltakelse i digital undervsning under covid-19-pandemien. Uniped. Tidsskrift for universitets-og høgskolepedagogikk.
Braun, V., & Clarke, V. (2008). Using thematic analysis in psychology. Qualitative Researche in Psychology.
Fossland, T., & Tømte, C. E. (2019). Deltaker eller tilskuer? En casestudie om vilkår for deltakelse og samarbeidslæring i et nettbasert masterprogram i økonomi og ledelse (MBA). Uniped, årgang 42, nr 1, ss. 41-59.
Hausstätter, R. S. (2007). Spesialpedagogiske grunnlagsproblemer- mellom ideologi og virkelighet. Fagbokforlaget.
Illeris, K. (2012). læring. Gyldendal akademisk.
Kunnskapsdepartementet. (u.d.). Meld.st. (2016-2017) Kultur for kvalitet i høyere utdanning.
Kunnskapsdepartementet. (u.d.). Meld.st.16 (2020-2021) Utdanning for omstilling- økt arbeidslivsrelevans i høyere utdanning.
Pettersen, R. (2005). Kvalitetslæring i høyere utdanning. innføring i problem- og praksisbasert didaktik. Universitetsforlaget.
Røisehagen, A., & Tvete, I. (2022). Den refleksive erfaringens magi: studentaktive læringsformer som veien mot den reflekterende praktiker? I H. Thuen, S. Myklestad, & S. Vik, pedagogikkens idè og oppdrag (ss. 183-). Fagbokforlaget.
Sfärd, A. (1998). On Two Metaphors for Learning and the Dangeres of Choosing Just One. Educational researcher 27(2), ss. 4-13.
Skjervheim, H. (1996). Deltakar og tilskodar og andre esseys. Aschehoug.
Wenger, E., & Lave, J. (1991). Situated learning., Legitimate Peripheral Participation.
Wittek, L. (2004). Læring i og mellom mennesker -en innføring i sosiokulturelle perspektiver. Cappelen akademisk forlag.


22. Research in Higher Education
Paper

Can a Brief E-learning Course Enhance Students’ Self-Regulated Learning?

Moiken Jessen, Klaus Lingel, Roland Stein

University of Würzburg, Germany

Presenting Author: Jessen, Moiken

Students in tertiary education are often considered competent and self-regulating learners. However, it has been shown, that under demanding circumstances, e.g., in distance-education, they are less likely to apply metacognitive strategies of self-regulated learning (SRL) (e.g. Broadbent & Poon, 2015), instead resorting to superficial cognitive strategies (e.g., Karpicke et al. 2009; Enders & Wienzierl, 2017; Dunlosky et al., 2017). Distance-learning settings pose particularly high demands on the self-regulation of learning, while at the same time minimizing instructors’ options to offer guidance. Is it possible to support students in their learning process with a short intervention in form of an e-learning course, specifically targeting the use of effective strategies of self-regulated learning? While in laboratory settings, interventions in the form of prompts have been shown to improve students’ learning competence, (e.g., Sonnenberg & Bannert (2015), little is known about supporting students in ecological-valid distance-settings. First studies find support that a promotion of SRL strategies online is promising (e.g., Endres at al., 2021). We designed a short e-learning course teaching students strategies of self-regulated learning and providing declarative and conditional knowledge on SRL strategies in order to promote the application of SRL-strategies in distance settings. E-learning courses have the following benefits: 1. easy administration (the same course can be administered repeatedly), 2. cross-discipline relevance (scientifically informed study behavior benefits students and educators from all fields), 3. adaptability to other languages given the universal context (can be use on large international scale), 4. an alleviation of the teachers’ duties.


Methodology, Methods, Research Instruments or Sources Used
The e-learning course, presented in Moodle, contains three phases (cf. Biwer et al.2020): 1. Raising awareness regarding effectiveness of learning-techniques, 2. Reflection on own learning, 3. Planning, detailed study-plan). The control-condition was a psychoeducational intervention on sleep-hygiene, following the same pattern: 1. Raising awareness on sleep habits, 2. Reflection on own sleeping habits, 3. Planning sleep-routine.
Participants (N=85) were randomly assigned to an experimental condition (n=36) or control condition A or B. The experimental group completed a pretest, the SRL-e-learning-course and a post-test. Participants in control-condition A (n=23) completed a pre-test, a psychoeducational e-learning course on sleep-hygiene and a post-test. Participants in control group B (n=26) only completed the sleep-hygiene course and post-test. The pre- and post-test assessed knowledge on effectiveness of SRL strategies, self-reported (planned and actual) application of cognitive and metacognitive strategies during exam preparation, motivation and effort. Grades were collected from the final exam.
Recruiting occurred during an introductory course for primary school teacher students in Germany during covid-19-restrictions. Students attended streamed live-lectures, were provided asynchronous screencasts, access to study material in Moodle and tutor lessons. At the end of the term, students took a graded exam. To assess the effects of the intervention, we first compared final grade as an indicator of learning outcome between the groups. We then investigated the effect of the intervention on the self-reported use of self-regulation strategies ‒ as an indicator of learning process using items of the LIST-questionnaire (Schiefele & Wild, 1994) (German version of the MSQL). We investigated effectiveness ratings as an indicator of declarative knowledge (Dunlosky et al., 2013).
Our analysis relies on a difference-in-differences-design (DiD) to estimate the effects of the intervention. In DiD, changes due to external factors are controlled by the difference in the reported changes of the control-groups. To avoid prompting effects of the pre-test, we constructed the difference between the post-test of control-group B and the pre-test of control-group A. Subtracting this difference from the difference of the reports of the experimental-group allows to eliminate changes due to external factors, identifying the causal effect of the intervention on the experimental-group. In our model
(1) Ƴ=β0+β1∗Treatment+β2∗Post+β3∗Treatment∗Post(+Ɛ)
Treatment and Post are dummy variables indicating whether the observation belongs to an individual in the treatment-group (treatment=1) or in the control-group (treatment=0) and whether the observation is after the treatment (post= 1) or before (post=0); the coefficient β3 corresponds to the causal effect of the intervention.

Conclusions, Expected Outcomes or Findings
Regarding the learning outcome, there were no significant differences in exam grades between the experimental group and control group B (MEG=2.93, SDEG=1.09; M CGB=2.43, SD CGB=1.04; t (30) =-1.24, p=.52). The effect size of the difference is d=– 0.47).
We next present the results of the DiD analysis, identifiying the causal effect of our intervention on strategy use (1) and effectiveness ratings (2). In this extended summary, we focus on the cases where the intervention yielded statistically significant effects (standardized regression coefficient β3):
1. Application of SRL strategies
a. Metacognitive control:
• I critically checked, what I have studied:  β= 0.38, SE .0.39 t=2.05 F(3/90)=1,95, p 0.44
• I asked myself questions about the material to check that I understood everything: β=0.35, SE 0.42, t=1.89, F(3/90)=1.48, p=0.62
• In order to identify gaps in my knowledge, I have recapped the most important contents without using my documents for help: β 0.40, SE = 0.46, t=2,17, F(3/90)=1,59, p=0.03
b. Cognitive rehearsal:
• I memorized the subject matter as much as possible using lecture notes or other recordings: β = -0.36, SE = 0.48, t=-1.95, F(3/90)=2.23, p=0,06
• I have memorized rules, technical terms or formulas: β = -0.37, SE = 0.43, t=-2,15, F(3/90)=6.23, p=0,04.
2. Effectiveness ratings:
a. Metacognitive control:
• To identify gaps in knowledge: recapitulate the most important content without taking documents for help: β=0.52, SE=0.29, t=-2.89, F(3/90)=4.18, p=0.01
b. cognitive rehearsal
• To read through the texts relevant to the exam again and again: β=-0.33, SE=0.32, t=-1.85, F(3/90)=4.65, p=0.07
• Memorize the subject matter using index cards or own notes as much as possible: β=-0.36, SE= 0.38, t=-1.95, F(3/90)=1.72, p=0.05

References
Biwer, F., Bruin, A. B. de, Schreurs, S., & oude Egbrink, M. G. (2020). Future Steps in Teaching Desirably Difficult Learning Strategies: Reflections from the Study Smart Program. Journal of Applied Research in Memory and Cognition, 9(4), 439–446. https://doi.org/10.1016/j.jarmac.2020.07.006
Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving Students' Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychological Science in the Public Interest : A Journal of the American Psychological Society, 14(1), 4–58. https://doi.org/10.1177/1529100612453266
Enders, N., & Weinzierl, C. (2017). Lernstrategienutzung beim E-Learning: Strategische Vorbereitung auf unterschiedliche Lern- und Prüfungsanlässe. ZeHf – Zeitschrift Für Empirische Hochschulforschung, 1(1), 5–23. https://doi.org/10.3224/zehf.v1i1.01
Endres, T., Leber, J., Böttger, C., Rovers, S., & Renkl, A. (2021). Improving Lifelong Learning by Fostering Students’ Learning Strategies at University. Psychology Learning & Teaching, 20(1), 144–160. https://doi.org/10.1177/1475725720952025
Foerst, N. M., Pfaffel, A., Klug, J., Spiel, C., & Schober, B. (2019). SRL in der Tasche? – Eine SRL-Interventionsstudie im App-Format. Unterrichtswissenschaft, 47(3), 337–366. https://doi.org/10.1007/s42010-019-00046-7
Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory (Hove, England), 17(4), 471–479. https://doi.org/10.1080/09658210802647009
Panadero, E. (2017). A Review of Self-regulated Learning: Six Models and Four Directions for Research. Frontiers in Psychology, 8, 422. https://doi.org/10.3389/fpsyg.2017.00422
Schiefele, U. & Wild, K. P. (1994). Lernstrategien im Studium:: Ergebnisse zur Faktorenstruktur und Reliabilität eines neuen Fragebogens. Zeitschrift Für Differentielle Und Diagnostische Psychologie. (15), 185–200.
Sonnenberg, C., & Bannert, M. (2015). Discovering the Effects of Metacognitive Prompts on the Sequential Structure of SRL-Processes Using Process Mining Techniques. 1929-7750, 2(1), 72–100. https://doi.org/10.18608/jla.2015.21.5


 
Contact and Legal Notice · Contact Address:
Privacy Statement · Conference: ECER 2023
Conference Software: ConfTool Pro 2.6.149+TC
© 2001–2024 by Dr. H. Weinreich, Hamburg, Germany