04. Inclusive Education
Paper
Digital Writing Support in Class 5
Lea Schröder
University of Bremen, Germany
Presenting Author: Schröder, Lea
In the course of inclusion and the digital transformation processes in schools and teaching, writing is taking on an increasingly significant role. However, the DESI study (Deutsch Englisch Schülerleistungen International) in Germany and the NAEP study (National Assessment of Educational Progress) in the USA show that many students fail to successfully write high-quality texts (National Center for Education Statistics, 2012, 2019; Neumann & Lehmann, 2008). There is a high demand for appropriate and effective support tools for inclusive education. For this reason, the digital application "Reise nach Narrativa digital - eine Welt voller Geschichten" (Journey to Narrativa digital - A World full of Stories) was developed for 5th grade students, which uses a structured set-up to guide students systematically and motivationally through the complex writing process from an initial collection of ideas to a finished story (Schröder & Vierbuchen, 2021). In this study, the question is investigated whether this digital support application supports heterogeneous learning groups in class 5 in writing their stories.
A quantitative longitudinal main study in a pre-post follow-up control group design was chosen as the research design to examine the effectiveness in terms of the positive development of quantitative and qualitative text quality as indicators of narrative writing competence. The web application was tested on a sample of n=243 students in seven middle schools in Lower Saxony in Germany over a period of eight weeks. The results show a mixed picture. The quantity of stories written improved only marginally for the pupils in the experimental group. The quality of the stories, which is subdivided into holistic and analytical text quality, shows a positive development trend at the post time compared to the control group, which decreases at the third measurement point. Students with a diagnosed special educational need for learning benefit especially from writing with the digital writing support tool, which was demonstrated by the significant improvement in the quantity of their stories. In addition, it was found that the characteristic of the students, such as gender and family language background, made no difference to the support.
Overall, the web application, as a meaningful didactic embedding in the classroom, can contribute to the support of narrative writing competence and helps the students in their writing process.
Methodology, Methods, Research Instruments or Sources UsedWithin the framework of a summative evaluation, the effectiveness of the digital support programme is examined. In order to be able to record the most important target areas of the study, procedures were used that are standardised and meaningful. In order to measure writing competence, the students in both groups were given a 45-minute writing assignment with three different pictures as writing stimuli to choose from at all three measurement times. They were evaluated with the Global Scale for Narrative Texts (Canz, 2015) for measuring holistic text quality and with the Textbeurteilungsmatrix Erzählendes Schreiben (Text Assessment Matrix Narrative Writing) (Schulden, 2022). The quantity of narrative texts was assessed using the commonly used Total Words Written (Hosp et al., 2016).
Furthermore, teachers were interviewed for the subgrouping and a questionnaire was filled out by the students. In this way, the subgroups of students with learning support needs and with a lack of motivation to write could be determined. Biographical data were also collected in the questionnaire, so that the influence of the characteristic correlations of gender and family language background on the support effect could be examined.
Conclusions, Expected Outcomes or FindingsThe quantity of the written stories hardly improved for the students in the experimental group. The quality of the stories, which is subdivided into holistic and analytical text quality, shows a positive development trend at the post time compared to the control group, which decreases at the third measurement time. Students with a diagnosed special educational need for learning benefit especially from writing with the digital writing support tool, which was demonstrated by the significant improvement in the quantity of their stories. In addition, it was found that the characteristic, such as gender and family language background, made no difference to the support. This is encouraging in that the web application can be used regardless of gender and family language background.
Overall, it can be seen that with a sensible didactic embedding in the lessons, the web application can contribute to the promotion of narrative writing competence and helps the students in their writing process. It can be concluded that the web application is an effective support tool. The study shows that the media competences of teachers and students play an important role for the success of the support.
ReferencesCanz, T. (2015). Validitätsaspekte bei der Messung von Schreibkompetenzen (Validity aspects in the measurement of writing competences) [Dissertation]. Humboldt Universität Berlin, Berlin.
Hosp, M. K., Hosp, J. L. & Howell, K. W. (2016). The ABCs of CBM: A practical guide to curriculum-based measurement. Guilford.
National Center for Education Statistics. (2012). The Nation's Report Card: Writing 2011. Washington. Institute of Education Sciences. https://nces.ed.gov/nationsreportcard/pdf/main2011/2012470.pdf
National Center for Education Statistics. (2019). Understanding Assessment Results. Scale Scores and NAEP Achievement Levels, National Center for Education Statistics. https://nces.ed.gov/nationsreportcard/gui-des/scores_achv.aspx
Neumann, A. & Lehmann, R. H. (2008). Schreiben Deutsch. (Writing German) In DESI-Konsortium (Hrsg.), Unterricht und Kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie (Teaching and competence acquisition in German and English. Results of the DESI study) (S. 89–103). Beltz.
Schröder, L. & Vierbuchen, M.‑C. (2021). Digitale Förderung der narrativen Schreibkompetenzen. (Digital support of narrative writing skills) In L. Schulz, I. Krstoski, M. Lüneberger & D. Wichmann (Hrsg.), Diklusive Lernwelten: Zeitgemäßes Lernen für alle Schülerinnen und Schüler (Diklusive learning worlds: Contemporary learning for all pupils) (1. Aufl., S. 190–195).
Schulden, M. (2022). Peergestützte Förderung der Schreibkompetenz: eine quantitativ-empirische Erhebung zur Untersuchung der Wirksamkeit eines Förderverfahrens für heterogene Lerngruppen in der 5. Jahrgangsstufe (Peer-supported promotion of writing competence: a quantitative-empirical survey to investigate the effectiveness of a promotion procedure for heterogeneous learning groups in the 5th grade) [Dissertation]. Universität Oldenburg, Oldenburg. http://oops.uni-oldenburg.de/5433/1/schpee22.pdf
04. Inclusive Education
Paper
Developing a Digital, Adaptive, Inclusive Reading Screening: Evaluating Test Modification's Impact on Students With and Without Learning Disabilities’ Performance
Nikola Ebenbeck, Markus Gebhardt
University of Regensburg, Germany
Presenting Author: Ebenbeck, Nikola
Assessments play an important role in the education of all students. They measure skills or knowledge in a standardized way and provide important information for further support (Lai & Schildkamp, 2013). In school performance, for example, they refer to crucial skills, such as reading. Reading is essential for academic success, and early identification of reading difficulties is important for providing appropriate support (Hakkarainen et al., 2013). Reading screenings therefore are well established in practice and can be used in combination with formative diagnostics (Fuchs et al. 2007). Supporting students with learning disabilities (LD) is an important aspect of inclusive education and early identification of reading difficulties is crucial for providing appropriate support (Fuchs & Fuchs, 2011). To measure more reliably, more quickly and thus more efficiently, the use of technology in education has grown significantly in recent years, and the integration of digital tools in assessment practices has the potential to improve the assessment process (Timmis et al. 2016). Adaptive assessments, in particular, present a perspective on the simultaneous testing of a heterogeneous, diverse student body (Stone & Davey, 2011).
The study is embedded in a larger project, in which a digital adaptive reading screening is to be developed freely available for teachers. Therefore, the study investigates the results of two data collections using different versions of the German inclusive reading screening “Les-In”. The initial version of the screening was developed as a paper-based test (Ebenbeck et al., 2022) and administered to a sample of 700 third graders, with 5% of the students having a LD. Results indicated that two out of four subtests did not measure as broad as expected (subtest 2: Mdnσ=-0.14, Mσ=0, SDσ=1.70; subtest 3: Mdnσ=-0.12, Mσ=0, SDσ=0.66). Also, ceiling effects in the summative scores (subtest 2: nitems=34, Mdnsum=31 , Msum=29.30, SDsum=3.22; subtest 3: nitems=25, Mdnsum=25, Msum=23.37, SD=4.08) indicate the missing of harder items. Because of those reasons, the subtests were modified. The modified tasks were then integrated into a digital version of the test on the web-based platform Levumi.de (Jungjohann et al., 2018). The digital test was administered to a sample of 400 second to fourth graders, with 7% of the students having a LD.
At the student level, the study examines the extent to which reading skills differ between students with and without LD and whether there is overlap between student groups. At the test level, the study examines whether the modification of the two tasks has added psychometrical value and whether the tasks can now more accurately measure students across a broader range of performance. For this purpose, the results of the digital screening are considered both individually and in comparison with the analog version. The results show that the change in tasks had the desired effect, and students with and without disabilities can now be measured more accurately across a broader range of performance (subtest 2: Mdnσ=0.10, Mσ=0, SDσ=0.89; subtest 3: Mdnσ=-0.39, Mσ=0, SDσ=0.81). The results also indicate that there are differences in the reading performance of students with and without LD, as students with LD show lower reading skills in the tasks. The extent to which the results can be used to expand the digital screening into an adaptive digital screening is further evaluated and discussed. Further discussed is the extent to which inclusive screening assessments should relate primarily to the low-ability domain, and the extent to which the value of diversity of students with and without disabilities and their individual learning levels and reading abilities can and must be considered in the development of assessments.
Methodology, Methods, Research Instruments or Sources UsedTwo data collections are evaluated and compared. In the first collection, 700 third graders from German elementary and special schools completed the first version of the screening as a paper-pencil test. The screening includes four subtests, namely “Phonological Awareness” (subtest 1; 36 items), “Vocabulary” (certainty of lexical recall; subtest 2; 36 items), “Flash Reading” (speed of lexical retrieval; subtest 3; 30 items) and “Sentence Comprehension” (subtest 4; 60 items). The analog subtests correlate between r=.26 and r=.52 (p<.001). All tasks are processed single-choice. Three subtests have a maximum processing time of five minutes and are therefore speeded tests. The subtest 3 has no time limit but ends when all items are completed. Students complete the screening simultaneously as a class. For subtests 2 and 3, the psychometric evaluations of this survey showed that the performance range is measured too narrowly and therefore the item pools must be expanded to include more difficult items. In the modified screening, subtests 2 and 3 were therefore changed. In subtest 2, more pseudowords were added to the item pool, and in subtest 3, the display duration of the words was varied. The screening’s modified version was digitalized as web-based computer- or tablet test. The digital subtests correlate between r=.34 and r=.55 (p<.001). In the second collection, 400 second to fourth graders of German elementary and special schools completed this digital and modified version of the screening. All students completed the screening on the tablet simultaneously in class. Subtests 1, 2 and 4 had a maximum completion time of five minutes. In subtest 3, all items in the pool were again processed without a maximum processing time.
It is examined how the number of correctly and incorrectly solved tasks changed after the modification of the screening. Students with and without LD are considered separately and compared in order to work out their individual abilities. Subsequently, the new version of the screening will be examined psychometrically. For this purpose, the fit to the one-dimensional Rasch model of each task of the screening is examined. Using graphical model tests, unfair items are identified and removed from the item pool. On the item level, the difficulty of the individual items and of the entire test is examined and compared with the analogous version of the screening. In this way, it is determined whether the modifications have had the desired effect on the difficulty.
Conclusions, Expected Outcomes or FindingsIn conclusion, the results of the study show that the change in tasks had the desired effect, and students with and without LD can be measured more accurately across a broader range of performance with the second Version of in the screening “Les-In”. The results also indicate that there are differences in the reading performance of students with and without LD, as students with LD show in average lower reading skills in the tasks. Nevertheless, there are also students without LD who show very weak reading performance. The goal of the project is to develop a digital adaptive reading screening for inclusive education. In the next step, the results presented in this study are therefore used to develop an adaptive version of the screening with help of various simulation studies. The item pools seem to be suitable for this purpose so far. The exact performance and suitability of the pools will become clear in further studies. The expanding to an adaptive screening would present a perspective on the simultaneous testing of a heterogeneous study body, as thus tests adapt their difficulty depending on the student’s answer pattern. This would lead to a shorter test length while maintaining an accurate measurement. Furthermore, the study also highlighted the potential of using technology in assessment practices to improve the assessment process and provide more efficient and reliable results for all students. Overall, the study emphasizes the importance of inclusive education and early identification of reading difficulties in order to provide appropriate support for all students.
ReferencesEbenbeck, N., Jungjohann, J., & Gebhardt, M. (2022). Testbeschreibung des Lesescreenings LES-IN für dritte inklusive Klassen. Beschreibung der Testkonstruktion sowie der Items der Screeningtests" Phonologische Bewusstheit"," Sicherheit im lexikalischen Abruf"," Geschwindigkeit im lexikalischen Abruf" und" Sinnkonstruierendes Satzlesen" in deutscher Sprache. Version 1.
Fuchs, L. S., Fuchs, D., Compton, D. L., Bryant, J. D., Hamlett, C. L., & Seethaler, P. M. (2007). Mathematics Screening and Progress Monitoring at First Grade: Implications for Responsiveness to Intervention. Exceptional Children, 73(3), 311–330. https://doi.org/10.1177/001440290707300303
Fuchs, D., & Fuchs, L. S. (2011). Responsiveness to Intervention: Multilevel Assessment and Instruction as Early Intervention and Disability Identification. The Reading Teacher, 63(3), 250–252. https://doi.org/10.1598/RT.63.3.10
Hakkarainen, A., Holopainen, L., & Savolainen, H. (2013). Mathematical and reading difficulties as predictors of school achievement and transition to secondary education. Scandinavian journal of educational research, 57(5), 488-506.
Jungjohann, J., DeVries, J. M., Gebhardt, M., & Mühling, A. (2018). Levumi: A web-based curriculum-based measurement to monitor learning progress in inclusive classrooms. In Computers Helping People with Special Needs: 16th International Conference, ICCHP 2018, Linz, Austria, July 11-13, 2018, Proceedings, Part I 16 (pp. 369-378). Springer International Publishing.
Lai, M. K., & Schildkamp, K. (2013). Data-based decision making: An overview. Data-based decision making in education: Challenges and opportunities, 9-21.
Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016). Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42(3), 454-476.
Stone, E., & Davey, T. (2011). Computer‐adaptive testing for students with disabilities: A review of the literature. ETS Research Report Series, 2011(2), i-24.
04. Inclusive Education
Paper
Teachers Producing, Digitalizing and Implementing Content Lesson Material: Results from the Project RegiNaDiff.
Daniela Ender1,2,3, Jessica Berger1,3, Lisa Paleczek1,3
1University of Graz, Austria; 2Private University College of Teacher Education Augustinum, Austria; 3Research Center of Inclusive Education (RCIE), Austria
Presenting Author: Ender, Daniela;
Berger, Jessica
Digitalization can support inclusive teaching (Autenrieth & Nickel, 2020; Brüggemann, 2019). To successfully use digital learning materials in class, teacher attitudes towards the use of digital tools, devices and materials in the classroom matter (Schmidt-Hertha, 2020) and teacher training is necessary (Buchner, 2018; Lorenz & Endberg, 2019). To successfully implement (digital) materials in lessons with a diverse student body, often, the material needs to be adapted according to student needs, which can be achieved through differentiation (Kurth et al., 2015).
This paper introduces the project "RegiNaDiff” (Regional, Sustainable and Differentiated in Class. Starting the green transformation in School; 11/2021 to 01/2023) that combines different opportunities to provide inclusive teaching and learning. In the project, we trained teachers to produce, differentiate and digitalize content lesson materials to then implement it in their classrooms in order to teach inclusively.
In the predecessor project RegioDiff (Discovering the regions of Styria: Differentiated materials for inclusive content lessons in Grade 4; Paleczek, 2020), we developed a digital learning environment that provided students with differentiated texts and evidence-based tasks (Hattie, 2008; Spörer et al., 2009) on different topics. The implementation was evaluated, and the learning environment was adapted according to teacher and student feedback. One of the feedbacks provided by the teachers was that they would need more topics that are compatible with the curriculum as well as meeting their students individual learning needs and interests.
Therefore, we trained the teachers to produce their own digital materials in the project RegiNaDiff. Guided in a teacher training, teachers wrote texts on different sustainability topics. Then they differentiated them into four difficulty levels, enriched the material with tasks known to support reading comprehension: reading strategy, reading comprehension and cooperative tasks as well as a glossary for difficult words were added. In RegiNaDiff, we developed an editor that enabled the teachers to digitalize and embed this differentiated and enriched material in the browser-based learning management system that then could be used in their classrooms.
Twelve teachers from Grades 4 and 5 participated in the project-based teacher training in spring 2022. Working cooperatively in pairs, they created a total of seven different topics (e.g., second hand, environmental pollution, electricity) with texts and tasks in four different difficulty levels. We accompanied this process with a mixed-methods approach to answer the following research questions:
a) What part of the process (writing, differentiating, digitalizing) was most challenging for the teachers and why?
b) How did the teachers experience working with the editor in digitalizing their own content lesson materials? What adaptations regarding the usability of the editor did they suggest?
c) How did teachers perceive the teacher training and continuous feedback from the project team? How can future teacher trainings on creating differentiated digitized materials be designed to support teachers in the best possible way?
d) How did teachers and students evaluate the materials? Which elements of the digital texts (e.g., reading comprehension tasks, words in the glossary, cooperative working) did they like or dislike and why?
Methodology, Methods, Research Instruments or Sources UsedIn order to obtain in-depth information regarding the teacher training (three half-days between 02/22 and 05/22) and the implementation of the materials, we used a mixed-methods approach consisting of questionnaires and interviews. Twelve teachers (age: M=31, SD=10,99, one male) participated in the teacher training. At the time of the training, they were teaching as a classroom teacher in primary (3rd and 4th grade: n=7) and secondary schools (5th grade: n=5), supporting teacher or German as a second language teacher. One teacher taught in both primary and secondary school. However, she had no whole classroom and therefore, she only participated in two training days and could not implement the materials in a class.
Through teacher questionnaires, information was collected on (1) previous experiences using digital media in the classroom and differentiating teaching materials, (2) the development of the materials and the clarity of the work assignment, and (3) the digitalization of the differentiated materials, the usability and intuitiveness of the editor regarding specific elements during the digitalization process (e.g., inserting pictures and audios, choosing the right task, differentiation in the editor) as well as how the teacher training and the support from the project team was perceived. The questions were answered either in an open response format or on a 5-point Likert scale (1=strongly disagree to 5=strongly agree).
To get deeper and more specific insights two rounds of semi-structured interviews were conducted (11:06h material) via online meeting tools in April and June 2022. The first interview was focusing on the development, differentiation, and digitalization of the materials in consideration of their previous knowledge and experiences. The second interview took place after the implementation of the materials in the classroom and gathered information on the conditions of the material implementation. In addition, they were asked for ideas and suggestions for adaptation regarding the project itself and the teacher training.
We also conducted group interviews with students (n=145) to find out (1) if they liked the materials, (2) if they were satisfied with the difficulty level they had worked on and with the diverse elements of the digital material (e.g., glossary words, tasks, pictures, cooperative tasks). In two student questionnaires, we learned about how they liked working with the tablets and the digital materials, and whether they liked the specific topic, learned something new and whether they would like to work with the digital material more often.
Conclusions, Expected Outcomes or FindingsChallenges
The teachers reported the differentiation to have been the biggest challenge. This related on the one hand to assessing the appropriateness of the text in relation to each difficulty level, and on the other hand to deciding which information to maintain in the easier levels. Another challenge was to appropriately describe the glossary words without making them even more difficult to understand.
Digitalization
The teachers liked working with the editor and perceived it to be very intuitive. However, they also made suggestions for further simplifications of the editor (e.g., adding a "back" button, as deleted content could not be restored). Some issues could subsequently be fixed (e.g., the retrospective change of task type) and the digitalization process could therefore be made less time-consuming.
Feedback and Support
The teachers expressed satisfaction regarding the teacher training. The continuous feedback from the project team on the various steps during the development process was experienced as important and helpful. In order to maintain and ensure support in the creation, differentiation, and digitisation of teaching materials, it proved necessary to keep offering the teacher trainings in the future.
Evaluation
The participating teachers are convinced that the texts are well suited for use in inclusive classrooms and appreciate the differentiation levels, enabling all students to work on the same topic.
Although there were challenges, the teachers were proud of their texts and rated them as high-quality (8 or 9).
(8 or 9).
The students liked working with the tablets and digital materials. They would appreciate working digitally more often and they recognized the importance of sustainability issues in their daily lives.
ReferencesAutenrieth, D. & Nickel, S. (2020). Kultur der Digitalität = Kultur der Partizipation: Herausforderungen für Gesellschaft, Schule und Unterricht des 21. Jahrhunderts. Medienimpulse, 58(4), 1–32. https://doi.org/10.21243/mi-04-20-13 [14.10.2022]
Brüggemann, M. (2019). Berufsfeld Grundschule. In I. Bosse, J.-R. Schluchter, & I. Zorn (Hrsg.), Handbuch Inklusion und Medienbildung (1. Aufl., S. 111–117). Weinheim: Beltz Juventa.
Buchner, J. (2018). Digital kompetent durch und mit Fachunterricht! Haushalt in Bildung & Forschung, (4), 16–32. https://doi.org/10.3224/hibifo.v7i4.02 [14.10.2022]
Hattie, J. & Zierer, K. (2019). Visible learning insights. Routledge. https://www.taylorfrancis.com/books/9781351002226
Kurth, J. A., Lyon, K. J. & Shogren, K. A. (2015). Supporting Students With Severe Disabilities in Inclusive Schools. Research and Practice for Persons with Severe Disabilities, 40(4), 261–274. https://doi.org/10.1177/1540796915594160
Lorenz, R. & Endberg, M. (2019). Welche professionellen Handlungskompetenzen benötigen Lehrpersonen im Kontext der Digitalisierung in der Schule? Theoretische Diskussion unter Berücksichtigung der Perspektive Lehramtsstudierender. MedienPädagogik: Zeitschrift für Theorie und Praxis der Medienbildung (S. 61–81).
Paleczek, L. (2020). How to Produce and Acquire Regional Knowledge Digitally and in Print: Conceptualisation of the RegioDiff-Project. Proceedings of the 19th European Conference on e-Learning, 611-614.
Schmidt-Hertha, B. (2020). Vermittlung medienpädagogischer Kompetenz in der Fort- und Weiterbildung von Lehrkräften. Zeitschrift für Pädagogik, 66, 191–207.
Spörer, N., Brunstein, J.C., & Kieschke, U. (2009). Improving students’ reading comprehension skills: Effects of strategy instruction and reciprocal teaching. Learning and Instruction, 19, 272–286.
|
