Session | ||
04 SES 11 E: Exploring Inclusive Data & Cases
Paper Session
| ||
Presentations | ||
04. Inclusive Education
Paper Exploring Distinctive Dimensions of PAX Good Behavior Game Program Implementation: A Qualitative Cross-Case Analysis within the Estonian Educational Context Tallinn University, Estonia Presenting Author:Evidence suggests that school-based Social-Emotional Learning (SEL) programs have a positive impact on both children's academic and social-emotional skills (Corcoran et al., 2011; Durlak et al., 2011; Cipriano et al. 2023). These programs, particularly, prove highly beneficial for students at risk of educational setbacks (Jones et al., 2011), and have proven an effective tool for inclusive education (Mitchell & Sutherland, 2020). Notably, the impact of these programs is intrinsically tied to the quality of their implementation by educators (i.e. program implementation fidelity, Durlak & Dupre, 2008; Humphrey et al., 2018). Despite extensive research identifying teacher and school context-related factors that influence the quality of SEL program implementation (Domitrovich et al., 2008; Durlak & Dupre, 2008; Kam et al., 2003), a consensus on which contextual factors are most pivotal or warrant the greatest emphasis remains elusive. Achieving such clarity is essential for addressing these factors with a concerted and informed approach, thus creating a more supportive context for implementing SEL programs effectively within schools and today’s inclusive reality. In 2023, Ulla and Poom-Valickis (2023) published a systematic review and identified four categories of contextual factors that can influence implementation quality: program support, school, teacher, and student level factors. Their analysis focused on the relative importance of these factors and found that the most frequent statistically significant factors included modeling activities during coaching and teacher-coach working relationship. The PAX Good Behavior Game (PAX GBG, Embry et al.., 2003) is an evidence-based prevention program implemented by teachers on the classroom level, which has been recognized by the Wallace Foundation as one of 33 leading SEL programs (Jones et al., 2021). PAX GBG is an SEL program implemented worldwide (Australia, Estonia, Ireland, Sweden, United States). The current study takes on a qualitative approach and focuses on two distinct groups of teachers, categorized by the level of implementation quality (fidelity) of PAX GBG as assessed through an observer-rated checklist over the course of the school year in Estonia. The cases under examination consist of teachers with high fidelity and low fidelity in implementing the PAX Good Behavior Game. Those cases were selected to investigate the conditions and experiences that shape a context, determining varying degrees of positive impact on children within an inclusive classroom setting. Given that previous research exploring contextual factors influencing the quality implementation of evidence-based SEL programs has predominantly been quantitative in nature (Ulla & Poom-Valickis, 2023), the qualitative cross-case study aims to elucidate, validate, or challenge the theoretical assumptions advanced in prior research, to determine the relevant conditions for carrying out inclusion through this classroom level toolkit that is utilized worldwide. The study thus seeks to add to the discussion about the relevance and conceptualization of SEL program implementation quality (Berkel et al., 2011; Dane & Schneider, 1998; Durlak, 2016) and the teacher and school related factors that may contribute to that (Domitrovich et al., 2008; Durlak & DuPre, 2008).
Research questions: RQ1: What are the characteristics of high and low fidelity cases? RQ2: In what ways do the high and low fidelity cases differ in terms of their implementation experience (including contextual factors), and how do these differences contribute to variations in the quality of implementation of the PAX Good Behavior Game program? Methodology, Methods, Research Instruments or Sources Used Sampling and data collection: In the academic year 2022/2023, all teachers participating in the PAX Good Behavior Game implementation cohort in Estonia were extended an invitation to partake in this study. The invitation requested their consent for the research team to access their implementation quality (fidelity) observation data, resulting in a potential sample of 129 eligible teachers. Remarkably, 28 teachers (constituting 22% of the population) consented to participate. To establish a selection criterion, all 129 teachers were quantitatively ranked based on their fidelity scores, derived from the observational scoring sheets, from highest (score=6.00) to lowest fidelity (score=-4,45). Subsequently, a final sample of 7 teachers was chosen, representing the top 33.33 percentile of implementation quality ranking, with scores ranging from 4.51 to 5.85. Additionally, 7 teachers were selected from the lower 20 percentile of implementation fidelity, where scores ranged from -1.00 to -2.1. Semi-structured interviews were conducted with all 14 teachers to capture their perspectives and insights on their implementation experiences. The interviews are analyzed using Qualitative Cross-Case analysis method (Miles & Huberman, 1994), as it allows to examine the similarities and differences across cases to reinforce validity, support generalizability, and promote theoretical predictions. The analysis is currently ongoing and will be finished by the time of the presentation. Conclusions, Expected Outcomes or Findings The findings will either validate or refute the hypotheses derived at through previous quantitative studies (Ulla & Poom-Valickis, 2023), namely that certain contextual characteristics, such as teacher-coach alliance or modeling of program activities are related to higher quality program implementation. As Proctor et al (2011) have stated: "Qualitative data, reflecting language used by various stakeholders as they think and talk about implementation processes, is important for validating implementation outcome constructs." The results of the current study may, thus, elucidate teachers' professional development choices or personal values that may predict a high or low fidelity program implementation process. Such factors may have not been previously operationalized or hypothesized in the quantitative study designs prevalent in the current literature. The results should offer a more profound understanding of teachers' SEL program implementation experiences that could lead to more quality inclusion of students in the classroom. References Cipriano, C., et al. (2023). The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions. Child Development, 94(5), 1181-1204. Corcoran, R.P., Cheung, A.C.K., Kim, E., & Xie, C. (2017). Effective universal school-based social and emotional learning programs for improving academic achievement: A systematic review and meta-analysis of 50 years of research. Educational Research Review, 56-72. Domitrovich, C. E., et al. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1(3), 6–28. Embry, D., Staatemeier, G., Richardson, C., Lauger, K., & Mitich, J. (2003). The PAX good behavior game (1st edn). Center City, MN: Hazelden. Durlak, J.A., Dupre , E.P. (2008). Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. American Journal of Community Psychology, 31, 327-350. Durlak, J.A., et al. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. Humphrey, N., Barlow, A., & Lendrum, A. (2018). Quality Matters: Implementation Moderates Student Outcomes in the PATHS Curriculum. Prevention Science, 19, 197-208. Jones, S.M., Brown, J.L., & Aber, J.L. (2011). Two-Year Impacts of a Universal School-Based Social-Emotional and Literacy Intervention: An Experiment in Translational Developmental Research. Child Development 28(2), 533-554. Jones, S.M., et al. (2021). Navigating SEL from the Inside Out. Looking Inside and Across 33 Leading SEL Programs: A Practical Resource for Schools and OST Providers. Preschool & Elementary Focus. Revised & Expanded Second Edition. Kam, C-M., Greenberg, M., & Walls, C.T. (2003). Examining the Role of Implementation Quality in School-Based Prevention Using the PATHS Curriculum. Prevention Science, 4(1), 55-63. Miles, M.B., & Huberman, A.M. (1994). Qualitative Data Analysis. Second Edition. SAGE publications. Mitchell, D., & Sutherland, D. (2020) What Really Works in Special and Inclusive Education : Using Evidence-Based Teaching Strategies. Third Edition. Taylor & Francis Group. Proctor, E., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and reserch agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76. Ulla, T. & Poom-Valickis, K. (2023a). Program support matters: A systematic review on teacher- and school related contextual factors facilitating the implementation of social-emotional learning programs. Frontiers in Education. 04. Inclusive Education
Paper Teachers’ Use of Learning Management Systems to Differentiate Instruction: A Mixed-Methods Study Humboldt-Universität Berlin, Germany Presenting Author:The presumed uncertainty in current education derives from a variety of recent changes and challenges in the educational sector. Therefore, it seems necessary to address these challenges in combination rather than viewing them as separate topics. Two of the current issues contributing to complexity arise from (1) a growing heterogeneity of students and (2) the increasing digitalization of the education system. The aim of this research is to combine these fields by analyzing teachers’ use of learning management systems to differentiate instruction through a mixed methods approach. With the increasingly diverse student population in schools, the establishment of inclusive classrooms has become a top international policy priority, emphasizing “concepts of efficiency, effectiveness, equity, and inclusion as a means of ensuring quality education for all” (Watkins, 2017, p. 1). In the sense of a broader understanding of inclusion that celebrates the diversity of all learners (ibid.), schools must become “more responsive to children with a diverse range of abilities, cultures, gender, religions, and other situations and issues that present in the classroom” (Loreman, 2017, p. 2). Differentiated instruction (DI) is considered as vehicle to achieve inclusive education that aims to meet students’ individual learning needs by maximizing learning opportunities. DI is defined as the intentional, systematically planned and reflected practices that enable teachers to meet the needs of all learners in heterogeneous classrooms (Letzel et al., 2020). Teachers can implement DI through a variety of instructional activities or didactical strategies such as, tiered assignments, student grouping, tutoring systems, staggered nonverbal material learning aids such as checklists, mastery learning and forms of open education like station-based work, interest-based centers, project-based learning, or portfolios. Digital technologies, such as learning management systems (LMS), have the potential to improve, facilitate and support teachers in differentiating their instruction to the various learning needs of students (Cha & Ahn, 2014; Edmunds & Hartnett, 2014). LMS serve as digital communication platforms supporting processes of teaching and learning by providing and organizing learning material, offering direct and indirect forms of online communication, allowing for data-based diagnostics and assessment as well as personalized and cooperative learning (Brägger & Koch, 2021). LMS, if used sensibly, can foster an inclusive, effective learning environments and fuel processes of school and classroom. LMS, as a basic educational infrastructure, have a long history and thus a more prevalent use in universities than schools. However, literature on the application of digital technologies and resources for DI in general education settings appear to be on the rise. Considering the potential that LMS can have to support the differentiating of teaching, there has been multiple literature outputs that serve as guidelines or practical examples for teachers (Cha & Ahn, 2014; Palahicky, 2015). Furthermore, empirical studies have also been undertaken to explore how LMS fosters the establishment of student-centered learning environment (Edmunds & Hartnett, 2014) and support the differentiation of instruction (Vargas-Parra et al., 2018). Despite this body of scientific literature, there is still little research that focuses on investigating the specific differentiation practices that teachers use within online learning environments such as LMS (Beck & Beasley, 2021). Against this background, the present study tackles this research gap and aims to examine how distinct DI practices are applied using LMS. The research question guiding this study is: Which DI practices do teachers apply within LMS and how often? Methodology, Methods, Research Instruments or Sources Used For the purpose of the study, a mixed-methods concurrent single-phase design, where both quantitative and qualitative data were simultaneously collected, was implemented (Creswell & Zhang, 2009). A total of 223 primary and secondary school teachers (62% female; mean age = 47.46 years; mean teaching experience = 17.10 years) participated in the study. The participants completed a voluntary online survey, which took approximately 15 to 20 minutes. Data were collected from February till April 2023. To quantitatively measure teachers’ differentiated practice using LMS, a questionnaire was developed based on the DI taxonomy by Pozas & Schneider (2019): tiered assignments, intentional composition of student groups, tutoring systems, staggered nonverbal learning aids, mastery learning and open education. The items could be responded by teachers using a 4-point Likert scale (1 = rarely to 4 = frequently). Qualitative data was collected through the following open-ended question: Could you please provide examples of how you have implemented differentiated instruction through the use of LMS? Quantitative data was analyzed using SPSS 27, whereas teachers’ (open) responses were analyzed using MAXQDA and following qualitative content analysis according to Kuckartz (2018). The tests of within-subject effects showed significant variations within the single use of DI practices in LMS, F(6.84,1217.07) = 14.95, p < 0.001, partial η2 = 0.08. In detail, teachers use LMS to differentiate their instruction predominantly using open education, tiered assignments (according to the difficulty of complexity level and differences in the task representation) as well as student grouping (e.g. cooperative learning). In contrast, teachers hardly differentiate their instruction by means of tutoring systems within LMS. However, when observing the overall means of the single DI practices, it becomes evident that teachers rarely differentiate their instruction in LMS. Qualitative data analysis was performed by using a category system following a deductive approach based on the six DI categories (Pozas & Schneider, 2019) as well as an inductive approach through data material. A total of 113 content units were coded from the material. After coding 25% by three individual researchers and reflecting upon the categories together, an inter-rater agreement of .88 (Cohen’s Kappa) was achieved. For the category of open education, a total of 72 codes segments were revealed. This category is followed by tiered assignments with 25 codes segments. For the case of tutoring systems, no segments were revealed for this category. Thus, the results from the qualitative analyses appear to confirm the quantitative results. Conclusions, Expected Outcomes or Findings Evidence from both studies reveal a similar trend, teachers use LMS to mainly differentiate their instruction using open education, tiered assignments and cooperative learning. In detail, the qualitative data shows that through the use of LMS teachers are able to open their instruction by establishing project-based learning, station learning, weekly plans and foster students’ autonomy. Moreover, through LMS, teachers can provide additional material and activities to students or design tasks with different complexity level. However, it is also clear that both studies in combination reveal that teachers hold a rather low variance of DI practices and rarely make use of LMS for differentiation purposes. This becomes even more interesting given the fact that teachers report that LMS provides more flexibilization of teaching and design in a differentiated manner. Results are further consistent with previous research that show that teachers mainly differentiate their instruction by means of tiered assignments (Smit & Humpert, 2012) and open education (Letzel & Otto, 2019) and have a low implementation of DI (Pozas et al., 2020). However, compared to studies were DI is implemented in an analog manner, it is clear there is a big room for improvement in digital learning environments. Given that DI is already a complex teaching task (Van Geel et al., 2019), it could be possible that teachers consider differentiating using LMS as even more challenging (Pozas et al., 2022). Thus, the results from this study not only serve as a basis for understanding teachers’ use of LMS for DI, but it also provides insights into the specific needs for professional development of teachers. In order for digital technologies and resources such as LMS to be able to support the academic outcomes of all students, it is imperative that teachers are able to use it effectively. References Beck, D. & Beasley, J. (2021). Identifying the differentiation practices of virtual school teachers. Education and Information Technologies, 26, 2191–2205. https://doi.org/10.1007/s10639-020-10332-y Brägger, G. & Koch, F. (2021). Potenziale von Lern- und Arbeitsplattformen für die Unterrichtsentwicklung [Potentials of learning and working platforms for teaching development]. In G. Brägger & H.-G. Rolff (Eds.), Pädagogik. Handbuch Lernen mit digitalen Medien [Pedagogy. Handbook on Learning with Digital Media] (p. 130–164). Beltz. Cha, H. J., & Ahn, M. L. (2014). Development of design guidelines for tools to promote differentiated instruction in classroom teaching. Asia Pacific Education Review, 15, 511-523. https://doi.org/10.1007/s12564-014-9337-6 Creswell, J. & Zhang, W. (2009). The application of Mixed Methods Designs to trauma research. Journal of Traumatic Stress, 22(6), 612-621. https://doi.org/10.1002/jts.20479 Edmunds, B., & Hartnett, M. (2014). Using a learning management system to personalise learning for primary school students. Journal of Open, Flexible and Distance Learning, 18(1), 11-29. Kuckartz, U. (2018). Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung (4. Auflage). Weinheim: Beltz Juventa. Letzel, V., & Otto, J. (2019). Differentiated instruction and its concrete implementation in school practice—a qualitative study. Zeitschrift für Bildungsforschung, 9, 375-393. Loreman, T. (2017). Pedagogy for Inclusive Education. Oxford Research Encyclopedia of Education. Palahicky, S. (2015). Utilizing learning management system (LMS) tools to achieve differentiated instruction. In Models for improving and optimizing online and blended learning in higher education (pp. 12-33). IGI Global. Pozas, M., Letzel, V., & Schneider, C. (2020). Teachers and differentiated instruction: exploring differentiation practices to address student diversity. Journal of Research in Special Educational Needs, 20(3), 217-230. Pozas, M., Letzel-Alt, V. & Schwab, S. (2022). The effects of differentiated instruction on teachers' stress and job satisfaction. Teaching and Teacher Education, 122. https://doi.org/10.1016/j.tate.2022.103962 Pozas, M. & Schneider, C. (2019). Shedding light into the convoluted terrain of differentiated instruction (DI): Proposal of a taxonomy of differentiated instruction in the heterogeneous classroom. Open Education Studies, (1), p. 73-90. https://doi.org/10.1515/edu-2019-0005 Smit, R., & Humpert, W. (2012). Differentiated instruction in small schools. Teaching and teacher education, 28(8), 1152-1162. van Geel, M., Keuning, T., Frèrejean, J., Dolmans, D., van Merriënboer, J., & Visscher, A. J. (2019). Capturing the complexity of differentiated instruction. School effectiveness and school improvement, 30(1), 51-67. Vargas-Parra, M. A., Rodríguez-Orejuela, J. A., & Herrera-Mosquera, L. (2018). Promotion of differentiated instruction through a virtual learning environment. Folios, (47), 165-177. Watkins, A. (2017). Inclusive Education and European Educational Policy. Oxford Research Encyclopedia of Education. Retrieved 9 Dec. 2021, from https://oxfordre.com/education/view/10.1093/acrefore/9780190264093.001.0001/acrefore-9780190264093-e-153. 04. Inclusive Education
Paper Infra-Data: Exploring the Untapped Educational Evidence from the Global South 1University of Glasgow; 2Taleemabad Presenting Author:As we gear our collective efforts towards achieving the Sustainable Development Goals, the notion of 'missing data' in education recurs throughout, especially in the context of the Global South. We are often unaware of the learning situation in many resource-constrained settings even as global data regimes continue to proliferate. In response, this paper introduces the concept of 'infra-data', based on our practice in Pakistan, to identify the often-overlooked wealth of educational evidence that lies beneath the surface of traditional metrics for education. Drawing from theoretical frameworks like James C. Scott's 'hidden transcripts' and Aníbal Quijano's 'coloniality of knowledge', this study illuminates infra-data as a window into the 'unseen' yet impactful educational practices that emerge from localized knowledge, pedagogies, and ways of thinking. Infra-data would allow us to explore the epistemic frames in which these practices are encoded. Utilizing Shaffer’s quantitative ethnography, we dive into infra-data, providing a window into diverse pedagogical approaches across the Global South. In this research, we delve into the case of Siyani Sahelian Program by Idara-e-Taleem-o-Aagahi, a second chance education program on accelerated learning reaching 50 thousand out-of-school adolescent girls across Punjab, Pakistan. By analyzing secondary data from Siyani Sahelian program, we aim to shed light on the definition, measurement, and integration of learning impact from the learner communities. This approach not only offers insights specific to the Pakistani context but also contributes to a broader understanding by providing a comparative perspective on localized vs global mainstream education narratives and perspectives. Methodology, Methods, Research Instruments or Sources Used The methodological approach for this research is anchored in Shaffer's quantitative ethnography, which allows for a rich, data-driven understanding of educational contexts. We will apply this methodology to identify and analyze infra-data from a sample of 20,000 learners within the Siyani Sahelian program. This infra-data is not a mere collection of numbers; it is a rich tapestry that weaves together various educational elements into a holistic narrative directly from the learner communities. Within the scope of the Siyani Sahelian program, our analysis will delve into a diverse range of infra-data components. This includes: • Demographic Information Recognizing the heterogeneity of learner populations, our methodology incorporates a detailed examination of Demographic Information. This aspect of infra-data collection encompasses an array of variables, including age, gender, socio-economic status, language, cultural background, and geographical location. The customization of education based on demographic insights ensures that interventions are not only contextually relevant but also equitable and inclusive, thereby contributing to the overarching goal of educational equality. • Student Learning Outcomes Our infra-data framework emphasizes the critical importance of Student Learning Outcomes, expanding beyond the limitations of standardized testing to encompass a spectrum of qualitative and quantitative data. This includes, but is not limited to, classroom-based assessments, project-based learning evaluations, and progressive, informal feedback mechanisms that capture the evolving academic and practical skill mastery of students. • Perceptions and Attitudes Perceptions and Attitudes form a qualitative component of infra-data that captures the subjective experiences and levels of satisfaction among students, educators, and parents. Through tools such as surveys, structured interviews, and focus group discussions, this data illuminates stakeholders' views on the educational interventions they experience • Enrollment Data Enrollment Data provides quantitative measures of student engagement with educational interventions, functioning as a proxy for the relevance and effectiveness of these initiatives. This encompasses attendance records, participation in educational activities, interactions with digital content, and completion rates of courses or assignments. • Practice Data The infra-data component of Practice Data entails a comprehensive documentation of the implementation of educational interventions, detailing the pedagogical strategies, curriculum adaptations, instructional materials, teacher training, and the integration of technology in the teaching and learning processes. This exhaustive record provides an overarching view of the educational landscape, offering insights into the efficacy of different teaching approaches and the contextual factors that contribute to or hinder the success of educational initiatives. Conclusions, Expected Outcomes or Findings In conclusion, we intend to explore the critical role of infra-data in enabling sustainable and meaningful education systems in the Global South. Infra-data stands as a vital resource in addressing specific challenges such as high out-of-school rates, gender disparities, and resource constraints prevalent in these regions. By embedding this data, deeply rooted in local contexts and practices, into educational policymaking, we can develop strategies that are not only informed by empirical evidence but also attuned to cultural and contextual nuances. This approach aligns closely with the aspirations of Sustainable Development Goal 4, emphasizing inclusive and equitable quality education. The integration of infra-data into educational planning and implementation promises to enrich the conceptual understanding of education in the Global South, leading to policies and practices that are truly inclusive and equitable. References UNESCO Institute for Statistics (UIS). (2020). The World Needs Almost 69 Million New Teachers to Reach the 2030 Education Goals. Scott, J. C. (1990). Domination and the Arts of Resistance: Hidden Transcripts. Yale University Press. Shaffer, D. W. (2017). Quantitative Ethnography. Cathcart Press. Quijano, A. (2000). Coloniality of Power and Eurocentrism in Latin America. International Sociology, 15(2), 215-232. https://doi.org/10.1177/0268580900015002005 |