Conference Agenda

Objectives and Connections to Theme:

Data systems can play a role in teacher confidence; contribute to improved, differentiated instruction; and ultimately improve schools. This presentation explores these ideas through the example of a government-led literacy initiative in the U.S.

Context:

Much has been written about the promise of teacher’s data use internationally (Poortman, Schildkamp, Lai, 2016). Furthermore in the 2018 PISA, at least 70% of students attended schools where student assessments guided instruction (Peña-López, 2020). Some studies connect teacher data use to student achievement (Reeves, & Honig, 2015; Poortman, & Schildkamp, 2016); and one recently found professional development increased teacher data use specifically in reading (van Kuijk, et al., 2016). However, data-use capacity has been questioned (Van Gasse, & Mol, 2021; Datnow, & Hubbard, 2016).

Investment in “the Science of Reading” in the U.S. provides opportunities to explore how “Multi-Tiered Systems of Support in Reading (MTSS-R)” influence teachers’ data-use (Fien, Chard, & Baker, 2021). Teachers use MTSS-R to screen students for reading difficulties, diagnose needs, place students in intervention groups, and monitor progress (Leonard, et al. 2019).

Since 2013, 31 states passed legislation about the Science of Reading and four have policies in process (Schwartz, 2023). States typically focus on evidence-based instruction including phonemic awareness, phonics, fluency, vocabulary, and comprehension, and also implement MTSS-R. Could implementing MTSS-R relate to teachers’ data use and confidence in teaching?

Questions, Methods, Data Sources:

Q1: To what extent are elementary teachers confident teaching reading content, and how does use of MTSS-R relate to confidence?

Q2: What do leader interviews and document reviews reveal about successful implementation and use of MTSS-R?

This presentation describes an evaluation of a state-funded reading initiative that requires use of MTSS-R. We surveyed elementary teachers in 24 districts (37% response rate, n=339) about confidence teaching 10 aspects of reading and about use of MTSS-R. We analyzed the data through descriptive statistics and Chi Squares. We used content analysis to examine district documents and leader interviews.

Results:

More teachers reported confidence teaching comprehension and vocabulary compared to other aspects of reading (Figure 1). Confidence teaching phonemic awareness, phonics, fluency, oral reasoning, and complex text was significantly related to reported use of MTSS-R (Table 1).

In interviews, district leaders believed schools made progress on MTSS-R but the systems were not fully implemented. Several noted that teachers weren’t always “on the same page” about assessments and their purposes. Documents showed that MTSS-R typically relied more heavily on measures of phonemic awareness, phonics, and fluency than on other aspects of reading.

Significance:

This presentation discusses

• How countries can learn from the promise (and implementation challenges) of data systems, such as MTSS-R in the U.S.

• How data systems may help teachers gain confidence, particularly in areas directly related to the measures in the data system (e.g. phonemic awareness, phonics, and fluency) or tangentially related areas where confidence was low (e.g. verbal reasoning and complex text)

Theoretical framework

The use of data, such as data from students’ perceptions of teaching quality (SPTQ), can help teachers improve their teaching quality and support students’ learning (Röhl, 2021). SPTQ data can help teachers identify areas of improvement and initiate improvement-oriented actions (Helms-Lorenz & Visscher, 2021).

However, teachers need a complex set of competencies (knowledge, skills, and beliefs) to use students’ feedback data (Röhl et al., 2021), which is called Data Literacy (Mandinach & Gummer, 2016). In this study we pay attention to beliefs about SPTQ data, because they allow teachers to engage in using their knowledge and skills to use this source of data to drive their professional growth. The theory of Planned Behavior (Ajzen, 1991) will be used to study the effect of a professional development program on teachers’ beliefs.

Objectives of research

In this presentation we will present preliminary results about a teacher professional development program aimed at improving teachers’ beliefs about students’ perceptions of teaching quality.

Research question

The research question that will guide the present study is: What are the effects of a teacher professional development program on teachers’ beliefs to use SPTQ data?

Methods

Teachers will participate in a four months teacher professional development program. The sample encompasses 20 Chilean schools, and around 60 teachers from 7-12th grade, from different subjects. We will use a pre-and post questionnaire about beliefs relative to SPTQ data, grounded in the Theory of Planned Behavior (Ajzen, 1991).

We will use a pre and post-intervention test to measure the impact of our teacher professional development program on teachers' beliefs regarding SPTQ data. The professional development includes workshops for the professional learning communities (PLC) coordinators, as well as a detailed manual. Each teacher will conduct a data-use cycle, which is a set of steps that helps to systematically improve instructional practices based on SPTQ data. Results will be discussed in their PLCs consisting of six to eight teachers, and the PLC coordinator.

Expected results

We expect the intervention will improve the three predictors of intention for using students’ perceptions of teaching quality data: attitudes, subjective norms, and perceived behavioural control.

Educational importance

This research addresses a crucial aspect of educational practice, which is the use of student feedback for instructional improvement. The study will illuminate how teachers working in professional learning communities can influence teachers' beliefs and subsequent utilization of SPTQ data. Besides, the results of this study will add to our knowledge of how teacher beliefs interact with data use, an area that has been recognized but not sufficiently explored.

Connection to the conference theme

Aligned with the ICSEI 2024 theme, this study examines how to develop competencies in teachers to improve their instructional practices. Teaching quality is a crucial component in the research on educational effectiveness (e.g., Muijs & Reynolds, 2017). This study will enrich the conversation about how to develop teacher professional development able to improve teachers’ beliefs, which is key for effective data use (Datnow & Hubbard, 2016; Mandinach & Gummer, 2016; Prenger & Schildkamp, 2018).

Theoretical framework

The use of data, such as data coming students’ perceptions of teaching practices (SPTQ), can help teachers improve their teaching quality and support students’ learning (Herbert et al., 2022; Röhl, 2021). However, teachers need support in improving their teaching using SPTQ data (Bijlsma et al., 2019). The greater the support, the more probable it is that teachers will optimise their teaching using SPTQ data (Röhl, 2021).

We designed a teacher professional development program to support teachers in improving their teaching quality through the use of data in professional learning communities (PLC).

Objective of research

The aim of this study is to share initial findings on the effects of a teacher professional development program on teaching quality.

Research question

The research question that will guide the present study is: What is the effect of a data use teacher professional development program on teaching quality?

Methods

Teachers will participate in a four months teacher professional development program. The sample encompasses 20 Chilean schools, and around 60 teachers from 7-12th grade, from different subjects. We will administer a pre - and post questionnaire in which students give their perceptions about teaching quality.

The teacher professional development program includes workshops for the professional learning communities coordinators, as well as a detailed manual. Each teacher will conduct a data-use cycle, which is a set of steps that helps to systematically improve instructional practices. They will discuss each of the steps of the data use cycle in their PLC.

Expected or preliminary results

We expect the four-month intervention will improve teaching quality in the dimensions targeted by teachers during the program.

Educational importance of this research

This research addresses a crucial aspect of educational practice, which is the use of student feedback for instructional improvement. The study will illuminate how teachers working in PLCs can improve their practices in the classroom using students’ perceptions of teaching quality as a source of feedback.

Connection to the conference theme

Aligned with the ICSEI 2024 theme, this study examines how to develop competencies in teachers to process students’ feedback and improve their instructional practices. Teaching quality is a crucial component in the research on educational effectiveness (e.g., Muijs & Reynolds, 2017). This study is the first in Chile to estimate the effect of a teacher professional development program on teaching quality by means of SPTQ. This study will enrich the conversation about how to develop effective teacher professional development to improve teaching quality.