Conference Agenda

Great advances have been made in how we conceptualize, operationalize and measure aspects of teaching quality (Charalambous et al., 2021). However, this field of research is fragmented. Scholars work in silos, drawing on their own specific framework despite what are often strong commonalities in ambition, terminology, and structural features across frameworks. We argue that classroom video provides an avenue to work across these silos, allowing multiple frameworks to be applied to the same videos. This provides a common ground for discussions across frameworks, facilitating communication and potentially the integration of different frameworks for understanding teaching.

This symposium uses classroom videos as a common ground to break out of our silos through analyzing the same videos with a broad range of frameworks. This symposium consists of 3 papers (with three additional papers in a linked symposium) that use unique frameworks to investigate teaching quality. The frameworks in this symposia stem from different traditions and are at different stages of development. The three quantitatively-oriented frameworks are International Comparative Analysis of Learning and Teaching (ICALT; van de Grift et al., 2007), Protocol for Language Arts Teaching Observation (PLATO; Grossman, 2015), and the Teacher Education and Development Study-Instruct framework (TEDS-Instruct; Schlesinger & Jentsch, 2016). The two qualitatively driven frameworks are the Model for analysing Teaching Quality derived from the Joint Action framework in Didactics (JAD-MTQ; Sensevy, 2014; Ligozat & Buyck, accepted), the praxeological documentary video analysis (DVA; Martens & Asbrand, 2022). Part 1 of the double symposium (this part) focuses on PLATO, JAD-MTQ, and a lens model comparison of the two frameworks. The quantitatively oriented frameworks pre-determine definitions of teaching quality based on rubric dimensions and performance categories. The qualitative approaches understand teaching as a situated practice developing within a dynamic system of social, material, and semiotic interactions.

This symposium’s ambition is to have participants reflect on how one’s framework shapes how one constructs an understanding of teaching and the limitations and benefits of each framework through comparing the decompositions of the focal lessons across frameworks. Through this, we hope to build common understandings across frameworks and break out of our silos. To this end, we have asked each individual paper to attend to three research questions:

1. What are key patterns of teaching quality (in the focal videos) according to this framework?
2. How are aspects of the frameworks shaping constructing decompositions of the focal videos?
3. What are benefits and challenges with using this framework in analyzing aspects of teaching quality?

The contributors provide an overview of their respective frameworks based on the following categories: purpose and the theoretical grounding of the observation framework, facets of teaching captured, specific focus, grain size (e.g., unit of analysis on time scales), and empirical evidence and use. Then, contributors analyze the same four videos of lower secondary mathematics and language arts lessons from Nordic classrooms. Each contributor presents patterns of findings derived and afforded by their respective framework. To that end, we especially discuss patterns of teaching quality and how differences in the above-mentioned categories might shape the construction of findings as well as limitations and affordances across frameworks.

The inclusion of both mathematics and language arts, as well as both quantitatively and qualitatively oriented frameworks, sets this work apart from past important efforts in this area (e.g., Charalambous & Praetorius, 2018)

In this paper we use the observation system Protocol for Language Arts Teaching Observation (PLATO; Grossman, 2015) as a lens into teaching quality in Norwegian lessons from two different subjects, mathematics and language arts. PLATO was developed at Stanford University in the USA and was originally a tool for studies aiming to link English Language Arts (ELA) instruction to student learning outcomes. Since then, it has been used for different research purposes and in different subjects (e.g., Cohen, 2018) and has been the main tool to describe teaching quality in several publications from the Nordic context (Klette et al., 2017; Tengberg et al., 2022). In this paper we demonstrate PLATO’s way of constructing patterns of teaching quality by focusing on aspects such as theoretical grounding, grain size, and discuss what type of information regarding teaching quality that PLATO may offer and for what purposes that might be useful. PLATO conceptualizes teaching quality in four domains (Instructional Scaffolding, Disciplinary Demand, Representation and Use of Content, and Classroom environment) that together consist of an ensemble of specific teacher practices (e.g., elements), all considered relevant for student learning. These practices are reflected in PLATO’s 12 elements and sub-elements, which are all independently rated on a 1-4 scale for every 15 minutes of a lesson. Together, the 12 elements provide a detailed and rich view of teaching patterns by pointing to whether the specific practices are present as well as the degree of quality of these practices. The findings reveal that the mathematics lessons receive consistently high scores on all PLATO while the patterns in the language arts lessons are more mixed of high and low scores fluctuating across different parts of the lessons. Grounded in this analysis of key patterns using PLATO’s lens of teaching quality, we present benefits and challenges with PLATO. Benefits include a detailed view of how different practices have different foci within and across lessons, while challenges include the way PLATO privileges some instructional formats above others and how to deal with arbitrary cut-off points. Finally, we discuss provoking questions such as whether everything we observe is equally important, and whether we can really determine normatively what patterns of high-quality teaching looks like across different lessons and tasks.

Cohen, J. (2018). Practices that cross disciplines?: Revisiting explicit instruction in elementary mathematics and English language arts. Teaching and Teacher Education, 69, 324–335. https://doi.org/10.1016/j.tate.2017.10.021 Grossman, P. (2015). Protocol for Language Arts Teaching Observations (PLATO 5.0). Palo Alto, CA: Stanford University Klette, K., Blikstad-Balas, M., & Roe, A. (2017). Linking instruction and student achievement: Research design for a new generation of classroom studies. Acta didactica, 11(3), 11-19 Tengberg, M., van Bommel, J., Nilsberth, M., Walkert, M., & Nissen, A. (2022). The Quality of Instruction in Swedish Lower Secondary Language Arts and Mathematics. Scandinavian Journal of Educational Research, 66(5), 760–777. https://doi.org/10.1080/00313831.2021.1910564

This paper presents a model (JAD-MTQ) for observing and analyzing classroom practices based on the Joint Action framework in Didactics (JAD; Sensevy, 2014; Sensevy & Mercier, 2007). This model aims at contributing to international debates on the conceptualization of teaching quality. In the French-speaking research, classroom qualitative studies carried out with the JAD framework typically investigate what and how knowledge contents develop in the teacher and students’ classroom interactions. Over the years, JAD has proved its capacity to analyze classroom practices in various subjects (mathematics, sciences, physical education, French language, etc.; e.g., Amade-Escot & Venturini, 2015; Ligozat et al., 2018). However the use of concepts from JAD is still open to different interpretations, depending on the research objectives pursued. The Model for analysing Teaching Quality based on JAD (JAD-MTQ) presented in this paper systematizes classroom observations according to three dimensions: selection of knowledge contents and tasks, structuration of learning situations and organisation of teacher and students’ interactions (Ligozat & Buyck, accepted). Each dimension is explored at a specific level of analysis, featured by a grain-size and a timescale of teaching unit (Tiberghien & Sensevy, 2012) and decomposed into a set of criteria, allowing to reduce the level of inference to be made from classroom video and transcripts. Similarly with findings from other frameworks presented in this symposium, JAD-MTQ rates the three dimensions of the mathematics lessons as high while the dimensions of the language arts lessons range from medium to very low. However these findings may be grounded in different rationales. In this paper, we highlight JAD-MTQ’s way of constructing patterns of teaching quality as relying upon the dual generic/specific nature of its criteria: they reflect certain aspects of teaching that are found in most classrooms (goals, instructional tasks, group works, classroom discussions, uptakes, etc,) but these criteria are also content-specific because to say something about them it is necessary to analyse the epistemic characteristics of instructional tasks. We argue that JAD-MTQ provides a content-based analysis of teaching quality with a set of dimensions and criteria that are not subject-specific. From this perspective, JAD-MTQ offers a didactic approach to teaching quality, in exploring systemically (according to the relations featuring the didactic system; Chevallard, 1985/1991; also see Schoenfeld, 2012) the power to learn certain specific knowledge contents afforded to the students in the classroom.

Amade-Escot, C., & Venturini, P. (2015). Joint Action in Didactics and Classroom Ecology : Comparing Theories using a Case Study in Physical Education. Interchange, 46(4), 413 437. Chevallard, Y. (1985/1991). La transposition didactique : Du savoir savant au savoir enseigné. La Pensée Sauvage, Ed. Ligozat, F., & Buyck, Y. (accepted). Comparative Didactics. Towards a « didactic » framework for analysing teaching quality. European Educational Research Journal. Ligozat, F., Lundqvist, E., & Amade-Escot, C. (2018). Analysing the continuity of teaching and learning in classroom actions : When the joint action framework in didactics meets the pragmatist approach to classroom discourses. European Educational Research Journal, 17(1), 147 169. https://doi.org/10.1177/1474904117701923 Schoenfeld, A. H. (2012). Problematizing the didactic triangle. ZDM, 44(5), 587 599. https://doi.org/10.1007/s11858-012-0395-0 Sensevy, G. (2014). Characterizing teaching effectiveness in the Joint Action Theory in Didactics : An exploratory study in primary school. Journal of Curriculum Studies, 46(5), 577 610. https://doi.org/10.1080/00220272.2014.931466 Sensevy, G., & Mercier, A. (Éds.). (2007). Agir Ensemble : L’action didactique conjointe du professeur et des élèves. Presses universitaires de Rennes. Tiberghien, A., & Sensevy, G. (2012). The Nature of Video Studies in Science Education. In D. Jorde & J. Dillon (Éds.), Science Education Research and Practice in Europe : Retrospective and Prospecctive. SensePublishers. https://doi.org/10.1007/978-94-6091-900-8_7

The growing interest in observationally assessing classroom instruction has lead to the proliferation of observation frameworks. In order to organize and synthesize results from studies using different observational frameworks, there is a need to understand how different frameworks decompose instruction. This paper adopts a lens model (Brunswik, 1952) to compare such frameworks. The lens model breaks down how frameworks decompose observable features of teaching into scores that are meant to characterize that instruction. Namely, each framework directs raters’ attention to specific pieces of evidence (and away from other evidence) while providing guidance on interpreting evidence and assembling evidence into overall scores. This highlights three specific areas where observation frameworks can be compared: (1) what specific pieces of evidence are identified?; (2) how is each piece of evidence interpreted?; and (3) how is evidence aggregated to create summary scores? The paper uses the lens model to compare how the Protocol for Language Arts Teaching Observation (PLATO; Grossman, 2015) and the Model for analysing Teaching Quality derived from the Joint Action framework in Didactics (JAD-MTQ; Sensevy, 2014; Ligozat & Buyck, accepted) make sense of one mathematics and one language arts lesson from Nordic lower secondary classrooms. This analysis shows how the two frameworks uniquely decompose teaching while acting as a model for comparisons of other frameworks. Overall, the two frameworks identify similar pieces of evidence and make similar interpretations of that evidence. In this way, the two frameworks are quite aligned, providing coherent understandings of instructional practice. However, the frameworks differ in scope and grain size. For example, PLATO considers only whether a teachers’ statement does or does not count as uptake while JAD-MTQ codes teacher statements within several different uptake categories. The largest difference between the frameworks, however, is in how they aggregate evidence to generate overall scores. Like other formalized frameworks, PLATO summary scores are based largely on the frequency and quality of the evidence for a category while JAD-MTQ interprets specific evidence in light of the broader instructional contexts in which that evidence occurs (i.e., meso- and macro-levels). Through demonstrating the lens model, this paper seeks to contribute a novel comparison of the PLATO and JAD-MTQ frameworks while also introducing a novel and fine-grained way to compare how observation frameworks decompose teaching. This can make an important contribution to harmonizing understandings of teaching quality across the many frameworks used in the European context. (Charalambous & Praetorius, 2020).

Brunswik, E. (1952). The Conceptual Framework of Psychology. University of Chicago Press. Charalambous, C. Y., & Praetorius, A.-K. (2020). Creating a forum for researching teaching and its quality more synergistically. Studies in Educational Evaluation, 67, 8. https://doi.org/10/gwsf Grossman, P. (2015). Protocol for Language Arts Teaching Observations (PLATO 5.0). Palo Alto, CA: Stanford University Cohen, J. (2018). Practices that cross disciplines?: Revisiting explicit instruction in elementary mathematics and English language arts. Teaching and Teacher Education, 69, 324–335. https://doi.org/10.1016/j.tate.2017.10.021 Klette, K., Blikstad-Balas, M., & Roe, A. (2017). Linking instruction and student achievement: Research design for a new generation of classroom studies. Acta didactica, 11(3), 11-19 Ligozat, F. & Buyck, Y. (accepted). Comparative Didactics. Towards a Didactic Model for Analyzing the Quality of Teaching. European Educational Research Journal. Tengberg, M., van Bommel, J., Nilsberth, M., Walkert, M., & Nissen, A. (2022). The Quality of Instruction in Swedish Lower Secondary Language Arts and Mathematics. Scandinavian Journal of Educational Research, 66(5), 760–777. https://doi.org/10.1080/00313831.2021.1910564 Sensevy, G. (2014). Characterizing teaching effectiveness in the Joint Action Theory in Didactics: An exploratory study in primary school. Journal of Curriculum Studies, 46(5).