In 2018, Ireland’s (then) Department of Education and Skills committed to developing a framework to evaluate the impact of teachers’ professional learning (TPL) (Department of Education and Skills, 2018). This paper describes that framework and practical lessons learned. The aim was to develop a research-based TPL evaluation framework, sufficient to evaluate the breadth of available TPL.
The need to evaluate TPL impact is widely recognised (Centers for Disease Control and Prevention, 2019; Guskey, 2000; Kirkpatrick, 1994). Despite some criticism, considerable attention has been given to using “level models” for this purpose (Coldwell & Simkins, 2011; Guskey, 2000; Kirkpatrick, 1994). However, some have argued that such models give inadequate consideration to the TPL’s incorporation of known effective features (Merchie et al., 2018). Compen et al. (2019) present a TPL evaluation model that comprises: contextual factors; key features of professional development; teacher quality; teaching behaviour; and student learning. The Compen et al. (2019) model provides the basis for Ireland’s TPL evaluation framework.
The project to develop a TPL evaluation framework (and supporting materials) ran from 2018 to 2023. With a budget in excess of €300k and overseen by a Steering Group, the project included:
- Desk-based research, including a systematic review of TPL evaluation frameworks
- A primary, post-primary and special school teacher survey
- TPL provider and learner consultation
- A case study of a specific TPL opportunity
- Development of the framework
This paper describes the TPL evaluation framework, linking to earlier project phases and reflecting on the development process.
Ireland’s framework comprises Context; Key features of TPL; Teacher outcomes; and Student, school or system outcomes. While the framework provides for all TPL to have teacher outcomes, not all TPL has outcomes for students, schools or the system. New elements introduced include the addition of “access, administration and data collection” as a key TPL feature and the inclusion of “reflective practice” as a teacher TPL outcome.
Work on this project highlights the need for a shared understanding from the outset of the meaning and purpose of a framework (Fynn et al., 2020; Nilsen, 2015), the potential role of a participatory negotiation process in determining the purpose of evaluation (Liket et al., 2014), and challenges in ensuring shared ontological perspectives in TPL evaluation (Coldwell & Simkins, 2011).
The framework is timely in Ireland, given the September 2023 launch of a new integrated provider for TPL. For wider policy and practice, a key contribution is the recognition in the framework that teachers undertake TPL for both their own benefit and to benefit students, schools and the system. The addition of “access, administration and data collection” and “reflective practice” as features to evaluate supports the development of theoretical models underpinning TPL evaluation.
Themes are: evidence-informed policy that respects and promotes teacher professionalism; and impact of research/policy/practice collaboration and partnerships. The framework recognises teachers as agentic professionals, balancing individual, school and system TPL priorities. The development process was characterised by a collaborative approach, involving multiple highly engaged stakeholders, thus highlighting the positive impact of effective collaboration on the system.