Conference Agenda

In 2018, Ireland’s (then) Department of Education and Skills committed to developing a framework to evaluate the impact of teachers’ professional learning (TPL) (Department of Education and Skills, 2018). This paper describes that framework and practical lessons learned. The aim was to develop a research-based TPL evaluation framework, sufficient to evaluate the breadth of available TPL.

The need to evaluate TPL impact is widely recognised (Centers for Disease Control and Prevention, 2019; Guskey, 2000; Kirkpatrick, 1994). Despite some criticism, considerable attention has been given to using “level models” for this purpose (Coldwell & Simkins, 2011; Guskey, 2000; Kirkpatrick, 1994). However, some have argued that such models give inadequate consideration to the TPL’s incorporation of known effective features (Merchie et al., 2018). Compen et al. (2019) present a TPL evaluation model that comprises: contextual factors; key features of professional development; teacher quality; teaching behaviour; and student learning. The Compen et al. (2019) model provides the basis for Ireland’s TPL evaluation framework.

The project to develop a TPL evaluation framework (and supporting materials) ran from 2018 to 2023. With a budget in excess of €300k and overseen by a Steering Group, the project included:

- Desk-based research, including a systematic review of TPL evaluation frameworks

- A primary, post-primary and special school teacher survey

- TPL provider and learner consultation

- A case study of a specific TPL opportunity

- Development of the framework

This paper describes the TPL evaluation framework, linking to earlier project phases and reflecting on the development process.

Ireland’s framework comprises Context; Key features of TPL; Teacher outcomes; and Student, school or system outcomes. While the framework provides for all TPL to have teacher outcomes, not all TPL has outcomes for students, schools or the system. New elements introduced include the addition of “access, administration and data collection” as a key TPL feature and the inclusion of “reflective practice” as a teacher TPL outcome.

Work on this project highlights the need for a shared understanding from the outset of the meaning and purpose of a framework (Fynn et al., 2020; Nilsen, 2015), the potential role of a participatory negotiation process in determining the purpose of evaluation (Liket et al., 2014), and challenges in ensuring shared ontological perspectives in TPL evaluation (Coldwell & Simkins, 2011).

The framework is timely in Ireland, given the September 2023 launch of a new integrated provider for TPL. For wider policy and practice, a key contribution is the recognition in the framework that teachers undertake TPL for both their own benefit and to benefit students, schools and the system. The addition of “access, administration and data collection” and “reflective practice” as features to evaluate supports the development of theoretical models underpinning TPL evaluation.

Themes are: evidence-informed policy that respects and promotes teacher professionalism; and impact of research/policy/practice collaboration and partnerships. The framework recognises teachers as agentic professionals, balancing individual, school and system TPL priorities. The development process was characterised by a collaborative approach, involving multiple highly engaged stakeholders, thus highlighting the positive impact of effective collaboration on the system.

The importance of school self-evaluation (SSE) as a school improvement mechanism has been acknowledged by increases in the number of countries with legal requirements for schools to conduct self-evaluation. Despite the provision of a range of support to schools to encourage their engagement with SSE, many implementation challenges have been highlighted in research from various jurisdictions, where SSE has been introduced. The provision of professional development (PD) to schools is a common response to such challenges, yet there is a dearth of research into the models of PD that might best support schools to apply the learning and address the common implementation issues that have been identified in the literature. This action research study explored a PD intervention for teachers leading SSE in Irish post-primary schools. The findings explored the experiences and perceptions of staff from 15 of the schools involved, and in particular focused on the features of the intervention that supported participants to apply the learning, by leading the SSE process in their respective schools. The findings indicate that this model of PD for teachers leading SSE may offer some useful solutions to the ongoing challenges experienced by schools in relation to SSE.

This paper sets out an original conceptual framework for place-based professional learning by teachers and schools in contemporary education systems. The framework has been tested and refined through a study of ‘local learning landscapes’ for teacher professional development in England.

Ensuring that all teachers and schools engage in high-quality Continuing Professional Development and Learning (CPDL) has long been a policy priority in school systems worldwide, given evidence that this is associated with improvements in children’s outcomes (Cordingley et al, 2015). Most recent research in this area has focused on evaluating formal professional development programmes and interventions provided by external (i.e. non-school) organisations (Sims et al, 2020). However, in practice, much professional learning takes place informally, through professional conversations and joint practice development activities, often facilitated by networks and learning communities as well as online (Hargreaves, 2010).

Meanwhile, in many systems globally, school leaders have been granted increased autonomy, for example taking on responsibility for the recruitment and professional development of staff. In these contexts, traditional place-based providers of CPDL, such as Local Authorities, have generally been rolled back, while school leaders have been encouraged to draw on a wider marketplace of CPDL provision (Steadman and Ellis, 2021). These developments might create space for agency and innovation in CPDL, but they also present risks in terms of coherence, quality and equity. For example, some schools and teachers might not engage due to a lack of capacity or awareness.

In this context, the conceptual framework sets out six core features of a ‘local learning landscape’:

• Local lens: Geographic localities do not have any intrinsic coherence unless the professionals who work there choose to imbue it.

• Many linked systems: Each local landscape is composed of multiple organisations and networks which might link together more or less tightly and in more or less formal ways.

• Professional learning: Individuals engage in formal and informal learning, both within their schools and through local and non-local networks as well as via formal courses and provision.

• Practices, tools and routines: Where professionals share practices (including theories and language), tools and routines, this can facilitate individual and collective learning.

• Bridging boundaries: Some individuals operate beyond their immediate organisation or context, helping to move knowledge and expertise around the locality.

• Sense making: Relevant leaders come together periodically to identify and tackle shared issues, taking time to explore underlying causes and to shape collaborative action.

The framework was developed by the authors informed by overlapping bodies of literature: socio-spatial theory; complexity theory; and organisational learning theories. It was tested and refined through empirical research across three localities in England (authors, 2023).

The paper introduces the framework and shares key findings from the study in England, using these to illustrate barriers to, and enablers of, coherence, quality and equity in CPDL for teachers in contemporary school systems. It argues that the framework offers a significant contribution to policy, practice and research in this area.