23. Policy Studies and Politics of Education
Paper
Contradictory Assumptions Regarding Evaluation Policy
Tirzah Margolin, Yariv Feniger
Ben-Gurion University of the Negev, Israel
Presenting Author: Margolin, Tirzah
The New Public Manegment (NPM) influences educational policy in many aspects. One of them deals with strengthen global governance by the reinforce the involvment of stakehoders from the business sector and philanthropy in educational policy. This involvement strives to adapt values of globalism, standards, and accountability as part of the educational evaluation policy (Lingard, Martino, & Rezai-Rashti, 2013).
The educational evaluation policy targets power relations between different stakeholders. The stakeholder's position and values are more often reveal in the way they identify structural problems and strategical difficulties in the education system. That effects both the educational evaluation policy and the expectancy from the education system to enact and implement the policy. The enactment of evaluation policy is more often challenging due to demands from the formal authorities and obstacles from inside the educational system (Adert-German, 2021; Ball, Magurie, Braun, & Hoskins, 2011; Camphuijsen, 2020).
The objective of this research is to comprehend the scope of evaluation policy and the effects of NPM on it. This highlights the influence of different ideologies and standards regarding educational evaluation policy. Moreover, the research challenges policy stakeholder's expectancy from enactment and implementation of the policy.
Our research deals with the educational policy in Israel since 2005. This was a significant year for Israeli education policy because of the Dovrat (Israel National Force for Education) Committee, which revealed their findings and recommendations.
This committee raised concerns regarding improving evaluation and measurement of the educational system, and adopting an evaluation policy that promotes accountability (Livni-Huberman, 2021; Resnik, 2011).
The Dovrat committee recommended establishing the National Authority for Measurement and Evaluation. It was also involved in the evaluation policy and its implementation in the educational system, including the national and international examinations. Another notable recommendation was creating the position of evaluation coordinator. A coordinator is a teacher specializing in the field of evaluation. This teacher trained to become the school evaluation coordinator and to enact the evaluation policy within the school.
Methodology, Methods, Research Instruments or Sources UsedThis research conducted using qualitative methods including extensive analysis of formal policy documents, such as legislation and governmental registers. We also included informal policy documents in our analysis. These documents included protocols and unofficial professional reports, due to the importance of unofficial documents for comprehensive understanding of different stakeholders’ perspectives.
The research included also interviews with politicians and other high-level decision makers in the Israeli education system.
The qualitative thematic analysis was based on Carol Bacchi’s critical methodology, which relies on Foucault's critical approach (Bacchi & Goodwin, 2016; Bacchi, 2020). Becchi's methodology identifies the problems and solutions implemented by the policy. The methodology — What’s the Problem Represented to be?’ (WPR) — highlights problems, and how they have been presents in the public domain. This encapsulates the influence of bureaucracy and the relationship amongst external stakeholders. Furthermore, Bacchi recommended considering stakeholders who were excluded from legislation.
Conclusions, Expected Outcomes or FindingsThe educational evaluation policy highlighted difficulties in the functioning of the Israeli educational system. Policy makers outlines the system inability to track achievements due to the lack of academic standardizations and characterize a problematic system that does not achieve global academic goals. They identified lack of the standardizations as a problem, and expressed the need for a centralized evaluation policy.
Conversely, there are those who promotes strict policy as a potential solution for the challenges of the educational system. They believe that systematic and public evaluation leads to competition, which strengthens the accountability of the system and can lead for pragmatic solution for the system challenges. For example, substandard results in national exams will lead principals to enact differently in order improve achievements.
The different approaches reflect the dispute about evaluation policy, due to the involvement of internal and external stakeholders. Within the educational system, there are different approaches regarding the role of evaluation. Moreover, power relationships influences the current dialogue regarding internal and external evaluation.
External stakeholders challenge the educational system to conform to measurable objectives and adopt NPM values. While the volatile situation in the government and the deficiency of the educational system creates instability in evaluation policy.
Some stakeholders request to use evaluation as a tool to strengthen the educational system. For example, the use of internal evaluation methods to improve learning and ongoing analysis for improving management and infrastructures. Yet, those stakeholders express distrust in the ability of schools to use evaluation for addressing challenges.
Under the volatile situation and different attitudes, unfortunately, the policy resulted in substandard outcomes. Stakeholders noted that attempt to define current evaluation policy, implemented with conflicting results.
Our research represents a variety of attitudes amongst stakeholders involved with evaluation policy. These values influence policy enactment and reinforced within the school system.
ReferencesAdert-German, T. (2021). Sustainable School Self evaluation: Enactments and Perceptions of Balancing Accountability and Improvement Goals. Educational Assessment, Evaluation and Accountability( (2021) 33:291–315), 291–315.
Bacchi, C. (2020). Policy as Discourse: What does it mean? Where does it get us? Discourse: Studies in the Cultural Politics of Education, 21(1), 45-57.
Bacchi, C., & Goodwin, S. (2016). Post-structural Policy Analyses. New York: Palgarve pivot.
Ball, S. J., Maguire, M., Braun, A., & Hoskins, K. (2011). Policy Actors: Doing Policy Work in Schools. Discourse: Studies in the Cultural Politics of Education(32:4), 625-639.
Camphuijsen, M. (2020). From Trust in the Profession to Trust in Results: A Multi-Level Analysis of Performance-Based Accountability in Norwegian Education. Barcelona: Department of Socialogy The Autonomous University of Barcelona.
Lingard, B., Martino, W., & Rezai-Rashti, G. (2013). Testing regimes, accountabilities and education policy: commensurate global and national developments. Journal of Educational Policy, 28, 2013 - Issue 5, 539-556.
Livni-Huberman, T. (2021). The Senior Planning and Strategy Division of the Ministry of Education - A Comparative Historical Examination of Policy Aspects During the Planning Action of the Ministry of Education (1948-2017). Tel Aviv University.
Resnik, J. (2011). The Construction of a Managerial Education Discourse and the Involvement of Philanthropic Entrepreneurs: The Case of Israel. Critical Studies in Education, 52:3, 251-266
23. Policy Studies and Politics of Education
Paper
Educational Reform Policy- A Professional Challenge for School Principals
Nirit Pariente, Noy Dali, Dorit Tubin, Heli Amossi
Ben-Gurion University of the Negev
Presenting Author: Pariente, Nirit;
Dali, Noy
In recent years, instability has become one of the permanent elements in everyday life, from the global economic crisis in 2008, through the climate crisis and the Arab Spring, which increased emissions and migration to Europe, culminating with the COVID-19 pandemic. One of the consequences of these emergencies was recognition of the advantage of local government over central government in effectively solving problems (Gupta et al., 2021). These events, along with changes of government, demographic shifts, and economic changes, are reflected in the characteristics of educational reform (Cohen, Spillane, & Peurach, 2018). Educational reforms usually seek to establish a new policy that will change teaching and learning patterns for the benefit of the schools and society (Sahlberg, 2016). As such, educational reform usually challenges the school principals’ basic concepts and assumptions, and can affect their professional skills. This study seeks to explore how a top-down reform affects principals’ professional skills.
The reform explored in this study is called “Gefen” (PAF – Pedagogical-Administrative Flexibility), designed by Ministry of Education policymakers, to be implemented by school principals. This reform reflects the growing accountability and crisis management trend at local leadership level. By means of a digital platform, this reform enables management of school budgets by selecting administrative and pedagogical alternatives, such as educational programs, consultancy and training, technological tools, etc. Choosing solutions requires principals to use professional skills to diagnose the schools’ needs, choose an appropriate intervention, and infer from the process and results (Abbott, 1988). In addition, to successfully meet the demands of the reform, principals have to develop supportive networks with colleges and experts to obtain information, resources, and tools (Eyal, 2019). The present study seeks to examine the relationship between educational reform and the professional skills of school principals.
Preliminary findings show that the main challenges the principals face are meeting budget requirements, timetables, and reports, while keeping to the reform budget. The principals deal with these challenges by participating in networks that provide sound advice, consultation, and quick solutions to the demands of the reform. Following the wisdom of the network enhances the pattern of compliance, and suppresses the professional skills needed to adapt the solutions to the school’s needs and context. Virtually no evidence was found of diagnostic processes regarding students’ or teachers’ needs. The principals preferred to delegate diagnosis responsibilities to middle leaders, such as the social education coordinator, or favored programs with popular topics, like student inclusion and resilience. At times, the guiding principle of “Every child gets something”, leads to purchasing a wide variety of programs that do not necessarily correspond to the students’ or teachers’ needs.
At the same time, when the principals chose programs, there was partial reference to an informed choice between alternatives. One reason for this is that the digital reform includes more than 5,000 different programs, making it difficult to choose. Consequently, in most instances, the principals look for recommendations from colleagues, or continue with programs already operating at the school.
Finally, little evidence was found to evaluate the quality of the program. Principals used accessible indicators, such as the students’ desire to participate or the absence of discipline problems, with no reference to whether the program’s pedagogical or administrative goals have been achieved. Possible explanations for the principals’ coping patterns are associated with the nature of reform implementation (top-down, all at once, without sufficient time), with “growing pains” of the new digital system, and with the principals’ lack of professional skills, which altogether result in them devoting their efforts to “solving the problem of the reform”, rather than to the problems the reform seeks to solve.
Methodology, Methods, Research Instruments or Sources UsedThe research design is a multiple-case study to examine the principals’ professional skills in coping with educational reform. A multiple-case study seeks to study a phenomenon in its social, cultural, and organizational contexts (Merriam, 1988). A “case” in the present study is defined as a principal’s decision and professional skills in implementing the reform.
The number of cases in a multiple-case study ranges from four to ten cases that can represent the phenomenon. In the present study, four categories of schools were selected, based on the size of the school (small, less than 500 students, and large – more than 1,000 students) and the principal’s seniority (young, up to five years, and veteran, more than ten years in office). This combination created four categories of four schools each, a total of sixteen schools.
As customary in case-study research, the data was collected using several practical tools to ensure the study’s reliability and obtain a deep understanding of the phenomenon (Merriam, 2015). The tools included two rounds of semi-structured interviews with the principals and relevant staff members (two or three in large schools), and dozens of reform implementation documents, such as the school curriculum, program contracts, internal feedback survey, and activity plans.
The data was analyzed in two stages: Within-case analysis of each school to identify the interactions between the reform, the context, and the principal’s professional skills, and Cross-case analysis to compare the patterns of similarities and differences between the cases (Yin, 2013). Concurrently, content analysis was conducted, based on an inductive thematic content analysis that emerged from the research data, and a deductive analysis derived from the theories of professions (Abbott, 1988) and expertise (Eyal, 2019).
This study adopts Guba and Lincoln’s (1985) approach to “trustworthiness” in qualitative research. Ethical standards were maintained by presenting the research objectives to the interviewees, protecting their privacy and anonymity, storing all data in password-protected digital folders, and obtaining informed consent from all the interviewees.
Conclusions, Expected Outcomes or FindingsThe study findings indicate that the principals’ focus their efforts on solving the “problem of the reform”, emphasizing its bureaucratic manifestations rather than what the reform sought to promote. This is because the reform was carried out in a rigorous timeframe, without adjustments to take into consideration the school’s calendar or needs. These constraints encouraged the principals to develop expertise by means of a colleague network, to find quick solutions for compliance with the demands of the reform, at the expense of utilizing the reform’s resources to promote their school’s unique goals, by diagnosing the school’s needs, choosing suitable programs, and inferring to promote student learning. This means that rather than promoting educational and pedagogical goals, the reform pushes principals to meet its demands.
Two key conclusions can be drawn: First, the characteristics of the reform prevented the principals from developing professional skills since it was top-down, under a strict time limit, and rigorous budget enforcement reduced the principals’ freedom. The findings are not surprising since developing principals’ professional skills is not one of the reform’s goals. The second conclusion is that the principals approached the reform with their preexisting professional skills, which largely depend on their seniority and the size of their school. This variance leads to differential reform assimilation; however, the vast majority emphasizes meeting bureaucratic requirements at the expense of educational or pedagogical quality.
There are theoretical and practical contributions: Theoretically, the findings demonstrate how the conditions of the reform promote or hinder principals’ professional skills of diagnosing the school’s needs, making an informed choice of intervention, and making inferences regarding the results of the intervention. Practically, the present study’s results can inform principal preparation programs regarding the reform, which can help in leading effective implementation, and developing the principals’ professionalism.
ReferencesAbbott, A. (1988). Professional work, ch. 2, 35-58. The system of professions. Chicago: The university of Chicago press.
Cohen, D. K., Spillane, J. P., & Peurach, D. J. (2018). The dilemmas of educational reform. Educational Researcher, 47(3), 204-212.
Eyal, G. (2019). The crisis of expertise. Polity Press.
Guba, E. G. & Lincoln, Y. S. (2000). Paradigmatic controversies, contradictions, and emerging confluence. In Denzin, N. K.& Lincoln, Y. S. (Eds.), Handbook of qualitative research, 163-188. Thousand Oaks: Sage.
Gupta, S., Nguyen, T., Raman, S., Lee, B., Lozano-Rojas, F., Bento, A., ... & Wing, C. (2021). Tracking public and private responses to the COVID-19 epidemic: evidence from state and local government actions. American Journal of Health Economics, 7(4), 361-404.
Merriam, S. B. (1988). Case study research in education: A qualitative approach. Jossey-Bass.
Merriam, S.B. (2015), Qualitative Research: A Guide to Design and Implementation, Jossey-Bass, San Francisco, CA.
Sahlberg, P. (2016). The global educational reform movement and its impact on schooling. The handbook of global education policy, 128-144.
Yin, R. K. (2013). Case Study Research: Design and Methods. Fifth edition. Thousand Oaks, CA: SAGE Publications.
23. Policy Studies and Politics of Education
Paper
New Spaces of Transnational Governance: A Case Study of the LOOP Consortium's Innovative Approach to Teacher Induction Policies in Europe
Marta Mateus de Almeida, Estela Costa, Joana Viana, Carolina Carvalho
Universidade de Lisboa Instituto de Educação
Presenting Author: Mateus de Almeida, Marta;
Costa, Estela
The study analyzes the emergence of new modes of regulation in education and the teaching profession, resulting in phenomena of multiregulation in education (Barroso, 2005). This involves the interaction of state and non-state actors at different levels and with different approaches, in new spaces, utilizing various governance instruments (Leuze Martens & Rusconi, 2007; Rizvi & Lingard, 2006). The study acknowledges the increasing intensity and complexity of transnational governance in education (Carvalho, 2016), leading to the creation of non-traditional spaces. One such example is the Erasmus+ support line for European projects, specifically Key Action 3 – Support for policy reform, Action Type – Policy experimentations, which includes the case of the European project - LOOP – Empowering Teachers.
The propositions put forth by this group are analyzed as an expression of transnational governance (Ozga & Lingard, 2007; Djelic & Sahlin-Anderson, 2006) of teacher training policies in unconventional forums. The identification of the issues that unite the actors in the LOOP consortium is based on the recognition of a global political matter that demands attention. This matter arises from two main factors: i) the shortage of teachers in Europe, which poses a pressing challenge and has significant impacts on educational systems, and ii) the necessity to attract and retain more qualified professionals, recognizing the critical early stages of the teaching profession (Almeida et al., 2018; Costa et al., 2019). In this context, the establishment of the consortium is justified, further fortified by the approval and funding of the proposal at the European level.
The consortium brings together representatives from Ministries of Education, academics, and teachers from six European countries (Croatia, Slovenia, Spain (Catalonia), Greece, Italy, and Portugal) to address a shared issue. The consortium's collective goal is to design a solution that can be implemented at a national level, incorporating perspectives from state authorities, teachers, and academic experts. The involvement of Ministries of Education adds legitimacy to the consortium as they are considered crucial stakeholders responsible for implementing the proposed solutions within their respective national contexts.
Therefore, as part of a larger project, this study has three main objectives: i) to document how the consortium organizes itself and assigns roles to different actors involved; ii) to analyze the issues raised as part of a critical analysis process, constructing a coherent argument that identifies and defines the key features of a shared problem and emphasizes the need for a collective response; and iii) to present the recommendations that emerge from the coordinated search for solutions to the identified problems, including proposals and suggestions for addressing the challenges at hand.
This effort encompasses a progressive exploration of potential approaches that should be adopted, following the established guidelines (Delvaux, 2009). It is crucial to analyze the tools and methods employed by the consortium to generate knowledge that supports the proposed solutions. For instance, the consortium utilizes various instruments in different national contexts, such as identifying teachers' needs and motivations during the initial phase in the consortium's six countries. Additionally, the consortium gathers 'best practices' and exemplary case studies from participating countries, which serve as inspiring models.
The generated knowledge, aiming to build the supporting 'argument' for the group's proposals, is based on 776 questionnaires collected by the LOOP consortium, applied to teachers (newly graduated and experienced) and school directors; 56 interviews conducted by LOOP teams and six focus groups (64 participants), and results obtained through a trial of the solution via the implementation of national pilot projects.
Methodology, Methods, Research Instruments or Sources UsedThis case study is of a phenomenological and interpretative nature, utilizing a documentary collection produced within the LOOP consortium. The study begins by establishing the overall argument for the relevance of the project, starting from the funding application, and culminating in the design of the solution. The latter includes the synthesis reports of questionnaire and interview results, an e-catalog of best practices, and justifications for the presented programs.
To analyze the documents, the study employs the content analysis technique (Bardin, 2009). Initially, the goal is to identify the reasons behind the participation of different actors in the consortium and the specific roles assigned to each group member. In the second phase, the study focuses on analyzing the narratives produced, which pertain to the identification of problems related to teacher induction policies and processes, as well as the formulation of solutions adaptable to national contexts.
Conclusions, Expected Outcomes or FindingsConcerning the consortium and the presence of diversified actors and their roles, various roles are identified: a) ministerial representatives perform roles of state authority representation; informational roles by providing data on national contexts and legitimizing roles by testing the feasibility of proposed recommendations; b) academic experts are tasked with supporting proposals based on scientific knowledge, and c) teachers are responsible for providing insights into professional challenges and needs and testing suggested solutions.
Despite the unique characteristics of each national/regional context, there is a clear convergence among the six contexts regarding the perception of the teaching profession and professional induction processes as shared problems that demand urgent responses from state authorities. The data collected support a set of assumptions previously agreed upon by the group regarding the preferred induction model - mentoring and peer induction. The use of academic expertise and the production of confirming evidence further reinforce these assumptions, such as the need to train mentors and develop induction programs.
The presented programs serve as guidelines for induction policies, offering solutions that address the identified needs of teachers and countries. These programs include both transversal and mandatory proposals and suggestions, allowing for the adaptation of measures to national and local contexts.
ReferencesAlmeida, M., Costa, E., Pinho, A.S., & Pipa, J. (2018). Atuar na indução de professores: Que implicações para os diretores escolares portugueses?. Revista Portuguesa de Educação, 31(2), 196-214
Bardin, L. (2009). Análise de conteúdo. Lisboa: Edições 70.
Carvalho, L. M. (2016). Intensificação e sofisticação dos processos da regulação transnacional em educação: o caso do PISA. Educ. Soc., 37 (136) 669-683, 2016.
Costa, E., Almeida, M. Pinho, A.S., & Pipa, J. (2019). School Leaders’ needs regarding beginning teachers’ induction in Belgium, Finland and Portugal. Eurasian Journal of Educational Research, 81, 57-78.
Delvaux, B. (2009). Qual é o papel do conhecimento na acção pública? Educ. Soc., 30 (109) 959-985.
Djelic, M.-L.; & Sahlin-Anderson, K. (2006). Institutional dinamics in a re-ordering world. In: M.-L Dejic & K. Sahlin-Anderson (Eds.). Transnational governance. Institutional dinamics of regulation. Cambridge: Cambridge University Press, 2006, p. 375-397.
Leuze E, Martens, K., & Rusconi,, A. (2007) Introduction. In K., Martens, & Rusconi, K. Leuze, K. (eds). New Arenas of Education Governance (p. 3-15). London: Routledge.
Ozga, J., & Lingard, B. (2007). Globalisation, education policy, and politics. In: B. Lingard & Ozga, J. (eds.) The Routlege/Falmer in Education Policy and Politics. New York: Routledge.
RizviI, F.& Lingard, B. (2006) .Globalisation and the changing nature of the OECD’s educational work. In: H. Lauder , P. Brown, J.A. Dillabough, A.H. Halsey A. H. (eds.), Education, Globalisation and Social Change. Oxford: Oxford University Press.
|