99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper
School Project. Meanings of School Quality for School Futures from an Ethnographic Perspective
Anna Chiara Angela Mastropasqua, Emilia Restiglian
University of Padova, Italy
Presenting Author: Mastropasqua, Anna Chiara Angela
It is since the time of the Sumerians (3500 BC) that the world has felt the need to pass on acquired knowledge in teaching and learning actions that have become increasingly structured and defined. A need that perhaps arose to make sense of the chaos of the outside world, to orientate, to delineate paths, and to educate the new generations so that they can make the world in which we live better. Undoubtedly the school represents, then as now, one of the pillars of the world and one of the elements that defines the quality of life in a country, and in our complex world, educational institutions are indispensable and necessary for everyone (Ingold, 2019). About quality, Elshaer (2012, p. 8) states: "Quality is a situation in which a set of intrinsic characteristics constantly satisfies the changing requirements of the organization’s customers and other stakeholders". And, if we want to relate this defining principle to the world of schools, we could say that school quality is what is achieved when the internal characteristics of the individual school under evaluation contribute to the satisfaction of the - constantly evolving - demand made on the school itself not only by students but also by society. But the picture of today's school appears dramatic. The data relating to access, attendance, and completion, as well as those relating to learning, scholastic well-being and teacher preparation, highlight numerous elements of weakness in the functioning of the world's schools. The UNESCO Statistical Institute (2022) states that 64 million children of primary school age do not attend school, most of them from marginalized groups, and of the 92% of children who start primary school, only 61% complete it (UNICEF, 2021). In addition to not being accessible to all, school also does not always teach. Even in countries with a high and medium level of economic development, the deficits in basic learning in reading, calculation and science are considerable. The results of PISA 2022, report that although about three out of four 15-year-old students demonstrate attainment of basic skills in reading and science in OECD countries, only 7% of students have attained the highest proficiency levels and only 69% of students are at least substantially proficient in mathematics (OECD, 2023).The data presented allows us to consider the needs and necessities for school intervention to improve the operations and direct the objectives in a new and changing society. Indeed, Sidorkin (2011) tells us that, at the crossroads where the world of mass education finds itself, we lack sufficient theoretical understanding to see where it can and cannot go next. Speaking about quality, the last theoretical aspect we wish to consider is the project Transforming our world: the 2030 Agenda for Sustainable Development of the United Nations (UN, 2015), with Goal 4: to provide quality, equitable and inclusive education, and learning opportunities for all. 2030 Agenda represents a milestone in the research, as it is the document that binds the four case studies selected, whose countries are signatories to the document, and is the script on which the research tools are based. The research we intend to present, therefore, aims to identify meanings and quality perspectives for the school of the future from the analysis of data collected in four case studies, which involved pupils, teachers, heads of school and families from four primary schools in the world, in comparison with the fundamentals of goal four of the 2030 Agenda (UN, 2015); and it has attempted to the tortuous but necessary path towards achieving a quality school by outlining some perspectives on quality for the future of schools.
Methodology, Methods, Research Instruments or Sources UsedThe research aims to identify meanings and quality perspectives for the school of the future starting from the analysis of data collected in multiple-case-studies which involved pupils, teachers, heads of school and families from four primary schools in the world, in comparison with the fundamentals of goal four of Agenda 2030 (UN, 2015). The subjects involved are part of 4 primary schools in the world located in Italy, the Russian Federation, Ghana and Sierra Leone, countries in different levels of economic development and whose public expenditure investment, in relation to GDP in education, is not proportional to their income and to total public expenditure. About the first factor, according to data compiled by the World Bank (2023), the countries of the world can be divided into four economic income classes: high-income countries, upper-middle-income countries, lower-middle-income countries and low-income countries. For each of these categories, a reference country was selected and then a primary school, the case study. The research methodology involved the conduct of multiple case studies in an ethnographic perspective in four mixed, public and government primary schools and, for each of these, the intentionally chosen sample consisted of pupils, teachers, school principals and families of pupils in the last two-year classes (school grades three, four, five or six depending on the school system of reference). Specifically, the selected schools are located in Padua (Italy), Moscow (Russian Federation), Elmina (Ghana) and Freetown (Sierra Leone) respectively. In total, the sample of subjects reached is 210 pupils, 21 teachers, 5 school managers and 122 families.
The multiplicity of research and observation tools, constructed within the trajectory of the 2030 Agenda, allows us to investigate the prospects of quality for the school of the future starting from a reasoning that involved the subjects in defining characteristics and meanings of the school of the present. The data analysis, which considered a specific model for data analysis, was based on the principles of phenomenological interpretative analysis (Pole & Morrison, 2003) and Computer-assisted qualitative data analysis (CAQDAS) with Atlas.ti.
Conclusions, Expected Outcomes or FindingsFrom the combination of what was codified for each primary school, an overall analysis was then reached. The results broaden the view considerably from the 10 sub-goals of Goal 4 of the 2030 Agenda, thus providing to the scientific community, but also to schools, with a bottom-up perspective, new meanings and new actions, to re-construct the school's future from a quality perspective. In particular, a dialogue was opened with respect to the accessibility of the school and the recognition of its importance; to the need for structures and learning spaces appropriate to education and to the needs of the people living in the school; to rethinking the conformation of the classroom environment, which can also be structured by discipline, by students' ability levels and by psycho-physical needs; to the adjustment of school times; to the involvement of pupils and families in decision-making processes; to the implementation of real, concrete lessons that help pupils orientate themselves in the future and to the care of pupils' education in multiple aspects of life; to the development of healthy relationships and a serene learning environment in which there is well-being for all involved, without exclusion. Finally, to the allocation of adequate financial aid.
The research therefore sought to move away from the usual to venture into the unprecedented. What has been arrived at can hopefully represent a tension towards change, towards new horizons and new minds, to reorient objectives, in terms of reflection and praxis, in the direction of an intercultural change, desirable or possible, in the face of the criticalities, the cracks, the limits, of today's school, to outline a new educational paradigm. Understood in this sense, the school is all to be built and in an always open building site.
ReferencesAtkinson, P. (1992). Understanding Ethnographic Texts. SAGE Publications.
Biesta, G. (2012). Making Sense of Education: Fifteen Contemporary Educational Theorists in their own Words. Springer Netherlands.
Biesta, G. J. J. (2017). The Rediscovery of Teaching. Routledge.
Burner T. (2018), “Why is educational change so difficult and how can we make it more effective?”, Forskning og Forandring, 1, 1: 22–134.
Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th edition). Routledge.
Common Worlds Research Collective, 2020. Learning to become with the world: Education for future survival. Paper commissioned for the Unesco. URL: https://unesdoc.unesco.org/ark:/48223/pf0000374032.locale=en (accessed on 20th September 2022).
Denzin, N. K. e Lincoln, Y. S. (2000). Handbook of qualitative research (Second). Sage Publications.
Elshaer, I. (2012). What is the Meaning of Quality? MPRA Paper 57345, University Library of Munich.
Hammersley, M., & Atkinson, P. (2010). Ethnography: Principles in practice (3rd edition). Routledge.
Harvey, L., & Green, D. (1993). Defining Quality. Assessment & Evaluation in Higher Education, 18(1), 9–34. https://doi.org/10.1080/0260293930180102
Ingold, T. (2017). Anthropology and/as Education. Routledge.
Mullis, I. V. S., & Martin, M. O. (2019). Pirls 2021 assessment frameworks. TIMSS & PIRLS.
OECD & Europäische Kommission (2020). Cities in the world: A new perspective on urbanisation. OECD Publishing.
OECD (2023). PISA 2022 Results (Volume I): The State of Learning and Equity in Education. OECD. https://doi.org/10.1787/53f23881-en
OECD. (2023a). Education at a Glance 2023: OECD Indicators. OECD. https://doi.org/10.1787/e13bef63-en
Packer, M. (2011). The science of qualitative research. Cambridge University Press.
Pole, C. J., & Morrison, M. (2003). Ethnography for education. Open university press.
Sidorkin, A. M. (2011). On the Essence of Education. Studies in Philosophy and Education, 30(5), 521–527. https://doi.org/10.1007/s11217-011-9258-3
Smith, J. A., Larkin, M., & Flowers, P. (2009). Interpretative phenomenological analysis: Theory, method, and research. SAGE.
UN General Assembly (2015). Transforming our world: The 2030 Agenda for Sustainable Development. A/RES/70/1. https://www.refworld.org/docid/57b6e3e44.html
UNESCO (2022). A Bayesian Cohort Model for Estimating SDG Indicator 4.1.4. Out-of-School Rates. A proposal by the UNESCO Institute for Statistics and the Global Education Monitoring Report.
https://www.unesco.org/gemreport/sites/default/files/medias/fichiers/2022/08/OOS_Proposal.pdf
UNICEF (2021), Protecting child rights in a time of crises. UNICEF Annual Report 2021, testo disponibile al sito: https://www.unicef.org/media/121251/file/UNICEF%20Annual%20Report%202021.pdf (accessed on 20th Settembre 2022).
Wolcott H. (1995), The Art of Fieldwork, Alta Mira Press, Lanham.
Yin, R. K. (2009). Case study research: Design and methods (4th Ed.). Sage.
99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper
Teacher Evaluation VS Teacher Attestation from Literature Review
Akerke Boltabekova1,2, Nazipa Ayubayeva1
1Kazakh National Women’s Teacher Training University; 2Nazarbayev Intellectual School of Physics and Mathematics in Almaty
Presenting Author: Boltabekova, Akerke
This literature review presented in this paper was completed to fulfill the assignment for my PhD study. As an emerging researcher I am interested in teachers’ teaching quality and how it is assessed. Therefore, I would like to present in this paper the most interesting finding I have encountered in my literature review journey. I have looked at numerous conducted research, scholarly written articles and books related to international and Kazakhstani practice of teaching quality assessment. As I am from Kazakhstan, it was in priority for me to look at its performance in teaching assessment quality and how it could be improved. However, to have wide and diverse view and to discover teaching assessment practices by leading countries in education, I have selected six countries to compare with Kazakhstan.
There is an interesting argument that the school’s quality cannot exceed its teacher’s quality (OECD, 2014). The idea of the statement could be interpreted as a teacher’s quality is crucial in fostering positive outcomes in the classroom. If a teacher’s quality is important, then how to determine whether a certain teacher is well-qualified? There are frameworks practised worldwide known as teacher appraisal, performance appraisal, and teacher evaluation. In the Kazakhstani context, it was given the name of teacher attestation by the Ministry of Education and Science (MoES, 2000). Therefore, I will use the term teacher evaluation to refer to international teacher assessment systems while teacher attestation will be used in relation to Kazakhstan’s teacher assessment system.
According to Danielson (2007), teacher evaluation serves as a foundation for assessing, improving, and maintaining the quality of teaching. The Kazakhstani teacher attestation is defined by Pak (2020) as a periodical process which takes place to identify the level of professional and qualification training of teachers to determine the level of pedagogical skills. Although both teacher evaluation and teacher attestation are generally implemented for assessment and development of teaching quality, the aims and structure may vary. For instance, countries such as Finland and Singapore are often cited for having effective teacher evaluation systems. Teachers in Finland are urged to reflect and learn continuously (Sahlberg, 2010). Professional learning communities, mentoring, and an emphasis on collaboration among teachers are all included in Singapore's model (Ong Kelly et al., 2008). Every system seeks to address the teachers' continuous growth and development.
In case of Kazakhstan, the reviewed teacher attestation was introduced and implemented in Kazakhstan in 2000 and its aim was to assess teachers' competencies, subject knowledge, and adherence to educational standards (MoES, 2000). Therefore, it was implemented to enhance the quality of education and professional development among educators. This assessment often included the review of instructional practices, student outcomes, and professional responsibilities. However, changes in the policy of teacher attestation as well as in its aim is a frequent practice in Kazakhstan. Therefore, in Kazakhstan as well as in other countries, teacher attestation and teacher evaluation systems have faced and been facing challenges.
In this context, this paper provides a systematic review and analysis of scholarly written works, policy documents, and empirical research. The evaluation systems of countries such as Finland, Canada, China, Japan, Singapore and South Korea were selected to be compared with the Kazakhstani teacher attestation. The paper is written to explore the aims of selected international teacher evaluations and Kazakhstani teacher attestation.
The systematic literature review aims to answer the following questions:
- What are the aims of Teacher Evaluation globally and in the Kazakhstani context?
Methodology, Methods, Research Instruments or Sources UsedThe process of the systematic literature review guided by research questions took the following steps. Initially, academic and scholarly related to the topic of interest articles from reliable journals were considered. The criteria of relativity and reliability were created in selecting articles. The criteria were based on the book Doing Your Own Research by Judith Bell (2010). According to Bell (2010) the keywords should be well grouped and organized before starting the search. In case of this research, the following keywords were identified and implemented: “teacher evaluation”, “teacher appraisal”, “performance appraisal”, “teacher attestation”, “Finland”, “Canada”, “China”, “Japan”, “Singapore”, “South Korea”, and “teacher evaluation critique”. The search included the university, national libraries in Kazakhstan as well as electronic databases as Google Scholar, ERIC, Taylor & Francis, Sage Publications, and SCOPUS. After the articles were accessed on the criteria of reliability and relativity, they were thematically grouped and divided. The initial number of articles was 60, however, after reviewing and analysing, 20 articles were removed, and 40 articles were opted and used in this systemic review. Moreover, apart from articles and empirical research, official standards by ministries were also included in the review according to the nationally standardised nature of teacher attestation.
The selected literature was divided according to the themes such as the Formation of Teacher Evaluation, Teacher Evaluation Systems Globally, Historical Background of Teacher Attestation, Aims of Teacher Evaluation and Teacher Attestation Systems, and Critiques towards Teacher Evaluation and Teacher Attestation.
However, there is an important aspect I have to mention as an emerging researcher who wants to be an ethically honest researcher. Unfortunately, through my current place of study and due to their absence of subscription and access to databases such as SCOPUS and other databases with peer-reviewed articles and scholarly works, I had to be involved in a process of obtaining the articles related to the topic of my interest through the website as Sci-Hub which is a shadow library website that provides free access to peer-reviewed and scholarly works. I do understand that my actions are unethical, however, in order to read reliable literature, I had to obtain them through the shadow library website. The reason why I have mentioned my such actions is because I would like to highlight my struggles as an emerging researcher of accessing reliable information, so in future universities will be obtaining official subscriptions not to make their students unethical as researchers.
Conclusions, Expected Outcomes or FindingsThe purpose of this paper is to introduce the topic of my interest and identify the gaps through the review of the literature. According to the review of the existing literature which were covered in this systematic review of literature several gaps have been identified.
It was identified that both goal teacher evaluation systems worldwide and the teacher attestation system in Kazakhstan are concerned about the quality of education. However, it was explored that the aims of the considered teacher assessment systems in this paper significantly vary. Moreover, some teacher evaluation systems face critiques and challenges as well as the attestation system of Kazakhstan. The literature review emphasises the importance of effective teacher evaluation systems that are contextually relevant, supportive, and growth oriented.
Moreover, undertaking this literature review has not only been done to fulfil the assignment, but has also been a personally enriching experience, by providing a valuable and deeper understanding of the topic of my interest. However, I have encountered challenges such as limitations of accessing peer-reviewed literature. Despite the hurdles, through this journey of literature review, I have learned important aspects as an emerging researcher such as being systematic, critical, able to analyse and interpret.
ReferencesDanielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Maharaj S. (2014). Administrators’ views on teacher evaluation: Examining Ontario’s teacher performance appraisal. Canadian Journal of Educational Administration and Policy, 152, 1-58.
Martinez, F., Taut, S., & Schaaf, K. (2016). Classroom observation for evaluating and improving teaching: An international perspective. Studies in Educational Evaluation, 49, 15-29.
OECD. (2014). Reviews of National Policies for Education: Secondary Education in Kazakhstan. OECD Publishing.
Ong Kelly, K., Yun Angela Ang, S., Ling Chong, W. and Sheng Hu, W. (2008), "Teacher appraisal and its outcomes in Singapore primary schools", Journal of Educational Administration, 46(1), 39-54.
Order of the Minister of Education and Science (MoES) of the Republic of Kazakhstan “On approval of the Rules on the certification of attestation staff,” dated April 28, 2000 No. 422. Registered with the Ministry of Justice of the Republic of Kazakhstan on July 13, 2000 No. 1193. Retrieved [5.12.2023] from [https://adilet.zan.kz/rus/docs/V000001193]
Order of the Minister of Education and Science (MoES) of the Republic of Kazakhstan “On approval of the Rules and conditions for certification of civil servants in the field of education and science,” dated January 27, 2016 No. 83. Registered with the Ministry of Justice of the Republic of Kazakhstan on February 29, 2016 No. 13317. Retrieved [23.10.2023] from [https://adilet.zan.kz/rus/archive/docs/V1600013317]
Pak, V. (2020). Teacher Appraisal System in one Nazarbayev Intellectual School in Kazakhstan: Teachers’ Perceptions and Experiences. Retrieved [22.11.2023] from [https://nur.nu.edu.kz/handle/123456789/4914]
Sahlberg, P. (2010). "The secret to Finland's success: Educating teachers." Teachers College Record, 112(10), 2603-2634.
Tarhan, H., Karaman, A., Lauri, K., & Aerila, J. A. (2019). Understanding teacher evaluation in Finland: A professional development framework. Australian Journal of Teacher Education (Online), 44(4), 33-50.
Whyte, J. B. (1986). Teacher assessment: a review of the performance appraisal literature with special reference to the implications for teacher appraisal. Research Papers in Education, 1(2), 137–163.
Yoo, J. (2018). Evaluating the new teacher evaluation system in South Korea: Case studies of successful implementation, adaptation, and transformation of mandated policy. Policy Futures in Education, 16(3), 277-290.
Zhang, X. F., & Ng, H. M. (2017). An effective model of teacher appraisal: Evidence from secondary schools in Shanghai, China. Educational management administration & leadership, 45(2), 196-218.
Zhumykbayeva, А., Ablayeva, М. (2023). “Teacher Attestation: Identifying the Factors Influencing Teacher Reflective Skills.” BULLETIN Series of Pedagogical Sciences, 79(3), 256–264.
99. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference)
Paper
Evaluating the Impact of a Policy in Education in Kazakhstan Using Synthetic Difference-in-Dierences
Marina Berdikhanova
Minerva University, United States
Presenting Author: Berdikhanova, Marina
There is substantial evidence supporting the importance of education and its significant positive effect on our lives. Specifically, it leads to higher income (Card, 1999), reduced crime rates (Machin et al., 2011), and better health (Conti & Hechman, 2010), and for women, in particular, it leads to lower birth rates and higher woman autonomy (Cygan-Rehm & Maeder, 2022; The World Bank, 2022). One of the common ways to assess the quality of education in a country, albeit subject to debate, is through standardized examinations. In the international arena, Kazakhstani students score lower than the Organisation for Economic Co-operation and Development’s (OECD) average on international tests like the Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study (TIMSS), with an overall declining national average over time. To improve the quality of education at primary, basic, and general secondary education levels, the Ministry of Education of Kazakhstan has implemented numerous educational programs in the past decade. However, there is little to no empirical evidence to support these programs and justify the allocated budget apart from success in the low-scale pilot studies. To address this challenge, I perform a policy evaluation of a shift to per capita funding in primary and secondary education levels, piloted in 2014 in partnership with UNESCO and launched in public schools in the capital Astana in 2018 and two other largest cities in the country, Almaty and Shymkent in the following year. Using the Difference-in-Differences, Synthetic Control and the newly introduced Synthetic Difference-in-Difference estimator by Arkhangelsky and colleagues (2021), I find no statistically significant evidence of the shift to per capita funding on the share of students who score below the threshold on a United National Testing (UNT) taken by high school graduates. While there certainly are limitations due to the data’s availability, the study’s short period, and the policy’s possible lagged effects, this paper is a significant step in using empirical research to inform policymaking and evidence-based social intervention in Kazakhstan.
Methodology, Methods, Research Instruments or Sources UsedQuestion
What is the impact of the shift to per capita funding in K-12 in Kazakhstan on student UNT scores in Astana?
Objective
To evaluate the impact of the shift to per capita funding in Astana using a quasi-experimental design and publicly available data, controlling for confounding variables across regions.
Program
In 2018, 76% of schools in Astana city had to undergo a mandatory transition from ”smeta” funding to per capita funding, followed by two other major cities, Almaty and Shymkent, in 2019 (the cities were, thus, excluded from the analysis).
Design, Setting and Units of Analysis
Using difference-in-differences (DID), synthetic control (SCM), and synthetic difference-in-difference (SDID) estimators, this study examines the effect of the shift to per capita funding in the city of Astana using UNT examination scores from 2014 until 2022 and compares it to the student outcomes of the remaining 13 regions (excluding an outlier).
Main Outcomes and Measures
The study uses UNT examination scores (specifically, the number of students who score below the national threshold to enrol in higher education institutions) as an outcome variable. Future studies will also use PISA and TIMSS scores to measure the policy’s success as soon as the results become available in 2023 and 2024.
Limitations
Due to inconsistency in data reporting, numerous other educational programs initiated by the government simultaneously, and possible lagged effects of the policy, the current study only lays the foundation for further analyses for impact evaluation. Future studies will significantly benefit from using scores from international examinations after sufficient time has passed.
Conclusions, Expected Outcomes or FindingsResults
The results from all three methods (DID, SC, SDID) show no impact of the program on student UNT scores (coefficients 0.04, 0.03, 0.03, respectively).
Findings
Using high school examination scores as a measure of success and controlling for demographic characteristics across regions, the shift to per capita funding does not affect student outcomes in public schools.
Meaning
Despite the government’s report of the program’s positive effect on student outcomes (8.6% increase), the results of this study show no effect of this program.
ReferencesAbadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of california’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505. doi: 10.1198/jasa.2009.ap08746
Abadie, A., Diamond, A., & Hainmueller, J. (2014). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495–510. doi: 10.1111/ajps.12116
Abadie, A., & Gardeazabal, J. (2003). The economic costs of conflict: A case study of the basque country. American Economic Review , 93 (1), 113–132. doi: 10.1257/000282803321455188
Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W., & Wager, S. (2021). Synthetic difference-in-differences. American Economic Review , 111 (12), 4088–4118. doi: 10.1257/aer.20190159
Asian Development Bank. (2018). Kazakhstan country gender assessment. Country Planning Documents, 1–109. doi: 10.22617/tcs179181
International Association for the Evaluation of Educational Achievement. (2023b). Trends in international mathematics and science study (timss). IEA. Re- trieved from
JSC Financial Center. (2022). Jsc financial center. Retrieved from https://
www.invest.fincenter.kz/
|
