Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

Please note that all times are shown in the time zone of the conference. The current conference time is: 1st June 2024, 09:14:49am GMT

 
Filter by Track or Type of Session 
Only Sessions at Location/Venue 
 
 
Session Overview
Location: Sir Alexander Stone Building, 204 [Floor 2]
Capacity: 55 persons
Date: Tuesday, 22/Aug/2023
1:15pm - 2:45pm11 SES 01 A: Teaching/Learning Methodologies and Approaches for Diverse Needs
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Rita Birzina
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Online Teaching and the Ensuing Challenges While Promoting Quality Inclusive Education

Heidi Flavian1, Dana Barak-Harel2

1Achva Academic College, Israel; 2Oranim Academic College

Presenting Author: Flavian, Heidi; Barak-Harel, Dana

Methods for teaching online were developed and gradually implemented in schools over the last two decades, but because of the COVID-19 pandemic that broke at the beginning of 2020, the development and adoption of all educational methods had to be carried much faster than planned (Kaur & Manroshan, 2020). Likewise, whereas educators and scholars continually search for innovative ways to improve inclusion processes, the pandemic added another challenge, since most teachers began teaching online rather than face-to-face in class (Flores & Swennen, 2020; Kaur, 2020). Likewise, educators had to reconsider the inclusion of learners with special needs in online lessons. Generally, teachers’ concerns relate to two main issues: the challenges of conducting online and hybrid teaching while not all learners have proper internet access, and the need to focus on new learning skills more than on the material content of the (Korkmaz & Toraman, 2020). Zhao & Watterston (2021), emphasized the teachers’ need to develop flexible curricula that allow learners with different learning abilities to cope with the materials, but at the same time, to define basic outcomes that all learners must meet. These educational changes and challenges highlight the need to better understand teachers’ perspectives regarding the processes they should conduct when including learners with special needs in online lessons.

Among the variety of learners with special needs that study in inclusive environments, the most common ones are those with Learning Disabilities (LD), given their high frequency in society, estimated as 5%-17% of the population (Grigorenko et. al., 2020). Accordingly, their inclusion takes place in many schools worldwide through the implementation of a variety of teaching strategies (Flavian & Uziely, 2022).

This study was initially developed out of the desire to better understand primary-school teachers’ perspectives, challenges, and advantages of teaching online during the lockdowns to heterogenous groups that included learners with LD, while also aiming to maintain high quality educational processes. There is no doubt that the role of teachers who include learners with LD in their classes is complex and challenging since they must adapt their teaching and learning for all learners. When teaching in person in schools, one can develop learning adjustments for all learners according to the learning environment, get help from other teachers and apply group learning. However, when teaching online via zoom and other similar platforms, this is not the case.

Aiming to contribute to quality online-teaching in inclusive classes with learners with LD, three main questions led this study throughout the interviews and the data analysis:

  1. How do including teachers with pupils with LD in their class perceive their role as disciplinary teachers during online teaching?
  2. How do including teachers with pupils with LD in their class perceive their role as disciplinary teachers promoting inclusion during online learning?
  3. What is the sense of self-efficacy among including teachers working remotely regarding their ability to optimally include pupils with LD?

Methodology, Methods, Research Instruments or Sources Used
This study is based on the qualitative-narrative approach, according to which researchers seek to learn about participants perspectives and beliefs (Ganzevoort, 2012). The participants of this study were 45 primary-school teachers, who were individually interviewed via Zoom technology. Following the qualitative approach for online interviewing (Salmon, 2014), all interviews were recorded and were immediately professionally transcribed. This method allowed researchers to gain relevant information to better understand the participants’ perspectives regarding the challenges they experienced and the teaching strategies they used to enable the inclusion of learners with LD while teaching online.
Data analysis was conducted in two stages, looking for main themes and ideas that reveal new and practical knowledge: first, the two researchers analyzed the transcripts separately. At this stage, the researchers mainly followed core themes from previous literature, while also highlighting other themes that emerged from the transcripts. In the second stage of data analysis, the researchers compared their results, discussed the new themes they had found and any minor differences that were found.

Conclusions, Expected Outcomes or Findings
Based on the participants’ answers, researchers found that online teaching challenges inclusion processes in general, but even more so when teachers aim to promote quality inclusion of learners with LD. Initial data analysis revealed that primary-school teachers believe that to promote quality inclusion in their classes, they need to teach in face-to-face sessions. They explained their attitude by emphasizing the importance of collecting data about their learners through unofficial ways, mainly based on their learners’ body-language. A common example that reoccurred in various ways was the fact that learners could mute themselves or turn off their cameras during online sessions, which was extremely challenging for teachers who wished to follow all their learners’ learning actions. On the other hand, teachers also emphasized the benefits of teaching and including learners with LD in online sessions, by referring to the virtual workspaces. They explained that these spaces allowed quiet learning areas for peer-learning and individual learning alongside one-on-one teaching, without distracting the other members of the class.
Promoting quality education is an ongoing process that should be based on stable educational theories, updated studies, and innovated learning procedures. There is no doubt that online teaching and learning provide teachers the opportunities to apply learning tools, but at the same time, there are challenges that need to be considered as well. Furthermore, teachers should be professionally prepared how to teach online learners with LD, because of the unique learning processes they require.  

References
Flavian, H., & Uziely, E. (2022). Determinants of teachers' attitudes towards inclusion of pupils with ADHD: The role of teacher education. Frontiers in Education. https://doi.org/10.3389/feduc.2022.941699
Flores, M. A., & Swennen, A. (2020). The COVID-19 pandemic and its effects on teacher education. European Journal of Teacher Education, 43(4), 453-456.‏
Ganzevoort, R. R. (2012). Narrative approaches. In: B. Miller-McLemore (Ed.) The Wiley-Blackwell companion to practical theology, 214-223.‏
Grigorenko, E. L., Compton, D. L., Fuchs, L. S., Wagner, R. K., Willcutt, E. G., & Fletcher, J. M. (2020). Understanding, educating, and supporting children with specific learning disabilities: 50 years of science and practice. American Psychologist, 75(1), 37.‏
Kaur, N. & Manroshan, S.B. (2020). The face of education and the faceless teacher post COVID-19.  Journal of Humanities and Social Sciences Research, Horizon, 2(S), 39-48.‏
Korkmaz, G., & Toraman, Ç. (2020). Are we ready for the post-COVID-19 educational practice? An investigation into what educators think as to online learning. International Journal of Technology in Education and Science, 4(4), 293-309.‏
Salmon, J. (2014). Qualitative online interviews, Strategies, design, and skills (2nd edition). Sage.
Zhao, Y., & Watterston, J. (2021). The changes we need: Education post COVID-19. Journal of Educational Change, 22(1), 3-12.‏


11. Educational Improvement and Quality Assurance
Paper

Teaching/Learning Methodologies for Solving Intergenerational Communication Problems in Tourism and Hospitality Enterprises

Ineta Luka1, Valerija Drozdova1, Gita Šakytė-Statnickė2, Laurencija Budrytė-Ausiejienė2

1Turiba University, Latvia; 2Klaipeda State University of Applied Science, Lithuania

Presenting Author: Luka, Ineta

Communication is a complex, social and never-ending process which takes place in a definite socio-cultural environment. Prior research shows various definitions of communication. It has been defined as a ‘verbal interchange of a thought or idea’ (Hoben, 1954, 77), ‘the transmission of information’ (Berelson, Steiner, 1964, 254), ‘the primary process by which human life is experienced’ (Craig, 1999), ‘a factor that helps develop, maintain, and change cultures’ (Littlejohn, Foss, 2008,4), an everyday ‘information exchange’ (Amaritei, 2013, 279), ‘the way in which people send to each other information and messages, inevitably with different intentions’ (Dragan, 2019, 176), ‘the evolution of physical, biochemical, cellular, community, and technological information exchange’ (Gontier, 2022). Although there are differences in how various scholars define communication, all of them underline that it is an information exchange between people and/or groups of people (Dragan, 2019). Thus, for an efficient communication the relationship between people is significant, including in organisational communication. However, in practice tourism enterprises face communication problems which stem from different cultures, generations, language skills.

Theoretical Framework of the research is formed by the theories of adult learning, generational differences and intergenerational communication specifics.

Nowadays, four different generations as described by Howe and Strass (2000) – Baby Boomers (born in 1943-1960), Generation X (born in 1961-1981), Generation Y (born in 1982-2000), Generation Z (born after 2000) are participating in the labour market. Generational differences are demonstrated in the way how people work, communicate and learn. Baby Boomers are hard-working (de Oliveira Lopes Melo, de Faria, Magri Lopes, 2019), work efficiently, prefer teamworking and in-person communication (Bejtkovsky, 2016). Generation X are autonomous but ‘less patient with activities demanding time’ (de Oliveira Lopes Melo, de Faria, Magri Lopes, 2019), cautious, conservative and value direct and immediate communication (Bejtkovsky, 2016). Generation Y easily accept changes, have good digital skills (Bencsik, Horváth-Csikós, Juhász, 2016), are realistic, confident, multi-tasking and use e-mail and voice-mail in communication (Bejtkovsky, 2016). Generation Z are practical, rather intelligent, brave and like online communication (Bencsik, Horváth-Csikós, Juhász, 2016), their interpersonal communication skills are at a low level (Bejtkovský, 2016).

Generational differences exist not only in communication, but also in their attitude towards learning. In general, adults are characterized by maturity, self-confidence, autonomy, decision-making, purposefulness, but they are less open-minded and receptive to change than children (Pappas, 2013; Kraus, 2016; Luka et.al, 2020). Therefore, specific teaching/learning methods must be applied, and the teaching/learning process has to be implemented in a dialogical manner, considering learners needs and specific requirements, including generational differences.

Baby Boomers highly value traditional education system, base their learning on their vast experience (Bencsik, Machova, 2016; Bencsik, Horváth-Csikós, Juhász, 2016), have a sense of urgency and learning must be meaningful to them (Rothwell, 2008). Generation X give preference to flexibility, short-time trainings, active participation and interactivity (Bencsik, Machova, 2016; Bencsik, Horváth-Csikós, Juhász, 2016), but they are not especially fond of teamworking (Kerry, Myron, 2009), so the learning process must be flexible and active. Generation Y like learning which incorporates IT skills. They have to see immediate result in attaining their learning outcomes (Bencsik, Machova, 2016; Bencsik, Horváth-Csikós, Juhász, 2016). Generation Z are not perfect listeners and lack interpersonal skills (Kirchmayer, Fratricová, 2020), they are always online (Bencsik, Horváth-Csikós, Juhász, 2016) and like to learn individually (Seemiller, Grace, 2016) and apply digital tools.

In ECER2022 the authors presented intergenerational communication problems discovered in tourism enterprises in Latvia, Lithuania and Sweden.

The aim of the research presented in ECER2023: identify generational differences in tourism and hospitality organisations and differences in communication and offer teaching/learning initiatives and methodologies to develop adult learners’ competences to solve intergenerational communication problems.


Methodology, Methods, Research Instruments or Sources Used
This research was conducted in Nordplus Adult Development project „NordTourNet-3: Solving Communication Problems of Different Generations in Tourism Companies” (NPAD-2020/10015; 2020-2023) implemented in Lithuania, Latvia and Sweden. The aim of the project: study communication differences among employees and customers of four different generations in tourism companies and create an educational game facilitating intergenerational communication problem solution in tourism enterprises and compile a training material package for educators of adult learners.
The research uses interpretivism paradigm to understand ‘individual cases and situation’ and meanings ‘that different actors bring to them’, and social phenomena are understood holistically (Coe, 2017, 6).
Qualitative exploratory research (Collis, Hussey, 2009) applying 12 semi-structured interviews conducted face-to-face onsite or using Zoom and Cisco Webex platforms and 9 unobtrusive social observations in tourism companies was done (Aurini, Heath, Howells, 2016). Data analysis: qualitative content analysis by implementing inductive coding process (Croucher, Cronn-Mills, 2019, 162).
Research question: What are the main generational differences and differences in communication in tourism and hospitality organisations and what teaching/learning initiatives and methodologies may be applied to develop adult learners’ communication competences to eliminate intergenerational communication problems?
Research sample: criterion sample (Aurini, Heath, Howells, 2016) was created for interviews – owners and top-level management representatives of tourism and hospitality enterprises with having at least five years of work experience in working with people of different generations, at least two years at a managerial position, and there were at least two employees from different generations in their enterprise. Opportunistic sample was created for observations.
The research period: January 2021 – June 2022.
Research limitations: None of the informants belonged to Generation Z. The employees of the enterprises selected for an analysis belonged to two or three generations only with Generation X represented in 11 tourism enterprises and Generation Y – also in 11 enterprises.

Conclusions, Expected Outcomes or Findings
All informants are directly confronted with generational differences in their organisation: responsibility, communication efficiency, stress management, self-esteem and self-confidence, loyalty to organisation, freedom and openness, attitude to work.  Baby Boomers and Generation X are more responsible and more loyal to their organisation and value work more than younger generations. Lack of language skills and digital skills hinders communication efficiency. Generation X can operate in stressful situations better than other generations. Generation Y has higher self-confidence and self-esteem level than other generations. Generation Z appreciate freedom and openness more than other generations.
The identified generational differences in communication process: 1) language barrier (younger generations don’t speak Russian, older generations don’t speak English); 2) different interests leading to insufficiently effective communication with other generations; 3) the need to follow a hierarchy (Baby Boomers have the largest respect for hierarchy, Generation X has the lowest); 4) differences in communication (Baby Boomers and Generation X prefer communication by phone or face-to-face; Generation Z – electronic communication; Generation Y uses all three types of communication); 5) different use of virtual space (Generation Y initiated a more active transition to a virtual space during the pandemic); 5) different use of IT (Generations X, Y, Z tend to find information on Internet); 6) different communication channels (Baby Boomers use face-to-face communication, Generation X – e-mails, Generation Y – e-mail, social media, Internet sites, Generation Z – social media and Internet sites more often than other channels); 7) different cultures (the cause for most communication problems).
Such teaching/learning initiatives may be applied to develop adult learners’ communication competences: Specific training that meets the specifics of the company's activities; Digital literacy competencies (including management of communication in social networks); Public speaking courses; Foreign language courses; Marketing and sales courses; Face-to-face courses in personal communication with customers.

References
*Amaritei, N.-C. (2013). Communication. International Journal of Communication Research, 3(3), pp.279-281.
*Aurini, J.D., Heath, M., Howells, S. (2016). The How to of Qualitative Research. LA: Sage.
*Bejtkovský, J. (2016). The Employees of Baby Boomers Generation, Generation X, Generation Y and Generation Z in Selected Czech Corporations as Conceivers of Development and Competitiveness in their Corporation. Journal of Competitiveness, 8(4), pp.105-123.
*Bencsik, A., Horváth-Csikós, G., Juhász, T. (2016). Y and Z Generations at Workplaces. Journal of Competitiveness, 8(3), pp.90-106.
*Bencsik, A., Machova, R. (2016). Knowledge Sharing Problems from the
Viewpoint of Intergeneration Management. In ICMLG2016 - 4th International Conference on Management, Leadership and Governance: ICMLG2016, pp.42-50.
*Berelson, B., Steiner, G.A. (1964). Human Behavior: An Inventory of Scientific Findings. NY: Harcourt Brace Jovanovich.
*Coe, R.J. (2017). The nature of educational research. R.J.Coe (Eds.), Research Methods & Methodologies in Education, (5-14), London: Sage.
*Collis, J., Hussey, R. (2009). Business Research. Basingstoke: Palgrave Macmillan.
*Craig, R.T. (1999). Communication Theory as a Field, Communication Theory, 9(2), pp.119-161.
*De Oliveira Lopes Melo, M.C., de Faria, V.S.P., Magri Lopes, A.L. (2019). Building professional identity: a study with female managers who are baby boomers, generation Xers, and millennials. Cad. EBAPE.BR, 17(Special Issue), pp.832-843.
*Croucher, S.M., Cronn-Mills, D. (2019). Understanding Communication Research Methods. NY: Routledge.  
*Dragan, A. (2019). Defining Managerial Communication, Limitations and
Challenges. Annals of “Dunarea de Jos” University of Galati Fascicle I. Economics and Applied Informatics, 3, pp.176-180.
*Gontier, N. (2022). Defining Communication and Language from Within a Pluralistic
Evolutionary Worldview. Topoi, 41, pp.609-622.
*Hoben, J.B. (1954). English Communication at Colgate Re-examined. Journal of Communication, 4(3), pp.76-83.
*Howe, N. and Strauss, W. (2000) Millennials Rising: The Next Great Generation. NY: Vintage.
* Kerry, G., Myron, E. (2009). Interaction among Undergraduate Students: Does Age Matter? College Student Journal, 43(4), A, pp.1125-1136
*Kirchmayer, Z., Fratričová, J. (2020). What motivates generation Z at work? Insights into motivation drivers of business students in Slovakia. Proceedings of the Innovation management and education excellence through vision, 6019-6030.
*Kraus, A. (2016). Perspectives on Performativity: Pedagogical Knowledge in Teacher Education. Münster: Waxmann.
* Littlejohn, S.W., Foss, K.A. (2008). Theories of Human Communication. USA: Thomson Wadsworth.  
*Luka, I., et.al. (2020). Quality Assurance in Adult Education in Latvia. In Flavian, H. (Ed.). From Pedagogy to Quality Assurance in Education: An International Perspective, Emerald Publishing, 155-174.
*Pappas, C. (2013). 8 Important Characteristics of Adult Learners.
*Rothwell, W.J. (2008). Adult Learning Basics. Alexandria, Virginia: ASTD Press.


11. Educational Improvement and Quality Assurance
Ignite Talk (20 slides in 5 minutes)

Using Diversity and Related Statistics in Educational Intervention Studies

Hermann Astleitner

Paris-Lodron-University of Salzburg, Austria

Presenting Author: Astleitner, Hermann

Diversity represents a major issue in all fields of educational theory and practice (e.g., Conners & Capell, 2021). From a social research perspective, it has been defined as “the distribution of population elements along a continuum of homogeneity to heterogeneity with respect to one or more variables” (Teachman, 1980). In educational intervention research, we have, for example, “interventions for diverse people” (e.g., Tincani et al., 2009), diversity as a “research strategy” (e.g., Bent-Goodley, 2021), or “design specifications” for diversity interventions (Vinkenburg, 2017). Diversity in educational intervention research is related to different facets like a research agenda, an evidence-based orientation, exploration and innovation, expanded effectiveness, interdisciplinary focus, error reduction, or statistical quality. For example, as a general social research strategy on diversity, multiple more or less stable personality characteristics should be measured that correlate with the dependent variables (control variables), that are related to the independent variables (interactions) or that might be related to alternative explanations (validation, exploration). In respect to data analysis and statistics, diversity has to be described, controlled, structured, and tested. Although diversity and related variability as well as variance represent the nucleus of statistical analyses in social research, strategies on how diversity can systematically be integrated into data analysis processes in educational intervention studies are still missing (e.g., Astleitner, 2020). Our goal in this paper is to identify, collect, and evaluate statistical concepts related to diversity which are essential in educational intervention research. Our perspective is one of quantitative social researchers who have comprehensive experience in the field of educational intervention studies. Our theoretical focus is based on the concept of diversity in social research settings (Schuelka et al., 2019) and on (methodological) models of educational intervention research (McBride, 2016).


Methodology, Methods, Research Instruments or Sources Used
Our method is a review of statistical methods (e.g., Tipton & Osen, 2018). First, we have reviewed literature and collected statistical concepts which are relevant for diversity and educational intervention research. Second, we have conceptually structured these concepts based on statistical procedures, definitions and goals, software for computations as well as potential use in intervention research. Third, we have formulated implications that allow to guide research and statistical analysis in educational intervention research in the future.
Conclusions, Expected Outcomes or Findings
We have found statistical concepts (as well as statistical software) on diversity related to dispersion indices (e.g., range), outliers (e.g., multivariate outliers), diversity indices (e.g., Simpson´s diversity index), or social cohesion indices (e.g., Herfindahl-Hirschman Index), analysis of covariance, aptitude-treatment-interaction-analysis, recursive partitioning methods, cluster analysis, latent class (clustering) analysis, and homogeneity of variance tests (e.g., Budescu & Budescu, 2012; Doove et al., 2014; Huitema, 2011; Kent et al., 2014; Leys et al., 2013; Schaeffer, 2016).
Using these concepts in a review of literature allows to identify numerous significant implications which can guide future activities in educational intervention research. We discuss issues related to educational interventions like changes in variances as side effects of interventions, disequalizing effects, handling outliers, diversity indices as sources for theory building, discovering different effectiveness patterns in different people, exploring participants who were particularly responsive or finding groups of people with similar characteristics before and after an intervention.
Within this paper, we present, up to our knowledge for the first time, a collection of well and less well-known statistical concepts and related implications that are important for handling diversity in educational intervention research. We have promoted a constructive, methodologically critical view of educational intervention research based on the concept of diversity (e.g., Mellenbergh, 2019). Our work aims to encourage the reflection and use of diversity-related tests as a standard in educational intervention studies as has been the case in other research disciplines (e.g., Magurran, 2003).

References
Astleitner, H. (Ed.). (2020). Intervention research in educational practice. Waxmann. https://www.waxmann.com/index.php?eID=download&buchnr=4197
Bent-Goodley, T. (2021). Diversity in interpersonal violence research. Journal of Interpersonal Violence, 36(11-12), 4937-4952. https://doi.org/10.1177/08862605211013003
Budescu, D. V., & Budescu, M. (2012). How to measure diversity when you must. Psychological Methods, 17(2), 215-227. https://doi.org/10.1037/a0027129
Conners, B. M., & Capell, S. T. (Eds.). (2021). Multiculturalism and diversity in applied behavior analysis. Bridging theory and application. Routledge.
Doove, L. L., Dusseldorp, E., Van Deun, K., & Van Mechelen, I. (2014). A comparison of five recursive partitioning methods to find person subgroups involved in meaningful treatment–subgroup interactions. Advances in Data Analysis and Classification, 8(4), 403-425. https://doi.org/10.1007/s11634-013-0159-x
Huitema, B. (2011). The analysis of covariance and alternatives: Statistical methods for experiments, quasi-experiments, and single-case studies. Wiley.
Kent, P., Jensen, R. K., & Kongsted, A. (2014). A comparison of three clustering methods for finding subgroups in MRI, SMS or clinical data: SPSS TwoStep Cluster analysis, Latent Gold and SNOB. BMC Medical Research Methodology, 14(1), 1-14. https://doi.org/10.1186/1471-2288-14-113
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764-766. https://doi.org/10.1016/j.jesp.2013.03.013
Magurran, A. E. (2003). Measuring biological diversity. Blackwell.
McBride, N. (2016). Intervention research. Springer.
Mellenbergh, G. J. (2019). Counteracting methodological errors in behavioral research. Springer.
Schaeffer, M. (2016). Diversity erfassen: Statistische Diversitätsindizes [Capturing diversity: Statistical diversity indices]. In P. Genkova & T. Ringeisen (Hrsg.), Handbuch Diversity Kompetenz (pp. 47-60). Springer.
Schuelka, M. J., Johnstone, C. J., Thomas, G., & Artiles, A. J. (Eds.). (2019). The SAGE handbook of inclusion and diversity in education. Sage.
Teachman, J. D. (1980). Analysis of population diversity. Sociological Methods & Research, 8, 341-362. https://doi.org/10.1177/004912418000800305
Tincani, M., Travers, J., & Boutot, A. (2009). Race, culture, and autism spectrum disorder: Understanding the role of diversity in successful educational interventions. Research and Practice for Persons with Severe Disabilities, 34(3-4), 81-90. https://doi.org/10.2511/rpsd.34.3-4.81
Tipton, E., & Olsen, R. B. (2018). A review of statistical methods for generalizing from evaluations of educational interventions. Educational Researcher, 47(8), 516-524. https://doi.org/10.3102/0013189X18781522
Vinkenburg, C. J. (2017). Engaging gatekeepers, optimizing decision making, and mitigating bias: Design specifications for systemic diversity interventions. The Journal of Applied Behavioral Science, 53(2), 212-234. https://doi.org/10.1177/0021886317703292


11. Educational Improvement and Quality Assurance
Ignite Talk (20 slides in 5 minutes)

Defeating a Protean Enemy: Red Tape in Education

Jeroen Lauwers

Flemish inspectorate of education, Belgium

Presenting Author: Lauwers, Jeroen

Red tape may arise at work when an authoritative entity – whether a school leader, a legislator, an inspector or a pedagogical school supporter – installs certain conventions, procedures or types of administration with the clear purpose of improving the quality of the work, yet reaches the opposite result. If the eventual executors of these measures – teachers, school leaders, administrative personnel – fail to perceive the purpose of what they are doing, even the best ideas may soon turn into a perception of red tape and provoke feelings of frustration, stress and tedium.

Red tape is all but unfamiliar in the field of education. Yet, as a topic for scholarly inquiry, it is relatively poorly studied. Nevertheless, the fruits of addressing this topic are quite clear. The less time teachers spend on red tape and the more time they get to spend on meaningful (inter)actions with their students, the more chances these students get to realize their optimal learning potential. Moreover, it has been evidenced that teachers who perceive a high rate of red tape in their job are more likely to quit their profession altogether.

Tackling the dynamics of red tape is not so easy. Its forms and sizes are very diverse. A very fine procedure at one school may turn out to cause a lot of frustration in another school. Red tape thus presents itself as a protean enemy, lurking in several areas of the school organization and constantly threatening to undermine people’s motivation to give it their all in the classroom.

To add to the problem, the origins of red tape are often hard to trace. Some red tape is produced by the interventions of the government, while other instances may be installed by a school leader or a coordinator at school. Some may even stem from the particular IT-tools used by the school to facilitate the work, or by the expectations of the students and their parents. Red tape may thus metaphorically be conceived of as an onion, of which one needs to peel quite a lot of layers before its core shows itself.

The Flemish inspectorate of education is determined to help school fight this common enemy. In a recent survey it held among over 7000 professionals, it was evidenced that while school leaders feel that they address the topic of red tape in several ways, teachers respond that they feel few results of these efforts. What seems to be lacking, thus, is a clear and precise dialogue at schools about where red tape is to be found, what the actual requirements of the government are, and how the school itself may deal with these requirements in a more functional and less patronizing fashion.

The ignite talk for this conference aims to highlight the importance of having a nuanced understanding of red tape. It addresses three essential questions:

  • What is red tape precisely?
  • What should schools do and avoid in tackling red tape?
  • How can an online tool as developed by the Flemish inspectorate improve the quality of dialogue and policy about red tape in Flemish schools? (more on that tool in the section about methodology)

By demonstrating how the tool is conceptualized, this talk aims to inspire researchers and practitioners in education to come up with novel and diverse methods to assist school world-wide in their battle against red tape. I thus hope to make a humble contribution to the overall quality of education and the job satisfaction of teachers, school leaders and other practitioners in the field of education.


Methodology, Methods, Research Instruments or Sources Used
Red tape revolves around the balance between effort and (positive) effect. When this balance is distorted, people feel frustrated with the poor effects of their efforts. Tackling red tape is thus not only about reducing people’s efforts, but also about raising the effects of their work.
In developing and launching the online tool called the ‘red tape calculator’, the Flemish inspectorate applied the following procedures:
1. On the basis of focus groups and extant secondary literature, we identified 34 areas in the school organization where red tape is most likely to occur. These 34 areas form the content of the questionnaire on which the red tape calculator is based.
2. School principles can ask for access to a personalized dashboard, from which they can invite the members of their particular school to participate in this survey.
3. We developed a functional grid in which school participants can mark the amount of effort they put into a certain area and the resulting effect of these efforts.
4. We programmed the calculator so that it can calculate the average balance between effort and effect for each of the areas as perceived by each member of the school community.
5. The calculator then provides a school report of the areas where the balance between effort and effect is quite sound, and areas where this balance is distorted.
6. This overview enables school leaders to address the causes of red in a more precise and strategic fashion.
The relevance of this online tool extends beyond individual schools, for the Flemish inspectorate also has access to the average scores of all participating Flemish schools. On the basis of this wider overview, the inspectorate is able to report on broader red tape tendencies in the schools to the Flemish government. This in turn allows for a more precise and purpose-driven political discussion about the actual need for certain decrees and other legal measures. The ultimate goal is, of course, to come to a legislative and practical framework that allows for the clear articulation of societal expectations towards the schools, but also for a maximum of autonomy in how teachers and school leaders deal with these expectations.

Conclusions, Expected Outcomes or Findings
As the ignite talk’s main purpose is to inspire listeners to address red tape in their local context, I will not dwell as much on the eventual results of this research in Flanders, but rather demonstrate how we devised the tool and how it assists us in tackling red tape in the different layers of the Flemish education system.
What we expect as an outcome from our engagement, is that schools will feel encouraged to claim the freedom they receive from the government to organize themselves in a functional way. At the aforementioned survey among over 7000 professionals in education, it was evidenced that more than 80% of the participants indicates that the level of red tape in Flemish education has increased in the past three years. It is to be expected that the combined effort from school leaders, inspectorate and legislators will result in a marked change of that trend (as we will measure again in a new survey in three years from now).
In the end, we hope that this purposeful effort to reduce red tape in education will also be positively linked to student performance, job satisfaction among school professionals and the attractiveness of working in education in Flanders.

References
Bozeman, B. (2000). Bureaucracy and Red Tape. Prentice Hall.
Campbell, J. W. (2017). Red tape and transformational leadership: An organizational echelons perspective. Journal of Organizational Change Management, 30(1), 76-90.
DeHart-Davis, L., Davis, R. S., & Mohr, Z. (2015). Green tape and job satisfaction: Can orga nizational rules make employees happy? Journal of Public Administration Research and Theory, 25(3), 849-876.
Department of Education Flanders (2016). Report: Operatie Tarra. www.onderwijs.vlaanderen.be/nl/operatie-tarra/rapport-operatie-tarra (in Dutch).
Eggers, W. (2007). Government 2.0: Using technology to improve education, cut red tape, reduce gridlock, and enhance democracy. Rowman & Littlefield Publishers.
Kenis, P. et al. (2013). Report: Kom op tegen planlast. www.onderwijs.vlaanderen.be/nl/operatie-tarra/rapport-kom-op-tegen-planlast-2013. Antwerp Management School (in Dutch).
Muylaert, J., Decramer, A. & Audenaert, M. (2022). How leaders’ red tape interacts with employees’ red tape from the lens of the job demands-resources model. Review of Public Personnel Administration, 1-26.
Struyven, K., & Vanthournout, G. (2014). Teachers’ exit decisions: An investigation into the reasons why newly qualified teachers fail to enter the teaching profession or why those who do enter do not continue teaching. Teaching and Teacher Education, 43, 37-45.
 
3:15pm - 4:45pm11 SES 02 A: School Improvement in Challenging Contexts
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Ineta Luka
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

What Factors Lead to School Improvement in Challenging Contexts? Contributions from a Case Study

Amelia Morales-Ocaña1, María José Latorre-Medina2

1University of Cádiz, Spain; 2University of Granada, Spain

Presenting Author: Morales-Ocaña, Amelia

In recent times, the study and analysis of schools located in socioeconomically challenging contexts have gained special attention. Improving schools in disadvantaged areas is today one of the most important issues on the agenda of researchers and policymakers, both nationally and internationally (Clarke & O'Donoghue, 2016; Harris & Chapman, 2002a, 2002b; Lupton, 2005; Maden, 2001; Muijs, 2003; Torres, Moncusí and Osvaldo, 2015; Olmo et al., 2021).

Research developed by Maden (2001) in the United Kingdom provided descriptions of good practices in schools with high vulnerability scenarios. With a similar research objective, Harris & Champan (2002a, 2002b) also highlighted the importance of distributed leadership in these contexts. Later, Lupton (2005) investigated the characteristics of these schools and established several characteristic elements. Among them, she highlighted the existence of continuous unpredictability together with the presence of low expectations. In addition, the author revealed that schools in difficult contexts have more problems taking on the same educational challenges than those in more favourable circumstances. On their side, Torres, Moncusí & Osvaldo (2015) showed that living in an unfavourable environment has a direct influence on the lives of the school population. Clarke & O'Donoghue (2016) also delved into the study of vulnerable schools located in the most challenging neighbourhoods.

Although the evidence from research on this subject shows that schools located in vulnerable contexts are characterized by being unstable and having students with low performance, low self-esteem and expectations (Lupton, 2005), there are also studies that reveal the ability of some schools to overcome and cope with adversity (Marujo et al., 2003; Olmo et al., 2021).

The works located within this second approach focus their attention on schools that have led to a transformation towards school improvement. Some of the key issues that have motivated this research field are as follows: how has the change been possible? what/who has brought it about? what does this school improvement materialise? how do they work in these schools? what kind of leadership is needed? How do they overcome adversities? (Day y Gu, 2015; Muijs, 2003; Hargreaves; 2007; Stoll, 2009).

The present study is part of this line of research, which has been developed within the framework of two larger research projects: I+D+i project "Communities of professional practice and learning improvement: intermediate leadership, networks and interrelationships. Schools in complex contexts" (Reference: PID2020-117020GB-I00), funded by the Agencia Estatal de Investigación of the Spanish Ministry of Science and Innovation (MCIN/AEI/10.13039/501100011033/) and I+D+i Project "Communities of professional practice and learning improvement: intermediate leadership, networks and interrelationships" (Reference: P20_00311), funded by the Andalusian Plan for Research, Development and Innovation of Spain.

The purpose of this study is to know, understand and analyse the factors that have allowed a school, located in Spain, to transform its circumstances in order to achieve educational quality and equity.


Methodology, Methods, Research Instruments or Sources Used
To understand the school in depth, we opted for the case study methodology (Stake, 2010), from the biographical-narrative approach (Bolívar, 2002).

The case was intentionally selected. We were looking for schools which despite having difficult circumstances (e.g. low socioeconomic index or previous low student engagement) were undergoing processes towards educational improvement. Taking these characteristics into account, the Provincial Directorate of the Ministry of Education and Vocational Training in Ceuta (Spain) suggested the school that is the subject of this case study.

It is a school run by the government that welcomes children from 3 to 12 years old. It is located in a low socioeconomic area in Ceuta (Spain) and is achieving increasingly better educational results in relation to other schools in similar contexts.

Data collection

We carried out two group interviews with the principal, the head of studies and three teachers selected as key informants. The interviews revolved around how an educational community is built and developed in a challenging framework, in order to gain more quality and equity.

We used the dialogic-communicative approach (Elboj and Gómez, 2015) trying to create a climate where participants could feel free to express and share their perceptions. In this regard, it should be noted that ethical considerations were taken into account. The participants' consent to collaborate in the study was provided and confidentiality was guaranteed.

Data analysis

A thematic content analysis was carried out (Díaz, 2018), using an inductive procedure, characterized by the construction of emerging categories from the content. This type of analysis allowed for an accurate and in-depth understanding of the interviews.


Conclusions, Expected Outcomes or Findings
The content analysis has made it possible to identify a set of ten central axes or thematic categories: "challenging context", "motivation for change", "origin of change", "change process", "leadership characteristics", "work team", "external support and resources", "teacher training", "achievements", "the school today". These emerging categories are the basis for future themes. So far they have allowed us to understand:

a) Why this school can be considered to have a challenging context
b) What has been the leitmotiv for the school to get involved in a transformation process
c) What situations and factors have promoted the transformation process
d) What leadership style exists in the school as well as its implication in the transformation process
e) What the change process consisted of and what achievements have been made as perceived by the participants
f) How the school is moving forward today to be more fair and equitable.  

The above information allows us to draw a current portrait of the case study which provides an answer to the main question posed for this research proposal.

References
Bolívar, A. (2002). “¿De nobis ipsis silemus?”: Epistemología de la investigación biográfico-narrativa en educación. Revista Electrónica de Investigación Educativa, 4(1). Recuperado de http://redie.uabc.mx/redie/article/view/49
Clarke, S. & O’Donoghue, T. (2016). Educational Leadership and Context: A Rendering of an Inseparable Relationship. British Journal of Educational Studies, 65(2), 167-182. https://doi.org/10.1080/00071005.2016.1199772
Day, C. y Gu, Q. (2015). Educadores resilientes, escuelas resilientes. Construir y sostener la calidad educativa en tiempos difíciles. Narcea.
Díaz, C. (2018). Investigación cualitativa y análisis de contenido temático. Orientación intelectual de revista Universum. Revista General de Información y Documentación, 28(1), 119-142. http://dx.doi.org/10.5209/RGID.60813
Elboj, C. y Gómez, J. (2015). El giro dialógico de las ciencias sociales: hacía la comprensión de una metodología dialógica. Acciones e investigaciones sociales, 12, 77- 94. Recuperado de https://dialnet.unirioja.es/servlet/articulo?codigo=206415
Hargreaves, A. (2007). Sustainable learning communities. In L. Stoll & K. S. Louis (Eds.), Professional learning communities: Divergence, depth and dilemmas (pp. 181-195). McGraw Hill (Open University Press).
Harris, A. & Chapman, C. (2002a). Democratic leadership for school improvement in challenging contexts. International Electronic Journal for Leadership in Learning, 6(9).
Harris, A. & Chapman, C. (2002b). Leadership in schools facing challenging circumstances. Management in Education, 16(1), 10-13.
Lupton, R. (2005). Social justice and school improvement: improving the quality of schooling in the poorest neighbourhoods. British Educational Research Journal, 31(5), 589-604. https://doi.org/10.1080/01411920500240759
Maden, M. (Ed.) (2001). Success Against the Odds: Five Years On. Routledge.
Marujo, H. A., Neto, L. M. y Perlorio, M. F. (2003). Pedagogía del optimismo: guía para lograr ambientes positivos y estimulantes. Narcea.
Muijs, D. (2003). La mejora y la eficacia de las escuelas en zonas desfavorecidas: resumen de resultados de investigación. REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 1(2), 0. Recuperado de:  https://www.redalyc.org/articulo.oa?id=55110203
Olmo, M., Sabino, I. M. y Domingo, J. (2021). Construir resiliencia en las escuelas desde la acción de liderazgo. DEDiCA. Revista de Educação e Humanidades, 18, 69-90. http://doi.org/10.30827/dreh.vi18.17002
Stake, R. E. (2010). Qualitative research: Studying how things work. The Guilford Press.
Stoll, L. (2009). Connecting learning communities: Capacity building for systemic change. En A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.), Second international handbook of educational change (pp. 469-484). Springer Heidelberg.
Torres, F., Moncusí, A. y Osvaldo, E. (2015). Crisis, convivencia multicultural y “efectos de barrio”. El caso de dos barrios de Valencia. Migraciones, 37, 217-238.


11. Educational Improvement and Quality Assurance
Paper

The Necessary and Sufficient Conditions for Turning Around Underperforming Schools

Stefan Arora-Jonsson, Ema Kristina Demir, Axel Norgren, Karl Wennberg

Stockholm School of Economics, Sweden

Presenting Author: Demir, Ema Kristina; Wennberg, Karl

Research on school improvement has accumulated an extensive list of factors facilitating the potential to turn around poorly performing schools (Leithwood et al., 2010; Murphy & Meyers, 2007). However, limited attention has been paid to contextual differences across schools, districts, and educational systems with distinct traditions and prerequisites. At the core of the problem lies the ever-present need to make trade-offs. While, in theory, it would be ideal to improve schools on all conditions identified as desirable for turnaround, this may not be feasible due to organisational capacity and resource constraints. Earlier empirical research on school turnaround relies either on qualitative case studies of one or several schools (Duke & Salmonowicz, 2010; Hallinger & Kantamara, 2001), or traditional regression methods (Boyne & Meier, 2009; Heissel & Ladd, 2018). What is missing is a case-oriented approach that addresses the complex causality related to school turnaround (van Der Steen et al., 2013). This study aims to address these contextual and complex causal patterns by making a systematic comparison of schools that do and do not make a turnaround across different school contexts (although generally within a Swedish intitutional context).


Methodology, Methods, Research Instruments or Sources Used
Using qualitative comparative analysis (QCA), we study 77 schools in Sweden over 10 years to unearth the existence of necessary and sufficient conditions for turnaround. The QCA analysis allows for analysing complex causal combinations as well as equifinal solutions (multiple paths to turnaround) (Ragin, 1987). We study conditions that could explain school turnaround as identified in the turnaround literature (Meyers & Smylie, 2017; Murphy & Meyers, 2007) as well as in Swedish school improvement studies (Jarl et al., 2017).
Conclusions, Expected Outcomes or Findings
We find that there are multiple paths to school turnaround, but we do not find evidence of a generalisable “silver bullet”. No single condition can solely explain school turnaround, but it is possible to turn schools around without ticking every box on the list. We however find two types of common turnaround processes, the first of which is bottom-up turnaround where teachers actively cooperate with each other and there is a principal who strongly focuses on the school’s core operations. This kind of turnaround however, seems to be conditional on the school being visibly underperforming. The second type we call a bypass-turnaround primarily driven by the school the organiser (the school district, i.e., the municipality) working actively with quality assurance of their schools in combination with high expectations at the school level. This type of turnaround takes place despite the absence of a principal focusing on core operations and regardless of whether teachers actively cooperate or not. The context also moderates the possible paths to a successful turnaround. While leadership from either the school principal or school district appears important across all contexts, schools in non-urban contexts could make quick gains by improving the collaborative culture among teachers. Schools in urban contexts face greater challenges in achieving such a collaborative culture and could require even more tailored solutions. We discuss implications for these findings for research on school improvement and education policy.
References
Boyne, G. A., & Meier, K. J. (2009). Environmental change, human resources and organizational turnaround. Journal of Management Studies, 46(5), 835-863.
Duke, D., & Salmonowicz, M. (2010). Key decisions of a first-year ‘turnaround’principal. Educational Management Administration & Leadership, 38(1), 33-58.
Hallinger, P., & Kantamara, P. (2001). Exploring the cultural context of school improvement in Thailand. School Effectiveness and School Improvement, 12(4), 385-408.
Heissel, J. A., & Ladd, H. F. (2018). School turnaround in North Carolina: A regression discontinuity analysis. Economics of Education Review, 62, 302-320.
Jarl, M., Blossing, U., & Andersson, K. (2017). Att organisera för skolframgång: strategier för en likvärdig skola. Natur & Kultur.
Leithwood, K., Harris, A., & Strauss, T. (2010). Leading school turnaround: How successful leaders transform low-performing schools. John Wiley & Sons.
Meyers, C. V., & Smylie, M. A. (2017). Five myths of school turnaround policy and practice. Leadership and Policy in Schools, 16(3), 502-523.
Murphy, J., & Meyers, C. V. (2007). Turning around failing schools: Leadership lessons from the organizational sciences. Corwin Press.
Ragin, C. C. (1987). The comparative method: Moving beyond qualitative and quantitative strategies. JSTOR.
van Der Steen, M., van Twist, M., Fenger, M., & Cointre, S. L. (2013). Complex causality in improving underperforming schools: A complex adaptive systems approach. Policy & Politics, 41(4), 551-567.


11. Educational Improvement and Quality Assurance
Paper

School Supervisors’ Beliefs on the Implementation of All-Day Schools in Germany – A Way of Supporting Heterogeneous Students?

Nicole Zaruba, Raphaela Porsch

Otto-von-Guericke-Universität Magdeburg, Germany

Presenting Author: Zaruba, Nicole; Porsch, Raphaela

For years, educational researchers and policymakers in Germany have strongly supported the idea of transforming schools into all-day schools. The leading argument for the reform is that all-day schools can foster heterogeneous students better than half-day schools. Research from large-scale assessment studies has shown – the latest study provided by Henschel et al. (2022) based on data from the IQB National Assessment Study in Year 4 – that students from families with a migrant background achieve lower proficiency scores than students from non-migrant families. The same results have been shown for students from families with a low socio-economic background (SES; e.g., Sachse et al., 2022). Both findings can be explained by the situation that parents provide different qualitative and quantitative support for their children. In addition, studies have demonstrated that parental homework involvement is significant for learning success (e.g., Dettmers et al., 2019). Thus, there is a higher probability that parents speaking German as their L1 and with a high SES are better suited to provide numerous learning opportunities outside school and help with homework and school preparation than parents with a migrant background and/or with a low SES.

In order to establish more equality, the reform for all-day schools is being pushed forward. The assumption is that all-day schools provide more learning opportunities for all students. Furthermore, instead of the traditional homework format, alternative concepts regularly exist at all-day schools, thus students are supposed to learn primarily in school. In this respect, the reform can contribute to greater educational equity. School development projects such as the transformation to an all-day school require that schools collaborate with multiple stakeholders. In Germany, in addition to principals, school supervisors play an important role in the management of schools. As representatives of the state, they legally have to supervise schools and, secondly, they have an advisory function. They advise principals, for example, on the implementation of reforms and give schools support in carrying out school development projects. Despite an extensive body of research on all-day schools (e.g., their effectiveness, see overview in Fischer & Kuhn, 2022), the role of school supervisors in the all-day school reform has not yet been considered. Since all-day schools are not compulsory in Germany yet (unlike Denmark, e.g., see Holm, 2015), the successful introduction highly depends on the various stakeholders and their beliefs on the innovation. Against this background, this study aims at exploring the beliefs of school supervisors on the implementation of all-day schools as one form of extended education. We focus on the concept of beliefs as they are considered to have a guiding function when it comes to the implementation of “new pedagogical approaches and reforms” (Fives & Buehl, 2012). Following Pajares (1992), we understand beliefs as an “individual’s judgment of the truth or falsity of a proposition” (p. 316). With regard to the reform under study, we ask whether school supervisors share the general positive assumption on the effects of all-day schools. In addition, we are interested in the factors that enable, facilitate or impede implementation of the reform from the perspective of school supervision.


Methodology, Methods, Research Instruments or Sources Used
In the context of schools, beliefs are usually assessed using self-report measures and interview techniques (Schraw & Olafson, 2015). Interviews are especially useful to assess individual and detailed perspectives on a topic allowing for “context sensitivity” and “conversational flexibility” (Brinkmann, 2018, p. 1000). To investigate the school supervisors’ perspectives, we conducted structured interviews with open questions. We asked for narrative accounts on the school supervisors’ job experiences regarding their task of advising schools in the reform processes to become all-day schools. The leading question was: “The all-day school reform is a challenge for all those involved in the process. Which role does the topic play in your job?”. We interviewed 12 school supervisors, half of them being female. We conducted the interviews from May to August 2022 via video conference system. They lasted from 45 to 100 minutes. The interviews were then recorded and transcribed. The methodology employed is based on a content analysis approach (Schreier, 2012), a method to analyse texts by categories. These categories allow a structured analysis of data and comparisons between different texts representing different cases. Using a consensual approach, a coding scheme was developed by the two authors that includes definitions, examples, and coding rules for all categories. Subsequently, we structured the transcripts by coding all transcripts using a combination of deductive and inductive categories that allow for a summary of the beliefs, opinions and topics across all interview cases.
Conclusions, Expected Outcomes or Findings
In the data, two main categories emerged during the inductive coding process: (1) school supervisors’ beliefs on the all-day school reform and their (2) perspectives on impeding, enabling and facilitating factors regarding the implementation of all-day schools. These broader concepts are structured by various subconcepts. Five subcategories could be assigned to the broader category (1) school supervisors’ beliefs on all-day schools. The first subcategory beliefs on the “right” concept of all-day schools encompasses opinions and explanations on what school supervisors believe is the most effective all-day school concept. All interviewees shared the opinion that all-day schools should have compulsory morning and afternoon lessons, i.e., a concept that does not only offer child-care and free-time activities after lessons in the morning but includes, for instance, lunch times together, school lessons in the afternoon and times for other pedagogically planned activities throughout the entire school day. Another subcategory describes the belief that all-day schools are a tool to support heterogenous students individually and to foster the competencies they need to succeed in school. In short, all-day schools are considered to be a tool to achieve more educational equity. Three more subcategories will be presented. Regarding the second broader category (2) perspectives on impeding, enabling and facilitating factors regarding the implementation of all-day schools six subcategories emerged in the coding process. They encompass views on the role of human and monetary resources, organizational challenges and the thoughts on professional cooperation and external support. All subcategories will be illustrated by quotations we considered representative for the school supervisors’ accounts. We discuss implications of the school supervisors’ beliefs in the context of their role in the all-day school reform with a special focus on the potential all-day schools can have in supporting heterogeneous students.
References
Brinkmann, S. (2018). The interview. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (pp. 997–1038). SAGE.
Dettmers, S., Yotyodying, S., & Jonkmann, K. (2019). Antecedents and Outcomes of Parental Homework Involvement: How Do Family-School Partnerships Affect Parental Homework Involvement and Student Outcomes? Frontiers in psychology, 10, 1048. https://doi.org/10.3389/fpsyg.2019.01048
Fischer, N., & Kuhn, H. P. (2022). Ganztagsschulforschung. In H. Reinders, D. Bergs-Winkels, A. Prochnow & I. Post (Eds.), Empirische Bildungsforschung (pp. 595–617). Springer Fachmedien.
Fives, H., & Buehl, M. M. (2012). Spring cleaning for the “messy” construct of teachers’ be- liefs: What are they? Which have been examined? What can they tell us? In K. R. Harris, S. Graham, T. Urdan, S. Graham, J. M. Royer & M. Zeidner (Eds.), APA educational psychology handbook, Vol 2: Individual differences and cultural and contextual factors (pp. 471–499). American Psychological Association.
Henschel, S., Heppt, B., Rjosk, C., & Weirich, S. (2022). Zuwanderungsbezogene Disparitäten. In P. Stanat , S. Schipolowski, R. Schneider, K. A. Sachse, S. Weirich & S. Henschel (Eds.), IQB-Bildungstrend 2021 (pp. 181–219). Waxmann.
Holm, L. (2015). Researching extended schooling ethnographically – with Danish all-day schools as examples. International journal for research on extended education, 3(1), 39–51.
Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of educational research, 62(3), 307–332.
Sachse, K. A., Jindra, C., Schumann, K., & Schipolowski, S. (2022). Soziale Disparitäten. In P. Stanat, S. Schipolowski, R. Schneider, K. A. Sachse, S. Weirich & S. Henschel (Eds.), IQB-Bildungstrend 2021 (pp. 151–180). Waxmann.
Schraw, G., & Olafson, L. (2015). Assessing teacher’s beliefs. Challenges and solutions. In H. Fives & M. Gregoire Gill (Eds.), International handbook of research on teachers’ beliefs (pp. 87–105). Routledge.
Schreier, M. (2012). Qualitative content analysis in practice. Sage.
 
5:15pm - 6:45pm11 SES 03 A: Systematic Quality Management in Education
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Rita Birzina
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Challenges in the Implementation of EMIS as a Litmus Test of the Quality of Education

Katarina Mićić, Ana Pešikan

University of Belgrade, Faculty of Philosophy, Department for Psychology

Presenting Author: Mićić, Katarina

Education Management Information System (EMIS) is a necessary tool for educational policy management, education reform, and monitoring and evaluation in education that enables the creation of valuable inputs for improving the quality of education (Abdul-Hamid, 2014; UNESCO, 2018; Bőjte, 2019). Introduction of an EMIS into a complex world of an educational system is a difficult task. One challenging aspect is defining an EMIS that can fit all scenarios and include all data needs and flows (Tolley & Shulruf, 2009), especially in a country such as Serbia, with over million students, three thousands of educational institutions with over 150 thousand of employees, and a diversity of educational programs held in several languages. Another concern is the social context of EMIS introduction, and how such novelty changes working routines and social relations in educational institutions (Kling, 2000).

Implementation of an information systems (IS) often face obstacles due to a gap between its design and the reality. Heeks (2002; 2006) argue that the size of this gap determines the probability of an IS successful implementation and its acceptance among users. He proposed a model for estimating the extent of this gap on six dimensions. Information dimension refers to a discrepancy between the envisioned set of data to be collected through an IS, and the data that users actually work with. Technology dimension assesses the difference between the required infrastructure for an IS to operate and the infrastructure available to users, in this case – to schools. Processes dimension deals with the difference in the processes envisioned to be automatized and the existing ones. Objectives and values dimension deals with the compatibility of the goals and values that an IS brings and enforces with the goals and values of its users. Staffing and skills accesses the gap between the required staff for operating the IS and their competence level and the available number of people and their actual competence level. Management system and structures is a dimension that is concerned with the discrepancy between the required managerial structure needed to guide the use of an IS and the one available. On the basis on this model, we analyzed the Serbian EMIS and looked into the causes of the gaps, which circled back to more broad and general structures in the decision-making process that pose barriers to other aspects of reform success as well.

After four unsuccessful attempts to build EMIS in Serbia in the last two decades, the Government of the Republic of Serbia started implementing EMIS, which has been developing since 2019. New EMIS includes data on all levels of education and collects data on schools, teachers, classes, and students (Rulebook on EMIS, 2019). However, during the first year of implementation, all groups of EMIS users – from those who collect and enter to those who use data – encountered difficulties. We utilized Heeks’ model as a lens to analyze the development and effectiveness of the implementation of EMIS. However, we discovered that challenges in the implementation of EMIS reflect subtle and wider obstacles to ensuring the quality of education. This study uses the case of EMIS implementation unravel these obstacles and to advocate for deconcentrated (Ivić, 2017) and participatory decision-making processes as an opposite to excessive top-down politics with poor communication of the change introduction in educational policy.


Methodology, Methods, Research Instruments or Sources Used
The study is a part of a project concerning support of reform implementation with a focus on the improvement of evidence-based policy making in Serbia, funded by European Union. The study was conducted during summer 2022. Methodological approach consisted of desk research, focus group discussions and interviews. Desk research was focused on analysis of relevant legal acts and bylaws. Five focus groups were conducted with a total of 104 representatives of educational institutions who enter data in EMIS and customer support administrators. Ten semi-structured interviews were conducted with users of educational data – three representatives of the government, five scientific researchers, and two representatives from other relevant institutions that use educational data. Findings from the desk research were presented to participants in both the focus groups discussions and the interviews and were used to guide the discussion. However, participants were eager to bring up other topics as well, so we would update the focus group and interview guide after each session. Transcripts of focus group discussions and interviews were recorded and transcribed. Transcripts were analyzed based on the Heeks’ dimensions in MAXQDA software.
Conclusions, Expected Outcomes or Findings
Our analyses revealed a moderate gap on the information dimension as the introduction of EMIS envisioned collection of data that haven’t been collected before and whose purpose has not been communicated to the school staff nor the parents. A moderate gap exists on the technology dimension as some rural areas haven’t been recognizes as not having adequate technical resources. We noted resistance regarding the processes the EMIS is envisioned to fulfill, as some actors don’t appreciate automatization of certain decisions, mostly regarding finances. A moderate gap was also noted in terms of objectives and values, as a large number of staff and parents fear being monitored, don’t appreciate the expansion of technology or want to avoid transparency. On the staff and skills dimension - analysis revealed that many persons who are delegated to work with EMIS are not competent for the role or satisfied with it, since their workload has increased while their compensation remained the same. Finally, regarding the management system and structures dimension – our analysis revealed the need for decentralization of some managerial processes, the most important one being data quality assurance.
  Looking for the ways of removing the barriers in EMIS implementation and closing the uncovered gaps, we underlined several weak spots in the ways EMIS was design and introduced. Some of the gaps could have been narrower in policy making processes were more participatory and included perspective from the EMIS direct users. Closely related to this is a need for deconcentrating the decision-making processes in order to obtain better insight (e.g., on the available infrastructure or staff skills) and feedback (e.g., on data quality and technical issues). Finally, gaps in the domains of information and objective and values could be shrunk by better transparency and communication of context and intents of implementing different novelties and measures.

References
Abdul‐Hamid, H. (2014).  What Matters Most for Education Management Information Systems:   A Framework Paper. SABER — SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS EDUCATION MANAGEMENT INFORMATION SYSTEMS, Working Paper Series Number 7, World Bank Group, jun 2014 http://wbgfiles.worldbank.org/documents/hdn/ed/saber/supporting_doc/Background/EMIS/Framework_SABER-EMIS.pdf


Bőjte, D. (2019). Mastering the Skills Gap at Systemic Level–The Education Management Information System: A Key Element for an Effective Digital Transformation in Education. Revista de Management Comparat International, 20(2), 131-143.

Heeks, R. (2002). Information Systems and Developing Countries: Failure, Success, and Local Improvisations. The Information Society, 18(2), 101–112. https://doi.org/10.1080/01972240290075039

Heeks, R. (2006). Health information systems: Failure, success and improvisation. International Journal of Medical Informatics, 75(2), 125–137. https://doi.org/10.1016/j.ijmedinf.2005.07.024

Ivic, I., & Pesikan, A. (2012). Education system reforms in an unstable political situation: the case of Serbia in the first decade of the 21st century. Ceps Journal, 2(2), 31-53.

Kling, R. (2000). Learning about information technologies and social change: The contribution of social informatics. The Information Society, 16(3), 217–232.

Pešikan, A., & Ivić, I. (2016). The sources of inequity in the education system of Serbia and how to combat them. Center for Educational Policy Studies Journal, 6(2), 101-124.

Pešikan, A., & Ivić, I. (2021). The impact of specific social factors on changes in education in Serbia. Center for Educational Policy Studies Journal, 11(2), 59-76.

Pešikan, A. (2022). Analiza stanja informacionog sistema u prosveti i preporuke za njegovo unapređenje. [Situation analysis of Serbian EMIS and recommendation for its improvement.]

Pravilnik o Jedinstvenom informacionom sistemu prosvete [Rulebook on Educational Menagement Information System] Službeni glasnik, 81/ 2019 http://www.pravno-informacioni-sistem.rs/SlGlasnikPortal/eli/rep/sgrs/ministarstva/pravilnik/2019/81/5/reg

Tolley, H., & Shulruf, B. (2009). From data to knowledge: The interaction between data management systems in educational institutions and the delivery of quality education. Computers & Education, 53(4), 1199–1206.

UNESCO. (2019).  https://learningportal.iiep.unesco.org/en/glossary/educational-management-information-system-emis


11. Educational Improvement and Quality Assurance
Paper

From National Policy to Local Practices – Systematic Quality Work in Education from the Perspective of Local Authorities

Tuija Muhonen, Anders Edvik, Magnus Erlandsson, Margareta Serder

Malmö University, Sweden

Presenting Author: Muhonen, Tuija; Serder, Margareta

Over the last 20 years systematic quality work has become the main tool for developing Swedish schools (Håkansson & Adolfson, 2022). According to the Education Act (2010:800), quality work – at the local educational authority level as well as in the schools themselves – should be conducted in a systematic and continuous way, with respect to planning, follow-up, analyses, and actions taken to develop education. By continuous assessments and evaluations, the goal of the systematic quality work is to identify and address issues that need improvement for students to achieve the educational goals (Swedish National Agency for Education, 2012a). A review of the literature reveals the problematic aspects of evaluation practices and quality management, such as the risk to focus on what is measurable rather than what is desirable as well as the diverse definitions of quality (Lundström, 2015). However, what the local quality systems consist of, how they have been designed, and what practices and perceptions of quality they entail is less understood.

Previous research has primarily focused on individual schools’ quality work (Håkansson, 2013; Jarl, et al., 2017) , while less attention has been paid to the way the local educational authorities conduct systematic quality work. Thus, the aim of this study is to fill this knowledge gap by investigating how the ideas of systematic quality work in the Swedish Education Act's requirements are interpreted, translated, and materialized at the local education authority level.

The following research questions will guide our study:

1. How do local educational authorities interpret and translate the systematic quality work regulations and requirements in the Education Act?

2. How do these interpretations och translations materialize in the local quality work practices?

Theoretically we approach the phenomena of systematic quality management within the Swedish school sector from an organizing (Czarniawska, 2014) and practice-oriented perspective (Gherardi, 2019; Nicolini, 2009; 2012). These theoretical perspectives provide us a framework to analyse how the institutionalized ideas (as mental images that are well spread within the society) of systematic quality management - through authorities, policies, regulations, and quality models - are translated and materialized (Czarniawska & Joerges, 1996) into the local quality work organization and practices. The latter refers to the practices of doing and saying something related to the ideas of systematic quality management in different social contexts and time (Gherardi, 2019; Nicolini, 2009; 2012). Although the national guidelines involve the entire school system, these are interpreted, translated, and materialized by actors operating in a local context, which means that quality is understood in different ways and that the systematic quality work is conducted in different ways. An organizational perspective also includes aspects related to the tensions that arise when different interests and logics collide (for example between political, administrative, and professional interests and logics; see Czarniawska, 2014).


Methodology, Methods, Research Instruments or Sources Used
In this project five Swedish municipalities have participated in a study of what systematic quality work means at a local education authority level and what practices materialize from the national regulations. At the heart of the study is the recognition that quality systems are locally designed to meet the national requirements, thereby allowing diverse interpretations and translations to occur.
The local education authorities can be understood as mediators, partly between state and municipal control, partly between needs and agendas at different levels in the chain of command. This understanding also characterizes the design of the study. The empirical data has been collected through three complementary methods: document studies, observations, and interviews. The document studies consist of analysing different central documents, e.g., quality reports, provided by the local education authorities covering the past two years. We have also observed meetings related to the systematic quality work (so called “quality-dialogues”). Besides the local education authorities, the key actors in these meetings were the principals, assistant principals, and teacher representatives of the school being followed up.  
The interviews were conducted with key persons in five different local Swedish education authorities individually by the authors. The duration of the interviews was approximately one hour, and they were conducted either face-to-face, via Zoom, or telephone. The interviews were based on an interview guide including questions about the participants’ role, their experiences, and activities in relation to the systematic quality work, the expected and actual effects, as well as challenges and potential for improvement of systematic quality work. The interviews were recorded with informed consent and were later transcribed verbatim.
All the research material described above is now gathered and will be analysed during the Spring 2023. As a tool for data analyses, we will apply Bacchi´s (2012) method “What is the problem represented to be?”.

Conclusions, Expected Outcomes or Findings
Equal education for all is includes three fundamental aspects: equal access to education, equal quality of education and the compensatory nature of education (Swedish National Agency for Education, 2012b). All students should receive an equivalent education, regardless of the area they live in, the socio-economic conditions they come from, or their functional variations. But study after study shows that Swedish students' schooling is not equal, and that who you are and where you live play a decisive role in the quality of the education you receive. Many of the last decade's school policy reforms and targeted initiatives have had as their overarching goal to address this lack of equality, so far with few concrete results. In order to break this trend, there has been an increasing focus on the local educational authorities’ responsibility for the individual school's shortcomings, quality, and development. Furthermore, lack of equality is a problem within rather than between different local educational authorities. Although there is paucity of research, the limited results show that schools are often isolated with their problems and that there is a lack of supportive structures and a functional systematic quality work (Jarl, et al., 2017; Swedish Schools Inspectorate, 2021).
The paper will present results from the ongoing study, results that we believe will have relevance both in the Swedish, Nordic and in a wider European context. Through our investigation we will contribute knowledge regarding how the National Educational Act's requirements for systematic quality work are interpreted, translated, and materialized at the local level, and how this in turn shapes, promotes or hinders the quality work of individual schools.

References
Bacchi, C. (2012). Introducing the ‘What’s the Problem Represented to be?’ approach. Engaging with Carol Bacchi: Strategic interventions and exchanges, 21-24.
Czarniawska, B. (2014). A theory of organizing. Edward Elgar Publishing.
Czarniawska, B., & Joerges, B. (1996). Travels of ideas. Translating organizational change, 56, 13-47.
Education Act, [Skollag] (2010), 2010:800. https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/skollag-2010800_sfs-2010-800
Gherardi, S. (2019). How to conduct a practice-based study: Problems and methods. Edward Elgar Publishing.
Jarl, M., Blossing, U. & Andersson, K. (2017). Organizing for School Success: Strategies for an Equal School. [Att organisera för skolframgång: Strategier för en likvärdig skola.] Stockholm: Natur & kultur.
Håkansson, J. (2013). Systematic quality work in preschools, schools and leisure-time centres: strategies and methods. [Systematiskt kvalitetsarbete i förskola, skola och fritidshem: strategier och metoder.] Lund: Studentlitteratur.
Håkansson, J., & Adolfsson, C. H. (2022). Local education authority’s quality management within a coupled school system: Strategies, actions, and tensions. Journal of Educational Change, 23(3), 291-314.
Lundström, U. (2015). Systematic quality work in Swedish schools: Intentions and dilemmas. Scandinavian Journal of Public Administration, 19(1), 23-44.
Nicolini, D. (2009). Zooming in and out: Studying practices by switching theoretical lenses and trailing connections. Organization studies, 30(12), 1391-1418.
Nicolini, D. (2012). Practice theory, work, and organization: An introduction. Oxford: Oxford University Press.
Swedish Schools Inspectorate (2021). Complaints against education. Organization and procedures of the Local education authorities [Klagomål mot utbildningen Huvudmäns organisation och tillvägagångssätt] file:///Users/imtumu/Downloads/klagomal-mot-utbildningen--kvalitetsgranskningsrapport-skolinspektionen-2020_tg.pdf
Swedish National Agency for Education [Skolverket] (2012)a. Systematic quality work – for the school system. The Swedish National Agency for Education's general advice with comments. [Systematiskt kvalitetsarbete – för skolväsendet. Skolverkets allmänna råd med kommentarer.]
Swedish National Agency for Education [Skolverket] (2012)b. Educational equity in the Swedish school system? A quantitative analysis of equity over time. Summary of Report 374.
 
Date: Wednesday, 23/Aug/2023
9:00am - 10:30am11 SES 04 A: Career in Teaching: Teachers' Motivation and Professionalism
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Dita Nimante
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Teacher goals and self-efficacy expectations among teachers in Austrian Summer Schools.

Maximilian Sailer, Christina Herrmann, Andrea Vorderobermeier, Christina Hansen

Universität Passau, Germany

Presenting Author: Herrmann, Christina

The Summer School is a two-week program supported by the Federal Ministry of Education, Science and Research (BMBWF) of the Republic of Austria that seeks to address the immediate educational disadvantages caused by the Covid-19-pandemic (BMBWF, 2022). It aims to promote students’ self-confidence and social skills by establishing opportunities of positive learning experiences. During the Summer School, students predominantly receive individual support and feedback for the subjects “German language”, “mathematics”. Participants of the Summer School are usually students with lacking German language skills and/or who performed poorly in the past school year. The program is conducted by teachers and student teachers. The latter group benefits from the Summer School by experiencing two weeks of “teaching practice” with associated classroom management to prepare them for the teaching profession (BMBWF, 2022).

According to the utilization-of-learning-opportunities model (e.g., Weinert & Helmke, 1997), there are many different variables and their interplay are accountable for learning success. Basically, the model aims to illustrate the multiple, multicausal, or even interdependent relations of variables in regard to learning success. Since learning success is evidently also influenced by affective dispositions of teachers and students, this research area was focused for this study. Student-related affective-motivational characteristics have been proven to influence school learning. Self-efficacy expectations might directly influence student behavior, but also influence goals, affective dispositions or perceptions in the social environment (Bandura, 2006; Bandura, 1997). Consequently, the question arises why some students remain confident and maintain their child-like optimism in coping with performance goals in school, while other students develop an anxious or even helpless attitude (Dweck & Leggett, 1988). At the student level, research has shown that teachers' self-efficacy expectations are positively related to student achievement and motivation. At the teacher level, self-efficacy expectations of teachers have been found to positively influence their own commitment to their work, the goals they set for themselves, and the level of job satisfaction they experience (Pfitzner-Eden, Thiel & Horsley, 2014). International studies have shown that the degree of self-efficacy among teachers varies across countries. Vieluf et al. (2013) were able to illustrate that teachers from Austria are in the upper range (higher values of self-efficacy) in the OECD compared to teachers from Iceland, Denmark, Ireland and Australia. In different studies with Austrian student teachers and teachers, Bach (2022) was able to show the connection between self-efficacy and professional experience. Interestingly, the self-efficacy expectation of student teachers after practical teaching experience is significantly higher than among experienced teachers. It is even higher for so-called teaching assistants, who are used to provide support in regular and extracurricular settings in Austria and make an important contribution to inclusive teaching.

Based on the available study results, the following research questions have emerged that require clarification in regard to the Summer School program.
- FF1: Which general and specific results on self-efficacy and teacher goals characterize Austrian Summer School teachers?

o H1: Professional teachers have higher self-efficacy expectancy scores than student teachers.

o H2: Teachers with ten or more years of professional experience have higher self-efficacy expectations than teachers who have less than ten years of professional experience.

o H3: Primary (elementary) school teachers have significantly higher self-efficacy expectations than teachers in secondary education.

- FF2: Are there discernable clusters of teacher characteristics in the data set?


Methodology, Methods, Research Instruments or Sources Used
To implement the objectives, various questionnaire instruments were developed, consisting of reliable, standardized scales and subscales. The study includes three different instruments, two of which were addressed to students. Another was addressed to teachers of the Summer School and is the focus of this presentation.
In total, the online teacher questionnaire consists of 63 items organized in nine scales and subscales. Seven scales are taken from the German “Fragebogen zur Erfassung von Lehrerzielen (FELZ)” (Questionnaire to capture teacher goals) based on the work of Rüprich & Urhahne (2015). The concept is based on the idea that teachers’ teaching goals influence students’ learning success. The questionnaire aims to represent teachers’ content-related goals and relate them to their experience and behavior (Rüprich, 2018). 33 items of the questionnaire are taken from the German inventory “Kompetenzentwicklung bei schulpraktischen Lerngelegenheiten (KliP)” (Competence development in practical school learning opportunities) (Gröschner & Schmitt 2012), which addresses the assessment of teachers’ pedagogical competencies. The self-assessment of competence can be seen as part of a person's own individual competence profile experience of self-efficacy (Gröschner & Müller, 2013). In addition, all 12 items of the “Scale for Teacher Self-Efficacy” (STSE) inventory (Pfitzner-Eden, 2016) were integrated into the instrument. The STSE measures teachers' self-efficacy expectations using the subscales Instructional Strategies; Classroom Management and Student Motivation. Cronbach's alpha (α) provides satisfactory values for the subscales FELZ (α = .92), KLiP (α = .95), Scale E (α = .78), and STSE (α = .90).
Teachers and student teachers from the states of Vienna, Vorarlberg and Upper Austria who taught in the Summer School were invited to participate in the survey immediately following the program’s conclusion. The teacher questionnaire could be accessed only electronically via the SoScisurvey platform. Access was granted through a QR code. A total of 362 teachers and 153 student teachers participated in the survey. The vast majority of student teachers (88%) stated that they participated in the Summer School voluntarily, that is, the measure did not constitute a compulsory internship. 70% of the professional teachers (n=254) have been teaching for less than 10 years. Nearly 24% of the teachers (n=89) have been in the teaching profession between 10 and 29 years. Almost 50% of the teachers work on primary school level (n=190), another 33% at a middle school (n=127), and 10% at general secondary schools (n=40).

Conclusions, Expected Outcomes or Findings
Overall, teachers of the Summer School score high on self-efficacy compared to findings of other studies that were previously outlined. The standard deviations can be classified as small to medium. The mean for teacher self-efficacy (STSE total scale) is slightly higher compared to the study by Bach (2022). However, there are differences in relation to the group of teachers, professional experience and type of school. Self-efficacy is significantly higher among teachers (M=4.16, SD=0.53, n=362) compared to student teachers (M=4.06, SD=0.54, n=153), t(513)=1.969, p=.025. Teachers with ten and more years of professional experience (M=4.11, SD=0.53, n=268) have significantly higher self-efficacy scores than teachers under ten years of professional experience (M=4. 28, SD=.49, n=96), t(362)=-2.630, p=.004. Teachers at elementary schools (M=4.21, SD=0.51, n=190) show significantly higher self-efficacy scores than teachers at secondary level (M=4.11, SD=0.54, n=173), t(361)=1.821, p=.035. In addition, a K-means cluster analysis was conducted which identified two groups. There are approximately the same number of subjects in both clusters (nModelEducators =235, nAmbitious=282). Based on the identifiable differences, the following clusters could be detected: the “Model Teachers” and the “Ambitious”. On average, “Model teachers” consistently scored higher on teachers´ goals as well the self-efficacy expectations compared to “Ambitious teachers”. The sample illustrates that Austrian Summer School teachers are highly motivated, almost idealistic to fulfill this challenging task.
References
Bach, A. (2022). Selbstwirksamkeit im Lehrberuf. Entstehung und Veränderung sowie Effekte auf Gesundheit und Unterricht. Münster, New York: Waxmann, https://doi.org/10.25656/01:24604
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.
Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. M. Pajares, & T. Urdan (Ed.), Self-efficacy beliefs of adolescents (pp.307–337). Greenwich: Information Age Publishing.
Bundesministerium Bildung, Wissenschaft und Forschung (BMBWF) (2022). Sommerschule.
Gröschner, A., & Müller, K. (2013): Bewertung praktischer Lerngelegenheiten durch Lehramtsstudierende – Betrachtungen zur Abbildbarkeit unterschiedlich dauernder Praxisphasen in Kompetenzselbsteinschätzungen. In: Gehrmann, A.; Kranz, B.; Pelzmann, S.; Reinartz, A. (Ed.), Formation und Transformation der Lehrerbildung. Entwicklungstrends und Forschungsbefunde (pp.119-133).Bad Heilbrunn: Klinkhardt.
Gröschner, A., & Schmitt, C. (2012). Kompetenzentwicklung im Praktikum? Entwicklung eines Instruments zur Erfassung von Kompetenzeinschätzungen und Ergebnisse einer Befragung von Lehramtsstudierenden im betreuten Blockpraktikum. Lehrerbildung auf dem Prüfstand, 5(2), 112–128. https://doi.org/10.25656/01:14733
Pfitzner-Eden, F., Thiel, F. & Horsley, J. (2014). An adapted measure of teacher self-efficacy for preservice teachers: Exploring its validity across two countries. Zeitschrift für Pädagogische Psychologie, 28(3), 83–92. https://doi.org/10.1024/1010-0652/a000125
Pfitzner-Eden, F. (2016). STSE - Scale for Teacher Self-Efficacy - deutsche adaptierte Fassung. https://doi.org/10.23668/PSYCHARCHIVES.451
Rüprich, C. (2018). Ziele und Handlungen von Lehrkräften – Eine Untersuchung zu Lehrerzielen, ihrer Struktur und den Zusammenhängen zu subjektivem Wohlbefinden, Unterrichtsqualität und der wahrgenommenen Wirksamkeit von Unterrichtsmethoden. Inauguraldissertation. Psychologie. Universität Passau
Rüprich, C., & Urhahne, D. (2015). Development of a questionnaire for the assessment of teacher goals from a content perspective. International Journal of Educational Research, 72, 173–184. https://doi.org/10.1016/j.ijer.2015.06.005
Vieluf, S., Kunter, M., Vijver F. J.R. van de (2013). Teacher self-efficay in cross-national perspective. Teaching and Teacher Education, 35, 92-103. https://doi.org/10.1016/j.tate.2013.05.006
Weinert, F. E. & Helmke, A. (1997). Entwicklung im Grundschulalter. Weinheim: Beltz Psychologie-Verl.-Union.


11. Educational Improvement and Quality Assurance
Paper

Service-Learning Contributions to Initial Teacher Training: Critical Aspects In Civic Engagement Promotion

Beatriz Gálvez, Prado Martín-Ondarza, Paloma Redondo, Lydia Serrano, Juan Luis Fuentes

Universidad Complutense de Madrid, Spain

Presenting Author: Fuentes, Juan Luis

The concern for youth civic commitment is shown in several recent years phenomena. On the one hand, educational systems of different countries have introduced civic or citizenship education in their official curricula, cross-curricular content and compulsory subjects, with diverse results (Davies and Chong, 2016; Johnson and Morris, 2012; MacLaughlin , 2000; Peterson, 2011). On the other hand, this interest concerns not only basic education, but also higher education, observing a change in the way the university links with the society in which it is located (Farnell, 2020). In this sense, the European Economic and Social Committee (2016) significantly has introduced the term “civic universities”, which goes beyond connections motivated by employment and economic growth, and underlines cultural and civic interconnection. Initiatives such as the TEFCE Project, whose objective is "To develop innovative and feasible policy tools at the university and European level for supporting, monitoring and assessing the community engagement of higher education institutions", the Living Knowledge Network (2023), the Foundation For Women in Science, or Horizon 2020 line 16: Science with and for Society, are some examples of this turn that higher education has been making.

Likewise, this scenario seems like responding to the alarm expressed by different authors, related with the questioning of the democratic system's quality. Jeffrey Stout (2004) warned of a progressive deterioration in the citizens' ethical-political training, which could pose a risk to democracy maintenance. More recently, Shoshana Zuboff (2018) finds in the current social technology a significant decrease in the citizens capacity for action, paradoxically hidden in the rhetoric of empowerment. For his part, Keith E. Whittington (2019) shows concern that the university is losing its status as an open space to dialogue, to exchange ideas and confrontate positions, typical of an academic community in a democratic setting.

In this situation, different initiatives arise from the institutions and university classrooms with pedagogical proposals focused on promoting the students' civic engagement. Service-Learning (S-L) is perhaps one of the most significant methodologies, that allows practical learning in social settings, while promoting the links with the nearby community, through a service that solves a real problem (Annette, 2005; Boston, 1997). Therefore, it represents an improvement both for the individuals themselves and for their social context (Arthur, Harrison and Taylor, 2015).

Such growing activity related to S-L, highlights the need to assess these programs and, specifically, the consequences, benefits and lessons that can be learnt, in order to improve the development of S-L proposals and the strengthening of this practice in education (Fuentes, Sirera and Redondo, 2022). But the adaptable and flexible nature of S-L, as well as its inherent ethical dimension (Alexander, 2016; Wright, Warren and Snow, 2020), gives rise to very heterogeneous proposals and interventions, which complicates the assessment and makes the tests standardization really difficult. Consequently, mixed assessment are needed, taking into account qualitative research that helps us to understand and explain the great diversity of capacities promoted by S-L.

The aim of this paper is to analyze the contributions of S-L methodology to the educator initial training, focusing on its capacity to promote ethical-civic engagement. Other critical aspects will also be analyzed by comparing four groups of different Degrees in Education that carried out projects with different social entities at Complutense University of Madrid, linked to curricular subjects.


Methodology, Methods, Research Instruments or Sources Used
We follow a qualitative paradigm corresponding to a service-learning case study that has been carried out at the Faculty of Education, Complutense University of Madrid (Spain) during the academic year 2020-2021. A mixed method research has been used for data collection and analysis. First, in order to evaluate the S-L methodology, a validated quantitative likert-type questionnaire (León, Sanchez & Belando, 2020) was applied. Second, an exploratory and descriptive content analysis of students’ reflexive essays and focus groups was carried out.

The sample group for the questionnaire consisted of 128 students from 4 different undergraduate degrees (Pedagogy, Primary Education, Pedagogy-Primary Education double degree, and Social Education Degree). The content analysis had a sample group that consisted of 139 students, with their corresponding reflective essays, which correspond to the total number of participants of the four focus groups.

The data collected through the questionnaire was analyzed using IBM SPSS Statistics 27, carrying out descriptive, inferential and correlation analyses. In the analysis of data, three dimensions were taken into account: training, learning, and service. An initial descriptive analysis was performed, afterwards correlational analysis were carried out in order to observe the relationship between the different dimensions and items. Finally, ANOVA and t-test analysis were made to see possible differences between variables such as gender or the degree to which they belong.

Regarding the qualitative content analysis of the reflective essays and focus groups, a descriptive and exploratory design was followed. A coding process was carried out using Atlas.ti 22 software, in which a mixed category system was used: deductive and inductive.

After the analysis a triangulation of the data obtained through the different techniques of data collection was carried out. The categories that we have obtained from it are: civic commitment, learning of curricular and experiential content through the service, teaching identity construction, teacher training impact, project evaluation, general satisfaction, transversal competencies acquisition, competential learning through theory-practice relationship, and consciousness about the relevance of ethical and civic values and emotions in education.

Finally, the information from the qualitative and quantitative results was integrated which allowed us to deepen its interpretation.

Conclusions, Expected Outcomes or Findings
As a general conclusion of the study, we found that S-L methodology in the initial educators training improves their formation regarding different dimensions. Considering the quantitative analyses, the average scores stand out at a very high level in the three dimensions studied: impact on training, learning and service. In addition, there is a statistical correlation between them. Analyzing all the sub-dimensions, they also have high scores, highlighting the implied training in the S-L methodology. The development of communication skills is associated particularly strongly with a better perception of professional and personal development. Also, it is related to project planning. Moreover, project planning is essential for a better perception of the training implied in the methodology, professional development and participation in the service.

Regarding the qualitative analysis, students show a better development of civic engagement, both as students and as future educators, thanks to the S-L methodology. They highlight the awareness of social commitment and responsibility as university students and particularly as education students. Another conclusion is that learning is different with service-learning than with a traditional methodology. It requires reflecting after the service to recognize the meaning of their learning deeping in the subject. On the other hand, carrying out the service in an activity close to their contexts increased their impact on them.

Both quantitative and qualitative analyses, indicate differences between the students of diverse Degrees. However, the difference between the number of participants of each degree and the diversity of entities make it difficult to compare in this regard. Furthermore, this diversity of activities offers a clear vision of advantages and disadvantages of specific characteristics of different projects.

References
Alexander, H. A.  (2016). Assessing virtue: measurement in moral education at home and abroad. Ethics and Education, 11(3), 310-325,

Annette, J. (2005). Character, Civic Renewal and Service Learning for Democratic Citizenship in Higher Education. British Journal of Educational Studies, 53(3), 326-340.

Arthur, J., Harrison, T. y Taylor, E. (2015). Building Character Through Youth Social Action. Research Report. Jubilee Centre for Character and Virtues - University of Birmingham.

Boston, B. (1997). Their best selves: Building character education and service learning together in the lives of young people. Council of Chief State School Officers.

Davies, I. and Chong, E. (2016). Current challenges for citizenship education in England. Asian Education and Development Studies, 5(1), 20-36.

European Economic and Social Committee (2016). Engaged universities shaping Europe. https://www.eesc.europa.eu/en/our-work/opinions-information-reports/opinions/engaged-universities-shaping-europe

Farnell, T. (2020). Community engagement in higher education. European Union.

Fuentes, J. L., Sirera, A. Redondo, P.  (2022). Towards a civically engaged teacher identity: qualitative analysis of a Service Learning project in the training of educators. En A. Gromkowska et al. (Ed.). In Search of Academic Excellence: Social Sciences and Humanities in Focus (pp. 83-107). Peter Lang.

Johnson, L. and Morris, P. (2012). Critical citizenship education in England and France: a comparative analysis. Comparative Education, 48(3), 283-301.

Living Knowledge (2023). Preamble https://livingknowledge.org/about-living-knowledge-network/

MacLaughlin, T. H. (2000). Citizenship Education in England: The Crick Report and Beyond. Journal of Philosophy of Education, 34:4, 541-570.

León, V., Sánchez, S. y Belando, M. (2020). Diseño y validación de un cuestionario para evaluar la metodología Aprendizaje-Servicio. Estudios sobre Educación, 39, 247-266.

Peterson, A. (2011). The common good and citizenship education in England: a moral enterprise? Journal of Moral Education, 40(1), 19-35.

Stout, J. (2004). Democracy and Tradition. Princeton University Press.

TEFCE (2018). Towards a European framework for Community Engagement in Higher Education. https://www.tefce.eu/project

Wright, J. C., Warren, M. T. and Snow, N. (2020). Understanding virtue: theory and measure. Oxford University Press.

Zuboff, S. (2018). The Age of Surveillance Capitalism. The Fight for a Human Future at the New Frontier of Power. PublicAffairs.
 
1:30pm - 3:00pm11 SES 06 A: Professional Identity and Education Quality
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Dita Nimante
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Effects of Meaningful Work, Percieved Organizational Support and Job Satisfaction on Organizational Commitment of Teachers

Filiz Akar

Yozgat Bozok University, Turkiye

Presenting Author: Akar, Filiz

Today, expectations from education and teachers have increased. International studies on learner and teacher competencies, emphasize changes in content and pedagogical approaches with the effect of technology (Bachy 2014). A generation ago, the teaching of a fixed curriculum was central to education in most countries, assuming that what teachers teach will last a lifetime; nowadays, in the conditions where the content is easily accessible in search engines and the information is digitized; Teachers need to manage complex ways of thinking, deeply understand students' motivations and emotions (Schleicher, 2012). Educational institutions faced with challenges such as hybrid education, digitalization of education, and the development of techno pedagogical approaches, have difficulties in finding and retaining qualified, committed and motivated teachers who will adapt to these changes. Because these changes can increase teachers' stress, burnout, and intention to quit. A meta-analysis study examining the psychological states of teachers reveals that the prevalence of burnout, stress, anxiety and depression in teachers is in high ranges (Agyapong, Obuobi-Donkor, Burback & Wei, 2022). The differences in the approach to work of Generation Y also make it difficult for their organizational commitment and retention in the organization (Pandita & Ray, 2018).
Organizational commitment is one of the important factors that keep employees in organizations and increase their job motivation. According to Meyer and Allen (1997), organizational commitment is “the employee's feeling of being connected to his organization”. Organizational commitment has three dimensions as "Adaptation", "Identification" and "Internalization" (O'Reilley & Chatman, 1986). Organizational commitment not only helps to keep employees in the organization, but also affects the success of organizations (Balay, 2000; Boylu, Pelit, & Güçer, 2007). Employees with high commitment make extra efforts for their jobs and organizations, play a role in maintaining the competitive advantage of the organization and focus on contributing to high performance (Kim & Brymer, 2011; Pandita & Ray, 2018). According to Khan (1990) ensuring organizational commitment depends on three conditions: the employee's finding their work meaningful, being safe in the organization and having a variety of skills that improve the person at work. Perceived organizational support is “the degree to which employees believe that their organization values their contributions and considers their well-being” (Robbins & Judge, 2013). According to studies, organizational support perceived by teachers affects organizational identification (Demir, 2015). The concept of meaningful work is expressed as a tool of “deep motivation”, “job satisfaction and organizational commitment”. It is the deep satisfaction that a person gets from having done the job and succeeded while doing his job, and the inner, deep motivation to do his job” (Chalofsky & Krishna,2009; Akar,2020).

Researches show that as emotional commitment increases, intention to quit and burnout decreases in teachers. In the competitive conditions where expectations from education increase and the way of doing work changes; Job satisfaction and organizational commitment of qualified teachers, who will adapt to changes and increase the effectiveness of themselves and the school by integrating information technology with pedagogical elements are important. In this study, it is aimed to determine the effect of teachers' perceived organizational support, meaningful work and job satisfaction level on organizational commitment. In the research: a) What are the teachers' levels of meaningful work, perceived organizational support and job satisfaction? b) Do teachers' opinions on meaningful work, organizational support, and job satisfaction levels differ significantly according to generation (X and Y), gender, seniority, school type and school size? c) Is meaningful work level, perception of organizational support and job satisfaction together a significant predictor of organizational commitment ? questions will be answered.


Methodology, Methods, Research Instruments or Sources Used
The research will be conducted with the relational screening method, as it includes determining the levels of meaningful work, perceived organizational support and job satisfaction, its relationship with various variables and the effect of meaningful work, perceived organizational support and job satisfaction on organizational commitment in teachers. Organizational commitment predicted (dependent) variable; meaningful work, perceived organizational support and job satisfaction were determined as predictive (independent) variables. The sample of the research consists of 400 teachers working in public schools in two cities in Turkey. Theoretical sample size chart was used for universes of different sizes to determine the sample size. Within the scope of the research, data are collected through the Meaningful Job Level Scale (Akar, 2020), Perceived Organizational Support Scale (Nayır, 2014), Job Satisfaction Scale (Tezer, 2001), Organizational Commitment Scale (Balay, 2000). It was decided by a simple random method that the measurement tools would be applied to the participants in which schools. In the analysis of the data, descriptive statistical techniques such as frequency, arithmetic mean and percentage was used to determine teachers' meaningful job, perception of organizational support, job satisfaction and organizational commitment levels. The t-test was used to analyze whether the opinions of the groups show a statistically significant difference according to the gender, generation (X and Y), education level variables; ANOVA (One-way analysis of variance) was used to analyze whether there is a significant difference according to seniority, branch, education level, school type and school size variables. Multiple regression analysis was used to determine the effect of meaningful work, perceived organizational support, and job satisfaction on organizational commitment.
Conclusions, Expected Outcomes or Findings
In this study, it was found that the teachers found their job mostly meaningful and; It was found that they perceived organizational support at a moderate level in the sub-dimensions of organizational justice and Organizational Awards and Work Conditions, and at a very high level in the sub-dimension of supervisors support. Perception of teachers' organizational commitment is low in the compliance sub-dimension, moderate in the identification sub-dimension and high in the internalization sub-dimension. There is a statistically significant difference between the opinions of teachers about meaningful work according to the variables of subject and school type. In the study, it was found that high school teachers found their jobs more meaningful compared to primary and secondary school teachers. The perceptions of school administrators on organizational support and organizational commitment are more positive than teachers. According to gender and school type, there is a significant difference between the perceptions of teachers on the organizational commitment dimension. Male teachers and teachers working in primary schools have higher compliance commitment than female teachers and teachers working in secondary and high schools. In the study, it was also revealed that organizational support and job satisfaction is an important predictors of organizational commitment. It is expected that the results of the study will be useful in understanding the psychological and professional motivations of teachers in their relationship with their work and organizations under changing conditions. Based on the results, it can be suggested that teachers should be developed, professional empowerment and coaching-mentoring, maintaining their well-being, ensuring their loyalty, and psycho-social programs could be developed in education and school management. Research can be conducted on the management of educators' well-being in prospective education management and leadership, teacher engagement and digitalization conditions.
References
Agyapong, B., Obuobi-Donkor, G., Burback, L., & Wei, Y. (2022). Stress, burnout, anxiety and  depression among teachers: a scoping review. International journal of environmental research and public health, 19(17), 10706.
Akar, F. (2020). Examining the meaningful work level of generation X and generation Y teachers. Ilkogretim Online - Elementary Education Online, 19 (3), pp. 1225-1241
Bachy, S. (2014). TPDK, a new definition of the TPACK model for a university setting. European Journal of Open, Distance and E-Learning, 17(2), 15-39.
Balay, R. (2000). The organizational commitment of administrators and teachers in private and public
secondart schools (Sample of Ankara province). Doctoral dissertation, Ankara University, Institute of
Social Sciences.
Boylu, Y., Pelit, E. ve Güçer, E.(2007). A Study on the Level Organisational Commitment of Academicians). Finance Politics & Economic Reviews, 44 (511), 56-74.
Chalofsky, N., & Krishna, V. (2009). Meaningfulness, commitment, and engagement: The  intersection of a deeper level of ıntrinsic motivation. Developing Human Resources, 11, 189-203.
Demir, K. (2015). The effect of organizational justice and perceived organizational support on organizational citizenship behaviors: The mediating role of organizational ıdentification. Eurasian Journal of Educational Research, 60, 131-148.
Kahn, W. A. (1990). Psychological conditions of personal engagement and disengagement at work. Academy of management journal, 33(4), 692-724.
Kim, W. G. & Brymer, R. A. (2011). The effects of ethical leadership on manager job satisfaction, commitment, behavioral outcomes, and firm performance. International Journal of Hospitality Management 30 , 1020– 1026.
Meyer, J. & Allen, N. (1997). Commitment in The Workplace, USA:Thousand Oaks SAGE Publications, CA.
Nayır, F. (2013). “Perceived Organizational Support Scale”- Short Form Validity-Reliability Study. Mehmet Akif Ersoy University Journal of Education Faculty, 1(28), 216 - 237
O'Reilly, C. A., & Chatman, J. (1986). Organizational commitment and psychological attachment: the effects of compliance, ıdentification and ınternalization on prosocial behavior. Journal of Applied Psychology,71, 492−499.
Pandita, D., & Ray, S. (2018). Talent management and employee engagement–a meta-analysis of their impact on talent retention. Industrial and Commercial Training, 50(4), 185-199.
Robbins, S. P.  ve Judge, T. A. (2013). Attitudes and Job Satisfaction (Çev. Edit. İlknur Özalp Türetken ve Gökçe Başbuğ). Organizational Behaviour. Nobel Publishing  
Schleicher, A. (2012). Preparing teachers and developing school leaders for the 21st century: Lessons from around the world. OECD Publishing. France.
Tezer, E. (2001). Validity and Relaibility of Job Satisfaction Scale. Turkish Psychological Counseling and Guidance Journal, 2(16), 33-39.


11. Educational Improvement and Quality Assurance
Paper

Internship Quality and Vocational Identity Status in Health Sciences Students

Ieva Rudzinska1, Buratin Khampirat2, Selma Mupeniwo Iipinge3

1Latvian Academy of Sport Education, Latvia; 2Suranaree University of Technology, Thailand; 3Namibia University of Science and Technology

Presenting Author: Rudzinska, Ieva; Khampirat, Buratin

Improving professional identity is crucial for enhancing students' mental health in the health sciences and addressing the scarcity of health professionals (Tang et al., 2022). Internship experiences have been found to be crucial in shaping the vocational identity development of university students (Gao et al., 2022; Jackson, 2017). Vocational identity is defined as an individual's sense of self in relation to their career that having a solid and clear understanding of one's objectives, passions, and skills (Holland et al., 1993). It is a key aspect of career development that can be influenced by various factors (Wong & Kaur, 2018), including the quality of internship experiences (Gamboa et al., 2021; Khampirat, 2020; Porfeli et al., 2011). The purpose of this study is to examine the relationship between internship quality and vocational identity status in university students.

Theory:

The Social Cognitive Career Theory (SCCT) (Lent et al., 1994) and the Career Construction Theory (CCT) (Savickas, 2005) were applied and integrated in this study.

SCCT (Lent et al., 1994) posits that individuals develop their vocational identity through a process of self-reflection and exploration, in which they consider their interests, abilities, and values, and how these relate to potential career options. This theory suggests that high-quality internship experiences can provide opportunities for students to gain knowledge, skills, and self-awareness that can enhance their vocational identity development (Pan et al., 2018).

CCT (Savickas, 2005) proposes that vocational identity formation is a dynamic, ongoing process that occurs through a series of career-related experiences, including internships. CCT states that internship experiences can provide students with opportunities to explore their interests and values and to develop a sense of self in relation to their careers (Chen et al., 2022).

Empirical evidence supports the relationship between internship quality and vocational identity status in university students. For example, a study by Gamboa et al. (2021) found that VET students who had high-quality internship experiences reported greater vocational identity exploration than those who had lower-quality internships. Another study by Ng et al. (2022) found that students who had high-quality internships were more likely to report a sense of vocational self-efficacy and career decision self-efficacy than those who had lower-quality internships.

In conclusion, this study aims to investigate the relationship between internship quality and vocational identity status in university students, drawing on the theories of SCCT and CCT. The results of this research may provide valuable insights for university educators and career counselors to enhance the quality of internship experiences and facilitate the vocational identity development of university students.


Methodology, Methods, Research Instruments or Sources Used
Participants:
This was a cross-sectional study. The participants were 117 health sciences students who were randomly sampled from universities in Thailand (male = 22, female = 95). All participants provided informed consent and were aware of their rights as research participants. Data was collected and analyzed in accordance with ethical guidelines and regulations.

Instruments:

          - The Vocational Identity Scale (VISA) is a measure of vocational identity developed by Porfeli et al. (2011). It is a self-report measure that assesses the degree to which an individual has a clear, stable, and committed sense of self in relation to their vocational interests, values, and goals. The VISA consists of 30 items that are rated on a 5-point Likert scale. The Cronbach’s alpha for the VISA subscales ranged from .802 to .831, with an overall Cronbach’s alpha of 0.918. The confirmatory factor analysis (CFA) results supported the second-order measurement model of VISA with a good model fit (chi-square (333) = 1329.057, chi-square/df = 3.991, CFI = 0.961, TLI = 0.949, RMSEA = 0.040 [0.038, 0.042], SRMR = 0.045).

          - The Internship Experience Questionnaire (InExp) is a tool used to measure the quality of internships. It was developed by combining the questionnaires of Luk and Chan (2020), Anjum (2020); Marinaș et al. (2018), Nghia and My Duyen (2018), and Khampirat and McRae (2016). The scale includes 20 items that are divided into four subscales: clear goals (4 items), university support (3 items), workplace support (3 items), perceived value (6 items) and assessment (4 items). Participants are asked to rate each item on a 5-point Likert scale, with 1 indicating "strongly disagree" and 5 indicating "strongly agree." The Cronbach’s alpha for the InExp subscales ranged from .761 to .878, with an overall Cronbach’s alpha of 0.934. The CFA results supported the second-order measurement model of InExp with a good model fit (chi-square (152) = 407.125, chi-square/df = 2.678, CFI = 0.986, TLI = 0.983, RMSEA = 0.030 [0.026, 0.033], SRMR = 0.021).

Data analysis:
Nominal and ordinal variables were described using frequency and percentage, whereas continuous variables were described using means and standard deviations. The variations in VISA and InExp rating between two or more groups were examined using the t-test and ANOVA. Pearson’s correlation was applied to explore the correlation between variables in this study. The influence of Internship Quality and VIS was analyzed via partial least squares structural equation modeling (PLS-PM, PLS-SEM).

Conclusions, Expected Outcomes or Findings
The results of this study provide evidence for the relationship between internship quality and vocational identity status in university students, particularly in the context of health sciences students in Thailand. The findings suggest that high-quality internship experiences can lead to greater vocational identity exploration, self-awareness, and self-efficacy among these students. These results are consistent with the predictions of the Social Cognitive Career Theory and Career Construction Theory, which posit that internship experiences can play a crucial role in vocational identity development.

The implications of these findings are particularly relevant for educators, practitioners, and career counselors in the health sciences field in Thailand, as they suggest that efforts to enhance the quality of internship experiences can have a positive impact on the vocational identity development of students. This includes providing opportunities for self-reflection, exposure to diverse career options, and guidance in the decision-making process.

Additionally, this research could also be applied to other fields and in other countries, including in the European/international dimension. As the theories used in this study are widely accepted across cultures and fields, the relationship between internship quality and vocational identity development is likely to be generalizable to a variety of contexts. Therefore, the findings of this study could be used to inform internship policies and practices in other fields and countries to promote vocational identity development among university students.

In conclusion, this study has provided valuable insights into the relationship between internship quality and vocational identity status in university students, specifically in the field of health sciences in Thailand. Furthermore, the findings of this study have the potential to be applied in other fields and countries to enhance the vocational identity development of university students.

References
Chen, H., Wu, Y., Jiang, L., Xu, B., Gao, X., & Cai, W. (2022). Future orientation and perceived employability of Chinese undergraduates: a moderated mediation model. Current Psychology.
 
Gamboa, V., Paixão, M. P., da Silva, J. T., & Taveira, M. d. C. (2021). Career Goals and Internship Quality Among VET Students. Journal of Career Development, 48(6), 910-925.

Gao, Z., Wei, X., Yang, L., Cui, D., Kong, L., Qi, L., & Zhang, P. (2022). Mediating role of career self-efficacy between clinical learning environment and professional identity in nursing students. Journal of Advanced Nursing, 78(4), 1012-1019.

Holland, J. L., Johnston, J. A., & Asama, N. F. (1993). The vocational identity scale: A diagnostic and treatment tool. Journal of Career Assessment, 1(1), 1-12.
 
Jackson, D. (2017). Developing pre-professional identity in undergraduates through work-integrated learning. Higher Education, 74(5), 833-853.

Khampirat, B. (2020). The relationship between paternal education, self-esteem, resilience, future orientation, and career aspirations. PLoS One, 15(12), e0243283.

Luk, L. Y. Y., & Chan, C. K. Y. (2020). Adaptation and validation of the Work Experience Questionnaire for investigating engineering students' internship experience. Journal of Engineering Education, 109(4), 801-820.

Marinaș, C. V., Goia , S. I., Igreț, R. Ș., & Marinaș, L. E. (2018). Predictors of quality internship programs—The case of Romanian business and administration university education. Sustainability, 10(12), 4741.

Ng, P. M. L., Wut, T. M., & Chan, J. K. Y. (2022). Enhancing perceived employability through work-integrated learning. Education + Training, 64(4), 559-576.

Nghia, T. L. H., & My Duyen, N. T. (2018). Internship-related learning outcomes and their influential factors. Education + Training, 60(1), 69-81.

Pan, J., Guan, Y., Wu, J., Han, L., Zhu, F., Fu, X., & Yu, J. (2018). The interplay of proactive personality and internship quality in Chinese university graduates' job search success: The role of career adaptability. Journal of Vocational Behavior, 109, 14-26.

Porfeli, E. J., Lee, B., Vondracek, F. W., & Weigold, I. K. (2011). A multi-dimensional measure of vocational identity status. Journal of Adolescence, 34(5), 853-871.

Savickas, M. L. (2005). The Theory and Practice of Career Construction. In D. Brown & R. W. Lent (Eds.), Career development and counseling: Putting theory and research to work. (pp. 42-70). John Wiley & Sons, Inc.

Tang, M., Sun, Y., Zhang, K., Luo, R., Liu, Y., Sun, H., & Zhou, F. (2022). Associated factors of professional identity among nursing undergraduates during COVID-19: A cross-sectional study. International Journal of Nursing Sciences, 9(1), 107-113.


11. Educational Improvement and Quality Assurance
Paper

Pursuit of Quality in Vocational Secondary Schools: deployment of management systems and reconfigurations at work

Silvia Sá

HEP Vaud, Switzerland

Presenting Author: Sá, Silvia

The school benches are crossed by a transposition of new measures of a managerial nature, with a progression of new modes of governance focused on accountability and results evaluation (Maroy & Pons, 2019; Taylor, 2007). Our contribution will highlight how the vocabularies and approach have subtly shifted from "quality control" to an incentive for "quality development" in a logic of collective action (Ozga et al., 2011). Launched in "project mode" and with action principles such as initiative and accountability, quality management systems encourage a logic of enrolment of everyone, at all levels of the organization.
In Switzerland, the institutionalization of quality management systems (QMS) has been brought into the regulatory basis (Vocational Education and Training Law, 2002) and has been integrated into Swiss secondary vocational schools, with certificates recommended at the cantonal level. In a context that has called for reforms and increasing complexity, it can be assumed that the introduction of quality certifications responds to a challenge of valorization and incorporation of quality and excellence standards. In the management of institutions, these systems represent the development of a tool for action and a culture of quality monitoring (quantified, comparative and cyclical information). In this context, these standardization systems are deployed as "soft constraints" and are based on mobilization principles. They are aligned with the rise of so-called "soft" governance and intended to be suggestive (Duru-Bellat, 2019). Self-evaluation and self-regulation (of institutions, actors; etc.) has become unavoidable and is combined with regulatory action through external mechanisms.

In this contribution, we will highlight how the deployment of QMS leads to a shift from endogenous professionalism (from within) to exogenous professionalism (from above), with repercussions on the definition of work and on activity (Boussard et al., 2010; Evetts, 2009). Organizational professionalism is clearly expanding in educational institutions (Tardif, 1999). It introduces forms of regulation, which no longer rely on the basis of "trust" or discretionary power of professionals (Evetts, 2009) and involves increased standardization of work procedures and practices. The use of new types of external experts (auditors who certify SQM) or the internal emergence of new roles within the intermediate professions, such as "delegate" or "quality manager", is part of the workings of these steering and regulation mechanisms.
This leads us to ask several research questions: what are the perceived impacts of these management mechanisms (understanding of meaning and relationship)? How does the deployment of a QMS affect the work and practices of different groups of school actors?


Methodology, Methods, Research Instruments or Sources Used
The choice of this field introduces the hypothesis of a particular resonance in the context of the implementation of vocational training, which is composed of a wide range of partners, actors and mechanisms. The specificities of vocational and post-compulsory training would constitute a favourable context for the development of quality management systems.
In order to understand the impact of this deployment, our method focuses on the actors who make, translate and interpret the meaning of the systems. By questioning the configurations "manufactured" by the instruments (Lascoumes & Le Galès, 2005), we will consider from the outset the emergence of effects of our own, which go beyond the initial intentions of the decision-makers and generate an appropriation and unexpected detour.
This contribution will present the results of an ongoing doctoral research project that began with an analysis of various documents and sources (official and legal documents; repositories of various standards; commission reports; charters; documents from professional unions; etc.). This qualitative research is based on nearly forty interviews conducted with members of the school management, quality delegates, deans, and teachers in 11 schools in one canton, as well as interviews with the general department. The purpose of these interviews was to review the history of the development of the quality system (at the school level or at the cantonal level) and to uncover the experience of this monitoring over the years. The questions concerned the biographical accounts of the people involved, the layout and relationship to the certification process, the choice of the quality label, the impact on professional activity, the collaborators and associated roles, etc.
We also conducted audit observations and interviews with auditors, seeking to understand the point of view of professionals who specialize in this external certification process for several quality standards (such as ISO, QSC, etc.).
We will present the impacts perceived by groups of actors and we will describe the meaning of and relationship with the process and the different associated socio-technical processes.

Conclusions, Expected Outcomes or Findings
Our results explore how intermediary actors collectively take hold of these devices, the logics of (re)negotiation at work (paradoxes; arbitrations made; logics of mediation; compromises) and emerging reconfigurations of work (impacts on their room for maneuver and discretionary power). At this point, the analysis of the different discourses, allows us to accentuate the intensification of a multi-regulation of education (Barroso, 2016; Carvalho, 2015), conceiving the quality certification of schools as a supplementary and complementary device to many other "political commands", which seek to ensure a remote control and the development of a tool managerial action.
Acting at the interface of the managerial and professional spheres (Resenterra et al., 2013), we will emphasize the extent to which actors are confronted with contradictory injunctions. In our results, particular attention will be paid to intermediary actors as "permanent passers" (Nay & Smith, 2002) between different regimes. Several interviewees clearly express this position of interface between different "worlds", in a quest for balance and a concern to preserve their teams.  
At this point, a few preliminary findings should be noted: the 11 schools cover a variety of domains and we note different discourses according to professional cultures and previous career paths (we are faced with teachers who often have a "second career", who bring other expertise and expectations into the school world); the development of the QMS is coupled with new computer tools in everyday school life; we observe mimetic and normative processes and mechanisms between schools (institutional isomorphism; DiMaggio & Powell, 1983); pedagogical autonomy seems to be little affected by the QMS, but other complementary instruments contribute to changes at the pedagogical level; auditing spaces reinforce the logic of reflective accountability (soft accountability).

References
Barroso, J. (2016). La régulation locale de l’éducation et la direction des établissements scolaires au Portugal. Recherche & formation, 78, 81‑94.
Boussard, V., Demazière, D., & Milburn, P. (2010). Introduction. Qu’est-ce qu’être professionel? In V. Boussard, D. Demazière, & P. Milburn, L’injonction au professionnalisme : Analyses d’une dynamique plurielle (p. 13‑22). Presses universitaires de Rennes.
Carvalho, L. M. (2015). As políticas públicas de educação sob o prisma da ação pública : Esboço de uma perspetiva de análise e inventário de estudos. Curriculo sem Fronteiras, 15, 314‑333.
DiMaggio, P. J., & Powell, W. W. (1983). The Iron Cage Revisited : Institutional Isomorphism and Collective Rationality in Organizational Fields. American Sociological Review, 48(2), 147.
Duru-Bellat, M. (2019). Évaluations, mesures ou classements ? A propos des enquêtes PISA. Revue francaise de linguistique appliquée, Vol. XXIV(1), 7‑19.
Evetts, J. (2009). The Management of Professionalism : A contemporary paradox. In S. Gewirtz, P. Mahony, I. Hextall, & A. Cribb (Éds.), Changing Teacher Professionalism : International Trends, Challenges and Ways Forward (p. 19‑30). Routledge, Taylor & Francis.
Lascoumes, P., & Le Galès, P. (2005). Introduction : L’action publique saisie par ses instruments. In P. Lascoumes & P. Le Galès (Éds.), Gouverner par les instruments (p. 11‑44). Presses de Sciences Po (P.F.N.S.P.).
Maroy, C., & Pons, X. (2019). Accountability Policies in Education : A Comparative and Multilevel Analysis in France and Quebec. Springer.
Nay, O., & Smith, A. (2002). Les intermédiaires en politique. Médiation et jeux d’institutions. In O. Nay & A. Smith (Éds.), Le gouvernement du compromis : Courtiers et généralistes dans l’action politique (p. 1‑21). Economica.
Ozga, J., Dahler-Larsen, P., Segerholm, C., & Simola, H. (2011). Introduction. In J. Ozga, P. Dahler-Larsen, C. Segerholm, & H. Simola, Fabricating Quality in Education : Data and Governance in Europe (p. 1‑8). Routledge.
Resenterra, F., Siggen, M., & Giauque, D. (2013). Les cadres intermédiaires entre contraintes managériales et défense des identités professionnelles : L’exemple des hôpitaux de Suisse romande. Humanisme et Entreprise, 315, 1‑24.
Taylor, I. (2007). Discretion and Control in Education : The Teacher as Street-level Bureaucrat. Educational Management Administration & Leadership, 35(4), 555‑572.
 
3:30pm - 5:00pm11 SES 07 A: Initiatives of Improving Students' Learning at Schools
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Ieva Rudzinska
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Process Facilitators’ Work for Improvement Capacity of School Organizers in the Collaboration for Best School (SBS) in Sweden

Marcia Håkansson Lindqvist1, Jaana Nehez2

1Mid Sweden University; 2Halmstad University

Presenting Author: Håkansson Lindqvist, Marcia; Nehez, Jaana

The Swedish Education Act (SFS 2010:800) sets out the right to an equal education, an education that is expected to be compensatory and to offset differences in pupils' different circumstances. Despite the Education Act, the Organisation for Economic Co-operation and Development (OECD) (2015) review, Improving Schools in Sweden, showed that differences in equivalence between and within schools in Sweden have increased and that many Swedish schools have failed to systematically address the problem. Furthermore, the OECD (2015) reported that many schools were left alone in their efforts to remedy the problems following the inspections by the Swedish Schools Inspectorate. Based on these findings, the Swedish government commissioned the Swedish National Agency for Education to initiate school improvement work (Utbildningsdepartementet, 2015; 2019). In 2016, the National Agency for Education presented a plan to carry out the assignment and since then, nearly 150 principals and almost 400 schools and preschools have now participated in the improvement work called the Collaboration for the Best School (SBS) (Skolverket, 2022). Thus, SBS is one of the most comprehensive initiatives undertaken in Sweden in recent times.

This paper presents the preliminary results of a study of the work of process facilitators in the Collaboration for Best Schools (SBS). The process facilitators have positions at the National Agency for Education, with an assignment to, in pairs, support school organizers and schools to improve schools. The study aims to develop knowledge about the practical work of process facilitators with school organizers, principals and teachers in order to promote their capacity for improvement. A need of such knowledge is highlighted by for example Blossing (2021). Based on sociocultural theories (Jakobsson; 2012; Säljö, 2005; Wartofsky, 1979)., the following questions were investigated: What tools do process facilitators use? How are these tools used? The research contribution of this paper provides insights in school improvement in the Swedish context, thus contributing to the national and international perspectives.

The study is based on observations of dialogues, so-called process meetings, between process facilitators and representatives of school organizers, principals and teachers respectively, on interviews with process facilitators and on document analysis. The results show that process facilitators use different categories of tools for different purposes such as focusing thoughts, deepening understanding, monitoring systematic quality work, eliciting commitment and promoting improvement capacity and ownership. Furthermore, they show that in the use of the identified tools there are critical incidents or turning points that have both positive and negative impacts on the development of participants' understanding and improvement capacity. These critical incidents relate to ownership, simplification, processing, repeated constructive feedback and model focus. The SBS model for how to conduct baseline analyses and formulate action plans guides the use of tools by process facilitators. It appears that the dominance of the model presents both opportunities and challenges for process facilitators to balance.

The conclusion is that there is potential for process facilitators to develop the use of tools to contribute to the development of understanding and improvement capacity of participants in SBS and that work is needed to support process facilitators in this. The study implies that the critical incidents that have a positive impact on the development of participants' understanding and improvement capacity can be strengthened, while the critical incidents that have a negative impact need to be worked away. Support could be in the form of skills development related to critical incidents in the use of tools by process facilitators.


Methodology, Methods, Research Instruments or Sources Used
Two process facilitator pairs were selected for observations in consultation with the National Agency for Education. They were observed during fourteen process meetings between January and August 2022. The meetings took place in the first phase of SBS in which the process facilitators are supporting the school organizers and schools in identifying what is needed to be improved, a baseline analysis. Each meeting lasted 120-150 minutes. Some were face-to-face meetings and others were digital. As observers, we participated digitally on all occasions.
Ten process facilitators were interviewed focusing on process facilitators' preparation and collaboration for process meetings, tools for process meetings, and challenges and opportunities in process meetings. The selection of interviewees was made in consultation with the National Agency for Education. The interviews, later transcribed, were conducted and recorded with the consent of the process facilitators during spring 2022.
A sample of documents (process meeting invitations, powerpoint presentations, templates and stencils) used by the process facilitators during observed process meetings were collected.
Artifacts and critical incidents were used as key analytical concepts. Regarding artifacts, we drew on the concepts of material and conceptual artifacts and on Wartofsky's (1979) division of artifacts into three levels: primary, secondary or tertiary level. To answer which tools process facilitators used, we analysed the observations and documents. We identified and categorized which material and conceptual artefacts process facilitators used in the observations. To analyse how they were used, we searched for critical incidents (Tripp, 1993) in the process facilitators' use of tools for the development of understanding and improvement capacity. Critical incidents were identified within each overall category of artifacts. Thus, we identified turning points related to the use of artefacts, i.e. when understanding was or was not developed or when the use of an artefact was or was not developed to a higher level (Wartofsky, 1979) and what in the use of the process facilitators contributed to the critical incident. Moreover, a thematic analysis (Braun & Clarke, 2006) of all critical incidents was conducted to make visible what the critical incidents were substantively about from a process facilitator perspective, such as 'taking over or transferring ownership'. Finally, to strengthen the credibility of the analysis, we examined how the process facilitators described their work in the interviews. This served as a member check (Lincoln & Guba, 1985) by which the analysis could be validated against the voices of the process facilitators.

Conclusions, Expected Outcomes or Findings
The conclusion is that there is potential for process facilitators to further support participants' development of understanding and capacity for improvement and that work is needed to support process facilitators in this. The study implies that the critical incidents identified in the study as having a positive impact on the development of participants' understanding and improvement capacity need to be predominant, while the critical incidents identified as having a negative impact should be minimised. Support to process facilitators could take the form of skills development related to critical incidents in the use of tools by process facilitators. Such support would enable them to use tools in a way that helps SBS participants to progress. This would be a successful way to create agency and ownership among school organizers and schools. Skills development on how to use the tools could further include exploring what it would mean for process facilitators to start from the models the participants already use instead of the SBS model, which is one way of working. What tools would process facilitators need to use then? In this respect, exchanges of experience between facilitators could contribute to a better understanding of the school organizers’ and schools’  prerequisites for the SBS work, as well as closer contact with the HEIs for support in the use of tools to develop understanding and improvement capacity among the participants in the SBS.

References
Blossing, U. (2021). Förbättringskapacitet som funktionalistisk strukturering eller en professionellt lärande gemenskapskultur [Improvement capacity as functionalistic structure or a professional learning community culture].  Pedagogisk Forskning i Sverige, 26(1), 70-93. https://doi.org/10.15626/pfs26.01.04

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.
doi:10.1191/1478088706qp063oa

Jakobsson, A. (2012). Sociokulturella perspektiv på lärande och utveckling [Sociocultural perspective on learning and development]. Pedagogisk Forskning i Sverige, 17(3-4), 152–170.  

Lincoln, Y. S. & Guba, E. G. (1985). Naturalistic inquiry. Sage.
https://www.ulfavtal.se/digitalAssets/709/c_709236-l_3-k_extern-utva-rdering-ulf.pdf

Organisation for Economic Co-Operation and Development. (2015). Improving schools in Sweden: An OECD perspective.
https://www.oecd.org/education/school/Improving-Schools-in-Sweden.pdf

Skolverket [National Agency for Edcuation]. (2022). Redovisning om uppdrag för Samverkan för bästa skola. Dnr: 2022:24. [Report on the Collaboration for Best School]
https://www.skolverket.se/getFile?file=9681

Säljö, R. (2005). Lärande och kulturella verktyg: Om lärprocesser och det kollektiva minnet [Learning and Cultural Tools: On learning processes and the collective memory]. Nordstedts.

The Swedish Education Act (SFS 2010:800).

Tripp, D. (1993). Critical incidents in teaching. Routledge.

Utbildningsdepartementet [Ministry of Education]. (2015). Uppdrag om Samverkan för bästa skola [Commission for Collaboration for Best School]. Diarenummer U2015/3357/S.

Utbildningsdepartementet. [Ministry of Education]. (2019). Uppdrag till Statens skolverk om samverkan för bästa skola [Comission to the National Agency for Education on Collaboration for Best School]. Diarenummer U2019/03786/S.

Wartofsky, M. (1979). Models. Representation and the scientific understanding. D. Reidel.


11. Educational Improvement and Quality Assurance
Paper

Relationship between Emotional Intelligence and Academic Performance in Secondary Education

Ainur Amanbayeva, Aigul Beskempirova, Baqlan Kojahmet, Yerlan Seitov, Bagdat Kerimzhan

Nazarbayev Intellectual Schools, Kazakhstan

Presenting Author: Amanbayeva, Ainur

This study is a quantitative analysis of empirical research on the relationship between emotional intelligence (EI) and academic performance (AP).

Emotional intelligence, as related with how an individual recognizes and manages his or her emotions, has made tremendous impact in scientific circles since the development of the concept. Countless researches have been done in order to assure its prominent role in an individual’s success in life. With the increasing demand for success in the developing world, the priority of emotional intelligence has also been regarded as an opportunity. Since emotional intelligence has been referred as crucial in many parts of people’s lives, it can be expanded into the world of language learning. In Kazakhstan, most people complain about not being able to master the English language. The problem underlying this may be the effects of emotional intelligence and its subscales on learning. For this reason, this study has been conducted to fill a gap in the literature of education by focusing on the relationship between emotional intelligence and academic performance.

The literature on emotional intelligence (EI) suggests that individuals with higher cognitive processing abilities tend to perform better on cognitive tasks (Saklofske et al., 2012). In secondary education, interpersonal and intrapersonal skills play a crucial role due to the social, contextual, and personal changes and stresses that occur during this period. Adolescents' emotional development and identity formation are significantly influenced by their peer groups, with the school environment being particularly relevant (Duncan et al., 2006; Eccles and Roeser, 2009; Monreal and Guitart, 2012).

People with higher emotional intelligence tend to be more resilient, adapting well to changes, managing stress effectively, and handling challenges more successfully (Schneider et al., 2013). Moreover, students who have better emotional management skills tend to experience greater happiness and have better social relationships (Eryilmaz, 2011). Improved interpersonal management is associated with larger social networks and higher quality friendships (Brackett et al., 2005). In the classroom setting, a larger social network can foster a conducive social environment for cooperative work, group learning, peer support (Hogan et al., 2010), and positive relationships with teachers (Di Fabio and Kenny, 2015). Both the classroom climate, including interactions with classmates and teachers, and a predisposition for learning-oriented abilities are linked to greater academic performance (Brackett et al., 2011; Johnson, 2016).

In summary, there are various plausible theoretical mechanisms that can explain the relationship between EI as a set of skills and optimal academic functioning in secondary education.

The current study is being conducted at Nazarbayev Intellectual school in Turkistan in 2022-2023 academic year. There are 55 10-graders (24 male and 21 female) participating in the study. The data is being collected using an

Emotional intelligence test by Lyusin that has been designed for the purpose of the study and the reliability value was reported as .84. The pre-test using Emotional intelligence test by Lyusin was conducted to students to measure the level of their Emotional intelligence. The pre-test result showed The data will be analyzed using SPSS 23.

The aim of this study is to show the relationship between Emotional intelligence and academic performance by examine the effectiveness of materials to increase students’ Emotional intelligence. The results will be analyzed according to the presets-posttest scores conducted before and after the treatment. It is predicted that the results will show statistically significant dıfferences between participants pretest and posttest scores, which means that the treatment will be found to be effective in terms of increasing secondary school students’ levels of emotional intelligence.


Methodology, Methods, Research Instruments or Sources Used
The current study is being conducted at Nazarbayev Intellectual school in Turkistan in 2022-2023 academic year. There are 55 10-graders (24 male and 21 female) participating in the study. Data was collected with the means of a survey method.
Emotional intelligence test by Lyusin  that has been designed for the purpose of the study and the reliability value was reported as .84. The pre-test using Emotional intelligence test by Lyusin was conducted to students to measure the level of their Emotional intelligence. The pre-test result showed The data will be analyzed using SPSS 23.
The quasi-experimental with a one-group pretest-posttest is being used as a research design in order to measure the effectiveness of using additional materials during classes aimed at developing secondary school students’ level of emotional intelligence to see if the students’ academic performance improved or not after the treatment. A one-group pretest-posttest design is a quasi-experimental design that is most often utilized by behavioral researchers to determine the effect of a treatment or intervention on a given sample (Allen, 2017). Data is being collected with the means of a survey method. The Emotional Intelligence scale used in the present study adapted from D.V.Lyusin (Люсин, 2006) was conducted among Grade 10 students of secondary school.
In the present study Lusin’s (2006) EmIn Emotional Intelligence questionaire is used. The questionnaire measures emotional intelligence (EI), the structure of which is highlighted interpersonal EI and intrapersonal EI gives the scores on the two subscales measuring different aspects of the interpersonal EI (UOPE subscale – understanding other people's emotions and a scale of MOPE – manage other people's emotions), and the picture on the subscales measuring different aspects of the intrapersonal EI scale (UYE - understanding your emotions, the scale of MYE - management of your emotions, the scale of EC (the ability to control symptoms of their emotion). The questionnaire consisted of 46 item in a form of a 4-point likert type was used as a data collection tool.

Conclusions, Expected Outcomes or Findings
We predict that specially prepared materials to develop students' Emotional Intelligence will improve students' academic performance. Saying exactly, those materials will help students to recognize others, the environment, understand it, openly express their thoughts, overcome internal stress, share it with others, increase self-confidence, make it possible to understand oneself, see other qualities in oneself and instill personal self-control, observation which can lead to enhance different skills required to improve learners academic performance.
Thus, emotional intelligence plays a fundamental role in various spheres of life; it has an effective contribution to language learning. The development and implementation of EI in students should be promoted, so students and teachers can have effective learning environment.

References
1. Abdolrezapour, P. (2013). The Relationship between Emotional Intelligence and EFL Learners’Writing Performance. Procedia - Social and Behavioral Sciences, 70, 331–339.
https://doi.org/10.1016/j.sbspro.2013.01.070
2. Gardner, H., & Hatch, T. (1989). Educational Implications of the Theory of Multiple Intelligences. Educational Researcher, 18(8), 4–10.
https://doi.org/10.3102/0013189X018008004
3. J.Mayer, P.Salovey, D. C., & Cherkasskiy, L. (2011). Emotional Intelligence John D . Mayer ,Peter Salovey , David R . Caruso
4. Lawrence, A. S. A., & Deepa, T. (2013). Emotional intelligence and academic achievement of high schoolstudents in Kanyakumari District. International Journal of Physical and Social Sciences, 3(2), 101–107.
5. Zarafshan, M., & Ardeshiri, M. (2012). THE RELATIONSHIP BETWEEN EMOTIONAL INTELLIGENCE, LANGUAGE LEARNING STRATEGIES AND ENGLISH
PROFICIENCY AMONG IRANIAN EFL UNIVERSITY STUDENTS.Journal of
Educational & Instructional Studies in the World,2(3).
6. Fahim, M., & Pishghadam, R. (2007). On the role of emotional, psychometric, and verbal intelligences in the academic achievement of university students majoring in English language.Iranian EFL Journal,4.
7. Lindebaum, D. (2013). Does emotional intelligence moderate the relationship between mentalhealth and job performance? An exploratory study. European Management Journal, 31(6), 538–548. https://doi.org/10.1016/j.emj.2012.08.002
8. Shatalebi, B., Sharifi, S., Saeedian, N., & Javadi, H. (2012). Examining the relationship between emotional intelligence and learning styles. Procedia - Social and Behavioral Sciences, 31(2011), 95–99. https://doi.org/10.1016/j.sbspro.2011.12.022
9. Zarezadeh, T. (2013). The Effect of Emotional Intelligence in English Language Learning. Procedia - Social and Behavioral Sciences, 84, 1286–1289. https://doi.org/10.1016/j.sbspro.2013.06.745


11. Educational Improvement and Quality Assurance
Paper

Classroom and School Impacts on Academic Achievement: A Systematic Review of Multilevel Modelling Studies in Primary Schooling

Sercan Erer1, Andreas Hadjar2,1, Susanne Backes1

1University of Luxembourg, Luxembourg; 2University of Fribourg, Switzerland

Presenting Author: Hadjar, Andreas

Understanding predictors of academic achievement has been among the essential research focus of sociologists of education. One of the most influential studies in the field dates to 1960s. James S. Coleman and his colleagues (1966) published their famous report on their scientific attempts to unveil determinants of academic achievement differences of students from different social backgrounds in the US. The results underlined the impact of students’ parental resources and classroom composition based on these resources, more than the impact of schools attended by these students. Despite of providing attention to the impact of individual and cumulative socioeconomic disadvantage of students, the report received critics on the lack of attention to possible school effects. Following the concerning critics, some researchers dedicated their large-scale work more into investigating school effects on academic achievement and ignited an increase in studies for the field of school effectiveness research (Burušić et al., 2016; Scheerens, 2016).

Throughout the years, the field not only yielded a list of significant effects emerging from classroom and school settings, but also highlighted the need for utilizing more precise methodological techniques to account for the nature of nested data while investigating these effects on academic achievement. As an invaluable example, John Hattie (2008)proposed evidences from abundant meta-analyses on how teachers, teaching practices, and schools pose impact on student learning. On the teacher and teaching practices, some of the underlined effects pointed the importance of teaching-occupation-related variables such as expectations from students, teacher training, quality of teaching, and professional development available to teachers, and of strategies related to feedback, learning intentions, and so on (Hattie, 2008). On the school, the importance of school characteristics, school and classroom compositions, school curriculum and classroom effects (Hattie, 2008). While such evidence brought undeniable contributions to the comprehension of multiple interacting effects, Creemers and Kyriakides (2008) introduced a dynamic model of how schools impact academic achievement. Within this model, natural clustering inside formal educational institutions, students within classrooms within schools within educational systems, is illustrated. Due to this natural multilayered structures of schools, the field underlined that data collected from students, classrooms and schools violate the observation independency assumption of traditional analytical techniques (de Leeuw & Meijer, 2008; Hox et al., 2018). Therefore, to model such nested data at multiple levels by controlling for observation dependency, the field recommended the utilization of multilevel modelling as the analytical technique to precisely examine the impacts of these classroom and school factors separately and in combination (Burušić et al., 2016).

Conducting research to probe possible classroom and school effects on academic achievement is especially important in the primary schooling level. The reason behind is ascertain, early in the schooling system, which classroom and school variables significantly explain variations in academic achievement, to be able to diminish the impact of cumulative advantage mechanism, or the Matthew effect (Merton, 1968), resulting in enduring achievement gaps and therefore unequal educational gains and opportunities (Diprete & Eirich, 2006). Besides, results from these scientific attempts potentially aid educational policy makers in developing more effective and efficient approaches to supporting learning, assist school administrations in diagnosing students at risk of lagging behind, and help researchers identify possible research gaps in the field. Consequently, regarding these considerations on the relevance of the classroom and school environment on academic achievement especially in the primary schooling and possible precise contributions from studies utilizing multilevel modelling, the main aim of this systematic review of the existing multilevel modelling studies is to derive and thematize the characteristics of the classroom and school settings impacting academic achievement in primary schooling, by applying strong methodologically-conceptually driven selection criteria.


Methodology, Methods, Research Instruments or Sources Used
The research methodology of this study is systematic review. As its research foci, the present review makes a set of inquiries into the following aspects over selected studies: their theoretical and methodological foundations, and significant classroom and school level variables impacting academic achievement in the primary schooling. Accordingly, the selection criteria were for studies to employ multilevel modelling, to examine data collected from primary schooling, and to contain classroom and school levels. After multiple trials, a search query was developed with grouped keywords and controlled terms (wildcards), but without a restriction on publication year or geography. Running the developed query in all fields available for searching on two prominent scientific databases during the last week of February 2021 gathered 112 studies from WoS and 96 studies from ERIC. After removing 25 duplicates, creation of the study pool was finalized with 183 unique study inputs.

Before the study screening process, it has been decided to distill studies for significant classroom and school variables separately. Screening for each level were conducted in two stages: title and abstract screening (TAS), and full-text review (FTR). During each stage for each level, two researchers out of three blindly screened the studies in accordance with the selection criteria and the remaining researcher acted as a conflict resolver to reduce the possible researcher bias in screening studies for inclusion. Before conflict resolution, for each step of screening, Cohen’s κ was run to check the agreement level between the screeners. For classroom, the initial yield rate after TAS was 14.75% and the test result revealed a substantial agreement (McHugh, 2012), κ = .649 (95% CI, .498 to .801), SE = .077, p < .01. The final yield rate following FTR was 8.74%, with again a substantial agreement (McHugh, 2012), κ = .743 (95% CI, .471 to 1.000), SE = .139, p < .01. For school, the initial yield rate after TAS was 26.23% and the test result revealed an almost perfect agreement (McHugh, 2012), κ = .874 (95% CI, .794 to .954), SE = .041, p < .01. The final yield rate following FTR was 19.67%, with a substantial agreement (McHugh, 2012), κ = .714 (95% CI, .480 to .948), SE = .119, p < .01. The data extraction was conducted with 16 and 36 studies respectively. Through inductive thematic analysis, the extracted data was examined to categorize the information under similar theoretical, methodological, and contextual/compositional aspects.

Conclusions, Expected Outcomes or Findings
The extracted information has been investigated in accordance with the research foci. Regarding the theoretical foundations of the selected studies, many studies focus on school effectiveness research and utilize the systematization of previous empirical findings rather than theoretical considerations. Most conceptual considerations focus on compositional effects on both levels. Very few studies provide linkages to general conceptual frameworks on educational inequalities such as Boudon (1974) and Bourdieu (1986) or Bourdieu & Passeron (1977). The overall reason behind might be the lack of theory particularly in school effectiveness research (Scheerens, 2016). From the methodological foundations, only a few studies focus on earlier grades of primary schooling (e.g., grades levels 1-3), and analyze differences in the respective learning growth of subject specific performances. Regarding outcomes, the focus is often on primary school students’ reading and/or mathematics performances.

On the classroom level, the inductive thematic analysis revealed that significant classroom effects are frequently examined regarding student composition, teacher-related determinants, classroom’s socio-physical conditions, and curriculum. On the school level, the analysis revealed that significant school effects are frequently examined regarding student composition, school’s socio-physical conditions, school management, school climate, teacher composition, and curriculum. Explicitly, the results on student composition from both levels indicated that the scholars probed compositional effects from the aspects of socioeconomic background, cognitive and behavioral outputs, ethnicity/language, special education needs, and gender.

The brief conclusion from the results underlines that students from advantageous backgrounds, being taught by more occupationally experienced teachers in more prosperous and positive socio-affective learning environments are estimated to have higher academic achievement. Ultimately, more multilevel modelling studies utilizing a longitudinal design, involving earlier grades of primary schooling, and focusing also on other subject-specific performances rather than reading and/or mathematics are needed. Yet, these findings potentially act as a scientific guideline for researchers.

References
Boudon, R. (1974). Education, Opportunity and Social Inequality: Changing Prospects in Western Society. Wiley.

Bourdieu, P. (1986). The forms of capital. In J. G. Richardson (Ed.), Handbook of theory and research for the sociology of education (pp. 241–258). Greenwood Press.

Bourdieu, P., & Passeron, J.-C. (1977). Reproduction in education, society and culture (3. pr). Sage.

Burušić, J., Babarović, T., & Velić, M. Š. (2016). School Effectiveness: An Overview of Conceptual, Methodological and Empirical Foundations. In N. Alfirević, J. Burušić, J. Pavičić, & R. Relja (Eds.), School Effectiveness and Educational Management: Towards a South-Eastern Europe Research and Public Policy Agenda (pp. 5–26). Springer International Publishing. https://doi.org/10.1007/978-3-319-29880-1_2

Creemers, B. P. M., & Kyriakidēs, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. Routledge.

Coleman, J. S., Campbell, E. A., Hobson, C., McPartland, J., Mood, A., Weinfeld, F., & York, R. (1966). Equality of educational opportunity. Washington, DC: U.S. Government Printing.

de Leeuw, J., & Meijer, E. (2008). Introduction to Multilevel Analysis. In J. de Leeuw & E. Meijer (Eds.), Handbook of Multilevel Analysis (pp. 1–75). Springer New York. https://doi.org/10.1007/978-0-387-73186-5_1

Diprete, T. A., & Eirich, G. M. (2006). Cumulative advantage as a mechanism for inequality: A review of theoretical and empirical developments. Annual Review of Sociology, 32, 271–297. https://doi.org/10.1146/annurev.soc.32.061604.123127

Hattie, J. (2008). Visible Learning. In Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Routledge. https://doi.org/10.4324/9780203887332

Hox, J., Moerbeek, M., & van de Schoot, R. (2018). Multilevel Analysis: Techniques and Applications (3, Ed.). Routledge.

McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 276–282. https://doi.org/10.11613/BM.2012.031

Merton, R. K. (1968). The Matthew Effect in Science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. https://doi.org/10.1126/science.159.3810.56

Scheerens, J. (2016). Educational Effectiveness and Ineffectiveness. In Educational Effectiveness and Ineffectiveness: A Critical Review of the Knowledge Base. Springer Netherlands. https://doi.org/10.1007/978-94-017-7459-8
 
5:15pm - 6:45pm11 SES 08 A: Developing Learners' Skills and Competences in Secondary Schools
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Buratin Khampirat
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Socio-Educational Action with Kinship Foster Care: Qualitative Analysis to Improve Quality from the Perspective of Social Educators

Tania García-Bermejo1, Juan Luis Fuentes2

1Universidad Nacional de Educación a Distancia, Spain; 2Universidad Complutense de Madrid, Spain

Presenting Author: Fuentes, Juan Luis

Kinship foster care is a measure for the protection of children and adolescents in which caregivers are direct relatives of the foster children. At present, it constitutes one of the measures most implemented and promoted by European countries (Lerch & Nordenmark, 2019; Sacur & Diogo, 2021).

Current research on kinship foster care families identify different problems. One of the most common is the intergenerational difference between children and caregivers (Del Valle et al., 2002; Montserrat, 2006), because most of these cases are formalized with grandparents (Molero et al., 2007; Montserrat, 2006; Wellard et al., 2017). This difference sometimes produces misunderstandings in families, which is frequently a source of conflict. Moreover, the age differences suppose a challenge for the physical capacity of the caregivers, which, sometimes, can hinder the correct care of children, considering that require a vitality in caregivers that the process of aging decreases considerably.

Considering academic aspects, it has been found that kinship foster care families obtain worse results than the average of the students (Jiménez & Palacios, 2008; Molero et al., 2007; Palacios & Jiménez, 2007), and that more than half are in courses lower than that which would correspond to them by age (Montserrat et al., 2013).

Difficulties that caregivers face when setting rules and limits for children are also highlighted (Molero et al., 2007; Palacios & Jiménez, 2007). Their style of authority is generally permissive, which is reactively changed to an authoritarian model when children become adolescents, without obtaining positive results. Furthermore, children do not understand this change in attitude in their caregivers, demanding a clear and coherent establishment of educational norms (Mateos et al., 2012).

Conflictive relationships between caregivers and parents are also noted, especially due to difficulties derived from the visits of parents with their children (Chateauneuf et al., 2017; Delgado et al., 2019), as well as health problems of adults belonging to these families (Del Valle et al., 2008; Mateos et al., 2012; Taylor & McQuillan, 2014).

Another significant problem is related to the construction of the identity and the elaboration of life history of children in foster care (García-Bermejo, 2017; Laklija, 2011; Mateos et al., 2012). It points out both difficulties of children to ask about their life trajectory, in many cases motivated by traumatic situations, and by the caregivers to facilitate the necessary transition in vital processes of children.

Along with the common problems of kinship foster care, an aspect linked to the quality of this educational intervention, are the theoretical models that support and give meaning to the used socio-educational methodologies. In this sense, there are currently different models that could answer to the specific structure and social circumstances of these families. Among them, the systemic model has acquired great relevance (Laszloffy, 2004; O´Hanlon & Beadle, 1996; Partridge et al., 2019), as well as others like mixed intervention model, cognitive-behavioral model or multifamily model, whose contributions deserve to be considered (Melendro, 2014).

However, previous research focus on the families, but not on the professionals who work with them, acquiring so a detailed description of its configuration, but without the pertinent perspective of educators. Hence, the research question of this study is: what educators can tell us, taking into account their own professional experience, to improve the quality of intervention with kinship foster care? As a consequence of that, the aim of this paper is to analyze the perceptions that social educators, who work with kinship foster care, have about the main difficulties presented by this kind of families, the most frequent demands, the key factors that promote a quality attention, and the most used theoretical model.


Methodology, Methods, Research Instruments or Sources Used
In order to respond to the objective of our research, we have opted for a qualitative design, which has been carried out with an ad hoc questionnaire. The instrument consists of two parts: the first asks about sociodemographic data (gender, age, professional experience with kinship foster care and complementary training), and the second include open questions about: frequent problems in kinship foster care families; common demands made by caregivers, children and parents to professionals; key factors to achieve a quality intervention and the theoretical model they consider most effective. This questionnaire was validated by seven experts from different universities and practitioners, and a pilot test was applied to a several educators with different qualifications, who currently work or had worked with kinship foster care.
The study population was all the educators of the Child Care Centers of Public Social Services at the City of Madrid, and the sample was 86.54% of such population. Data was analyzed with Atlas.ti software.
Information was coded in such a way that it would allow a more accurate analysis (Massot et al., 2004), systematizing and homogenizing terms of each of the responses of social educators. This categorization process was carried out in an inductive way (Rodríguez et al., 2005), that is, from reading the answers to the questionnaires and not previously defining them. To be as objective as possible and to provide credibility and validity to the study, a triangulation of researchers was also developed carried out (Flick, 2014).

Conclusions, Expected Outcomes or Findings
Results show that 88.9% of educators are women between 36 and 45 years old. In 75.6% of cases, they have been working with kinship foster care for an average of more than 8 years. 64.44% and 37.77% have expert training or a master's degree in systemic family therapy and family mediation, respectively.

The two main problems that educators identify in kinship foster care are: difficulties in the elaboration of life histories of children (51.11%), followed by the conflictive relationships between caregivers and parents (37.78%).

The most important demands that children make to the professionals are: to maintain a closer relationship with their parents, with 37.78% of educators pointing it out; and to know their own life history, with 35.56%. Caregivers demand more support in the education of adolescents (44.44%), and more financial aids (37.78%). Parents demand to increase visits with their children (51.11%), followed significantly very far by the aim of family reunification (17.78%).

Educators emphasize that creation of an educational alliance with caregivers and children is what promotes a successful intervention with them, while with parents they consider that it is necessary to give them a proper space in the intervention. Additionally, they state that the work of elaborating the life history with the shelters is essential for a successful intervention (84.4%).

Finally, when asked to indicate the preferred theoretical model, they answer that the systemic model is the most appropriate for this kind of situations (48.88%); a mixed intervention model: systemic, cognitive-behavioral and psychoanalytic is a second option between educators (11.11%); and the cognitive-behavioral model (4.44%) or the multifamily model are stressed just by a few of the participants.

References
Balsells Bailon, M. Á., Urrea Monclús, A., Ponce Alifonso, C., Vaquero Tió, E. y Navajas Hurtado, A. (2019). Claves de acción socioeducativa para promover la participación de las familias en procesos de acogimiento. Educación XX1, 22(1), 401-423, https://doi.org/10.5944/educxx1.21501

Chateauneuf, D., Turcotte, D. y Drapeau, S. (2017). The relationship between foster care families and birth families in a child welfare context: The determining factors. Child & Family Social Work, 23(1), 71-79.

Delgado, P., Pinto, V. S., Carvalho, J. y Gilligan, R. (2019). Family contact in foster care in Portugal. The views of children in foster care and other key actors. Child & Family Social Work, 24(1), 98-105.

García-Bermejo, T. (2020). Reflexión teórica para la orientación psicopedagógica en los acogimientos en familia extensa. Revista Española de Orientación y Psicopedagogía, 31(2), 19-28.

Laklija, M. (2011). Foster care models in Europe: results of a conducted survey. Forum. https://www.udomiteljizadjecu.hr/wp-content/uploads/documents/publikacije/Foster%20care%20models%20in%20Europe%20-%20results%20of%20a%20conducted%20survey%20-%20Maja%20Laklija.pdf

Laszloffy, T. A. (2004). Rethinking Family Development Theory: Teaching with the Systemic Family Development (SFD) Model. Family Relations, 51(3), 206-214.

Lerch, V. y Nordenmark, A. (2019). Feasibility Study for a Child Guarantee Target Group Discussion Paper on Children in Alternative Care. European Commission.

O’Hanlon B. and Beadle S. (1996). A Field Guide to PossibilityLand: possibility therapy methods. BT Press.

Melendro, M. (Coord.) (2014). Estrategias eficaces de intervención socioeducativa con adolescentes en riesgo de exclusión. UNED - Universidad Nacional de Educación a Distancia.

Partridge, K., Dugmore, P., Mahaffey, H., Chidgey, M. y Owen, J. (2019). ‘Step by step, side by side’: the quest to create relational artistry through systemic practice within children’s social care. Journal of Family Therapy, 41(3), 321-342.

Pérez Serrano, G., Poza Vilches, F. y Fernández García, A. (2016). Criterios para una intervención de calidad con jóvenes en dificultad social. Revista Española de Pedagogía, 63(1), 51-69.

Sacur, B. M. y Diogo, E. (2021). The EU Strategy on the Rights of the Child and the European Child Guarantee—Evidence-Based Recommendations for Alternative Care. Children, 8, 1181. https://www.mdpi.com/2227-9067/8/12/1181

Taylor, B. J. & McQuillan, K. (2014). Perspectives of Foster Parents and Social Workers on Foster Placement Disruption. Child Care in Practice, 20(2), 232-249. https://doi.org/10.1080/13575279.2013.859567

Wellard, S., Meakings, S., Farmer, E. y Hunt, J. (2017). Growing up in Kinship Care: Experiences as Adolescents and Outcomes in Young Adulthood. Grandparents Plus.


11. Educational Improvement and Quality Assurance
Paper

The Influence of Рhenomenal-Оriented Learning on the Solution of Educational Problems of Secondary School Students

Svetlana Popkova1, Gulmira Yestepbergenova2, Sairangul Sembayeva3

1Nazarbayev Intellectual School, Kazakhstan; 2Nazarbayev Intellectual School, Kazakhstan; 3Nazarbayev Intellectual School, Kazakhstan

Presenting Author: Popkova, Svetlana; Yestepbergenova, Gulmira

The purpose of the study is to investigate the influence of phenomenally–oriented learning on studying a variety of topics in secondary school students. It was important for us to conduct this study to ensure the reliability of the method used. The question of our research is to what extent phenomenally oriented learning, within the framework of interdisciplinary interaction, influences the solution of various educational problems in secondary school students in the 7th grade.

Hypotheses put forward:

A) The study of phenomena in the classroom at school can contribute to solving a variety of educational problems.

C) The use of phenomenally oriented learning has a positive effect on the development of meta-subject skills among students in secondary school, in grade 7.

This is a study among a small group of 24 students conducted during September - November 2022 by students at the intellectual school of Almaty.

Evaluation method. The results pre- and post-testing were analyzed through a comparative method. The metacognitive skills of students were evaluated when solving problems:

  • self-regulation,
  • internal motivation,
  • argumentation,
  • interpretation.

The tests conducted before and after the study showed positive dynamics, and many students improved their results. The study showed that phenomenon-based learning had provided a more meaningful learning experience by actively engaging learners to solve problems that have a real-world context. It also helps learners to make connections across different disciplines. The results of the study show positive dynamics of changes in the level of self-regulation, internal motivation, argumentation, and interpretation of students.

When conducting the study, we considered how the authenticity of various educational problems of modern society can be attractive to students if you think about how to make the learning process such that it does not lead to entertainment and only superficial experiences. An important part of a teacher's job is to create positive cognitive dissonance through competing worldviews/roles [1]. Considering the solution of educational problems by students as an opportunity to study the features of the surrounding world from different points of view, it becomes clear the rationale for the use of phenomenally oriented teaching in school lessons, in interdisciplinary interaction, from the perspective of several subject contents that differ from each other [2].

Education in our school begins with the 7th grade. Students come to our school with different background knowledge, from different educational institutions of the city, region, and republic, with different features of the curricula. We noticed that students of grades 7 unproductively use the subject knowledge and skills acquired in various subjects, and as a result, they cannot offer solutions to any educational problem. Students have poorly developed self-regulation, low internal motivation, ineffective argumentation and practically no interpretation. Many scientists and educational researchers have emphasized the role of metacognitive processes in the regulation of cognitive activity, in particular intellectual activity. According to John Flavell, the concept of "metacognition" is defined as "the ability to analyze one's own thinking strategies and manage one's cognitive activity" [3].

According to A. Brown's model of metacognition, the regulation of cognition, as a category of metacognition, includes planning in the form of choosing a strategy for solving educational tasks [4]. The role of metacognitive processes is considered very important in the educational activities of students, since they understand and evaluate their own learning process, considering their own solutions to educational tasks and problems [5]. Phenomenally–oriented learning is just the interdisciplinary approach to learning that we need, which is student–oriented and based on research and problem solving. The purpose of phenomenon-based learning is to prepare students to solve problems related to real life [6].


Methodology, Methods, Research Instruments or Sources Used
Phenomenon-based learning is a learner-centred, multidisciplinary instructional approach that is based on student inquiry and problem solving [7]. Understanding various phenomena can become an important component of the learning process. The phenomena that teachers choose and include in their practice, linking science with life, should be attractive in their diversity for students and based on their life experience [8].
To understand how phenomenally oriented learning can help Grade 7 students solve educational problems related to real life, we have identified the following methods:
● situational.
● partially - search engine.
Emotional saturation through immersion in the situation of the problem will motivate students to cognitive activity to solve problems [9]. To determine the level of students' metacognitive skills in the form of self–regulation, internal motivation, argumentation, and interpretation, we conducted two tests with students - preliminary and subsequent.
The study consists of 3 stages:
● Pretest.
● The study of the phenomenon of "Water" in the form of a problem.
● Post - test.

Pre-test. Students were asked to answer preliminary questions that determine the level of self-regulation, internal motivation, argumentation, and interpretation. The questionnaire consists of 24 questions. As survey tools, the questions of Glazer's and T.D. Dudovskiy’s tests "Identification of the orientation and level of internal motivation of students' learning activities" were used selectively. During the pre-test, we did not offer students to study information about the phenomenon of "Water" yet.
The study of the phenomenon of "Water" in the form of a problem. After the testing, a multi-subject block was held for students, consisting of 4 lessons in the subjects: Biology, Physical education, Art, and English. All four lessons were aimed at studying the phenomenon of "Water", from various subject positions:
The Biology lesson began the study of the phenomenon, and it considered the properties, functions, and qualities of water.
The lesson of Physical culture considered the ability of water to transport harmful substances from the human body, with increased physical exertion, along with sweat.
At the English lesson, students analyzed several quotes about water and studied the physical and chemical properties of water.
The art lesson considered the creation of mock-ups of robots, considering their functional purposes when storing and delivering clean drinking water at school.
Post – test. When summing up the results of changes, we noted different outcomes from each other of the levels of self-regulation, internal motivation, argumentation, and interpretation available to students.



Conclusions, Expected Outcomes or Findings
At the end of the study, we obtained positive comparative results, the level of metacognitive skills in grade 7 students increased after using the situational method, in the form of immersion in a problematic situation to study the phenomenon of water, also after using a partially search method, in the form of independent work on the topic of the phenomenon.
Questions to determine the level of self-regulation were aimed at the existing skills of students planning their working time. The questions to determine the level of argumentation were aimed at measuring the skills to confirm and prove one's opinion and the various ideas that arise, in the form of answers to questions. The questions on the definition of interpretation demonstrated the existing level of developed skills to interpret the various information received into the knowledge necessary to solve the problem. In this area, the application of existing subject knowledge, skills and life experience, when answering questions. Questions to determine internal motivation were aimed at the desire to actively search for the necessary information to do work chores or assignments in the classroom.
Thus, the post - test showed that out of 24, four students did not show a positive trend in the level of self-regulation, and the remaining 22 students have a positive change. It should be noted that these four students showed positive results in measuring the level of internal motivation. The level of interpretation has increased in comparison with pre - test. In 8 students, the level of argumentation changed in a positive way during the post - test, compared with the pre - test.
We intend to continue researching the impact of phenomenally oriented learning in the future, among students of several schools of the same age.


References
Ермилова Е.В., Попкова С.В., Мейрханова З.Д., Сембаева С.К., Методические рекомендации «Мастерская Web Qwest» для учителей НИШ, рекомендованное Научно-методическим советом Центра педагогического мастерства АОО, Назарбаев Интеллектуальные Школы», г. Нур-Султан, 2021 г., стр. 8.
Мейрханова З.Д. Ермилова Е.В. Попкова С.В., Методическое пособие «Проблемно – ориентированное обучение на уроках искусства» для учителей НИШ, рекомендованное Научно-методическим советом Центра педагогического мастерства АОО, Назарбаев Интеллектуальные Школы», г. Нур-Султан, 2020 г., стр. 4 – 5.
Flavell J.H. Metacognitive aspects of problem solv ing // The nature of intelligence. Hillsdale / Ed. by L.B. Resnick. – N.Y., 1976.
Brown, A. L. (1978). Knowing When, Where, and How to Remember: A Problem of Metacognition. Advances in Instructional Psychology, 1, 77-165.
Hartman H.J. Metacognition in Learning and Instruction: Theory, Research, and Practice, Kluwer Academic  Publishers,  Dordrecht,  The  Netherlands, 2001.
https://www.valamis.com/hub/phenomenon-based-learning - электронный ресурс, время обращения – 21.11.2022
https://www.nextgenscience.org/ - Next generation science Standards - электронный ресурс, время обращения 10.09.22
Ли, О. (2020). Делать повседневные явления феноменальными. Наука и дети, 58 (1), 56-61.
Cabrera, M. A. M., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1‐2), 103.


11. Educational Improvement and Quality Assurance
Paper

Supporting Second Language Learners in Science Education: Insights from a Science and Literacy Teaching Project

Sofie Johansson1, Clas Olander2

1University of Gothenburg, Sweden; 2University of Malmö, Sweden

Presenting Author: Johansson, Sofie

Due to the recent surge of immigration to Sweden, it is estimated that approximately 25% of students aged 7-19 are second language learners. This heterogeneous group of students poses a challenge for teachers across all subjects, due to diverse individual differences, varying levels of language proficiency and the unique linguistic characteristics of each subject.

Enhancing science literacy can be facilitated by incorporating a focus on comprehending words in context, as proposed by Logan and Kieffer (2021). Furthermore, functional science literacy, as defined by Norris and Phillips (2003), encompasses not only an understanding of science terms, but also the ability to effectively communicate, read, and write using these terms in non-technical settings. The project, Science and Literacy Teaching (SALT), funded by the Swedish Research Council, attempted to identify the best strategies for supporting science teachers in working with second language learners.


Methodology, Methods, Research Instruments or Sources Used
A model of word types was constructed, drawing upon the categorization of words into everyday common words, academic words, and scientific words, as proposed by scholars such as Nation (2013), Hulstijn (2012) among others.
In order to assess the vocabulary difficulties faced by both first language (L1) and second language (L2) students, a sample of 232 participants was administered vocabulary tests that consisted of seven types of words. Three of these word types were neutral in terms of subject matter, while the other four were subject-specific and drawn from textbook texts in the fields of biology, physics, and chemistry. The tests were designed to examine students' receptive knowledge of the words, including their forms, meanings, and usage in authentic contexts. The tests involved matching test words with their correct meanings or synonyms in order to evaluate the depth of students' knowledge.

Conclusions, Expected Outcomes or Findings
We found that there are four types of word groups that require special attention from teachers. In summary, L2-students had particular challenges with general language comprehension, such as academic words and homonymous words with subject-specific meanings. These types of words were not difficult for L1 students. Finally, all students had difficulty with homonymous words with academic and subject-specific meanings, as well as general subject-related words that were new to all students.

This research provides insights into the challenges that second language learners face in science education, and the findings can inform the development of pedagogical strategies to support their learning. By addressing these challenges, the research makes a contribution to the field of multilingualism by improving our understanding of how to support second language learners in acquiring science literacy.

References
Hulstijn, J.H. (2012). The construct of language proficiency in the study of bilingualism from a cognitive perspective. Bilingualism, 15(2), Cambridge.
Logan, J. K., & Kieffer, M. J. (2021). Investigating the longitudinal and concurrent relationships between polysemous word knowledge and reading comprehension among Spanish-English bilingual middle school students. Reading and Writing, 34, 301-335.
Nation I.S.P. (2013), Learning Vocabulary in Another Language. Cambridge. Applied Linguistics. Cambridge University Press.
Norris, S. P., & Phillips, L. M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science education, 87(2), 224-240.
 
Date: Thursday, 24/Aug/2023
9:00am - 10:30am11 SES 09 A: School Performance and Quality Models
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Andra Fernate
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

The school improvement wheel: Shaping Transitional Processes in Dynamic Societies

Frank Brückel1, Heike Beuschlein1, Rachel Guerra2, Reto Kuster1, Susanna Larcher1, Regula Spirig1

1Pädagogische Hochschule Zürich, Switzerland; 2Schulamt Fürstentum Liechtenstein, Liechtenstein

Presenting Author: Brückel, Frank; Beuschlein, Heike

Remark: An earlier version of the project has been presented two years ago. We previously presented preliminary results of this study.

Our rapidly changing society influences the work of schools. Today, public schools in different countries are under enormous pressure to adapt and change as they need to to deal with technological improvements and cultural changes that leads to the increase heterogenization of societies (cf. e.g. OECD 2019, 2016; Imlig, Lehmann & Manz 2018; European Union 2016; Altrichter & Maag-Merki 2016).

In spite of the abundant literature about the conceptual and practical notions and processes of educational change, there is still a need to learn more about the ways to conceptualize and put into practice transitional change processes in individual schools and how to conceptualize and put into practice transitional change in individual schools and school district (Steffens, Heinrich, Dobbelstein, 2016; Reinbach 2016). Transitional change processes refer to the steps and strategies that a district takes to implement significant changes in its operations and policies such as curriculum reform, or the implementation of new technologies. Transitional change processes can be complex and multi-faceted, involving a range of different stakeholders, such as students, teachers, administrators, and community members.

For principals, teacher leaders and project managers in charge of implementation processes at different levels, these accelerated changes raise the question about how to bring successful and sustainable implementation that is accepted and carried through by all those involved?

Many leaders have assumed and taken on willingly the challenge of implementing numerous, simultaneous and, in part, highly complex transitional processes. They are often confronted with scepticism, reserve, doubt and even outright boycott, often by politicians, staff and parents (Rasmussen 2017; Landert 2014).

This is where the project Shaping Transitional Processes in Dynamic Societies comes in. This project was conducted by a research group of the Zurich University of Teacher Education and the education authority of the Principality of Liechtenstein. The project aims at developing relevant and helpful support services and practices for school principals and project leaders with which current and future reforms and improvements can be mastered alongside the demanding daily school routine.

In order to answer the questions and to advance the research process of the project as transparently and comprehensibly as possible, the approach of Design Based Research (DBR) was used (Euler and Sloane 2014; Euler 2017). Due to its methodological design, the chosen approach opens up the possibility to think and design practical school improvement processes in complementarity with school improvement research (Rau, Gerber u. Grell 2022, 353) and thus to consider both the relevance for practice and the further development of theoretical findings.


Methodology, Methods, Research Instruments or Sources Used
The project follows an approach by Euler and Sloane (2014), which provides five different phases and is guided by the following design principles: The research started with a literature review on school improvement followed by interviews with principals, teachers and members of school authorities. Af-terwards the results were then triangulated: Results compared, terminology reviewed, sharpened and condensed, redundancies cleaned up, individual factors combined and subcategories formulated. A challenge was to understand the statements beyond the cultural context, to assign them to the factors accordingly, to compare and superimpose English and German terminology in order to gain a deeper understanding of how changes can be mastered successfully. At the end of this work there were sev-en factors with their sub-factors.
After finishing the literature review, the evaluation of the interviews and triangulation of the results, a prototype was developed. For the design of this prototype it was important that the factors can be presented and described so that
- the entire spectrum of factors important to school improvement are represented.
- the description is comprehensible and accessible without diminishing complexities,
- it supports principals and project leaders in constructively coping with transitional processes in their own schools
- it supports principals and project leaders in competently initiating transitional processes in their schools, to recognise snags early on and handle them constructively.
At the end of this step, a first prototype of a model was available at the end of 2020 (Brückel, Kuster et al. 2022).
To test whether the model and the material are relevant to stakeholders, dialogue workshops (Brück-el, Larcher et al. 2019; Bohm 2008) with school and project leaders (20 to 25 participants) were held.
The aim of the workshops was to test
- how target groups receive the model and the material (f.eg. are they using the framework to manage their change projects?),
- whether they are considered helpful and goal-oriented,
- what could be missing, and
- how the material can be supplemented and improved.
Each workshop was divided into two phases: in phase 1, each individual examines the model and the material for him or herself; in phase 2, the model’s and material’s practical sustainability was dis-cussed in a group. A diversified group is critical to these workshops to ensure that all relevant perspectives are considered. Finally, the result is a science-based model that meets the demands of practice.

Conclusions, Expected Outcomes or Findings
The paper presents the project and its objectives, shows how the design-based research process was conducted, and discusses the model and the school improvement factors that emerged from research as being salient. The results show that there are certain school improvement factors which are rele-vant and consistent in any change process. These factors can be determined beyond a national con-text and are not culturally driven: Learning of students, mindset, communication and cooperation, induvial and organisational competences, framework conditions, process design, multi-level school system,+ leadership and dynamics.
Some limitations became apparent during the research process for example, there is no consistent wording for school development, school improvement or educational change. In consequence a litera-ture review must address the question of which papers are considered. Due to a lack of resources, only a limited number of interviews were analyzed.
And finally the question of how to translate research results into a model which is relevant for princi-pals and project leaders has to be discussed critically.

References
Altrichter Herbert & Maag Merki Katharina (2016). Handbuch Neue Steuerung im Schulsystem. Wies-baden: Springer VS.
Bohm David (2008). Der Dialog. Das offene Gespräch am Ende der Diskussionen. Stuttgart: Klett-Cotta.
Brückel, Frank, Larcher, Susanna, Kuster, Reto, Spirig Regula, Guerra, Rachel, Annen, Luzia (2022). Das Schulentwicklungsrad. Eine Reflexionshilfe für die Führung schulischer Veränderungsprozes-se. #schuleverantworten 2022_2, 46–56, https://doi.org/10.53349/sv.2022.i2.a199
Brückel Frank, Larcher Susanna, Annen Luzia, Kuster Reto (2019). Entwicklung von praxisnahen Arbeitsmaterialien im Kontext Tagesschule/ Tagesstrukturen. In Sabine Maschke, Gunhild Schulz-Gade, Ludig Stecher (Hrsg.), Jahrbuch Ganztagsschule, Frankfurt: Debus Verlag, S. 212-228
European Union (2016). Smarter, greener, more inclusive? Indicators to support the Europe 2020 strategy. Luxembourg: Publications Office of the European Union.
Imlig, Flavian, Lehmann, Lukas & Manz, Karin (Hrsg.) (2018). Schule und Reform. Veränderungsab-sichten, Wandel und Folgeprobleme. Wiesbaden: Springer VS.
Landert, Charles (2014). Die Berufszufriedenheit der Deutschschweizer Lehrerinnen und Lehrer. Be-richt zur vierten Studie des Dachverbandes Lehrerinnen und Lehrer Schweiz (LCH). Zugriff unter http://www.lch.ch/fileadmin/files/documents/Medienmitteilungen/141209_MK_Berufszufriedenheitsstudie_Berufsauftrag/141209_05_Studie_Charles_Landert_zur_Berufszufriedenheit.pdf  [08.05.2015].
OECD (2016). Trends shaping education. Paris: OECD Publishing.
OECD (2019). OECD Future of Education and Skills 2030 Concept. http://www.oecd.org/education/2030-project/teaching-and-learning/learning/learning-compass-2030/OECD_Learning_Compass_2030_concept_note.pdf [30.12.2020]  
Rasmussen, Jens (2017). When Constructions of the Future Meet Curriculum Development and Teaching Practice. Hauptvortrag am ECER Kongress Copenhagen, 23.08.2017.
Reinbacher Paul (2016). Ein theoretischer Bezugsrahmen für "Schulentwicklung". In Schweizerische Zeitschrift für Bildungswissenschaften 38 (2016) 2, S. 295-318.
Steffens Ulrich, Heinrich Martin & Dobbelstein Peter (2019). Praxistransfer Schul- und Unterrichtsfor-schung – eine Problemskizze. In Claudia Schreiner, Christian Wiesner, Simone Breit, Peter Dob-belstein, Martin Heinrich & Ulrich Steffens (Hrsg.), Praxistransfer Schul- und Unterrichtsentwick-lung, S. 11 – 26


11. Educational Improvement and Quality Assurance
Paper

Making Sense of School Performance Feedback: Which Attributions Do Teachers and School Leaders Make?

Evelyn Goffin1,2, Rianne Janssen2, Jan Vanhoof1

1University of Antwerp, Belgium; 2KU Leuven, Belgium

Presenting Author: Goffin, Evelyn; Vanhoof, Jan

School performance feedback (SPF) systems present educational professionals with student achievement data in order to support self-evaluation and data-based decision making (Schildkamp & Teddlie, 2008; Visscher & Coe, 2003). However, in order to arrive at (formative) conclusions based on SPF, recipients need to make sense of the (summative) data they are presented with (Schildkamp, 2019; van der Kleij et al., 2015). Attribution, or reflecting on the causes of (learning) outcomes, is an integral part of this sensemaking process (Coburn & Turner, 2011). In line with the basic propositions of attribution theory (Weiner, 1985, 2010), the nature of educators’ causal explanations for student outcomes has been found to affect their emotions and subsequent (instructional) behavior (Wang & Hall, 2018).

Research finds that educational professionals often struggle to pinpoint factors that (may) have led to certain outcomes, particularly when those outcomes are unfavorable (Verhaeghe et al., 2010). Moreover, in defiance of ideals relating to data-based decision making, student performance is often attributed to external causes such as student characteristics, rather than matters internal to educational professionals, such as instruction (Evans et al., 2019; Schildkamp et al., 2016). This is especially apparent in cases of student failure (Wang & Hall, 2018). Consequently, it can be difficult to formulate productive decisions and constructive actions based on SPF (Schildkamp et al., 2016).

In the present study, we examine educational professionals’ causal explanations for results presented in a SPF report from a low-stakes national assessment (NA) in Flanders, Belgium. Like typical external standardized assessments, the Flemish NA relate school performance to standards and to the performance of reference groups (AERA et al., 2014; Visscher & Coe, 2003). We investigate educational professionals’ attributions of these data, with a particular interest in the locus of causality of the attributions they make. To what extent is the SPF interpreted introspectively (i.e., with regard to aspects of school policy and instructional practice that can be improved or sustained) and to what extent is school performance ascribed to external factors (such as aspects of the test itself, or input from students)?

Our review of the literature suggests that perceptions of school leaders remain somewhat underexposed in studies on attribution in educational data use. However, SPF intends to inform both school policy and instructional practice. Consequently, we will not only focus on teachers’ attributions in the present study, but also on causal explanations made by school leaders. Furthermore, we will examine causal explanations for both outcomes perceived as favorable, and those perceived as unfavorable. Perhaps in line with the very term “diagnosis”, we find that the attributions and attributional processes discussed in empirical literature are predominantly focused on explanations for student failure (van Gasse & Mol, 2021; Verhaeghe et al., 2010) and not so much for student success. However, school improvement is not only a narrative of identifying problems, but also of fostering what works.

In summary, the three research questions we address, are:

RQ1 To which internal and external factors do teachers and school leaders attribute their school’s performance on a national assessment?

RQ2 Do attributions differ according to the attributor’s work role?

RQ3 Do attributions differ according to the perceived favorability of the result?

We adopt a qualitative approach and make use of authentic educational data, because our aim is to illuminate how individuals and groups make meaning of something they experience (here, their schools’ SPF) from their own perspectives (Savin-Baden & Major, 2013).


Methodology, Methods, Research Instruments or Sources Used
Data were collected in Flanders, the Dutch-speaking region of Belgium. Periodically, national assessments (NA) are organized in order to monitor whether attainment targets are met on system-level and in order to explore whether school-, class- of student-level variables explain differences in achievement. These NA are conducted in representative samples of schools, who afterwards receive a personalized SPF report. School results are never publicized, nor do outcomes carry any formal consequences for participants.
Participants for the present study were recruited from the Flemish primary schools that had taken part in the May 2019 NA of People and Society (formerly a subdomain of the World Studies curriculum) in the sixth grade. In pursuit of maximum variation (Savin-Baden & Major, 2013), schools were categorized into four profiles based on aspects of their criterion-referenced and norm-referenced results on one focal test: Spatial use, traffic and mobility. Approximately one week after having received the SPF report, in the autumn of 2020, a random selection of schools within each profile was approached.
In total, semi-structured interviews were conducted with 22 participants (11 school leaders and 11 sixth grade teachers) from 13 schools. The interview protocol included open-ended questions about participants’ appraisal of the schools’ results and about how they causally explained these results. These questions followed a think-aloud section in which participants were asked to describe and interpret the tables and graphs in their schools’ SPF report. Due to societal restrictions relating to the COVID19-pandemic, the interviews were conducted online using video-conferencing software. Recordings were transcribed verbatim.
Data were coded with NVivo. Framework analysis served as an overall analytical method, as it is fit to both organize and interpret data, allows for a combination of inductive and deductive techniques, and facilitates the development of matrices to condense findings and explore patterns (Gale et al., 2013). In order to identify trends, the qualitative interview data were also ‘quantitized’ (Sandelowski et al., 2009).

Conclusions, Expected Outcomes or Findings
Participants attribute their schools’ performance to a wide array of factors on the level of the school, the classroom, the student, and the assessment itself. School-level factors and class-level factors can be categorized as internal or external, based on the source of the attribution. We see, for instance, that school leaders reflect on factors such as teacher professionalism: a factor internal to teachers but external to themselves. Overall, school-level factors (such as the curricular line for the subject that was tested) and student-level factors (such as pupils’ cognitive capacity) are invoked most frequently, especially by school leaders and teachers, respectively.
Throughout the dataset, external attributions dominate. We might relate this to educational professionals' professional attitude: to where does my responsibility for student outcomes extend? Overall, we find few differences in attributional patterns between participants from schools that scored well and from those that scored poorer. However, reservations and concerns about (the design and the conditions of) the assessment – an external factor – are uttered primarily to explain negative results.
Most participants mention a whole range of factors when making causal ascriptions for their schools’ results in the SPF report. This suggests that educational professionals acknowledge that learning outcomes are the product of different building blocks. However, it also establishes why it is not easy or straightforward to formulate an unambiguous analysis and an actionable diagnosis based on SPF. Finally, the finding that teachers and school leaders (even within schools) emphasize different factors to interpret the (same) outcomes, illustrates the importance of collective sensemaking in order to piece together a complete story.
These insights are relevant not only for research on data-informed decision-making in schools, but also for educational professionals themselves, as well as for those who train them, and for those who design, offer and mandate assessments.

References
AERA, APA, & NCME. (2014). Standards for Educational and Psychological Testing. American Educational Research Association.
Coburn, C. E., & Turner, E. O. (2011). Research on Data Use: A Framework and Analysis. Measurement: Interdisciplinary Research & Perspective, 9(4), 173–206.
Evans, M., Teasdale, R. M., Gannon-Slater, N., Londe, P. G. la, Crenshaw, H. L., Greene, J. C., & Schwandt, T. A. (2019). How Did that Happen? Teachers’ Explanations for Low Test Scores. Teachers College Record: The Voice of Scholarship in Education, 121(2), 1–40.
Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 117.
Sandelowski, M., Voils, C. I., & Knafl, G. (2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208–222.
Savin-Baden, M., & Major, C. H. (2013). Qualitative research: The essential guide to theory and practice. Routledge.
Schildkamp, K. (2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61(3), 257–273.
Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016). Data teams for school improvement. School Effectiveness and School Improvement, 27(2), 228–254.
Schildkamp, K., & Teddlie, C. (2008). School performance feedback systems in the USA and in The Netherlands: a comparison. Educational Research and Evaluation, 14(3), 255–282.
van der Kleij, F. M., Vermeulen, J. A., Schildkamp, K., & Eggen, T. J. H. M. (2015). Integrating data-based decision making, Assessment for Learning and diagnostic testing in formative assessment. Assessment in Education: Principles, Policy & Practice, 22(3), 324–343.
Van Gasse, R., & Mol, M. (2021). Student guidance decisions at team meetings: do teachers use data for rational decision making? Studia Paedagogica, 26(4), 99–117.
Verhaeghe, G., Vanhoof, J., Valcke, M., & van Petegem, P. (2010). Using school performance feedback: perceptions of primary school principals. School Effectiveness and School Improvement, 21(2), 167–188.
Visscher, A. J., & Coe, R. (2003). School Performance Feedback Systems: Conceptualisation, Analysis, and Reflection. School Effectiveness and School Improvement, 14(3), 321–349.
Wang, H., & Hall, N. C. (2018). A Systematic Review of Teachers’ Causal Attributions: Prevalence, Correlates, and Consequences. Frontiers in Psychology, 9(DEC), 1–22.
Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92(4), 548–573.
Weiner, B. (2010). The Development of an Attribution-Based Theory of Motivation: A History of Ideas. Educational Psychologist, 45(1), 28–36.


11. Educational Improvement and Quality Assurance
Paper

Validating the Kazakhstani Teacher Observational Protocol: A Pilot Study

Janet Helmer1, Matthew Courtney1, Bridget Goodman1, Kathy Malone2, Kulzhan Beysembayeva3, Filiz Polat1

1Nazarbayev University, Kazakhstan; 2Univeristy of Hawaii, USA; 3Eurasian National Uniiiversity, Kazakhstan

Presenting Author: Helmer, Janet; Goodman, Bridget

This three-year study aims to design and validate the Kazakhstani Teacher Observational Protocol (KTOP) aimed at gauging the degree to which teachers/ instructors in Kazakhstan successfully implement reform-based practices in high school and undergraduate STEM lessons.

The 2011-2020 Educational Strategy for Kazakhstan outlined the goal of further development of the ‘training system and professional development’ with the primary objective being to improve teachers' learning and professional mastery in the Kazakhstani school system. Education reform has been considered necessary as the secondary curriculum was perceived as overly dense, with a focus on rote learning that resulted in only providing superficial knowledge rather than deep mastery of topics (Fimyar, Yakavets, & Bridges, 2014).

Education reform across contexts has focused on teachers' critical role in improving education quality (Schleicher, 2016). The question remains as to how initiated changes are validated or measured. Teacher observation instruments have been developed to measure effective teaching (Mantzicopoulos, et al., 2018). In addition to being used to provide standardized data to monitor education, these instruments can help to determine if professional development programs or the use of reformed curricula are producing changes in teaching practices (MacIsaac et al., 2001). Classroom observation instruments can be constructive tools for aiding in the evaluation of teachers and designing professional development (Evenhouse et al., 2018). Classroom observation allows observers to gather information on student and teacher behaviours and the classroom environment within an authentic setting, which is one way that links theory and practice in order to better understand the classroom environment (Snyder, 2012). Observations have been used to gather data on teachers’ integration of technology (Helmer et al., 2018); student-teacher interactions (Darling-Hammond, 2006); and explicit subject-area learning (Waxman et al., 2009). In Kazakhstan, the fast pace and wide range of reforms raises questions about teachers’ readiness to implement reforms in the classroom. Moreover, historically “open lessons” where teachers and administrators observe lessons may be rehearsed without systematic criteria for evaluation.

To address these issues, this paper reports on the pilot study where researchers collected data to determine how well the KTOP is suited to the reform-based practice in Kazakhstan. The reform-based practice was conceptualized by way of the following six subscales: lesson design and implementation, methods/teaching strategies, communicative interactions, student-teacher relations, assessment interactions, and integration of content and language in teaching.

. The specific questions addressed:

RQ1: What is the overall level of inter-rater reliability of the Kazakhstan Teaching Observational Protocol (KTOP) instrument?

RQ2: What is the level of inter-rater reliability of the KTOP instrument’s (a) lesson design and implementation (5 items), (b) methods/teaching strategies (5 items), (c) classroom culture (communicative interactions; 6 items), (d) classroom culture (student-teacher relationships; 5 items), (e) assessment interactions (3 items), and (f) integrating content and language in teaching (2 items) subscales?

RQ3: How well targeted is the KTOP instrument for identifying the higher- and lower-reform-based teaching practice in the Kazakhstani high school and higher educational contexts?


Methodology, Methods, Research Instruments or Sources Used
The Protocol was adapted from the Reformed Observation Teaching Protocol (RTOP) (Sawada et al., 2002). The same 21st-century reform practices on which the RTOP is based are included in many current international educational reform efforts, including those of Kazakhstan (e.g., Yakavets & Dzhadrina, 2014). In the development of this tool, the team considered the best attributes of other tools such as the COPUS (Smith et al., 2013) COPED (Wheeler et al., 2019), and CLIL (deGraaff et al., 2007) with additional sections added to consider the teaching of STEM in English and inclusive teaching practices as these are both current priorities in Kazakhstani educational reforms.

Instrumentation
A total of 26 items were used in the overall KTOP instrument. The 26 items were comprised of the six separate scales mentioned in RQ2. Each item was anchored by 0 = Never occurred and 4 = Very descriptive.

Sampling
This study employed convenience sampling with schools that volunteered to participate in trialling the observation instrument in their classes. A total of 25 unique lessons delivered by 13 unique teachers from five distinct schools were of focus for the current study. Lesson observations took place between February and May 2022.

On average, the 13 unique teachers who participated in the current study had acquired 10.78 (SD = 6.44) years of teaching experience. The teachers were all drawn from educational institutions in the Astana, Kazakhstan region (three high schools and two universities). The lessons observed included the following eight different subject areas: physics, calculus, math, chemistry, ICT, computer-aided engineering, algebra, biomechanical engineering, and general math. All classes were taught in English.

For each lesson, two trained observers were assigned to judge each of the 25 lessons. For this study, of the total 50 judgements, individual observers contributed to a different total number of lessons.

Conclusions, Expected Outcomes or Findings
Overall Level of Inter-Rater Reliability (RQ1)
The intra-class correlation for the 26-item instrument was estimated at .72 (N = 650) which can be considered acceptable (0.50 < icc ≤ .749; lower 95% CI = .667, upper 95% CI = .755).

Level of Inter-Rater Reliability of the KTOP Instrument’s Six Subscales (RQ2)

Table 1
Inter-Rater Reliability of the Six KTOP Subscales
Abbr. Scale ICC L 95% CI U 95% CI
LDAI Lesson design and implementation .689 .558 .782
MTS Methods/teaching strategies .683 .549 .778
CC(CI) Classroom culture (communication interactions) .701 .588 .783
CC(STR) Classroom culture (student-teacher relationships) .577 .398 .703
AI Assessment interactions .730 .574 .829
ICLIT Integrating content and language in teaching .785 .622 .878
Note. Abbr. = scale abbreviation; L/U 95% CI = upper/lower 95% confidence interval for the ICC estimate; acceptable ICC underlined, good ICCs in bold.


All ICC subscales were considered moderate, with the exception of the ICLIT, which exhibited a slightly higher level of “good” inter-rater reliability (ICC = .785).

 KTOP Targeting (RQ3)
    
In terms of targeting, the items did not function optimally. As explained, each item had five levels (0-4). However, the lowest level (zero) was not ‘reached’ by students for 21 of the total 26 items. However, items did discriminate well, with only one item exhibiting negative item-rest point biserial correlation (MTS.8, r = -.040) and only one item, CC(CI).16 was only slightly underfitting to the Rasch model with an outfit value of 1.61 (p < .05).


 Given the positive results of the pilot study, the tool will be translated into Kazakh and used with a larger sample of teachers/instructors, as the psychometric analysis in the current study is very much underpowered.


References
Darling-Hammond, L. (2006). Constructing 21st-century teacher education. Journal of Teacher Education, 57(3), 300-314.

de Graaff, R., Koopman, G. J., Anikina, Y & Westhoff, G. (2007). An Observation Tool for Effective L2 Pedagogy in Content and Language Integrated Learning (CLIL), International Journal of Bilingual Education and Bilingualism, 10(5), 603-624, DOI: 10.2167/ beb462.0

Evenhouse, D., Zadoks, A., Silva de Freitas, C. C., Patel, N., Kandakatla, R., Stites, N. & DeBoer, J. (2018). Video coding of classroom observations for research and instructional support in an innovative learning environment. Australasian Journal of Engineering Education, 23(2), 95-105.

Fimyar, O., Yakavets, N. and Bridges, D. (2014). Educational Reform in Kazakhstan: the contemporary policy agenda. In D. Bridges (Ed.) (2014) Educational reform and internationalisation: the case of school reform in Kazakhstan. Cambridge: Cambridge University Press.
 
MacIsaac, D., Sawada, D., & Falconer, K. (2001, April). Using the Reformed Teaching Observation Protocol (RTOP) as a Catalyst for Self-Reflective Change in Secondary Science Teaching. Paper presented at the American Educational Research Association, Seattle, WA

Mantzicopoulos, P., Patrick, H., Strati, A., & Watson, J. S. (2018). Predicting kindergarteners' achievement and motivation from observational measures of teaching effectiveness. The Journal of Experimental Education, 86(2), 214-232.

Sawada, D., Piburn, M. D., Judson, E., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol. School Science and Mathematics, 102(6), 245-253.

Schleicher, A. (2016). Teaching excellence through professional learning and policy reform: Lessons from around the world, International Summit on the Teaching Profession. Paris: OECD Publishing. http://dx.doi.org/10.1787/978926452059-en

van der Lans, R. M., van de Grift, W. J., & Van Veen, K. (2018). Developing an instrument for teacher feedback: Using the Rasch model to explore teachers' development of effective teaching strategies and behaviors. The Journal of Experimental Education, 86(2), 247-264.

Waxman, H. C., Padrón, Y. N., Franco-Fuenmayor, S. E., & Huang, S. L. (2009). Observing classroom instruction for ELLs from student, teacher, and classroom perspectives. TABE Journal, 11(1), 63-95.

Wheeler, L. B., Navy, S. L., Maeng, J. L., Whitworth, B. A. (2019). Development and validation of the Classroom Observation Protocol for Engineering Design (COPED). Journal of Research in Science Teaching, 56, 1285-1305.


Yakavets, N., & Dzhadrina, M. (2014). Educational reform in Kazakhstan: Entering the world arena. In D. Bridges (ed.), Educational reform and internationalisation: The case of school reform in Kazakhstan (pp. 28-52). Cambridge, UK: Cambridge University Press.
 
12:15pm - 1:15pm11 SES 10.5 A: NW 11 Network Meeting
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Ineta Luka
NW 11 Network Meeting
 
11. Educational Improvement and Quality Assurance
Paper

NW 11 Network Meeting

Ineta Luka

Turiba University, Latvia

Presenting Author: Luka, Ineta

.


Methodology, Methods, Research Instruments or Sources Used
.
Conclusions, Expected Outcomes or Findings
.
References
.
 
1:30pm - 3:00pm11 SES 11 A: Quality of Teacher Education
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Buratin Khampirat
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Investigating the Factors Influencing Teaching Choice: an Exploratory Study with Student Teachers in a Distance Education Context

Oihana Llovet-Díaz, Patricia Gómez-Hernández, María-Mar Román-García, Raúl González-Fernández, María-Luz Cacheiro-González, Ernesto López-Gómez

UNED, Spain

Presenting Author: Llovet-Díaz, Oihana; Cacheiro-González, María-Luz

In a challenging context for the teaching profession and also for teacher education (Carrillo & Flores, 2022), it seems pertinent to highlight the relevance of teachers' motivations for choosing to teach. The factors that are sources of motivation for choosing to teach cover a wide range of nuances that are worth systematically exploring. In this regard, over the last decades, much research has been conducted on the factors that influence teaching choice in the international context (Fray and Gore, 2018; Heinz, 2015; Watt et al., 2017; Navarro et al., 2021). That is why identifying and analyzing the motivations for choosing to teach is a prolific line of research (Shang et al., 2022). Although, a scoping review of the literature shows that the investigations are contextualized in face-to-face teacher education.

From this perspective, the objective of this research is to identify the factors influencing teaching choice through an exploratory study with student teachers in a distance education context. The hypothesis that we posed is that the factors that influence the teaching choice of students in a distance university are different than those motivational profiles that traditional students have (face-to-face education) because the profile of the distance student is very heterogeneous.

Acknowledgments: Teaching Innovation Project supported by the Vicerrectorado de Digitalización e Innovación de la UNED (Spain): "Factors that influence teacher choice: an exploratory study with university students of the Degree in Early Childhood Education" (FIED-EI).


Methodology, Methods, Research Instruments or Sources Used
To address the research objective we present a quantitative, exploratory, and descriptive study. This study collected data from student teachers (teacher candidates) who completed the FIT-Choice scale (Watt & Richardson, 2007).

The research team selected the FIT-Choice scale because it is an international reference to identify the determinants of motivation for career choice, as well as the perception of the teaching profession. This is an internationally validated scale (Navarro et al. 2021) and in its Spanish version (Gratacós and López, 2016), which measures the following factors: perceived teaching abilities, intrinsic value, job security, time for family, job transferability, shape future of children, enhance social equity, make a social contribution, work with children, prior teaching and learning experiences and social influences. All these factors consisted of 50 items, and scoring was performed on a 7-point rating scale where “1” means “not important” and “7” means “very important” (Watt and Richardson, 2007; Gratacós and López, 2016).

Data collection is currently taking place (approximately 200 students will participate), through the FIT Choice scale in an online form. Research participants are previously informed about the objective of the study. Participation will be voluntary and anonymity and confidentiality will be ensured.


Conclusions, Expected Outcomes or Findings
The data obtained has been analyzed using SPSS 24.0, through descriptive techniques (mean and standard deviation) and the study of significant differences (inferential statistics) between the variables and factors. Due to it is ongoing research, recently supported by the Office of the Vice President for Innovation of our university, we cannot provide a preview of the results (the project is in the data collection phase). However, our research plan is to have the report by the end of June 2023.

The main expected result will be the identification of the most relevant motivations of teacher candidates in a distance education context, these results will be widely discussed considering previous research. The implications for the practice of a better understanding of different motivational profiles are focused on providing support and mentoring, promoting belief changes, and establishing initiatives to improve the attractiveness of the teaching profession, among others.

References
Carrillo, C., & Flores, Mª. A. (2022). Online teaching and learning practices in teacher education after COVID-19: lessons learnt from the literature. Journal of Education for Teaching, 1-13. https://doi.org/10.1080/02607476.2022.2153018

Gratacós, G. y López-Jurado, M. (2016). Validación de la versión en español de la escala de los factores que influyen en la elección de los estudios de educación (FIT-choice). Revista de Educación, 372, 87-105. https://dx.doi.org/10.4438/1988-592X-RE-2015-372-316

Fray, L., & Gore, J. (2018). Why people choose teaching: A scoping review of empirical studies, 2007–2016. Teaching and Teacher Education, 75, 153-163. https://dx.doi.org/10.1016/j.tate.2018.06.009

Heinz, M. (2015). Why choose teaching? An international review of empirical studies exploring student teachers’ career motivations and levels of commitment to teaching. Educational Research and Evaluation, 21(3), 258-297. https://dx.doi.org/10.1080/13803611.2015.1018278

Navarro, E., López, E., Asensio, I. I., Expósito, E., Carpintero, M. E., y Ruiz, C. (2021). Metaanálisis de generalización de la fiabilidad del cuestionario FIT-Choice (Factores que influyen en la elección de la enseñanza como carrera). Revista de educación, 393, 231-260. https://hdl.handle.net/11162/210363

Nocito, G., Sastre, S., Gratacós, G., & López-Gómez, E. (2022). “Conoce el atractivo de la profesión docente”: Impacto de un programa de orientación profesional dirigido a estudiantes de Bachillerato. Revista de Investigación Educativa, 40(2), 385-402.

Shang, W., Yu, T., Wang, J., Sun, D., & Su, J. (2022). Why choose to become a teacher in China? A large-sample study using the Factors Influencing Teaching Choice scale. Asia-Pacific Journal of Teacher Education, 50(4), 406-423.

Watt, H. M. G.; Richardson, P. W.; & Smith, K. (2017) Global perspectives on teacher motivation. Cambridge University Press. https://dx.doi.org/ 10.1017/9781316225202

Watt, H. M. & Richardson, P. W. (2007). Motivational factors influencing teaching as a career choice: Development and validation of the FIT-Choice scale. The Journal of experimental education, 75(3), 167-202. https://dx.doi.org/10.3200/JEXE.75.3.167-202


11. Educational Improvement and Quality Assurance
Paper

Choosing a STEM Subjects Teacher Profession: Views of Science Faculties Students

Rita Birzina, Dagnija Cedere, Jazeps Logins

University of Latvia, Latvia

Presenting Author: Birzina, Rita

The role of teachers is becoming increasingly important as Europe addresses its educational, social and economic challenges (Eurydice, 2018). At the same time, the most widespread problems in Europe and the world are teacher shortage, which is no longer a myth (Martin, & Mulvihill, 2016), but is real, large and growing, and worse than we thought (García, & Weiss, 2019). The European Commision (2015) pointed out problems of teacher shortages: shortage in some subjects, in some geographical areas, ageing teachers, high drop-out rates from the teaching profession, insufficient numbers of students in teacher studies programmes and high student attrition. The pandemic has also had an impact on the number of existing teachers, through increased anxiety and changes in workload (Darling-Hammond & Hyler 2020). The shortage of teachers, especially those for STEM (science, technology, engineering and mathematics), is a well-known global problem recognized by many (Diekman, & Benson-Greenwald, 2018).

What are the main factors influencing the teacher shortage? In this research the factors are divided into three groups: work environment/circumstances, personal and academic/professional.

Factors related to the working environment/circumstances determine a teacher's job satisfaction in school. They are: lack of recognition, poor remuneration/advancement opportunities, and loss of autonomy (Aragon 2016). Few researchers noted the characteristics of schools: type of school, class size (Cowan, et al., 2016), a perceived lack of respect for teachers (Barth,et al., 2016), and teacher workload, teacher cooperation (Toropova, Myrberg, & Johansson, 2021). In Latvia, teachers' job satisfaction is most influenced by a positive and democratic school culture - teacher relationships, teacher-student relationships and teacher-principal relationships (Geske & Ozola, 2015:206). In the LIZDAs study (2016): the biggest difficulties that teachers face in their work are the lack of respect from education policy makers, children's permissiveness, increased media interest in negative events in school life, stress and professional burnout

Personal factors are mainly related to teachers' perceptions of the teaching profession and their motivation to work in schools. Kyriacou and Coulthard’s (2000) study on undergraduates’ views of teaching as a career choice indicated three categories of the most motivating factors: altruistic reasons (desire to benefit society), intrinsic reasons (interest in subject matter and expertise), and extrinsic reasons (extended work breaks, level of pay, etc.). As the current shortage of teachers in STEM subjects calls for stimulating students' interest and motivation to learn, the teacher is given the role of inspirer. To teach, motivation can be seen as a multidimensional construct that includes (social influence, positive prior teaching and learning experiences, perceptions of teaching ability, intrinsic value, personal and social utility values), perceptions of the teaching profession (perceptions of task demands and returns), and evaluations of social withdrawal experiences and satisfaction with the teaching profession choice (Kuijpers, Dam, & Janssen, 2022).

Academic/professional factors have been attributed to the teacher's performance in the classroom: Do my knowledge, skills, and attributes fit with those demanded by the profession? (Klassen, Granger, & Bardach, 2022). It means that all teachers need the skills (explain a subject in a way that students understand, use a variety of teaching methods) and knowledge (subject content, pedagogy and psychology) to accomplish their immediate goals as a teacher.

The teacher shortage is becoming more acute today, with fewer and fewer students choosing not to become teachers. This situation is particularly problematic in STEM education, so the aim of the study was to find out the views of STEM faculties students on the choice of science teaching as a profession. In order to achieve this goal, the research question was posed: what factors determine the choice of STEM students to become/not to become a STEM subjects teacher?


Methodology, Methods, Research Instruments or Sources Used
Using the QuestionPro e-platform, 289 students (female (N = 200; male (N = 89) of Bachelor's and Master's degree programmes at the Faculties of Biology, Chemistry, Physics, Mathematics and Optometry and Geography and Earth Sciences of the University of Latvia were surveyed in 2022.  
The questionnaire was structured in two parts: general and conceptual. In the general part, closed-ended questions were used to establish the student's identity: demographic data, faculty, level of study, course of study, his/her choice of a teaching profession and expected salary. In the conceptual part, open/closed questions were used to identify students' views on the advantages, disadvantages and problems of the teaching profession. Finally, an open-ended question was asked to find out the conditions that should be fulfilled in order to study and work as a teacher.
The data obtained were processed using SPSS and AQUAD statistical data processing software. A coding system was created according to the questions of the conceptual part, which was later expanded based on the context of the open questions. Descriptive statistics, Spearman rank correlation non-parametric test, Mann-Whitney U test for two-group comparison and Chi-Square test for multiple-group comparison were used for data interpretation.

Conclusions, Expected Outcomes or Findings
The study concluded that the main factors determining the choice of a teaching profession were characterised by predominantly common socio-economic beliefs about the prestige of the teaching profession and low salaries. The main disadvantages in choosing a teacher profession are inadequate salaries (M=3.74; 94%), workload (M=3.39; 85%), limited personal growth opportunities (M=3.18; 80%) and low prestige (M=2.95; 74%).
Students consider the ability to public speak in front of an audience to be the greatest benefit of choosing teaching as a career (M=3.55; 89%). Opportunity to inspire young people (M=3.38; 85%) and ability to teach complex things simply (M=3.18; 79%) indicates students' desire to develop young people's interest in STEM subjects. It means that students were positive about the role of teachers in generating interest among young people in studying science. This is evidenced by the moderately strong correlation between the variables that are important in science (inspiring students/ability to teach complex subjects, r=0.44; ability to teach in a way that students can understand/sufficient depth and depth of subject knowledge, r=0.54). This suggests that students, few of whom have had the opportunity to be a teacher, have a reasonably good understanding of the job of a science teacher. Of the 289 students, 104 (36%) have seriously considered becoming a teacher, 14 have worked in a school, 19 are already working in a school and 89 (31%) could also teach young people in a school. Only 50 (17%) would categorically not want to work in a school.
There are no significant differences in the perceptions of students from different STEM faculties about the teaching profession. Students are able to assess the strengths and weaknesses of the teaching profession by assessing the school as a working environment, the teacher's personal perceptions and motivation to work in a school, and the teacher's professional/academic work.

References
Aragon, S. (2016). Teacher Shortages: What We Know. Teacher Shortage Series. Education Commission of the States. Denver, CO 8020
Barth, P., Dillon, N., Hull, J., & Higgins, B. H. (2016). Fixing the Holes in the Teacher Pipeline: An Overview of Teacher Shortages. Center for Public Education.
Cowan, J., Goldhaber, D., Hayes, K., & Theobald, R. (2016). Missing elements in the discussion of teacher shortages. Educational Researcher, 45(8), 460-462.
Darling-Hammond, L., & Hyler, M. E. (2020). Preparing educators for the time of COVID… and beyond. European Journal of Teacher Education, 43(4), 457-465.
Diekman, A. B., & Benson-Greenwald, T. M. (2018). Fixing STEM workforce and teacher shortages: How goal congruity can inform individuals and institutions. Policy Insights from the Behavioral and Brain Sciences, 5(1), 11-18
Eurydice. (2018). The teaching profession in Europe: Practices, perceptions, and policies. Eurydice Report. European Commission/EACEA/
European Commission. (2015). 2015 Joint Report of the Council and the Commission on the implementation of the Strategic Framework for European cooperation in education and training (ET 2020) - New priorities for European cooperation in education and training. https://eurlex.europa.eu/legalcontent/EN/TXT/?uri=CELEX:52015XG1215(02)
Geske, A., & Ozola, A. (2015). Teachers’ Job Satisfaction: Findings from TALIS 2013 Study. In Society. Integration. Education. Proceedings of the International Scientific Conference (Vol. 2, pp. 56-62).
Ingersoll, R. M. (2002). The teacher shortage: A case of wrong diagnosis and wrong prescription. NASSP bulletin, 86(631), 16-31
Kyriacou, C., & Coulthard, M. (2000). Undergraduates' views of teaching as a career choice. Journal of education for Teaching, 26(2), 117-126.
Klassen, R. M., Granger, H., & Bardach, L. (2022). Attracting prospective STEM teachers using realistic job previews: A mixed methods study. European Journal of Teacher Education, 1-23.
Kuijpers, A. J., Dam, M., & Janssen, F. J. (2022). STEM students’ career choice for teaching: studying career choice processes using personal projects. Teacher Development, 1-20.
Kunz, J., Hubbard, K., Beverly, L., Cloyd, M., & Bancroft, A. (2020). What Motivates Stem Students to Try Teacher Recruiting Programs?. Kappa Delta Pi Record, 56(4), 154-159.
LIZDA (2016). Skolotāja profesijas prestižs Latvijā. Latvijas Izglītības un zinātnes darbinieku arodbiedrība. / The prestige of the teaching profession in Latvia. Latvian Education and Science Employees' Trade Union/. https://www.lizda.lv/wp-content/uploads/2019/08/Skolotaju-prestizs.pdf
Martin, L. E., & Mulvihill, T. M. (2016). Voices in Education: Teacher Shortage: Myth or Reality?. The Teacher Educator, 51(3), 175-184.
Toropova, A., Myrberg, E., & Johansson, S. (2021). Teacher job satisfaction: the importance of school working conditions and teacher characteristics. Educational review, 73(1), 71-97.


11. Educational Improvement and Quality Assurance
Paper

Teachers' Readiness to Use Formal Performance Data to Improve Student Learning and the Impact of School Culture

Glen Molenberghs, Roos Van Gasse, Sven De Maeyer, Jan Vanhoof

Universiteit Antwerpen, Belgium

Presenting Author: Vanhoof, Jan

Background, rationale and research questions:

Although the empirical evidence on the educational impact of the systematic use of formal performance data from central tests is quite strong (Datnow & Park, 2018), the Flemish(1) education system is one of the few European education systems in which no form of central testing is widely implemented (OECD, 2013). As of school year 2023-2024, all Flemish pupils will also take central tests during their school career, which (in contrast to numerous other education systems) aim to take a strong development-oriented perspective.

While policymakers and governments expect teachers to use data to improve student learning, teachers still appear reluctant to integrate this data into their teaching practices (Schelling & Rubenstein, 2021). In this regard, a numerous number of descriptive studies provide in-depth insight into influencing factors of data use (Schildkamp, Poortman, Luyten, & Ebbeler, 2017). However, most of these studies only consider (a small number of) psychological factors to a limited extent. Since data use is essentially a human endeavour, it is important, in order to fully benefit from the rich potential of data use, to also study psychological aspects (Schildkamp, Poortman, Ebbeler, & Pieters, 2019).

Because the use of formal performance data from central tests for educational improvement can be considered a (relative) educational change, certainly in Flanders, but also to some extent in an international context, and to select psychological factors driving teachers’ data use for educational improvement, we drew inspiration from the literature related to ‘change readiness’. Armenakis, Harris and Mossholder (1993) consider teachers’ readiness as ‘one's beliefs, attitudes and goals regarding the extent to which change is needed and their perceptions of individual and organisational ability to successfully implement those changes’. The readiness to use formal performance data concerns, in other words, both ‘willing’ and 'being able' to change. Each of these dimensions explains an aspect of readiness (Rafferty, Jimmieson, & Armenakis, 2013) and is highlighted in the literature because of their role in successful educational change processes (Armenakis & Bedeian, 1999). Because readiness can be considered a predictor of behavior (Armenakis, Harris, & Mossholder, 1993), we expect that a positive teachers’ readiness (i.e. a positive appraisal of willing and being able to use formal performance data to improve student learning) may contribute to effective (future) data use. Consequently, this study firstly examines the extent to which teachers are ready to use formal performance data (from central tests) to improve student learning (RQ1).

Moreover, data use does not occur in isolation (Schildkamp et al., 2019). By including school-level factors we account for the fact that teachers’ readiness does not occur in isolation en can be impacted by a data use stimulating school culture (Prenger & Schildkamp, 2018). As a consequence we secondly study to what extent characteristics of a data use stimulating school culture have an impact on teachers’ readiness (RQ2).

In sum, the present study aims to address the aforementioned knowledge gap by quantitatively studying Flemish teachers' readiness to engage with formal achievement data (from central tests) to improve student learning and school culture’s impact.

(1) Flanders is the Dutch-speaking part of Belgium.


Methodology, Methods, Research Instruments or Sources Used
Operationalisation of concepts:

In the operationalisation of the dependent variable ‘readiness’, we have focused on the ‘will’ and ‘able’ part of this concept. The ‘will part’ of readiness was operationalised in terms of emotions or affective appraisal towards data use (Jimerson, 2014) and in terms of the usefulness of the data to improve students’ learning (Vanhoof, Vanlommel, Thijs, & Vanderlocht, 2014). The ‘able part’ of readiness was operationalised by self-efficacy (Bandura, 1997) and by teachers’ perceptions about having the necessary time for data use (Jimerson, 2016).

These central predictors of teachers’ readiness can be impacted by teachers’ perception of a data use stimulating school culture (Prenger & Schildkamp, 2018). In this study, a data use stimulating school culture was operationalised by shared goals towards data use (Vanhoof, Verhaeghe, Van Petegem, & Valcke, 2012), by internal support and collaboration in data use (Schildkamp et al., 2019), by expectations regarding data use (Vanhoof et al., 2014), by experience in the use of standardised tests, by transformational leadership (Yu, Leithwood, & Jantzi, 2002) and by level of education.

Instrument and sample:

To answer the research questions, we administered an online survey. The content in this survey was  both compiled from existing, validated scales and adopted items from previous research on data use and central tests. All items were statements to be scored on a 5-point Likert scale with a possibility to opt out. Finally, 611 Flemish teachers from 45 schools participated in the survey.

Data analysis:

To measure teachers’ individual perceptions of their readiness to engage with formal performance data to improve student learning and of a data use stimulating school culture, we constructed scales. By applying CFA, we examined the construct validity of each scale. In addition, a Cronbach's alpha was calculated for each multi-item scale as a measure of internal consistency.

Based on these scales, we applied descriptive statistics for RQ1.

In order to answer RQ2, we built and tested a path model. We started out with a model in wich shared goals, support and collaboration, expectations and use of standardised tests mediate the effect on teachers’ readiness to engage with formal performance data to improve student learning of transformational leaderschip and education level. Based on the modification indices we gradually added covariances and eliminated non-significant parameters in pursuit of a parsiminous final model with optimal fit. The path analysis was conducted in R with the lavaan-package (Rosseel, 2012).

Conclusions, Expected Outcomes or Findings
Conclusion:

Teachers’ readiness to use formal performance data to improve students’ learning can be considered an important bridge to effective data use (Armenakis et al., 1993). Yet, we found that teachers perceive only limited readiness: they have limited positive attitudes towards the use of formal performance data from central tests, they rather feel self-efficace for data use but at the same they do not perceive sufficient time to do so.

However, the path model shows that the perception of a higher level of a data use stimulating school culture has a positive impact on teachers’ readiness. In this, positive attitudes towards data use can be promoted if teachers perceive expectations regarding the use of formal performance data from central tests as clear. In addition, the can part of readiness can be promoted by perceiving supportive relationships and collaboration. This finding suggest that teachers in particular engage with data use if they perceive data use as a team event.

The many positive indirect effects of transformational leadership on teachers’ readiness we found, point to the important role of the school leader in cultivating a data use stimulating culture within school teams. Moreover, teachers' readiness to engage with formal performance data is no one-size-fits-all story. Primary school teachers appear to perceive a more stimulating data use culture, and consequently a higher degree of readiness than secondary school teachers.

By studying teachers’ readiness and promoting school-level factors, this study (further) strengthen the bridge to the use of formal performance data to improve student learning. The Flemish context with the prospect of implementing central tests charachterised by a development-oriented perspective, provided a particularly appropriate case.

References
Armenakis, A. A., & Bedeian, A. G. (1999). Organizational change: A review of theory and research in the 1990s. Journal of management, 25(3), 293-315.
Armenakis, A. A., Harris, S. G., & Mossholder, K. W. (1993). Creating readiness for organizational change. Human relations, 46(6), 681-703.
Bandura, A. (1997). Self Eflicacy. The Exercise of Control. New York: Freeman.
Datnow, A., & Park, V. (2018). Opening or closing doors for students? Equity and data use in schools. Journal of Educational Change, 19(2), 131-152.
Jimerson, J. B. (2014). Thinking about data: Exploring the development of mental models for “data use” among teachers and school leaders. Studies in Educational Evaluation, 42, 5-14.
Jimerson, J. B. (2016). How are we approaching data-informed practice? Development of the Survey of Data Use and Professional Learning. Educational Assessment Evaluation and Accountability, 28(1), 61-87. doi:10.1007/s11092-015-9222-9
OECD. (2013). Synergies for better learning. An international perspective on evaluation and assessment. Paris: OECD.
Prenger, R., & Schildkamp, K. (2018). Data-based decision making for teacher and student learning: a psychological perspective on the role of the teacher. Educational Psychology, 38(6), 734-752. doi:10.1080/01443410.2018.1426834
Rafferty, A. E., Jimmieson, N. L., & Armenakis, A. A. (2013). Change readiness: A multilevel review. Journal of management, 39(1), 110-135.
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of statistical software, 48, 1-36.
Schelling, N., & Rubenstein, L. D. (2021). Elementary teachers’ perceptions of data-driven decision-making. Educational Assessment, Evaluation and Accountability, 33(2), 317-344.
Schildkamp, K., Poortman, C., Luyten, H., & Ebbeler, J. (2017). Factors promoting and hindering data-based decision making in schools. School Effectiveness and School Improvement, 28(2), 242-258. doi:10.1080/09243453.2016.1256901
Schildkamp, K., Poortman, C. L., Ebbeler, J., & Pieters, J. M. (2019). How school leaders can build effective data teams: Five building blocks for a new wave of data-informed decision making. Journal of Educational Change, 20(3), 283-325. doi:10.1007/s10833-019-09345-3
Vanhoof, J., Vanlommel, K., Thijs, S., & Vanderlocht, H. (2014). Data use by Flemish school principals: impact of attitude, self-efficacy and external expectations. Educational Studies, 40(1), 48-62. doi:10.1080/03055698.2013.830245
Vanhoof, J., Verhaeghe, G., Van Petegem, P., & Valcke, M. (2012). Flemish primary teachers' use of school performance feedback and the relationship with school characteristics. Educational Research, 54(4), 431-449. doi:10.1080/00131881.2012.734726
Yu, H., Leithwood, K., & Jantzi, D. (2002). The effects of transformational leadership on teachers’ commitment to change in Hong Kong. Journal of Educational Administration, 40(4), 368-389.
 
3:30pm - 5:00pm11 SES 12 A: Application of Digital Technologies in Education
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Ieva Rudzinska
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Evaluation of Learning Management System of Culture-Based Multilingual Blended-Learning Course for Adult Learners Implemented during Covid-19 Pandemic

Ineta Luka

Turiba University, Latvia

Presenting Author: Luka, Ineta

Due to the Covid-19 pandemic, the world experienced a technological boom in education, especially in formal education. The popularity of e-learning, online learning, blended-learnig (BL) has increased with applying them to non-formal adult education (Wiliamson et al., 2020). Various BL courses and tools have been created for adult non-formal learning, for example, applying BL for adult learners’ literacy classrooms (Rosen, 2019), specific professional development (Macumber, 2021), organizational learning (Sutherland, Porter, 2019), mindfulness and well-being courses (McGarvie, 2020), adults’ social inclusion and social capital (Cocquyt et al., 2019), developing language skills (Erickson, 2019), adult engagement in learning communities (Abedini, Abedi, Zowgi, 2021) and others.

The current research was conducted in the Erasmus+ project “Cultural knowledge and language competences as means to develop 21st century skills” involving six EU countries: Croatia, Latvia, Slovenia, Romania, Poland, Czechia (Project No.2018-1-HR-01-KA204-047430; 2018-2021).

The project aims to develop adult learners’ 21st century skills and foster learners’ knowledge of the rich European cultural heritage and its values by applying innovative learning approaches and materials consequently improving people’s education level and bringing them closer to cultural heritage, history and the common values of Europe, enhancing their overall development and employability.

3 comprehensive outputs have been created: two BL courses and LLL guidelines for adult educators. In ECER2019 the course construct was presented (Luka, 2019), in ECER2021 the results of the English language course implementation (O1 Output) were presented (Luka, 2021a, b), in ECER2022 implementation of the “Culture-based multilingual blended-learning course for adult learners” in 10 languages (EN, HR, LV, Sl, PL, CZ, HU, RO, DE, FR) in the 6 countries was introduced (Output 2) (Luka, 2023). The contribution of ECER2023 deals with the evaluation of the learning management system (LMS) (Output 1 & 2), since learning environment is a crucial part of BL. The course employs generic model by Wang (2008) comprising three components: pedagogy (makes learning meaningful, authentic and relevant to learners), social interaction (ensures interaction with peers, teachers, experts), technology (interaction with content and with people is implemented through the interaction with the interface). This model enables securing learner engagement (Shi et al., 2021).

Technology-supported learning environments in education have been in the focus of researchers for more than twenty years (Müller & Wulf, 2021).Recent research on LMS implies that LMS must be user-friendly (Hofmeister & Pilz, 2020), interactive (Gao et al., 2020) and innovative (Patra et al., 2021). It has to offer synchronous and asynchronous interaction opportunities (Ruscanda et al., 2021), consider learners’ learning styles (Carrillo, Flores, 2020) and provide a feedback to learners (Ionescu, 2020). Hence, it is crucial to create such a LMS that would support the use of various teaching/learning methods (pedagogical content), involve learners in interaction (social component), and secure the learning process technologically (technological component).

The topicality of this research is underlined by the target groups of learners – alongside with regular adult learners, adults with certain barriers to learning (cultural, social, econonomic, geographic, and having learning difficulties) and the course implementation during the pandemic period predominantly in an online-only BL format. Thus, the BL course and the LMS required the incorporation of specific features. Hence, the evaluation of the LMS after the course acquisition is very important.

The aim of this paper is to evaluate the suitability of the LMS for the implementation of a BL adult learning course for various target groups in the six partner countries during the COVID-19 pandemic, when novel ways of teaching/learning had to be found.

The research question: How do the learners evaluate the LMS and is it suitable for implementation of a BL adult learning course for adults with barriers to learning?


Methodology, Methods, Research Instruments or Sources Used
The research period: March 2020 – January 2021.
Comparative research design (Boeren, 2019) was appied to evaluate the results overall and compare them among the six partner countries. The course participants chose modules according to their interests. Each module required 20-30 hours of work. After the course, participants filled in a paper-based questionnaire comprising 3 parts: 1) socio-demographic data (11 questions), 2) evaluation of the LMS (6 questions), content of the modules (9 questions) and 9 questions evaluating their skills’ development (all 5-point Likert scale), 3) evaluation of learning styles of learners (12 ranking questions – from 1 to 4). This contribution analyses the evaluation of the LMS. The tool measured the usefulness and visuality of the LMS, how interesting it was, the ease of understanding how to operate it, if learners will suggest the LMS to other learners and if they had used a similar LMS before.
Data were analysed by IBM SPSS Statistics 23 software employing descriptive statistics (frequencies, percentages, means and modes), Cronbach’s Alpha reliability test (α=0.924), inferential statistics tests (to elicit significant differences among the six countries and among different groups of learners).
Research sample: 638 adult learners – 83 (13%) from Croatia, 111 (17.4%) from Latvia, 122 (19.1%) from Poland, 111 (17.4%) from Romania, 104 (16.3%) from Slovenia, 107 (16.8%) from Czechia. 410 (64.3%) female, 228 (35.7%) male, aged 18-65 and older. 1/3 of the course participants (209 learners out of 638 or 32.8%) were learners with barriers to learning: in total 67 learners (10.5%) had cultural and social barriers to learning, 73 learners (11.4%) had geographic barriers, 41 learners (6.4%) had economic obstacles to learning and 28 learners (4.4%) were with learning difficulties requiring special educational treatment. 429 (67.2%) were regular adult learners.
In accordance to Brancati (2018), research ethics was observed guaranteeing participants’ anonymity, voluntary participation and causing not physical nor psychological harm to them.
Research limitations: The LMS and the course were designed in 2019 but held during  the pandemic period. The strict restrictions introduced into the countries because of the pandemic, influenced the course implementation and the results. In Latvia, Czechia and Croatia the face-to-face stage was implemented partly in the classroom, partly using video conferencing apps, in Poland and Romania the course was implemented as complete online-only BL course, Slovenia was the only country wherein the course was implemented in a traditional BL format.

Conclusions, Expected Outcomes or Findings
Learners found the LMS useful (Mean=4.1113, Mode=4.00), visually appealing (Mean=3.7868, Mode=4.00), interesting (Mean=4.0235, mode=4.00). Although they had not used a similar LMS before (Mean=2.5956, Mode=2.00), it didn’t take them too long to understand how to use the LMS. Consequently, they were ready to suggest the LMS to other learners (Mean=3.5956, Mode=4.00). Thus, overall findings indicate the fulfilment of the requirements of a user-friendly LMS.
Significant differences were discovered among the countries concerning all questions (p=0.000). Romanian learners found the LMS significantly more useful (Mean Rank=372.59) and visually appealing (Mean Rank=360.15) than Polish learners (Mean Rank=283.28 and 245.26). Czech learners found it significantly more interesting (Mean Rank=398.84) than Polish (Mean Rank=238.84) and Croatian (Mean Rank=251.64) learners. Slovenian learners found it easier to understand the LMS use. Romanian and Czech learners were more ready to suggest the LMS to other learners.
Differences among the countries may be explained by their specific target groups of learners. Learners with barriers to learning (Mean=4.0622) found the LMS significantly more interesting than regular learners (Mean=4.0047). At the same time fewer of them had used a similar LMS before (Mean Rank=2.2584) compared with regular learners (Mean Rank=2.7599) and it was more difficult for them (Mean Rank=3.0766) than for regular learners (Mean Rank=2.5221).
Learners with cultural, social, economic obstacles found the LMS significantly more useful and visually appealing than other groups of learners. Learners with economic barriers found the LMS significantly more interesting than others. Seniors with learning difficulties gave the lowest evaluation in all the criteria. Learners with geographic obstacles understood the LMS use much faster than other groups of learners. Learners with cultural, social, geographic obstacles were more willing to recommend the LMS to other learners.
To conclude, this LMS is suitable to all groups of learners but some groups need more pedagogical and technological assistance.

References
*Abedini, A., et.al. (2021). Adult learning in online communities of practice: A systematic review. BJET, 52(4),pp.1663-1694.
*Boeren, E. (2019). International and Comparative Research Design. L.Hamilton, J.Ravenscroft (Eds.) Building Research Design in Education (131-150). London:Bloomsbury Academic.
*Brancati, D. (2018). Social Scientific Research. LA:SAGE.
*Carrillo, C., Flores, M.A. (2020). COVID-19 and teacher education: a literature review of online teaching and learning practices. European Journal of Teacher Education, 43(4),pp.466–487.
*Cocquyt, C., et.al. (2019). Examining the role of learning support in blended learning for adults’ social inclusion and social capital. Computers and Education, 142,pp.1-19.
*Erickson, B. (2019). Blended Learning among Adult English as a Second Language Programs. Culminating Projects in Education Administration and Leadership.
*Gao, B.W., et.al. (2020). The effect of blended learning platform and engagement on students’ satisfaction – the case from the tourism management teaching. Journal of Hospitality, Leisure, Sport & Tourism Education, 27, 100272.
*Ionescu, C.A., et.al. (2020). Sustainability Analysis of the E-Learning Education System during Pandemic Period COVID-19 in Romania. Sustainability, 12, 9030.
*Luka, I. (2019). Creating a Culture-Based Language Learning Course for Developing Adult Learners’ 21st Century Skills. Journal of Education Culture and Society, 10(2), pp.151-169. https://doi.org/10.15503/jecs20192.151.169
*Luka, I. (2021a). Developing Adult Learners’ Language Competence in Culture-Based Blended-Learning Course. Acta Universitatis Sapientiae, Philologica, 13, 2, pp.71-92, DOI:10.2478/ausp-2021-0014
*Luka, I. (2021b). European cultural heritage and skills development course for adult learners’ self-development. Journal of Education Culture and Society, 12(2), pp.505-526, https://doi.org/10.15503/jecs2021.2.505.526
*Luka, I. (2023). Implementation of a blended learning course for adult learners during the COVID-19 pandemic. Quality Assurance in Education, 31(1), pp.91-106. https://doi.org/10.1108/QAE-03-2022-0061
*Macumber, C. (2021). Effective Implementation of Blended Learning. Iowa:Northwestern Cpllege.
*McGarvie, S. (2020). Attentive amelioration: developing and evaluating an applied mindfulness programme for psychologists.
*Rosen, D.J. (2019). Blended Learning Program Development. Adult Literacy Educaation, 1(2),pp.84-86.
*Rucsanda, M.D., et.al. (2021). Students' Attitudes Toward Online Music Education During the COVID 19 Lockdown. Frontiers in Psychology, 12, 753785.
*Shi, Y., et.al. (2021). Investigating relationships among blended synchronous learning environments, students’ motivation, and cognitive engagement: A mixed methods study, Computers & Education, 168, 104193.
*Sutherland, R., Colin, P. (2019). Reporting from the Test Kitchen: Blended Leaarning at New Leaders. NY: Nw Leaders.
*Wang, Q. (2008). A generic model for guiding the integration of ICT into teaching and learning. Innovation in Education and Teaching International, 45(4),pp.411-419.
*Williamson, B., et.al. (2020). Pandemic politics, pedagogies and practices: digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45(2),pp.107-114.


11. Educational Improvement and Quality Assurance
Paper

Using Technology Pedagogy Content Knowledge Framework for Pedagogical Efficiency in Digital Learning

Mudassir Arafat

International College of Cosmetology, Latvia

Presenting Author: Arafat, Mudassir

Modern-day twenty-first-century education is now being constructed around the pillars of a framework that supports and dwells on Information and Communication Technology in the learning process. Nowadays, teachers are supposed to develop a pedagogical approach that integrates and blends with technology in learning. Often seen many digital learning tools are complex demanding special training, thus forcing teachers not to practice it in their pedagogical approach.

TPACK is a framework that highlights the labyrinth involved with the adoption of a particular digital learning tool and its use between the three basic components of knowledge (technology, pedagogy, and content). This particular article takes a deeper dive into the TPACK framework and how essential it is to integrate technology into education with ease.

Hernawati & Jailani (2019) argue that the birth of the internet has changed the way we perceive learning. According to Schoenl & Fusarelli (2008), it is stated that the exercise of ICT as a medium for instructional tools and the pedagogical capabilities of teachers are the reasons to support the learning organization as well as teachers to face the problems that might arise whilst developing the students’ learning skills.

This study aims at helping HEIs understand and validate the practice of their digital learning tools by teachers and learners. The aim of the research: evaluate the digital efficiency of teachers and learners with digital tools and identify the steps to be taken in order to make the learners and teachers digitally capable to practice the digital learning tools with ease.

As described by Harris & Matt Koehler (2009), Technological Pedagogical Content Knowledge (TPACK) framework aims to identify the concept of digital learning needed by teachers for novelties in their pedagogical approach. TPACK simultaneously aims to answer the likeliness of confusion, as well as the complex stacked type of the teacher's information.

Chai. et.al (2013) have reviewed 74 journal papers that investigated ICT integration with the TPACK framework. The understanding of the TPACKs must be learned by the teachers to efficiently enforce ICT in the classroom. TPACK is a framework for teachers, educators, and students that addresses the issue of incorporating technology into learning which has been exercised apart from content learning and pedagogical education.

Further discussions: As stated by Steinle & Lynda Ball (2013), every teacher has their own TPACK capabilities to harness when it comes to digital tools for learning, such as a mathematics teacher needs to have the skills of a well-connected CK and PCK, that a teacher can have the freedom to enter tweak or modify pre-lecture and post-lecture for interactive, monitored and effective learning.

Case studies & arguments: study conducted in Tanzania by Patrick et.al (2016), Africa showed that more than 50% of teachers in terms of preparedness for classroom technology integration were not prepared with the content or the knowledge of the ICT for education at their research institution. With such a percentage index and the lack of technological knowledge in modern-day society, a teacher would make knowledge transferred to other learners completely obsolete.

Final statements: There is still a fair share of critics for TPACK, but the framework is still relevant. Various communities implement different aspects of TPACK globally. Mary C. Herring (2016) indicates that TPACK’s official website has over 6,000 registered users who have common shared interests that together have compiled a detailed bibliography of TPACK-related literature involving over 600 articles


Methodology, Methods, Research Instruments or Sources Used
The study uses the SLR method (systematic literature review) for its analysis and expert interviews, and the conclusion is based on the literature arguments and analysis supported by expert evaluation.

Literature review is a method that helps to compose, clarify and focus the subject of a study. Literature reviews are useful when the aim is to provide an overview of the body of current literature in a certain area or to examine the validity or accuracy of a theory or competing theories (Snyder, 2019). In addition, Tranfield, Denyer, & Smart (2003) claim that SLR is useful when establishing research questions and need answers regarding specific points or issues in the field.

The SLR aims in the direction to give an overview of how the exercise of digital learning tools for education can be exploited to its optimum potential by incorporating the TPACK framework capability of the teachers and how it can influence on the students’ performance. Additionally, semi-structured expert interviews were conducted to validate the results of the SLR.

Sampling: For this article, the data sampling is done as per the academic needs. The cited articles and journals that have been analyzed and studied are the ones that describe the benefits of the TPACK framework and talked about its limitations. Articles that discussed case studies were given higher priority, as they give an anchored validation to the systematic literature review. The emphasis was given to articles with keywords (TPACK) technological pedagogical content knowledge and more stress was given to cite literature from recent years and not older than 2010, however, in some cases where and when the model was created by Davis, the article does require citations and references from previous years.

Two experts were selected for the interview according to a purposeful sampling procedure. The criteria for selection: doctoral/Ph.D. degree in education, practical teaching experience at various education levels for more than 15 years, highly developed digital skills, give courses to other teachers.

Research Question: How can teachers’ learners & educational organizations use their digital learning tools to the best capabilities using the TPACK Framework to help?

The expert interviews were analyzed by the medium of thematic analysis. Meaningful patterns from the semi-structured expert interviews were collected. The collected patterns helped in analyzing the pedagogical transitions and adaptations, teachers have to undergo with the changing digital, pedagogical and content trends in learning.
Research period: June 2021 to January 2023.  

Conclusions, Expected Outcomes or Findings
Based on the SLR and the semi-structured expert interview results, it may be concluded that ICTs with the help of the TPACK framework can be used as validating tools, self-knowledge evaluation tools, and knowledge-enhancing tools to foster pivotal and creative philosophy, and genuine and creative issues resolving. Teachers will need regular training with changing ICT trends to master their skills to formulate lessons that are integrated with technology even with the TPACK framework. In order to exploit the ICT to its optimum teachers must incorporate the TPACK framework, which involves selecting the right content in accordance with the digital platforms, ascertaining the most suitable pedagogical approach and method, and judging the learners as per their individual learning patterns.

Learning that allows student participation by teachers, creates learning/teaching possibilities that are in accordance with developing trends. It's only through a simultaneous change in the technological, human service, and management domain that the full potential of digital learning can be realized.

TPACK framework is a blend of three fundamental forms of learning: There is, Technology (TK), Pedagogy (PK), and Content (CK). The TPACK approach goes way ahead in penetrating these three learning forms in the division.

According to Chai. et.al. (2013), the exercise of a particular technology incorporated for the purpose of learning can put an impression of a new form or an approach in other words different pedagogical style. Technological Pedagogical Content Knowledge (TPACK) is the foundation of productive and efficient teaching practices with the support of technology, which usually needs the knowledge of portraying the concepts by using technology; Pedagogical styles, approaches, and patterns apply technology to highlight the content; the knowledge about things that are complicated or simple to learn or understand making it simpler for the learners to overcome problems with the help of technology and innovations.  

References
Hernawati, & Jailani, &. (2019). Mathematics mobile learning with TPACK framework. Journal of Physics: Conference Series 5th ICMSE2018, 1-8. doi:10.1088/1742-6596/1321/2/022126
Schoenl, & Fusarelli, &. (2008, Jan 1). Innovation, NCLB, and the Fear Factor: The Challenge of Leading 21st-Century Schools in an Era of Accountability. SAGE Journals, 22(1). doi:10.1177/0895904807311291
Harris, J., & Matt Koehler, &. (2009). What Is Technological Pedagogical Content Knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70. Retrieved March 20, 2022, from https://www.researchgate.net/publication/241616400_What_Is_Technological_Pedagogical_Content_Knowledge
Chai.et.al, C. S. (2013). A Review of Technological Pedagogical Content Knowledge. Journal of Educational Technology & Society, 16(1), 31-51. Retrieved March 28, 2022, from https://www.researchgate.net/publication/290044779_A_Review_of_Technological_Pedagogical_Content_Knowledge
Steinle, V., & Lynda Ball, C. B. (2013). mathematics education: yesterday, today and tomorrow. proceedings of the 36th annual conference of the Mathematics Education Research Group of Australasia, 2, 816. doi:9780734048448
Patrick.et.al. (2016). Classroom ICT integration in Tanzania: Opportunities and challenges from the perspectives of TPACK and SAMR models. International Journal of Education and Development using Information and Communication Technology, 12(1), 107-128. doi:1814-0556
Mary C. Herring, M. J. (2016). Handbook of Technological Pedagogical Content Knowledge (TPACK) for Educators. New York, USA: Routledge. doi:ISBN 9781315771328
Snyder, H. (2019, November). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333-339. doi:https://doi.org/10.1016/j.jbusres.2019.07.039
Tranfield, D., Denyer, D., & Smart , P. (2003, Sep 16). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management,, 14, 207-222. doi:10.1111/1467-8551.00375


11. Educational Improvement and Quality Assurance
Paper

Quality Assurance of Digital Education

Amy Quintelier1, Kurt Maenhout2

1Flemish Inspectorate of Education, Belgium; 2Flemish Inspectorate of Education, Belgium

Presenting Author: Quintelier, Amy

Research suggests that, in line with social developments, schools are using more intensively digital technologies (OECD, 2021). However, an improvement in infrastructure does not automatically lead to a well-considered use of that technology in education (Albion & Tondeur, 2018). We must therefore be aware that the use of ICT does not automatically have added value. The ultimate goal of ICT use in teaching-learning processes is to ensure that more and can be learned more deeply.

From a theoretical perspective, the extent to which schools develop a policy on the technical and didactical use of ICT and the extent to which schools strengthen educational practice by means of digital education, formed the basis of this study. Research also showed that different preconditions on pedagogical, organisational and technological aspects can influence the quality of the offered digital education. A suitable, reliable and secure digital infrastructure as well as a digitally competent school team are just two of these important preconditions.

In addition to subject-specific knowledge, the literature points to teachers’ technological and didactic ICT knowledge and skills to provide quality digital education, although time and space to professionalise is important too. Lawrence and Tar (2018) emphasize the importance of a positive attitude towards digital education among teachers in relation to the quality of the lessons offered.

From a parental perspective, a positive involvement in learning is a beneficial effect on the child's learning gains. This involvement can be strengthened by the use of digital technologies. Parents who are sufficiently digitally competent support their children often better with digital homework and make more use of digital communication channels in the school (del Carmen Ramírez-Rueda et al., 2021).

We embarked on this topic with the aim of providing a broad overview of how Flemish primary and secondary schools organize digital education in their school. Three questions were identified during the literature study as being in need of attention: (1) To what extent do the Flemish schools of compulsory education pursue a high-quality ICT policy? (2) To what extent do the Flemish schools of compulsory education strengthen the teaching learning practice through digital education? (3) To what extent are the preconditions for high-quality digital education fulfilled in the Flemish schools of compulsory education?

To evaluate the implementation of Digisprong, a government initiative to strengthen digitalisation in the Flemish schools of compulsory education, the Inspectorate of education collected data on how schools shape digital education. A mobile survey via the VOI.CE app was used to gather the perspectives of different education stakeholders school leaders, teachers, ICT coordinators, pupils and parents).

The survey (N=7481) provides an interesting picture of the state of the digital education offered in Flemish schools. Halfway through the rollout of the Digisprong, a large majority of schools state they have a vision on ICT, although they do not always have an effective strategy to realize that vision. The results indicate also differences between schools. To close the gap between the needs and needs regarding school members’ digital competences, a good professionalisation policy is needed that takes into account the needs and requirements of the whole school team.

With a view to a sustainable strengthening of quality development in schools, we look at both the strengths as well as areas for improvement. At the same time, we want to educate with the help of this research raising awareness to develop a sustainable ICT policy aimed at strengthening the digital teaching learning practice.


Methodology, Methods, Research Instruments or Sources Used
The Flemish Inspectorate of Education was instructed to participate annually in the evaluation of the implementation of Digisprong, a government initiative to strengthen digitization in the Flemish schools of compulsory education. Three questions were identified during the literature study as being in need of attention: (1) To what extent do the Flemish schools of compulsory education pursue a high-quality ICT policy? (2) To what extent do the Flemish schools of compulsory education strengthen the teaching learning practice through digital education? (3) To what extent are the preconditions for high-quality digital education fulfilled in the Flemish schools of compulsory education?
For the first time, the Flemish Inspectorate made an explicit choice for a mobile-first approach in which respondents are encouraged to participate in the survey via the VOI.CE app, a mobile communication channel (app) between the education inspectorate and various education partners (school board, teachers, ICT-coordinators, pupils and parents), that can be downloaded on a smartphone. The reason why the Inspectorate used a mobile application to send the survey is of two kinds. On the one hand, the organization supports the Flemish policy memorandum that opts for an extensive digitization of all services in Flanders. On the other hand, the organisation thinks it is important to enabling digital inclusion.
The mobile survey ran from 30 May to 17 June 2022 and contained items that measured (1) the ICT policy in schools, (2) the ICT use in educational practice, (3) digital competences, (4) attitudes towards the use of ICT and (5) infrastructure, time and support regarding the use of ICT in the schools. The respondents were asked to express their degree of agreement/disagreement with each statement by choosing one of the four answers provided by the Likert scale: strongly agree, agree, disagree, and strongly disagree. The survey involved 7481 respondents (2237 educational professionals, 2443 pupils, 2801 parents).  We have provided means, standard deviations and t-test p-values for each of the questions, using R Studio.

Conclusions, Expected Outcomes or Findings
In general, the strategic policy to implement ICT in a well-thought-out way appear to be well in most schools, so does the necessary professionalization in the schools. Yet, we detected several speeds in schools. For example, not every school has a vision on ICT that takes into account the local context and public of the school. In addition, we also observed differences in terms of the school teams’ digital competencies. A good professionalisation policy that takes into account the needs and requirements of the individual school team members can strengthen these competences. In addition, the results emphasized a positive relationship between the digital competences and attitudes of teachers towards ICT. The majority of parents and pupils feel less involved on the content of the ICT policy plans of their schools. There is room for improvement regarding the schools’ communication with pupils and parents about these policy plans too. The results point also to differences between pupils and parents in terms of digital competences and attitudes towards ICT.
Regarding the second research question, both principals and teachers indicated that agreements about the didactical use of ICT in schools are often non-existing. The lack of agreements on evaluation practice in almost three quarters of the schools is a good example of this. We found a similar result for pupils’ reflection practices. Nevertheless, research point to the importance of actively involving pupils in their evaluation and reflection on the use of digital learning resources to gain an insight into their own learning process.
Finally, considering the preconditions for high-quality digital education, principals and teachers are not so much concerned about their competences and attitudes, but highlighted the need for more time and space in order to strengthen their digital competencies because their current time for learning is limited by other professional duties.

References
Albion, P. R., & Tondeur, J. (2018). Section introduction: Professional learning and development of teachers. Second handbook of information technology in primary and secondary education, 377-379.
Barometer Digitale Inclusie. (2020). Koning Boudewijnstichting. Geraadpleegd op 19 oktober 2022, van https://kbs-frb.be/nl/barometer-digitale-inclusie-2022 Belet, Margot. "The importance of relevance to student lives: The impact of content and media in introduction to sociology." Teaching Sociology 46, no. 3 (2018): 208-224.
Cohen, L., Manion, L., & Morrison, K. (2017). Research Methods in Education (8th ed.). Routledge. https://doi.org/10.4324/9781315456539
Cussó-Calabuig, R., Farran, X.C. & Bosch-Capblanch, X. Effects of intensive use of computers in secondary school on gender differences in attitudes towards ICT: A systematic review. Educ Inf Technol 23, 2111–2139 (2018). https://doi.org/10.1007/s10639-018-9706-6
del Carmen Ramírez-Rueda, M., Cózar-Gutiérrez, R., Colmenero, M. J. R., & González-Calero, J. A. (2021). Towards a coordinated vision of ICT in education: A comparative analysis of Preschool and Primary Education teachers’ and parents’ perceptions. Teaching and Teacher Education, 100, 103300.
Digitale transformatie in het Vlaamse onderwijssysteem: hervorming van ICT-teams op school. (z.d.). www.vlaanderen.be. Geraadpleegd op 19 oktober 2022, van https://www.vlaanderen.be/kenniscentrum-digisprong/themas/ict-coordinatie/digitale transformatie-in-het-vlaamse-onderwijssysteem-hervorming-van-ict-teams-op-school
Kreijns, K., Xu, K., & Weidlich, J. (2021). Social presence: Conceptualization and measurement. Educational Psychology Review, 1-32.
Maderick, J. A., Zhang, S., Hartley, K., & Marchand, G. (2016). Preservice teachers and self-assessing digital competence. Journal of Educational Computing Research, 54(3), 326-351. Marler, W. (2018). Mobile phones and inequality: Findings, trends, and future directions. New Media & Society, 20(9), 3498-3520.
Otterborn, A., Schönborn, K., & Hultén, M. (2019). Surveying preschool teachers’ use of digital tablets: general and technology education related findings. International journal of technology and design education, 29(4), 717-737.
Salikhova, Nailya R., Martin F. Lynch, and Albina B. Salikhova. "Psychological Aspects of Digital Learning: A Self-Determination Theory Perspective." Contemporary Educational Technology 12, no. 2 (2020).
Tsetsi, E., & Rains, S. A. (2017). Smartphone Internet access and use: Extending the digital divide and usage gap. Mobile Media & Communication, 5(3), 239-255. Vlaamse onderwijsraad (Vlor) (2022).
Advies over de relatie tussen onderwijs, de overheid en de Ed Tech-markt. Beschikbaar op www.vlor.be
Voogt, J., Fisser, P., Pareja Roblin, N., Tondeur, J., & van Braak, J. (2013). Technological pedagogical content knowledge–a review of the literature. Journal of computer assisted learning, 29(2), 109-121.
Zijlstra, T., Hoogendoorn-Lanser, S., & Wijgergangs, K. (2017, October). The impact of survey completion with a mobile device in a longitudinal transport study. In European Transport Conference (pp. 4-6).
 
5:15pm - 6:45pm11 SES 13 A: School Financing and School Policy
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Mudassir Arafat
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Input Stratification? The Case of Tracking & School Resources

Maximilian Brinkmann, Nakia El-Sayed, Janna Teltemann

University of Hildesheim, Germany

Presenting Author: Brinkmann, Maximilian

Early between school tracking (i.e. tracking) is an institutional set-up that has strong implications for students and their future outcomes. While there is increasing consensus that tracking does not increase average levels of achievement but average levels of inequality (e.g., Terrin & Triventi 2022, meta-analysis), we know little about the underlying mechanisms. The focus of this study is a potential mechanism that is as prominent as it is understudied: the role of unequal school resources in tracked systems. In other words, while higher tracks (i.e. academic tracks) benefit from above-average resources, lower tracks (i.e. vocational tracks) are disadvantaged. We call such a situation input-stratification, because the total input (resources) assigned to the education system is stratified across tracks (c.p. Esser 2016).

Input stratification has been discussed as a potential mechanism throughout the literature (e.g. Betts 2011; van de Werfhorst 2021, Terrin & Triventi 2022), but there are hardly any studies providing evidence on the unequal distribution of resources across tracks. For instance, van de Werfhorst (2021; 1214) and Terrin & Triventi (2020, meta-analysis; 7) still cite the prominent paper of Brunello & Checchi (2007). While Brunello & Checchi (2007; 795) only provide student-teacher ratios in 2004 for a handful of countries, ostensibly because resources are not the focus of their study, they apparently provide the best evidence on input-stratification to this date. The scarcity of evidence may be explained by the inherent difficulty of estimating (causal) effects of school resources (e.g. Gibbons & McNally 2013) combined with the scarcity of high-quality data that allows for comparison across countries. Thus, a consequence is that we do not know whether a) input stratification (unequal resources) actually exists in tracked systems and b) whether it drives unequal levels of achievement in different tracks. Although answering b) is outside the scope of this study, we argue that it is still valuable to know about a) since b) presupposes a). Our approach is driven by two goals.

First, we want to provide a thorough discussion on the role of school resources in tracked systems. While the topic of school resources is often mentioned in the literature, it is hardly spelled out with its theoretical implications. The unequal distribution of resources across tracks could be a mechanism that explains why tracked systems fail to show increased (average) achievement but increased social inequalities (i.e. Terrin & Triventi 2022). Accordingly, students on higher (lower) tracks benefit (are disadvantaged) through resources above (below) average. But since tracks are segregated by social status (Strello et al. 2022), high (low) status students are more (less) likely to benefit from above-average resources, explaining the increased social inequality in tracked systems.

Given the limited space, we merely note that we incorporate the existing literature on school resources in our study (e.g. Krüger 2003 vs. Hanushek 2003). We distinguish between explicit resources (e.g. official government funding) and implicit resources which are indirect consequences of the institutional set-up (e.g. self-selection of more capable or motivated teachers). Further we discuss whether differences in resource allocation across tracks should be seen as a bug or a feature of tracked systems (i.e. vocational vs. academic training; c.p. Esser 2021).

Second, we want to assemble data sources that are either informative about resource levels 1) across tracks within a country or 2) across tracked and untracked countries. Noting the inherent difficulties of estimating the effects of school resources, we will restrict our analysis to a descriptive analysis in order to answer the question whether the existence of input-stratification is plausible given the existing evidence.


Methodology, Methods, Research Instruments or Sources Used
Research plan:
In general we want to restrict our analysis to a thorough descriptive analysis of tracked (and untracked) education systems. Our analysis comprises two parts: First, using administrative data from the German speaking tracking countries (Germany, Austria, Switzerland, Luxembourg) we will discuss educational spending per student, teacher-student ratios, teaching hours and teacher qualifications across tracks in a case-study like fashion. Second, for our main analysis we will use large-scale assessment data (LSA) from the last 25 years.

Unfortunately, measures of resources in PIRLS & TIMSS 4 (primary school) and PISA (secondary school) are largely incomparable which hampers efforts to compare change over time from primary to secondary school. However, it is possible to compute teacher-student ratios from LSA data (e.g. Woessmann & West 2006), which is a resource indicator commonly used in the school resource literature (e.g. Gibbons & McNally 2013). This allows us to track the change in the variance of teacher-student ratios in tracked and untracked countries as they move from primary (no country is tracked) to secondary school (some countries have administered tracking).

Lastly we will use PISA data to compute a broader set of resource indicators which resonates with our idea of explicit and implicit resources. Using PISA has the advantage that we can draw on a broad set of variables and that we can directly identify the school track (as compared to TIMSS 8). Unfortunately, however, PISA has only administered (short) teacher questionnaires since 2015. To remedy this shortcoming, we aim to match PISA with TALIS data, which provides in-depth data on the teacher and school principal level. Taken together, this allows us to compute different indicators of explicit and implicit school resources (e.g. material resources, student-teacher ratios, teacher qualifications, teacher motivation, parental support) across track types.


Conclusions, Expected Outcomes or Findings
We are still in the process of data collection (cleaning & assembling). The limited evidence on school resources indicates large differences in student-teacher ratios in five tracked systems (Brunello & Checchi, 2007). However, Brunello & Checchi also report administrative data from Austria that shows that spending per student is higher in vocational tracks.

Overall, we expect more mixed evidence when it comes to explicit resources (i.e. official allocation of resources through the government) as compared to implicit resources (i.e. differences in resources as a consequence of the institutional set-up). Tracking is often understood as a form of stratification, inducing an implicit “better” or “worse” into the system. Further, it is theoretically plausible (e.g. Boudon 1974) and empirically validated (Strello et al 2022) that tracked systems are segregated by social status. We argue that this could lead to differences in implicit resources because involved actors take this stratification and segregation into account. More able or motivated teachers could self-select into higher tracks, parental support via booster clubs is likely to depend on the average social status of parents at the school and so on.

References
Betts, J. R. (2011). The economics of tracking in education. In Handbook of the Economics of Education (Vol. 3, pp. 341-381). Elsevier.

Boudon, R. (1974). Education, opportunity, and social inequality: Changing prospects in western society.

Brunello, G., & Checchi, D. (2007). Does school tracking affect equality of opportunity? New international evidence. Economic policy, 22(52), 782-861.

Esser, H. (2016). Bildungssysteme und ethnische Bildungsungleichheiten. Ethnische Ungleichheiten im Bildungsverlauf: Mechanismen, Befunde, Debatten, 331-396.
[English: “Education systems and ethnic educational inequalities” in “Ethnic inequality along the educational pathway: Mechanisms, Results, Debates”]

Esser, H. (2021). » Wie kaum in einem anderen Land...«?: Die Differenzierung der Bildungswege nach Fähigkeiten und Leistungen und ihre Wirkung auf Bildungserfolg,-ungleichheit und-gerechtigkeit. Band 1: Theoretische Grundlagen. Campus Verlag.
[English: “‘Hardly any other country…’?: Differentiation of educational pathways according to aptitude and performance and their consequences for educational attainment, inequality and justice. Volume one: Theoretical foundations” ]

Gibbons, S., & McNally, S. (2013). The effects of resources across school phases: A summary of recent evidence.

Hanushek, E. A. (2003). The failure of input‐based schooling policies. The economic journal, 113(485), F64-F98.

Krueger, A. B. (2003). Economic considerations and class size. The economic journal, 113(485), F34-F63.


Strello, A., Strietholt, R., & Steinmann, I. (2022). Does tracking increase segregation? International evidence on the effects of between-school tracking on social segregation across schools. Research in Social Stratification and Mobility, 78, 100689.

Terrin, E., & Triventi, M. (2022). The effect of school tracking on student achievement and inequality: A meta-analysis. Review of Educational Research, 00346543221100850.

van de Werfhorst, H. G. (2021). Sorting or mixing? Multi‐track and single‐track schools and social inequalities in a differentiated educational system. British Educational Research Journal, 47(5), 1209-1236.

Woessmann, L., & West, M. (2006). Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS. European Economic Review, 50(3), 695-736.


11. Educational Improvement and Quality Assurance
Paper

What Data to Use for Planning Educational Reforms? A Meta-analysis of Educational Interventions' Research in post-Soviet Countries

Roman Zviagintsev1, Julia Kersha2

1University of Vienna; 2HSE University

Presenting Author: Zviagintsev, Roman; Kersha, Julia

The education system is a key social institution of any modern state, critical to the socio-economic and cultural development of society. For this institution to work effectively in a complex and uncertain world, its management must be data-driven (Burns et al., 2016). Informed transformation of educational content and the systematic use of new technologies in pedagogical practice are important characteristics of effective educational systems (Nelson & Campbell, 2017; Wiseman, 2010). Only research can determine "what works," similar to what is successfully implemented in, for example, health care (Davies, 1999).

The lack of effective use of research in educational policy and practice is regularly highlighted, and mechanisms are sought to strengthen the impact of research on decision-making in education systems (OECD,2022). The importance of and need for evidence-based decision-making was particularly vivid during the pandemic, highlighting how the existing gap between research and policy decisions may have dramatic consequences (Stuart & Dowdy, 2021).

A fairly wide pool of interventions designed to increase educational outcomes exists and well documented in international research. Prominent examples of cataloging this type of information are, for example, the following electronic resources:

- The U.S. Department of Education's Institute of Educational Sciences WWC open repository[1];

- repository of practices to help Every Student Succeeds Act, Evidence for ESSA[2];

- BEE Encyclopaedia of Educational Practices, created by Johns Hopkins University[3].

Webpages such as these appear due to the fallacy of the view that any educational projects are useful and effective. Moreover, evaluations of their effectiveness are often highly contradictory. It is not uncommon to find that proposed interventions do not lead to any worthwhile outcome (Lortie-Forgues & Inglis, 2019). However, the very fact that educational programs are evaluated for their effectiveness is extremely important in optimizing resources for their implementation and scaling.

The movement toward evidence-based education is most evident in English-speaking countries (Dekker & Meeter, 2022). However, there is growing interest in China (Slavin et al., 2021), Sweden (Segerholm et al., 2022), the Netherlands (Wubbels & van Tartwijk, 2017), Italy (Mincu & Romiti, 2022), France (Bressoux et al., 2019)[4]. To a much lesser extent, we see the development of this movement in the Post-Soviet states. The importance of the studies has been only increasing and is especially evident since these countries began to participate in PISA, the results of the first waves of which opened up possibilities for analyzing the connection between differences in the quality of educational outcomes and the reforms implemented (Khavenson & Carnoy, 2016).

Of particular interest is the ability of countries to generate and use their own "contextualized" research data about "what works" in education. Here one encounters a problem — in general, there are few works characterizing the landscape of educational research in the post-Soviet space (Chankseliani, 2017; Hernández-Torrano et al., 2021). We found no systematic review or meta-analysis of publications summarizing research on the impact of educational interventions on school students' academic outcomes that could allow policymakers and practitioners to construct informed educational policies.

The purpose of our paper is to evaluate the effectiveness of educational improvement programs in the Post-Soviet states, as there is a clear lack of systematization of such information in the presence of a clear demand for data-driven reforms. Our paper simultaneously addresses two tasks leading to the overall goal — a meta-analysis of studies summarizing the experience of post-Soviet countries in terms of interventions aimed at improving educational outcomes, and the search for a basis for building educational policies that would consider the current state of affairs.

[1] https://ies.ed.gov/ncee/wwc/

[2] https://www.evidenceforessa.org/page/about

[3] https://bestevidence.org/

[4] References are omitted in the list due to the lack of space.


Methodology, Methods, Research Instruments or Sources Used
To answer the research question meta-analysis methodology was used. While conducting meta-analysis, we carefully followed the PRISMA statement guidelines (Page et al., 2021). To create the database, we used the following selection criteria:
• the study must be devoted to evaluating the effectiveness of the program aimed to improve the academic achievements of schoolchildren or develop their skills facilitating successful learning;
• the study should be conducted in the form of an experiment (the presence of randomization procedure in the study was coded, but was not a strict exclusion criterion);
• schoolchildren must be the general population of the study;
• the study is conducted on the sample of schoolchildren from one of the 15 post-Soviet countries;
• the text is published in Russian or English;
• the research must be a published article in a peer-reviewed journal or a defended PhD thesis;
• year of publication from 1992 to 2023.
To achieve the research goal, we systematically searched in these 4 scientific databases: Scopus, Web of Science, ProQuest, Google Scholar (the search was conducted in September 2022) . We were looking for the keywords that characterize the study design (experiment, control, rct etc.) and the dependent variable (achievement, performance, learning outcomes etc.). A total number of the publications suitable for analysis was 262. All of them were screened using Rayyan software . At the last step, we selected 27 papers for the full-text analysis. Three members of the research group went through the procedure of coding. The key parameters recognized in the coding scheme were: authors, year, type of publication, country, sample characteristics, sample size, dependent variable, type of the intervention, duration of the intervention, presence of randomization.
We chose Cohen's d as the key statistic for an effect size calculation. We used an online calculator  to convert the statistics published in studies to d. Afterwards, we carried out a classical version of the statistical meta-analysis with the random effects model in JASP . In addition, we assessed the homogeneity of the studies and checked whether the results demonstrate any kind of publication bias. To estimate the bias, we used the graphical method and the Egger’s test, as well as the selection models.

Conclusions, Expected Outcomes or Findings
Our final results are based on 28 effect sizes from 27 publications with a total sample of 14,853 schoolchildren. According to the age of the respondents, the studies covered samples from grades 1 to 9. It should be noted that there were no papers with high school students as the general population. Regarding the program type, vast majority of them were classified as pedagogical technologies. The overall mean effect size of the studies is 0.48 with a 95% confidence interval of 0.36 to 0.60 (and a range of 0.02-1.55). At the same time, we see that the effect size varies greatly across the studies — indicator of heterogeneity equals 82% (i2). If we consider three studies in which proper randomization was carried out, the effect size of the interventions decreases to 0.07 and becomes insignificant.
We are going to build our discussion around the particular limitations and general barriers one the way of carrying high quality research. For example, we can conclude from the available research that in those countries all along there were and are: difficulties in accessing data when conducting research (Jonbekova, 2020); specifics of research culture and methodology, especially experimental research (Gromyko & Davydov, 1998); problems with standards of reviewing, publishing, academic integrity (Kuzhabekova & Mukhamejanova, 2017), a general low level of integration into international science.
It is important to note that the idea of "what works" is only possible in a situation where the goals of the education system are clear (Hammersley, 2005), but many countries were dealing with much more severe issues since the collapse of the USSR. Politicians' words about the need for research are often just a blind "fashion" following. The very statement “we need data-driven policy” in a situation where there are almost no data is, at the very least, deceitful.

References
Burns, T., Köster, F., & Fuster, M. (2016). Education Governance in Action. OECD. https://doi.org/10.1787/9789264262829-en
Chankseliani, M. (2017). Charting the development of knowledge on Soviet and post-Soviet education through the pages of comparative and international education journals. Comparative Education, 53(2), 265–283. https://doi.org/10.1080/03050068.2017.1293407
Davies, P. (1999). What is Evidence-based Education? British Journal of Educational Studies, 47(2), 108–121. https://doi.org/10.1111/1467-8527.00106
Dekker, I., & Meeter, M. (2022). Evidence-based education: Objections and future directions. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.941410
Gromyko, Iu. V., & Davydov, V. V. (1998). The Conception of Experimental Work in Education Ideas for a formative experiment. Journal of Russian & East European Psychology, 36(4), 72–82. https://doi.org/10.2753/RPO1061-0405360472
Hernández-Torrano, D., Karabassova, L., Izekenova, Z., & Courtney, M. G. R. (2021). Mapping education research in post-Soviet countries: A bibliometric analysis. International Journal of Educational Development, 87, 102502. https://doi.org/10.1016/j.ijedudev.2021.102502
Jonbekova, D. (2020). Educational research in Central Asia: methodological and ethical dilemmas in Kazakhstan, Kyrgyzstan and Tajikistan. Compare: A Journal of Comparative and International Education, 50(3), 352–370. https://doi.org/10.1080/03057925.2018.1511371
Khavenson, T., & Carnoy, M. (2016). The unintended and intended academic consequences of educational reforms: the cases of Post-Soviet Estonia, Latvia and Russia. Oxford Review of Education, 42(2), 178–199. https://doi.org/10.1080/03054985.2016.1157063
Kuzhabekova, A., & Mukhamejanova, D. (2017). Productive researchers in countries with limited research capacity. Studies in Graduate and Postdoctoral Education, 8(1), 30–47. https://doi.org/10.1108/SGPE-08-2016-0018
Lortie-Forgues, H., & Inglis, M. (2019). Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned? Educational Researcher, 48(3), 158–166. https://doi.org/10.3102/0013189X19832850
Nelson, J., & Campbell, C. (2017). Evidence-informed practice in education: meanings and applications. Educational Research, 59(2), 127–135. https://doi.org/10.1080/00131881.2017.1314115
OECD. (2022). Who Cares about Using Education Research in Policy and Practice? OECD. https://doi.org/10.1787/d7ff793d-en
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, n71. https://doi.org/10.1136/bmj.n71
Stuart, E. A., & Dowdy, D. W. (2021). Evidence-based COVID-19 policy-making in schools. Nature Medicine, 27(12), 2078–2079. https://doi.org/10.1038/s41591-021-01585-2
Wiseman, A. W. (2010). The Uses of Evidence for Educational Policymaking: Global Contexts and International Trends. Review of Research in Education, 34(1), 1–24. https://doi.org/10.3102/0091732X09350472


11. Educational Improvement and Quality Assurance
Paper

An Explanatory Quantitative Study of the Funding Policy Supporting British Academies and American Charter Schools

Tyrone Bynoe

St. Bonaventure University, United States of America

Presenting Author: Bynoe, Tyrone

Choice theory has shaped much of public-school policy throughout the world as countries have adopted free market principles to establish educational alternatives to state and traditional public schools with the goal of improving chronically failing schools. While public dollars are being used globally to support private school enterprises, public dollars are also being used to finance public school alternatives to the state and traditional public-school offerings. On both sides of the Atlantic Ocean during the 1990s, school choice theory promoted public school alternatives to state or traditional public schools with the emergence of Multi-Academy Trusts in the United Kingdom and the rise of Charter Schools in the United States. Beginning with the Education Reform Act of 1988 -- which allowed for the formation of city technical academies -- and since the “academisation” of schools in 2010, the number of multi-academy trusts have soared from 407 in 2011 to 6493 during 2017 (Male). Concurrently since the first charter school in Minnesota in 1992, the number of charter schools in the United States have climbed to 7,038, and 44 states now have charter school laws. Both school growth trends in choice schools have had the unflinching endorsement of respective national governments.

Theoretical Framework: Choice Theory is the study’s theoretical framework. Choice theory is the antithesis of the governmental centralization of public schools, as conceived by German fiscal influence in the United States educational policy during the early twentieth century (Seligman). Choice Theory seeks equitable and efficient resource distribution through parental selection and market-fueled competition. Whether promoted by Milton Friedman in the 1950s (Friedman) or Adam Smith during the 1770s, Choice Theory application advocates a variety of resource distribution schemes in contemporary school finance policy, featuring tuition tax credits, vouchers, funding portability, and charter schools. Given this framework, my study will evaluate how the theoretical framework has effectively or ineffectively been implemented in school finance policy within British multi-academy trusts and US charter schools through an equity and efficiency analysis. Due to the researcher’s limitations to aggregate data on all multi-academy trusts and charter schools in both respective countries, an explanatory quantitative study will be conducted among choice schools in these countries’ largest municipalities: London (or Inner London - its schools within fourteen local education authorities) and New York City (NYC’s Department of Education). Given this theoretical backdrop, the paper’s main research questions include:

1. To what extent do the distribution of per pupil expenditures of London multi-academy trusts and New York City (NYC) charter schools vary when compared to the distribution of these per pupil expenditures in state schools and the traditional public schools in each of these municipalities?

2. To what extent does the per pupil funding formula allocate to the magnitude of student need in both London multi-academy trusts and NYC charter schools when compared to this needs-based allocation in the traditional public and state schools in each of these municipalities?

3. Given the school spending data from the Income and Expenditure Reports in England for state and choice schools and the School Based Expenditure Reports in the NYC public schools, to what extent can one track the usage of resources to determine what percentage of resources are being allocated for total uses on instruction – minus capital outlays, security, transportation, building up-keep, and other non-instructional needs?

4. Are multi-academy trusts and charter schools respectively in London and NYC getting more bang for the “pound” or “buck” when analyzing the relationship between student gain scores and per pupil expenditures in these schools, especially when compared to analyzing the same data of state and traditional schools in these two municipalities?


Methodology, Methods, Research Instruments or Sources Used
Methods:
Question 1 will use descriptive statistics and school-finance equity measures to analyze the spending variability in different parts of the distribution for each set of schools.  Question 2 will use an ordinary least square regression to determine if the unstandardized beta coefficients, partial correlations and effect-sizes of this statistical model reinforce or refute real allocation of need based equalization policies in both sets of schools within both municipalities.  Question 3 will use a functional analysis model, which will track spending at various decentralized levels of schooling in each set of schools within the respective municipalities to analyze the extent the spending is being allocation to instruction.   Question 4 will derive effective size from hypothesis testing and correlations to evaluate the efficiency of the funding policy as a function of student gain scores.  
The study’s unit of analysis is at the school level, and subsequently embraces the new school-finance perspective, which calls for more meaningful input-output analysis at the school level (Grubb & Huerta).  School finance data will be aggregated at the pupil level for the years 2011-2012 to 2016-2017.  The British spending data will be collected from the British Department of Education’s (DfE’s) Local Authority and School Expenditure reports.  This specific British per pupil school finance data source contains revenue and expenditure data on the primary and secondary state schools of Inner London’s fourteen local education authorities.  From this same agency, per pupil school finance data on multi-academy trusts will be aggregated from the Income and Expenditure Reports in England for only years 2011-2012 to 2016-2017, focusing only on financial data of multi-academy trusts in the 14 Inner London local education authorities.  The NYC school finance data will be aggregated from its Education Department’s School Based Expenditure Reports, which contains rich per pupil finance data on all NYC public schools, including charter schools, by school and function respectively.  All data will be reported in Excel spreadsheets, and then transported to SPSS for statistical analysis.  British school level student performance data will originate from its accountability program (i.e. A-level exams of secondary schools), and NYC school-level student performance will be aggregated from the publicly available state accountability data system.

Conclusions, Expected Outcomes or Findings
Anticipated findings:   It is hypothesized that expenditures patterns will vary between choice schools and state/traditional schools in respective Inner London and NYC municipalities using horizontal and vertical equity measures.   Horizontal equity measures will include the coefficient of variation (measuring spending variability within the middle 68 percent of the distribution), McLoone Index (measuring spending variability under the median), Verstegen Index (measuring spending variability above the median) as well as other measures.      
               Secondly, vertical equity measures will generate findings featuring an interpretation of the beta coefficients of an ordinary least squared regression to evaluate whether the actual funding levels of the need equalization formula are being allocated according to the formula’s design.
Thirdly, it is anticipated that some measure of decentralized resource allocation will be found when using the functional analysis model to track funding at specific levels of schooling.  
Additionally, analyses between spending and student performance will either validate evidence toward trends of economies of scale and efficiency or show evidence for a diseconomies of scale and inefficiency.

References
References

Bynoe, T. (2018). A historical and conceptual overview of school-finance equalization models – a book chapter. In BenDavid-Hadar, I. (eds). Education, equity and economy. New York, NY: Springer, Inc

Bynoe, T. & Armstead, A. (2019).  American charter schools and British academy trusts:  An comparative perspective on the school choice movement since the 1990s.  In Storey, V. (ed.). Pathways to school leadership: Negotiating context and diversity in England and the United States.  Charlotte, NC:  Information Age Publishing, Inc (In-Press).

Bynoe, T. & Feil, J. (November, 2016). School finance equity: Lessons learned from Michigan’s charter school spending patterns, School Business Affairs Journal. 82(10), 19-22.

Department for Education.  (2016). Educational excellence everywhere:  Presented to Parliament by the secretary of education by command of her Majesty.   (Cm 9230).  Retrieved from:  http://www.educationengland.org.uk/documents/pdfs/2016-white-paper.pdf
Friedman, M. (1955). The role of government in education.  Retrieve from: http://homepage.fudan.edu.cn/jfeng/files/2011/08/role-of-government-in-education_Friedman.pdf
Grubb, W.N. & Huerta, L.A. (2001).  Straw into gold, resources into results:  Spinning out the implications of the improved school finance.  Journal of Education Finance, 31(14), 334-359.
Hoxby, C. M. (2003). School choice and school productivity: Could school choice be a tide that lifts all boats? In C.M. Hoxby (Ed.), The economics of school choice (pp. 287-341). Chicago, IL: University of Chicago Press.
Ladd, H. & Fiske, E. (November 2016).  Report:  Lessons for US charter schools from the growth of academies in England.  Brookings.   Retrieved from:  https://www.brookings.edu/research/lessons-for-us-charter-schools-from-the-growth-of-academies-in-england/
Male, T. (September, 2017).  Multi-academic trusts:  A background briefing paper.  London, United Kingdom:  The London Centre for Leadership in Learning.  Retrieved from:  http://www.lcll.org.uk/uploads/2/1/4/7/21470046/2017_multi-academy_trusts_-_a_background_briefing_paper_-_trevor_male.pdf
Seligman, E. (December 1908). Progressive taxation in theory and practice, 3rd Series, American Economic Association Quarterly.  9(4).
West, A. (2015). Education policy and governance in England under the coalition government (2010-15):  Academies, the pupil premium and free early education.  London Review of Education, 13, 21-36.
West, A. & Wolfe, D. (2018).   Academies, the school system in England and a vision for the future:  Clare Market Papers No. 23.  London, England:  Education Research Group-Department of Social Policy-London School of Economics and Political Science.  Retrieved from:  http://www.lse.ac.uk/social-policy/Assets/Documents/PDF/Research-reports/Academies-Vision-Report.pdf
 
Date: Friday, 25/Aug/2023
9:00am - 10:30am11 SES 14 A: Quality of Education Systems
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Mudassir Arafat
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

Constructing Educational Systems in the Global South – Role and impacts of international organisations through the views of Finnish education experts

Íris Santos, Elias Pekkola

Tampere University, Finland

Presenting Author: Santos, Íris

International organisations and their role and impacts on local/national and global policymaking in the sector of education have been object of discussion in a vast number of academic studies during the past decades (e.g. McNeely 1995; Verger, Novalli, and Altinyelken 2018; Martens and Windzio 2022). However, the perspectives of the practitioners working within these international organisations are rarely analysed. In this study we analyse the role and impacts of international organisations in the differentiation between education systems through the understandings of education experts in these organisations. Empirically the paper builds on thematic semi-structured interviews with Finnish education experts working in international organisations. Our research question is: How are the international organisations impacting the reduction, construction or continuity of differentiation between the educational systems of the Global North and the Global South through development collaboration initiatives?

We develop our analysis by combining the onto-epistemic lenses of complexity thinking (e.g. Cilliers 1998) with the Luhmaniann concept of differentiation (e.g. Luhmann 1982; Baraldi, Corsi, and Esposito 2021;) and utilise the perspective of sensemaking as a tool to guide the analysis (e.g. Weick, Sutcliffe, and Obstfeld 2005; Brown, Colville, and Pye 2015). The combination of different perspectives and concepts has been done successfully in previous literature (e.g. Zahariadis 1998; Howlett et al. 2016; Santos 2022) demonstrating that different theoretical lenses can benefit from being aggregated, not only leading to a better understanding of the phenomenon under analysis, but also by contributing to advancements of each of the different theories utilised, while also innovating the ways in which theories are utilised in qualitative research. In the case of this study, this theoretical aggregation enables the development of a more holistic analysis of the dynamics of development cooperation in the education sector, contributing not only to a better understanding of these dynamics but also of how the Finnish education experts involved in them make sense of what happens around their professional environment.

The study starts with two assumptions a) that there is an imaginary, blurry divide between the so-called Global North (often described as developed countries, also donor countries) and the so-called Global South (frequently see as the underdeveloped, peripherical countries, which are the receivers of international support); and b) that the initiatives developed by international organisations impact the development of education globally, but more intensively in the Global South as it has been discussed in earlier research (e.g. Verger, Novelli, and Altinyelken 2018).

Hence, while bundles of studies, analyse the role of international organisations from distant, macro-economic perspectives. We study the sensemaking of the (Finnish) education experts working in international organizations to understand, from an internal perspective, the roles and impacts of international organisations in development cooperation, with the aim of comprehending if these organisations indeed maintain or even increase differentiations North-South or if they actually manage to reduce this differentiation and harmonise education systems, access and quality globally, as it is the aim Sustainable Development Goal 4 – Quality of education. This study contributes to complementary fields of research, namely Global Education Policies studies and Development studies.

Our findings indicate that interviewees understand the complexity and ambiguity of development cooperation and that international organisations, by keeping the leadership of development cooperation initiatives often maintain and can even increase differentiations among education systems along the divide North-South.


Methodology, Methods, Research Instruments or Sources Used
This study utilised a convenience sample (Etikan, Musa, and Alkassim 2015); selected in the context of a larger project that analysed (1) the roles and impacts of national experts in development cooperation in the sector of education and (2) how these experts understand to be the roles and impacts of their organisations in the development of education locally/nationally and globally. We interviewed 31 Finnish education experts working (or that until recently have worked) in international organisations (inter-governmental and non-governmental). The interviewees selection was done in two steps. First, a survey was conducted to already identified education experts potentially interested in participating in an interview. Second, through the agreed interviews a snowballing method was utilised to identify other Finnish experts involved in development cooperation in the sector of education within international organisations. This strategy was chosen because one of the criteria for the identification of expertise is the consensus among peers that one is an expert in a specific field (Chi 2006, 22–23). The sample represents a group of international experts placed in a variety of expert positions. The nationality of the interviewees is taken into account to understand their work, but the analysis is not done from a “national perspective” but rather from an organizational perspective into global development of education.
The study deployed a qualitative methodological approach started with a review of earlier literature - e.g. international organisations’ reports (e.g. World bank and UNESCO) earlier academic literature on development cooperation and international organisations, and previous reports discussing the Finnish participation in international cooperation in the sector of education, which allowed us to understand how these publications discuss the roles and impacts of international organisations in education development and how Finland position itself within these international dynamics.
The data obtained from the interviews was analysed qualitatively using content analysis (Schreier 2014), inductively and deductively. Thus, a set of categories were created beforehand based on readings of previous literature which seemed to be needed to retrieve the information necessary to respond to the research question (e.g. “significant organisations in development cooperation for education”, “roles of international organisations”, “experts views on their own role and influence in education development”, “problems and challenges in development cooperation initiatives”. These categories were then complemented with the information offered by the interviewees (e.g. “significant organisations in development cooperation for education: UNESCO”, “roles of international organisations: develop international awareness”.

Conclusions, Expected Outcomes or Findings
The analysis revealed that the participants have holistic understandings of the world of development cooperation in education and that they perceive international organisations’ roles and impacts in the development of education locally, nationally and globally to be complex, ambiguous and paradoxical. On the one hand, international organisations are crucial in supporting the development of education systems in the countries of the Global South, on the other hand, these organisations’ initiatives, if not contextualised in the place of implementation and if not led by these contexts’ actors, often have unintentional and unpredictable outcomes that contribute to the maintenance and even increasing of differentiation between the education systems of the Global North (where donors usually are) and the education systems of the Global South (where development cooperation initiatives take place). While international organisations might, in all good intentions, be aiming at harmonising high-quality education globally, aspects such as the ones mentioned above lead to dynamics that directly or indirectly maintain and in some cases, increase the differentiation between education systems in both sides of the divide South-North.
Furthermore, in line with earlier studies (e.g. Sultana 2019; Menashy 2017), and mentioned in the interviews, international organisations need to change the assumption that they have the valid knowledge about how education must be developed. To reduce differentiations between education systems in the Global North and the Global South, dynamics of development cooperation in education need to become co-constructed long-term partnerships where the leadership and, therefore, the power of decision is on the local/national actors and their knowledge and priorities are recognised as pivotal. This transformation requires, thus, a change in the established power dynamics and a democratic synergy of agendas (Centeno 2017) between governments of the receiving countries and international organisations, in order to grant these systems’ sustainable development.

References
Baraldi, C., Corsi, G., & Esposito, E. (2021). Unlocking Luhmann: A keyword introduction to systems theory. Frankfurt am Main: Bielefeld University Press.

Brown, Andrew, Ian Colville, and Annie Pye. 2015. “Making Sense of Sensemaking in Organisation Studies.” Organization Studies 36 (2): 265-277. doi: 10.1177/0170840614559259.

Centeno, Vera. 2017. The OECD’s Educational Agendas: Framed from Above, Fed from Below, Determined in Interaction. A Study on the Recurrent Education Agenda. Berlin: Peter Lang.

Chi, Michelene T. H. 2006. “Two Approaches to the Study of Experts’ Characteristics.” In The Cambridge Handbook of Expertise and Expert Performance, edited by K. Anders Ericsson, Neil Charness, Paul J. Feltovich, and Robert R. Hoffman, 21–30. Cambridge: Cambridge University Press.

Cilliers, Paul. 1998. Complexity and Postmodernism: Understanding Complex Systems. London: Taylor & Francis.

Etikan, Ilker, Sulaiman Abubakar Musa, and Rukayya Sunusi Alkassim. 2015. “Comparison of Convenience Sampling and Purposive Sampling.” American Journal of Theoretical and Applied Statistics 5 (1): 1-4. doi: 10.11648/j.ajtas.20160501.11.

Howlett, Michael, Allan McConnell, and Anthony Perl. 2016. “Moving Policy Theory Forward: Connecting Multiple Stream and Advocacy Coalition Frameworks to Policy Cycle Models of Analysis.” Australian Journal of Public Administration 76 (1): 65-79. doi: 10.1111/1467-8500.12191.

Luhmann, Niklas. 1981. “The World Society as a Social System.” International Journal of General Systems 8: 131-138.

Martens, Kerstin, and Michael Windzio, eds. 2022. Global Pathways to Education: Cultural Spheres, Networks and International Organisations. Cham: Palgrave Macmillan.

McNeely, Connie. 1995. “Prescribing National Policies: The Role of International Organisations.” Comparative Education Review 39 (4): 483-507.

Menashy, Francine. 2018. “Multi-stakeholder Aid to Education: Power in the Context of Partnership.” Globalisation, Societies and Education 16 (1): 13–26.
doi.org/10.1080/14767724.2017.1356702.

Santos, Íris. 2022. “Externalisations in the Portuguese Parliament and Print Media: A Complexity Approach to Education Policymaking Processes.” PhD Diss., Tampere University and University of Lisbon.

Schreier, Margrit. 2014. “Qualitative Content Analysis.” In The SAGE Handbook of Qualitative Data Analysis, edited by Uwe Flick, 170–183. London: Sage.

Sultana, Farhana. 2019. “Decolonizing development education and the pursuit of social justice.” Human Geography 12 (3): 31- 46.

Verger, Antoni., Mario Novelli, and Hülya Kosar Altinyelken, eds. 2018. Policy and International Development: New Agendas, Issues and Policies. London: Bloomsbury.
Weick, Karl E., Kathleen M. Sutcliffe, and David Obstfeld. 2005. “Organizing and the Process of Sensemaking.” Organization Science 16 (4): 409-421. doi 10.1287/orsc.1050.0133.

Zahariadis, Nicolaos. (1998). Comparing Three Lenses of Policy Choice. Policy Studies Journal 26 (3): 434-448.


11. Educational Improvement and Quality Assurance
Paper

The practice of System Transformation

Pauline Taylor-Guy, Michelle Lasen, Fabienne Van Der Kleij

ACER, Australia

Presenting Author: Taylor-Guy, Pauline; Lasen, Michelle

Education systems globally are grappling with the challenge of how to best prepare children and young people for life and work in the context of a rapidly changing and uncertain future. This focus has become more urgent given the impact of the recent COVID-19 pandemic and the imperative for rapid economic recovery, promoting societal cohesion and building healthy, resilient citizens. In this context, contemporary thinking and research is no longer focussing on system reform, as in reshaping what is already in place, but rather system transformation which requires a fundamental rethink of the purpose and goals of education (Sengeh & Winthrop, 2022) and the ways in which learning is organised to ensure that every child is learning successfully (Masters, 2022). This presentation illustrates the research process and outcomes of doing system transformation in partnership with a national education system. The research contributes new methodological understandings to the practice of system transformation relevant to a broad international audience.

Education systems around the world are grappling with the challenge of how to best prepare children and young people for life and work in the context of a rapidly changing and uncertain future. This focus has become more urgent given the impact of the recent COVID-19 pandemic and the imperative for rapid economic recovery, societal cohesion and building healthy, resilient citizens. In this context, thinking and research is no longer focussing on system reform, as in reshaping what is already in place, but rather system transformation, which requires fundamental rethink of the purpose and goals of education (Sengeh & Winthrop, 2022) and the ways in which learning is organised to ensure every child is learning successfully (Masters, 2022).The system transformation literature (see Fullan, 2009; 2011; Fuller & Kim, 2022; Masters, in press; Sengeh &Winthrop, 2022; Winthrop et al., 2021) reflects common themes including developing a shared vision for the purpose of education, the importance of participatory approaches and, focussing system efforts on the teaching and learning core to ensure all students learn successfully.

Masters (in press), building on a review of practices across five of the world’s top performing systems, including two European systems (Finland and Estonia), conceptualises a learning system as six interconnecting components: a quality curriculum; informative assessment processes; highly effective teaching; comprehensive student support; strong leadership of learning and; a supportive learning ecosystem. At the centre of the learning system is a clear purpose- preparing young people for life and work and ensuring every student learns successfully. Sengeh and Winthrop (2022) conceptualise system transformation similarly. In their model, which they call a participatory approach to transformation, they refer to 3Ps: Purpose, Pedagogy and Position. Purpose is self-explanatory. This is the notion that, in a particular context, a shared vision of the purpose of education needs to be developed. Pedagogy refers to a sharp focus on the teaching and learning core. Position relates to the cohesion between different system elements to support the pedagogical core.

In 2022, ACER partnered with a national education system to implement a major system transformation initiative focussing on five core areas: curriculum implementation, quality teaching, assessment processes and practices, educational leadership, and school and system school improvement.

Consequently, the overarching research questions were:

  1. What are international benchmarks describing international good practice in the areas of this system’s focus focus?
  2. How does the current state of the system compare to international best practice as identified international benchmarks?
  3. What key strategic actions need to be undertaken to work toward international best practice?

Methodology, Methods, Research Instruments or Sources Used
The methodology consisted of three interconnecting phases. In each of these phases, extensive consultations took place to ensure context-appropriateness of the research methods and proposed solutions.

In Phase 1 ACER conducted a review of effective international policies and practices for each aspect of the system based on contemporary research evidence. In addition to drawing on recent international research, the recent review of high-performing systems by Masters (in press) was a key foundational resource. This review examined the practices across five high-performing systems, including two European systems, Finland and Estonia. This provided ‘international benchmark’ policies and practices as aspirations for reform and a foundational reference point for the research project. International benchmarks describe specific, observable aspects of education policy/practice. They serve as aspirational goals for learning systems globally.

In Phase 2, the aim was to gain a deep understanding of the system’s current state.  ACER systematically undertook a detailed review of the system’s s existing policies and practices. This included an analysis of findings from analyses of multiple and rich data sources. Data sources included public school policy, curriculum, and other documents, and system-level quantitative datasets. A school case study component, included interviews with principals and teachers, analysis of school and teacher documents, and extensive consultations with relevant school-based staff. Findings were triangulated against the international benchmarks as an analytical frame to interrogate elements of their learning system that are aligned with good practice, as well as elements that are misaligned, inconsistent, absent, or nascent. Findings were distilled into a gap analysis. Opportunities and priorities for transformation were identified collaboratively and articulated in a set of national benchmarks with realistic targets and timelines for working toward international best practice.

In Phase 3, to support the system in reform, ACER developed a focused set of frameworks to guide short- to medium-term strategic planning and reform. These frameworks were developed based on prior project work and refined through iterative rounds of consultation with key stakeholders.

Conclusions, Expected Outcomes or Findings
This work provides valuable insights for education systems wanting to enhance educational outcomes or those who aim to undertake system transformation.
The first phase of the project resulted in an extensive literature review and set of 34 international benchmarks. This review drew upon contemporary international evidence, and its findings can be applied to a diverse range of systems. In the second phase, a detailed description of the current state of the system was generated, which provided critical insights into strengths and areas for improvement. This informed the development of national benchmarks for the short, medium and long term. High impact and cross-cutting strategies were then identified to develop a coherent system-level focus in implementation of these national benchmarks, and, ultimately, national benchmarks. The final phase resulted in a set of implementation frameworks in the areas of curriculum implementation, pedagogical practices, assessment, leadership, and system transformation. Founded upon our original theoretical framing, our goal has been to ensure a coherent, aligned learning system underpinned by a set of key principles. These principles include better identifying and targeting individual learner needs; promoting holistic student development and wellbeing; and focusing teaching and learning on long-term student growth. This research has resulted in the development of a methodology for work of this kind, as well as frameworks and tools that can be applied to similar work for other school systems across the world. This approach enabled taking account of the multi-layered nature of systems and effectiveness of practices across these layers, and their interrelationships. We encourage systems to consider this work when undertaking system reviews and reform.

References
Australian Council for Educational Research. (2016). National school improvement tool. https://research.acer.edu.au/cgi/viewcontent.cgi?article=1019&context=tll_misc

Care, E., Kim, H., Vista, A., & Anderson, K. (2018). Education System Alignment for 21st Century Skills: Focus on Assessment. Center for Universal Education at The Brookings Institution, 1-39.

Datnow, A., Park, V., Peurach, D., & Spillane, J. (2022). Education system reform journeys: Toward holistic outcomes. Brookings Institution. https://www.brookings.edu/research/transforming-education-for-holistic-student-development/

Fullan, M. (2009). Large-scale reform comes of age. Journal of Educational Change, 10(2–3), 101–113. https://doi.org/10.1007/s10833-009-9108-z

Fullan, M. (2011). Choosing the wrong drivers for whole system reform (Seminar Series Paper No. 204). Centre for Strategic Education. https://michaelfullan.ca/wp-content/uploads/2016/06/13396088160.pdf

Goddard, C., Chung, C. K., Keiffenheim, E., & Temperley, J. (n.d.). A new education story: Three drivers to transform education systems. Big Change. https://neweducationstory.big-change.org/

Liu, S. (2020). Neoliberalism, globalization, and “elite” education in China: Becoming international. Routledge. https://doi.org/10.4324/9780429449963

Masters, G.n. (in press). Building a world-class learning system. National Center on Education and the Economy, Center for Strategic Education, and ACER.

Mourshed, M., Chijioke, C., & Barber, M. (2010). How the world’s most improved school systems keep getting better. McKinsey & Company. https://www.mckinsey.com/~/media/mckinsey/industries/public%20and%20social%20sector/our%20insights/how%20the%20worlds%20most%20improved%20school%20systems%20keep%20getting%20better/how_the_worlds_most_improved_school_systems_keep_getting_better.pdf

National Center on Education and the Economy. (2020). The design of high-performing education systems: A framework for policy and practice. https://www.ncsl.org/Portals/1/Documents/educ/International_Ed_Study_Group_2020/Framework-10-19.2.pdf


11. Educational Improvement and Quality Assurance
Paper

Repurposing School Improvement Networks for Teacher-led Change

Thomas Cowhitt

University of Glasgow, United Kingdom

Presenting Author: Cowhitt, Thomas

We find ourselves at the beginning of what might become an exciting new era in the field of Educational Change. Andy Hargreaves and Dennis Shirley anticipated a new era, what they called a Fourth Way, that, “pushes beyond standardization, data-driven decision making, and target-obsessed distractions to forge an equal and interactive partnership among the people, the profession, and their government” (2009, p. 71). Many different stakeholders are now working in partnership to improve schools, including, among others, educators, university-affiliated researchers, parent groups, for-profit and charity organisations, and government agencies. This plurality of participants has helped a more diverse research landscape emerge. A welcomed departure from a technocratic model of evidence-based education, which narrowly assumed that, “the only relevant research questions [were] questions about the effectiveness of educational means and techniques” (Biesta, 2007, p. 5). Instead, researchers are developing new collaborative research strategies (Penuel et al., 2020) and refining working arrangements, such as Research Practice Partnerships (Coburn et al., 2021; Coburn & Penuel, 2016) and Networked Improvement Communities (Bryk et al., 2011; Russell et al., 2019), to support engagement between multiple stakeholders and build collective capacity for sustained improvement across large school systems (Fullan, 2010).

However, as Stephen Ball aptly points out, “policy works by accretion and sedimentation rather than revolution; new policies add to and overlay old ones, with the effect that new principles and innovations are merged and conflated with older rationales and previous practices” (Ball, 2021, p. 63). Festering just beneath the surface of this supposed collaborative landscape is a policy strata where accountability was, and in many areas, remains the cornerstone of education policy (Smith & Benavot, 2019). Dominant school improvement models emanating from this policy context have similar characteristics- they involve mandates for teachers in the form of prescribed practices and specified outcome targets, and involve routine standardized testing to support comparisons of student academic achievement across schools (Afdal & Afdal, 2019; Biesta, 2009; DeLuca & Johnson, 2017; Fuller et al., 2008; Jackson & Temperley, 2007; Taubman & Savona, 2009). Even in national contexts like England where schools appear to be gaining autonomy through networked governance (Goldsmith & Kettl, 2009), organizational replication reigns (Peurach & Glazer, 2012) as government agencies continue to steer education provision, albeit from a distance (Whitty & Wisby, 2016), resulting in “coercive autonomy” (Greany & Higham, 2018) and schools well within the “shadow of hierarchy” (Davies, 2011).

This presentation reports on findings from a Mixed Methods Social Network Analysis project that traced the activities of a teacher innovator as they scaled up their own educational improvement initiative from a single-school pilot into a thirty-school regional program. The research is part of a larger effort to explore how teacher-led educational change can thrive in contexts dominated by networked governance. Specifically, this research asks- How can classroom teachers repurpose School Improvement Networks to replicate their own improvement initiatives in schools?

In England, School Improvement Networks, or educational systems in which a central hub organization works with outlet schools to enact change (Peurach & Glazer, 2012), is becoming the predominant model for large-scale school improvement. For example, the Education Endowment Foundation’s (EEF) Research Schools Network and the Department for Education (DfE) curriculum hubs and Teaching School Hubs are all systems where a lead school connects with outlet schools across a region to deliver common approaches for learning and teaching. Furthermore, by 2030, all schools in England are expected to be members of Multi-academy Trusts (MATs), or networks of schools, sometimes with dozens of members, controlled by lead schools and early joiners. Teachers can repurpose the relational infrastructure of networked governance to lead large-scale school improvement initiatives.


Methodology, Methods, Research Instruments or Sources Used
A Mixed Methods Social Network Analysis (MMSNA) design was used for this research (Crossley & Edwards, 2016; Domínguez & Hollstein, 2014; Froehlich et al., 2019). In the initial phase of the research, the frequency of school signups for the teacher-led initiative were recorded. This was done using time-stamped data from a google signup form. Interviews were then conducted with the program creator to understand their recruitment activities in the lead-up to surges in new program enrolments. Qualitative explanations of recruitment activities were then verified by additional data collection methods such as document analysis or further interviews with recruitment collaborators. Visualization tools from Social Network Analysis (SNA) were then used to construct network schemas, which detail the micro-steps of social processes using node and edge diagrams (Stadtfeld & Block, 2017).

The aim of the MMSNA design was to identify highly efficient interorganizational recruitment mechanisms for practitioners wanting to scale up their own improvement initiatives. Social mechanisms are constellations of entities and their activities which regularly lead to a social phenomenon of interest (Hedstrom, 2005). The phenomenon of interest was a new program signup. The MMSNA protocol helped identify mechanistic evidence, or the empirical fingerprints (Beach, 2016) of various actors and their activities that led to clusters of new program signups.

The MMSNA protocol resulted in several different types of relevant data being collected. To present the resulting recruitment mechanism, interactive network visualization tools were used to generate a joint display. Using interactivity to increase the quantity and diversity of data available to readers of network diagrams is a new development in network visualization. Until now, researchers have primarily deployed interactivity to support exploratory data analysis of large networks at various scales. The use of interactivity in this manner allows researchers to easily zoom in and zoom out of large networks to examine interesting structural configurations like small node clusters and structural holes within whole networks (Pirch et al., 2021). For this research, interactivity was used to demonstrate the explanatory potential of network visualizations by repurposing node and edge labels, along with other components of network diagrams, to display rich qualitative data about the formation of new relationships, which users could call upon by hovering or clicking their cursor, thereby preventing visual clutter and the reduction of perceptual efficiency.

Conclusions, Expected Outcomes or Findings
This research resulted in the discovery of new network mechanism, termed here for the first time as Star Coalescence. Star Coalescence is a novel network mechanism that describes a series of activities resulting in two collaborators developing new and enduring relationships with each other’s previously separate alters. At the core of this network mechanism is a tendency for separate contacts of colleagues to become shared associates. However, Star Coalescence represents a more complex social phenomenon because the mechanism can trigger a cascade of triad closures between two sparsely connected professional networks. Causing the formation of many new triads between previously disconnected alters means this network mechanism has the potential to facilitate many interorganizational recruitment events.

In this example of Star Coalescence, the program creator managed to solicit the help of a Maths Hub coordinator with their regional recruitment efforts. While the remit of Maths Hubs is expanding, their primary objective is to engage with maths teachers at schools within an assigned region to implement a particular pedagogy known as Teaching for Mastery. The forty regional Maths Hubs, overseen by the National Centre for Excellence in the Teaching of Mathematics (NCETM), have managed to create substantive regional School Improvement Networks. Soliciting the help of a Maths Hub Coordinator meant repurposing the existing relational infrastructure of School Improvement Networks, originally created to support organizational replication of a maths pedagogy scheme, to instead scale up their own school improvement initiative. After successfully running a single-school pilot, the program creator managed to scale their initiative up to a thirty-school regional implementation. Half of their signups were the result of their coordination with the Maths Hub Coordinator.

References
Ball, S. J. (2021). The Education Debate (Fourth). Bristol University Press.
Beach, D. (2016). It’s all about mechanisms – what process-tracing case studies should be tracing. New Political Economy, 21(5), 463–472. https://doi.org/10.1080/13563467.2015.1134466
Biesta, G. (2007). Why ‘What Works’ Won’t Work: Evidence-Based Practice and the Democratic Deficit in Educational Research. Educational Theory, 57(1), 1–22.
Biesta, G. (2009). Good education in an age of measurement: On the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability, 21(1), 33–46. https://doi.org/10.1007/s11092-008-9064-9
Bryk, A. S., Gomez, L. M., & Grunow, A. (2011). Getting ideas into action: Building networked improvement communities in education. In Frontiers in sociology of education (pp. 127–162). Springer.
Crossley, N., & Edwards, G. (2016). Cases, Mechanisms and the Real: The Theory and Methodology of Mixed-Method Social Network Analysis. Sociological Research Online, 21(2), 13.
Froehlich, D. E., Rehm, M., & Rienties, B. C. (2019). Mixed Methods Social Network Analysis: Theories and Methodologies in Learning and Education. Routledge.
Fullan, M. (2010). All Systems Go: The Change Imperative for Whole System Reform. Corwin Press. http://ebookcentral.proquest.com/lib/gla/detail.action?docID=996270
Goldsmith, S., & Kettl, D. F. (2009). Unlocking the Power of Networks: Keys to High-Performance Government. Brookings Institution Press.
Greany, T., & Higham, R. (2018). Hierarchy, Markets and Networks: Analysing the ‘self-improving school-led system’ agenda in England and the implications for schools. UCL Institute of Education Press.
Hargreaves, A., & Shirley, D. (2009). The Fourth Way: The Inspiring Future for Educational Change. Corwin Press, Ontario Principals’ Council and the NSDC.
Hedstrom, P. (2005). Dissecting the Social: On the Principles of Analytical Sociology. Cambridge University Press.
Peurach, D. J., & Glazer, J. L. (2012). Reconsidering replication: New perspectives on large-scale school improvement. Journal of Educational Change, 13(2), 155–190.
Pirch, S., Müller, F., Iofinova, E., Pazmandi, J., Hütter, C. V. R., Chiettini, M., Sin, C., Boztug, K., Podkosova, I., Kaufmann, H., & Menche, J. (2021). The VRNetzer platform enables interactive network analysis in Virtual Reality. Nature Communications, 12(1), Article 1. https://doi.org/10.1038/s41467-021-22570-w
Stadtfeld, C., & Block, P. (2017). Interactions, Actors, and Time: Dynamic Network Actor Models for Relational Events. Sociological Science, 4, 318–352. https://doi.org/10.15195/v4.a14
Whitty, G., & Wisby, E. (2016). Education in England—A testbed for network governance? Oxford Review of Education, 42(3), 316–329. https://doi.org/10.1080/03054985.2016.1184873
 
1:30pm - 3:00pm11 SES 16 A: Educational Technologies and Quality Assurance
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Andra Fernate
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

KNOWLO Project Promoting Knowledge Sharing Culture in Vocational and Higher Educational Institutions in Europe

Mudassir Arafat

International College of Cosmetology, Latvia

Presenting Author: Arafat, Mudassir

Undoubtedly the progress of any nation highly depends upon the proficiencies of its younger generation (Taleba & Hassanzadehb, 2015). Modern learning organizations and modern learning techniques have been continuously emphasized by the EU to harness the capabilities & exploit the potential of generation Z in the diverse areas of education in relation to ICT (Reding, 2003).

Previously, various researchers have focused on harnessing and exploiting new technologies that equip learners to learn with flexibility, comfort, and interaction. Nowadays learners are equipped with smart handheld or desktop devices to enable them to access digital learning materials with ease. Smart education and Modern Smart Learning organizations have become a concept that has acquired huge attention and is considered to be a necessity in this modern age. (Zhu, Yu, & Riezebos, 2016)

To overcome these changing trends and prepare the masses for a modern and technically advanced job market the European Union’s educational support program Erasmus+ has been working extensively on fostering many research, exchange, and development programs. KNOWLO project is one such program funded & supported by Erasmus+.

The aim of the KNOWLO Project is to develop a framework that helps traditional VET & HEIs transform into modern learning organizations.

This paper aims to offer crucial support to any traditional learning organization on the path of transforming into a modern learning organization. This paper can be treated as a case study towards identifying various aspects of consideration when undergoing transformation.

The KNOWLO project has 5 results to develop namely R1 to R5 where R1 is a Transformational Framework, R2 Self-Assessment tool for organizations, R3 Learning & sharing platform for VET & HEIs, R4 Resource Database, and R5 promotion of a Smart sharing culture.

This paper highlights crucial criteria developed under Result 1 for the transformational framework.

A SMART learning organization is a place where resources such as academics and intellect are used to their optimum (Vveinhardt & Minkute-Henrickson, 2015). One of the challenges is how to self-assess if an organization is traditional or smart to foster a knowledge-sharing culture.

The most common tool to assess is a survey that involves a crucial set of criteria necessary for the transformation.

A smart learning organization can possess various aspects of a traditional learning organization, however, the aspects that make it smart are features such as organizational self-awareness, communication, diversity, inclusion, emotional intelligence, and digitalization (Uskov et.al, 2019).

According to (Botella et.al, 2017) emphasis has mostly been given to student self-assessment in terms of their educational competencies and ability with digital learning and psychology. According to (J.W.Gikandi et.al, 2011) it is stated that online instruction, in general, is considered more beneficial compared to traditional mode. However, it is stressed that teaching and learning processes require to be centrally assessed so that they can provide learners with opportunities to demonstrate their abilities and capabilities of an organization to foster such developments.

The systematic review by Al-Kurdi et.al (2018) highlights the fact that there are limited contributions in the understanding of knowledge-sharing culture in HEIs in comparison to other sectors. According to (Taylor & James, 2001) Organizational transformations do not happen automatically, it needs proactive human mediation. Stakeholders’ participation often benefits such transformations in organizations. Lee (2018) claims that the impact of knowledge sharing on people has not been given due consideration. Hence, the KNOWLO project is committed to working in these areas, helping learning organizations with a framework that comprises all the essential criteria necessary for a Smart Learning organization.


Methodology, Methods, Research Instruments or Sources Used
This article contributes to this research segment by presenting real-time results of an ongoing Erasmus+ funded educational project, where the "International College of Cosmetology (ICC) in Riga, Latvia", is the project coordinator, with project partners from Latvia, Slovakia, Italy, and the Czech Republic. The project is ongoing and is subject to be finalized by December 2023. As of current the data collected and analyzed is for Result 1, the development of smart framework criteria. The data gathered has been analyzed per different regions of the EU, enabling us to understand which EU regions are lagging behind technological transformations and other aspects of the transformation model.

The social-emotional intelligence model (Channell, 2021), the Technology Acceptance Model, and various other Models have been used to define the framework criteria. It substantiated that those various aspects of the criteria demand the incorporation of various paradigms.

Research involves 2 purposeful samples: in total 265 stakeholders as survey respondents from VET & HEIs from the partner countries. KNOWLO Consortium partners details: 1. International College of Cosmetology (Riga, Latvia), 2. Eurofortis It (Riga, Latvia), 3. Catholic University of Ruzomberok (Slovakia), 4. Harmony Academy (Tarnava, Slovakia), 5. Schola Empirica (Prague, Czech Republic), and 6. Euroreso (Italy).

Research question: Are the teachers exercising the use of Modern Technologies in teaching to its maximum?

Research methods: 1) data collection – stakeholders’ survey (closed-ended questions) for quantitative results and for qualitative results interviews. Five, structured Interviews of stakeholders’ from HEIs in Latvia, Slovakia, and the Czech Republic were done. The Interviewees were a professor from the Catholic University of Ruzomberok, Slovakia, a Language School manager from Harmony Academy Slovakia, and a Communication & Marketing Manager from the International School of Latvia, and the remaining two were adult learners from the International College of Cosmetology in Riga Latvia. Mixed methods of research were used. For qualitative data analysis, narrative analysis was used and for the quantitative data – descriptive statistics were used!

Research process: Stakeholders’ who participated in the KNOWLO survey were specifically targeted according to the project proposal and guidelines. Respondents answered questions that highlighted Organizational self-awareness, sustainable goals, Digitalization of learning practices in a Global Context, communication, cooperation, and people. The structured interviews helped understating, the stakeholders’ self-assessment patterns and tools exercised to evaluate results.

Research period: November 2021 – November 2022.

Conclusions, Expected Outcomes or Findings
As stated above, this report only represents the development of Result 1, Transformational Framework. Key criteria established for the Stakeholders survey under R1 for the framework are 1. Organizational self-awareness, strategy, and development 2. Learning, communication, and cooperation 3. Organization and its people 4. Clients 5. Sustainability and Product Orientation 6. Digital transformation, global context, and value creation, and 7. Results & Benchmarking.

As each established criterion answered respective questions, criteria number 6. Digital transformation, in a global context, and value creation, are analyzed in this report in context with the research question (Are the teachers exercising the use of Modern Technologies in teaching to its maximum?).  

The results analyzed highlighted that teachers from the Czech Republic were more likely to agree, that they use technology to its maximum whilst teaching than teachers from Italy. The Czech Republican teachers were not convinced that the assessment of students prior to training digitally can help in utilizing technology to its maximum.

Stakeholders from all the partner countries agreed on two things unanimously there is not enough technical equipment in the organizations and not enough government funding to support the use of technology to the fullest.

The Knowlo framework that will help traditional HEIs & VETs transform into modern learning organization starts with digital transformation, the qualitative results show that stakeholders agree that new methods/forms of learning and individual approach to learners is essential. Other aspects that the stakeholder's survey highlight 60% believe that organizational vision must be clear to all 60% that constructive feedbacks help an organization excel 67%.
  
KNOWLO project is ongoing and after the successful completion of the Project, an effective self-evaluation tool and framework will be made available for learning organizations seeking transformation from traditional to Modern learning organizations

References
Al-Kurdi et.al, R. E.-H. (2018. gada 5. March). Knowledge sharing in higher education institutions: a systematic review. Journal of Enterprise Information Management, 31(2), 226-246. doi:10.1108/JEIM-09-2017-0129
Botella et.al, E. P. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74-98. doi:10.1016/j.edurev.2017.08.004
Channell, M. (2021. gada 13. October). Daniel Goleman’s Emotional Intelligence In Leadership: How To Improve Motivation In Your Team. Ielādēts no https://www.tsw.co.uk/blog/leadership-and-management/daniel-goleman-emotional-intelligence/#:~:text=Daniel%20Goleman's%20emotional%20intelligence%20theory,happier%20and%20healthier%20working%20culture
J.W.Gikandi et.al, D. N. (2011. gada December). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333-2351. doi:10.1016/j.compedu.2011.06.004
Lee, J. (2018 . gada 14 . May ). The Effects of Knowledge Sharing on Individual Creativity in Higher Education Institutions: Socio-Technical View. Division of Interdisciplinary Wellness Studies, Soonchunhyang University, 22 Soonchunhyang-ro, Asan, Chungnam 31538, Korea, 1-16. doi:https://doi.org/10.3390/admsci8020021
Reding, V. (2003). e-learning for Europe. European Council, Education & Culture. Brussels: Publications.eu.int. Ielādēts no http://europa.eu.int
Taleba, Z., & Hassanzadehb, &. F. (2015. gada 16th . January). Toward Smart School: A Comparison between Smart School and Traditional School for Mathematics Learning. Procedia - Social and Behavioral Sciences, 171 , 90-95. doi:https://doi.org/10.1016/j.sbspro.2015.01.093
Taylor & James, C. (2001). Fifth generation distance education. Instructional Science and Technology, 4(2), 1-14. Ielādēts no http://www.usq.edu.au/e-jist/
Uskov et.al, V. L. (2019. gada June). A Validation of “Smartness Features—Main Components” Matrix by Real-World Examples and Best Practices from Universities Worldwide. Smart Education and e-Learning 2019, 144, 3-17. doi: 978-981-13-8259-8
Vveinhardt, o., & Minkute-Henrickson, R. (2015). Transformation of a learning organization into a smart organization: expansion of human resource by intellectual capital. Proceedings of the EDULEARN15 Conference, 172-181. doi:978-84-606-8243-1
Zhu, Z.-T., Yu, M.-H., & Riezebos, &. P. (2016. gada 31. March). A research framework of smart education. Smart Learning Environments. doi:10.1186/s40561-016-0026-2


11. Educational Improvement and Quality Assurance
Paper

A Comprehensive Systematic Review of AI in NLP, EDM, and LA for Feedback in K-12 Education

Burcu Toptas1, Munevver Ilgun Dibek2

1Ankara University, Turkiye; 2TED University, Turkiye

Presenting Author: Toptas, Burcu; Ilgun Dibek, Munevver

AI, or artificial intelligence, refers to computing systems that can perform tasks similar to those of humans, such as adapting, learning, and using data for complex processing (Popenici & Kerr, 2017). There are various branches and sub-branches of AI, but for feedback purposes, the most relevant ones are natural language processing (NLP), educational data mining (EDM), and learning analytics (LA) (Gardner et al., 2021). NLP is beneficial for feedback because it can analyze linguistic components of students’ written work and provide feedback on writing quality, syntactic complexity, and grammatical errors. EDM allows for data-supported feedback through data visualization and can also provide verbal feedback using NLP or manual input from instructors. LA uses student activity data to provide personalized feedback through an interactive dashboard. Feedback can be either semi-automatic or fully automatic, depending on the system used (Wongvorachan et al., 2022). AI has been incorporated into NLP, EDM, and LA, leading to the development of complex systems that can provide students with timely and individualized feedback. As a result, both their performance and learning process can be improved. It has been demonstrated that AI-based feedback systems are more effective and efficient than more conventional forms of feedback.

It has been demonstrated that incorporating AI into feedback improves student motivation and engagement, which results in higher learning outcomes (Alazmi & AlZoubi, 2020). Moreover, it has been revealed that AI-based feedback systems are economical and scale to large classrooms, making them appropriate for use in both traditional and online learning environments (Chang et al., 2020). Thus, the integration of AI in feedback is not only improving the learning experience of students but also transforming the traditional methods of feedback in education. According to Zawacki-Richter et al. (2019), the incorporation of AI in K-12 education has seen significant growth in recent years. Crompton and Song (2021) also note that the use of AI offers numerous possibilities for enhancing teaching and learning. One way AI is being utilized is in the automatic grading of essays, as reported by Yang et al. (2019). Additionally, AI can provide swift feedback to students, as stated by Benotti et al. (2018), and can adjust instruction to meet the unique needs of each student, as highlighted by Arnett (2016).

A systematic review of the application of AI and robotics in K-12 education was carried out by Hrastinski et al. (2019), with an emphasis on the relationship between teachers and students. However, the scope of the study was limited to papers from one international symposium and solely on robotics, rather than AI more broadly. Furthermore, it did not examine the potential of AI in enhancing feedback practices in K-12 education. Zafari et al.'s (2022) and Crompton et al.’s (2022) studies examined the current state of AI integration in K-12 education, with a focus on its general use, not just its use for feedback.

The aim of this study is to address the call by scholars (Banihashem et al., 2022) to investigate the role of NLP, EDM, and LA in enhancing feedback practices in K-12 education. This paper will provide researchers and educators with a deeper insight into the application of NLP, EDM, and LA for feedback purposes. In this regard, the present systematic review seeks to answer the following research questions:

1. What are the primary reasons behind the utilization of NLP, EDM, and LA in feedback studies in K-12 education?

2. What types of data are utilized in studies on NLP, EDM, and LA to provide feedback in K-12 education?

3. What NLP, EDM, and LA tools and techniques are employed by studies to facilitate feedback in K-12 education?


Methodology, Methods, Research Instruments or Sources Used
We followed the PRISMA framework (identification, screening, eligibility and inclusion) to conduct the systematic review.
Search strategy
We searched the most relevant terms and synonyms which are overlapping the concepts that the present study focused on by identifying the prior systematics reviews (e.g., Banihashem et al., 2022; Zafari et al., 2022) on K-12 education, artificial intelligence or feedback practices. The search string included the following terms: ("learning analytic*" OR "educational data mining" OR "artificial intelligence" OR "natural language processing") AND (feedback OR "formative assessment" OR feedforward) AND ("K-12 student*" OR "K-12" OR "K-12 education" OR "primary school*" OR "primary education" OR  “kindergarten*" OR “pre-primary” OR “middle school*" OR "secondary education" OR "secondary school*" OR “high school*” OR “1st  grade”  OR  “2nd  grade”  OR  “3rd  grade”  OR  “4th  grade”  OR  “5th  grade”  OR  “6th  grade”  OR  “7th  grade”  OR  “8th  grade”  OR  “9th  grade”  OR  “10th  grade”  OR  “11th  grade”  OR  “12th  grade”  OR  "grade  1"  OR “grade  2"  OR  "grade  3"  OR  "grade  4"  OR  "grade  5"  OR  "grade  6"  OR  "grade  7"  OR  "grade  8"  OR “grade  9"  OR  "grade  10"  OR  "grade  11"  OR  "grade  12"). Web of Science (WOS), ERIC, and IEEE databases were chosen considering their reputation and inclusion of numerous research studies on the topics that were addressed in the current study.
Criteria for inclusion
The following criteria were used to determine which articles were included: (a) journal articles published between 2013 and 2023; (b) articles written in English language, (c) peer-reviewed journals to ensure quality, and (d) empirical studies. However, conference proceedings were excluded.
Identification of relevant publication
During the initial screening phase in the selected databases (WOS [n=72], ERIC [n=55], and IEEE [n=443]), a total of 570 were identified. After eliminating duplicates (n=22) and non-peer-reviewed articles (n=10), a pool of publications (n=538) remained. In the second phase, the titles and abstracts were screened against our inclusion criteria, and 459 papers did not meet the criteria, 79 papers were further evaluated through full-text screening. The final pool of papers was then used for quality appraisal.
Quality appraisal
We adopted quality appraisal criteria from Theelen et al. (2019), based on the work of Savin-Baden and Major (2007) for evaluating qualitative studies and NICE (2012) for evaluating quantitative studies.

Conclusions, Expected Outcomes or Findings
Preliminary Findings and Expected Outcomes
The Rayyan program is utilized for reviewing research papers. It's a free online tool for scientists conducting systematic reviews and similar projects. Initially, we used it as a blind version for accuracy purposes. We independently evaluated the papers by marking "include" or writing reasons for “exclusion”. After both authors finished evaluating the articles, we switched to the unblinded version and resolved conflicts. We disagreed on five papers, with one author wanting to include while the other excluded. We ultimately included only three of these five studies. At least one of us used the word "maybe" in evaluating 34 papers, so we also evaluated each paper as a team. We used the label "maybe" because the abstract was not clear on which AI techniques used. After our team review, we chose 16 of them for further evaluation because the full text can aid in labeling the technique. Out of the articles where the technique was identifiable, 42 utilized LA, 16 used NLP, and 5 employed EDM. We employ Nvivo to conduct content analyses relevant to our research questions. We will also conclude our review by highlighting key challenges and opportunities for future research.


References
Alazmi, B., & AlZoubi, A. (2020). The role of artificial intelligence in education. Journal of Education and e-Learning Research, 7(2), 19-30.

Arnett, T. (2016). Teaching in the machine age: How innovation can make bad teachers good and good teachers better. Christensen Institute.

Banihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. (2022). A systematic review of the role of learning analytics in enhancing feedback practices in higher education. Educational Research Review, 37.

Benotti, L., Martinez, M.C., & Schapachnik, F. (2018). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192.

Chang, K.E., Huang, Y.M., & Chen, W.H. (2020). A review of AI-based feedback systems for education. JETDE, 3(1), 1-14.

Crompton, H., Jones, M.V., & Burke, D. (2022). Affordances and challenges of artificial intelligence in K-12 education: a systematic review. JRTE, 1-21.

Crompton, H., & Song, D. (2021). The potential of artificial intelligence in higher education. Revista Virtual Universidad Católica Del Norte, 62, 1–4.

Gardner, J., O’Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: Breakthrough? Or buncombe and ballyhoo?. Journal of Computer Assisted Learning, 37(5), 1207–1216.

Hrastinski, S., Olofsson, A.D., Arkenback, C., Ekström, S., Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L-M., Fuentes, A., Gustafsson, U., Humble, N., Mozelius, P., Sundgren, M., & Utterberg, M. (2019). Critical imaginaries and reflections on artificial intelligence and robots in post digital K-12 education. Postdigital Science and Education, 1(2), 427–445.

Popenici, S.A.D. & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. RPTEL, 12(22).

Theelen, H., Van den Beemt, A., & den Brok, P. (2019). Classroom simulations in teacher education to support preservice teachers’ interpersonal competence: A systematic literature review. Computers & Education, 129, 14-26.

Wongvorachan,T., Lai, K.W, Bulut, O. Tsai, Y. & Chen, G. (2022). Artificial Intelligence: Transforming the Future of Feedback in Education. Journal of Applied Testing Technology, 23(1), 1-22.

Yang, Y., Xia, L., & Zhao, Q. (2019). An automated grader for Chinese essay combining shallow and deep semantic attributes. IEEE Access 7.

Zafari, M., Bazargani, J.S.,Sadeghi-Niaraki, A., & Choi, S.M. (2022). Artificial intelligence applications in K-12 education: A systematic literature review. IEEE Access, 10.

Zawacki-Richter, O., Marín, V.I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? IJETHE, 16(1), 1–27.
 
3:30pm - 5:00pm11 SES 17 A: Learners' Academic Achievement in Secondary Schools
Location: Sir Alexander Stone Building, 204 [Floor 2]
Session Chair: Mudassir Arafat
Paper Session
 
11. Educational Improvement and Quality Assurance
Paper

AP or IB? Urban/Rural Differences in Access to Advanced Coursework, High School Achievement, and Postsecondary Outcomes

Dong Wook Jeong1, Thomas F. Luschei2

1Seoul National University, Korea, Republic of (South Korea); 2Claremont Graduate University, United States of America

Presenting Author: Jeong, Dong Wook; Luschei, Thomas F.

Educators, education researchers, and policymakers have increasingly pointed to the importance of advanced coursework to prepare high school students to enter and succeed in college. Research has found ample evidence of the importance of such coursework, which can include Advanced Placement (AP) and International Baccalaureate (IB) courses. The positive advantages of taking such coursework appears to accrue while students are in high school and to increase their likelihood of entering and succeeding in postsecondary education (Coca et al, 2012; Conger et al., 2021; Perna et al., 2015).

Unfortunately, access to advanced coursework is not uniform across students or schools in the United States. Studies of access to IB coursework have found that despite growing efforts by the International Baccalaureate Organization to expand and “democratize” IB course offerings, lower-income and ethnic minority students do not have equal access to them (Perna et al., 2015). There is also evidence that both IB and AP coursework are less available to students in remote and rural schools, relative to students in cities and suburban areas (Gagnon & Mattingly, 2016; Thier & Beach, 2021). However, because IB is offered on a much smaller scale than AP, IB courses are much less common than AP courses in rural schools (Thier & Beach, 2021). Assuming that IB and AP programs provide a clear advantage over non-advanced coursework, this pattern has negative implications for the educational achievement and attainment of students in rural areas, as well as for rural development (Saw & Agger, 2021).

Despite the clear advantages of advanced coursework for students’ high school and postsecondary success, it is not clear whether all advanced coursework has the same benefit for students. Many studies that examine the link between advanced course-taking and student success combine AP and IB into one group, as if they offered the same educational experience and rigor (e.g., Long et al., 2012). In fact, there are important differences between AP and IB in their missions, structures, and potential utility for high school students (Saavedra, 2014; Thier & Beach, 2021). For example, whereas AP offers discrete, subject-based courses, IB provides a comprehensive program of study leading to an internationally recognized high school diploma. Further, IB’s emphasis on global understanding may offer an additional benefit to students in rural areas, who have less opportunity to develop “international mindedness” than their peers in large cities (Thier & Beach, 2021). Unfortunately, very few studies have directly compared the relative impact of AP and IB coursework on students’ high school and postsecondary success.

Our objective is to examine differential access to and impact of advanced coursework—specifically IB and AP courses—for high school students across communities of different sizes (urban, suburban, town, and rural) in the United States. Although AP and IB are commonly grouped together under the category of “advanced coursework,” we examine differences between these two curricular and programmatic approaches in terms of availability, course-taking, and impact on high school and post-secondary outcomes. Using data from the US National Center for Education Statistics’ High School Longitudinal Study (HSLS 2009), which allow us to generate nationally representative results, we examine the following research questions:

  1. Does access to AP and IB courses vary across schools in communities of different sizes (urban, suburban, rural, towns) in the United States?
  2. How do students in these schools differ in terms of whether and which advanced courses they take?
  3. Do students in these schools differ in terms of the relative impact of AP and IB courses on their high school achievement and post-secondary outcomes?

Methodology, Methods, Research Instruments or Sources Used
For our analysis, we draw on the rich, nationally representative HSLS:09 data, administered by the NCES. NCES also conducted HSLS follow-up studies in 2012, 2013, and 2016. We also use data from the HSLS high school and postsecondary transcript studies, generated in 2013 and 2017 respectively.  Together, these data allow us to examine and compare AP and IB courses in terms of high school- and college-level outcomes.

To conduct our analysis, we must contend with the fact that students are not randomly assigned to AP or IB programs. In fact, many schools have entry requirements, such as a minimum GPA or test score performance, for students to enroll in these courses in the first place (Saavedra, 2014).  This suggests the possibility of pre-existing differences between program participants and non-participants, leading to sample selection bias (Dickson et al., 2018). Further, neither AP nor IB programs are evenly distributed across urban and non-urban schools, nor do these programs uniformly enroll students within schools.

We employ several strategies to mitigate the potential for sample selection bias in our estimates. First, we first limit our analytic sample to 12th graders in 2013 who ever participated in either AP or IB programs only. This approach allows us to: (1) directly compare the relative effectiveness of each of these programs and (2) reduce or eliminate the first type of bias, between program participants and non-participants. Second, we use a differences-in-differences (DID) approach to account for between-school bias. Using the DID approach, we subtract differences in the outcomes of AP students between urban and non-urban areas from differences in the outcomes of IB students between the two areas. The underlying idea is to cancel out any pre-existing differences in characteristics, whether observed or unobserved, of IB and AP students and programs between urban and non-urban schools. Last, we introduce propensity score matching to adjust for any remaining pre-existing differences between IB and AP participants based on the observed characteristics of individuals. We then replicate a differences-in-differences estimation for matched samples of IB and AP students between urban and rural schools. To test the robustness of our results, we employ other propensity score methods including propensity score stratification and inverse probability weighting.

Conclusions, Expected Outcomes or Findings
For Research Question 1, we find that although IB and AP courses are both less likely to be offered in towns and rural areas relative to cities, the differences between towns/rural areas and cities are much greater in terms of IB availability. However, when schools do offer IB courses, students are more likely to take them, compared to AP students.

The results of Research Question 2 adds evidence to these conclusions. When examining course-taking patterns of students in schools that offer both IB and AP courses, we find that students take on average more IB units. Finally, our findings regarding the relative impact of IB and AP courses on students’ high school and postsecondary outcomes (Research Question 3), we find that whereas IB students in towns have a small advantage in high school-level outcomes (primarily in terms of GPA in IB/AP courses), IB holds a consistent advantage in terms of postsecondary outcomes for students in towns and rural areas. These include enrollment in a four-year college (towns and rural areas), credits earned in the first year of college (rural areas), first-year GPA (rural areas), total credits (rural areas), and accumulated GPA (rural areas). However, our analysis finds systematic differences between IB and AP students that are driven primarily by school-level supply constraints. After adjusting for these differences by limiting our sample and applying propensity score matching, we continue to find a relative IB advantage for students in towns and rural areas in terms of IB/AP GPA (towns), four-year college enrollment (rural areas) and accumulated GPA (towns). Our sensitivity analysis using propensity score stratification and inverse probability weighting finds similar results, suggesting that our key findings related to IB advantages for town and rural students are robust across different specifications for 12th grade GPA, four-year college enrollment, and accumulated college GPA.

References
Coca, V., Johnson, D., Kelley-Kemple, T., Roderick, M., Moeller, E., Williams, N., & Moragne, K. (2012). Working to my potential: The postsecondary experiences of CPS students in the International Baccalaureate Diploma Programme. Chicago, IL: Consortium on Chicago School Research.

Conger, D., Kennedy, A. I., Long, M. C., & McGhee, R. (2021). The effect of Advanced Placement science on students’ skills, confidence, and stress. Journal of Human Resources, 56(1), 93-124.

Dickson, Anisah, Laura B. Perry, and Susan Ledger. "Impacts of International Baccalaureate programmes on teaching and learning: A review of the literature." Journal of Research in International Education 17, no. 3 (2018): 240-261.

Gagnon, D. J., & Mattingly, M. J. (2016). Advanced placement and rural schools: Access, success, and exploring alternatives. Journal of Advanced Academics, 27(4), 266-284.

Long, M. C., Conger, D., & Iatarola, P. (2012). Effects of high school course-taking on secondary and postsecondary success. American Educational Research Journal, 49(2), 285-322.

Perna, L. W., May, H., Yee, A., Ransom, T., Rodriguez, A., & Fester, R. (2015). Unequal access to rigorous high school curricula: An exploration of the opportunity to benefit from the International Baccalaureate Diploma Programme (IBDP). Educational Policy, 29(2), 402-425.

Saavedra, A. (2014). The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools. Teachers College Record, 116(4).

Saw, G. K., & Agger, C. A. (2021). STEM pathways of rural and small-town students: Opportunities to learn, aspirations, preparation, and college enrollment. Educational Researcher, 50(9), 595-606.

Thier, M., & Beach, P. T. (2021). Still where, not if, you’re poor: International Baccalaureate opportunities to learn international-mindedness and proximity to US cities. Journal of Advanced Academics, 32(2), 178-206.


11. Educational Improvement and Quality Assurance
Paper

Agency of Schoolchildren and Their Particiation in Extracurricular Activities

Mikhail Goshin, Pavel Sorokin, Sergey Kosaretsky

Higher School of Economics, Russian Federation

Presenting Author: Goshin, Mikhail

The rapidly changing social reality of contemporary world brings new challenges to an individual (including children) in everyday life. One of the most important issues is the growing demand for agency, that is, the ability to make decisions and be proactive in various contexts and spheres of public and personal life without external control or support typical for the institutional systems (including educational systems). The concept is "agency" generally implies the ability of a person to proactively influence the environment, social structures, including the creation of new forms of interaction in various spheres of public life (Sorokin, Zykova, 2021). Agency can be considered as an umbrella concept combining such constructs as subjectivity, autonomy, independence, initiative, self-determination, self-regulation, proactive behavior, social impact, cooperation, etc. [Udehn, 2002; Sorokin, Frumin, 2022; Cavazzoni et al., 2021].

The field of education has a special role in the formation and development of these personal qualities and behavioral models. The Organization for Economic Co-operation and Development (OECD) in the framework of the project "The Future of Education and Skills 2030" emphasizes the importance of forming agency among students, understood as a proactive position in relation to their educational trajectory. The OECD also considers the so-called "transformative competencies" as a special educational result, which are necessary for individual success in the modern world and for social well-being.

Unfortunately, contemporary school remains to a large extent what Goffman called a "total institution", assuming a rigid system of rules and regulations, in which it is quite difficult to find opportunities for proactivity. Under these circumstances, participation in extracurricular activities becomes of special importance for developing children’s agency. The available literature shows that extracurricular activities of schoolchildren have great potential in terms of the formation of agency, since its format implies, in comparison with formal school education, more room for independence, initiative, and free choice [Lareau, Weininger, 2008; Peterson et al., 2013; Kosaretsky et al., 2019]. A number of studies have shown that extracurricular activities make a significant contribution to the formation of such qualities as perseverance, independence, self-confidence, creativity, and social activity [Fletcher, Nickerson, Wright, 2003; Baker, 2008; Durlak, Weissberg, Pachan, 2010; Baharom, Sharfuddin, Iqbal, 2017].

At present, literature lacks tools for assessing agency of children and adolescents in formal school and extracurricular education (Sorokin, Froumin, 2022; Cavazzoni, 2021), especially given high diversity between different educational contexts, as well as different ages, etc. Also, there are no general models for analysis and interpretation of the empirical results related to various possible manifestations of agency. In particular, lacking are models allowing to correlate different types and levels of children’s agency with external factors, including educational practices.

The present study is aimed at filling this research gap and investigating the relationship between agency (proactive behavior of certain manifestations) of schoolchildren with academic results in formal school education and with participation in various extracurricular activities.

Research questions:

- How the agency of schoolchildren manifests itself in various fields, such as extracurricular activities, the circle of peers, family, economic activities? What is the difference between groups of respondents with different educational characteristics in terms of manifestations of agency?

- How is the extent (or level) of proactive behavior (agency) of schoolchildren relates to their academic performance in school curriculum subjects? How strong is the correlation?

- How is the level of proactive behavior (agency) of schoolchildren relates to their involvement in extracurricular activities both at school and outside of school? Involvement in which extracurricular activities is most correlated with the level of agency of schoolchildren?


Methodology, Methods, Research Instruments or Sources Used
The data were obtained as a result of the survey of students of formal education institutions (schools) in the city of Yaroslavl (N = 4,914 Mage of children = 12.6, SD = 1.7; 50.3% female), aimed at studying the effects of participation in extracurricular activities at school and outside of school on the development of the so-called “soft skills”, academic results, curiosity, and proactive behavior (agency). The sample of the study is random and representative for all schools in Yaroslavl (which if a relatively big city in Russia with approximately 600 000 of population).
Among the study participants, 48.6% noted that the mother or stepmother has tertiary education, and 31.26% of the respondents have both parents with tertiary education. Next, 37.8% of the respondents have a father or stepfather with tertiary education. At the same time, 0.8% of the respondents noted that they do not have "this parent – mother or stepmother," and 5.8% of the respondents – "there is no parent – father or stepfather." Thus, it can be argued that the vast majority of the study participants live in full families, and 48.8% of the respondents live with a brother or sister.
More than half of the respondents (65%) attend some extracurricular activities. On average, the study participants attend 2 extracurricular courses. At the same time, half (50.5%) of those participating in extracurricular activities attend only 1 course. About a third of the respondents (30.3%) attend 2 courses. There is no gender asymmetry in the number of courses among the study participants. The average time spent on extracurricular activities is 7.74 hours per week.
The questionnaires had special section aimed at assessing proactive behavior (agency).  In particular, the respondents were asked whether their parents participated in choosing of extracurricular activities (or the decision was made solely by a child him- or herself), whether they tend to take the initiative and make decisions about everyday activities in the company of peers, how the decisions are made about joint activities with parents, and where from they obtain most of the money for their personal expenses (for instance, do they have a paid working experience). The respondents' responses for each category were ranked and as a result, a total integral indicator (agency index) was calculated, i.e.,  the level of agency for each respondent.
Finally, information about children's participation in extracurricular activities was juxtaposed with the agency index.

Conclusions, Expected Outcomes or Findings
The level of agency increases depending on the child’s age reflected through the corresponding stages of education, that is the transition of a child to the next educational level from junior school to secondary school and then to high school.
A non-linear relationship between the level of agency and academic performance at school was identified. On the one hand, the higher the grades of academic performance in various subjects, the lower the proportion of the respondents who are characterized by a lack of agency. However, the highest academic scores are more likely to be characteristic of the respondents who demonstrate an average level of agency. Remarkably, the highest level of agency did not correspond with the highest academic scores at school.  
At the same time, the participation of schoolchildren in extracurricular activities demonstrates a strong positive relationship with the level of agency. Only 28.8% of the respondents with a low level of agency are involved in extracurricular activities.  Among schoolchildren with a high level of agency, 83.3% are involved in extracurricular activities. The respondents with a low level of agency attend an average of 1.9 courses. The respondents with a high level of agency attend on average 2.8 courses. The respondents with a low level of agency spend an average of 7.4 hours per week on extracurricular activities, and 12.1 hours per week with a high agency level.
The higher the level of agency, the higher the proportion of the respondents involved in extracurricular activities of all types.  It is important to note that, with an increase in the level of agency, the involvement of schoolchildren in the courses that prepare children for technical colleges, university, tourism, and crafts in school increases most significantly. Similar increases are observed in courses on crafts, tourism, science, and technology performed outside of school.

References
Baharom M. N., Sharfuddin M., Iqbal J. (2017) A Systematic Review on the Deviant Workplace Behavior. Review of Public Administration and Management, 5(3), 1–8.
Baker C. N. (2008) Under-Represented College Students and Extracurricular Involvement: The Effects of Various Student Organizations on Academic Performance. Social Psychology of Education, 11(3), 273-298.
Cavazzoni, F., Fiorini, A., & Veronese, G. (2021). How Do We Assess How Agentic We Are? A Literature Review of Existing Instruments to Evaluate and Measure Individuals' Agency. Social Indicators Research, 159(3), 1125-1153.
Durlak J. A., Weissberg R. P., Pachan M. (2010) A Meta-Analysis of After-School Programs that Seek to Promote Personal and Social Skills in Children and Adolescents. American Journal of Community Psychology, 45(3). P. 294-309.
Fletcher A. C., Nickerson P., Wright K.L. (2003). Structured Leisure Activities in Middle Childhood: Links to Well‐Being. Journal of Community Psychology, 31(6), 641– 659.
Kosaretsky S. G., Goshin M. Ye., Belikov A. A., Kudryavtseva M. A., Maksimova A. S., Poplavskaya A. A., Yankevich S. V., Petlin A. V., Zhulyabina N. M. (2019). Extracurricular Education of Children: Unified Diversity. Moscow: HSE.
Lareau A. & Weininger E. B. (2008). Class and the Transition to Adulthood. Social Class: How Does It Work.
Peterson T., Fowler S., Dunham T. F. (2013) Creating the Recent Force Field: A Growing Infrastructure for Quality Afterschool and Summer Learning Opportunities. Expanding Minds and Opportunities. Washington, DC: Collaborative Communications Group.
Sorokin P. S., Froumin I. D. (2022) Education As a Source for Transformative Agency: Theoretical and Practical Issues. Voprosy Obrazovaniya / Educational Studies Moscow, 1, 116-137.
Sorokin P. S., Zykova A. V. (2021) «Transformative Agency» as a Subject of Research and Development in the 21st Century. Monitoring of Public Opinion:  Economic and Social Changes, 5, 216–241.
Udehn L. (2002) The Changing Face of Methodological Individualism. Annual Review of Sociology, 28(1), 479–507.
 

 
Contact and Legal Notice · Contact Address:
Privacy Statement · Conference: ECER 2023
Conference Software: ConfTool Pro 2.6.150+TC
© 2001–2024 by Dr. H. Weinreich, Hamburg, Germany