Conference Agenda

Across education systems, pressures to standardise assessment collide with traditions of teacher autonomy. This tension raises a crucial question: What counts as valid assessment?

The fundamental nature of assessment is often ignored in policy debates and public discourse. To address this gap, Jönsson (2020) has proposed a distinction between two approaches: assessments based on qualitative (or evaluative) judgments made by assessors and those grounded in quantitative measurement principles informed by psychometrics. In this study, we refer to these paradigms as judgment and measurement.

Psychometrics traces its origins to the latter half of the 19th century, with major advances occurring during the 20th century in relation to the measurement of human intelligence. From this foundation, numerous branches have emerged, including tests of scholastic aptitude, personality, and academic achievement (e.g., Black, 1998; Urbina, 2004). Qualitative judgments (Sadler, 1989), also called evaluative judgments (Boud et al., 2018), are used to appreciate the quality of student performance. According to Sadler (1989), a qualitative judgment is made by a knowledgeable expert and is not reducible to a formula that nonexperts can apply.

The most notable difference between evaluative judgment and a psychometric perspective lies in their focus. In psychometrics, assessment is treated as an inferential process, targeting an invisible theoretical construct such as knowledge. Consequently, psychometric approaches depend on indirect indicators, aggregating data from multiple items. In contrast, evaluative judgment centers on the quality of performance itself. Assessing an essay or report is a direct act, requiring no inference about hidden traits.

Building on this conceptual distinction, this study examines how these paradigms framed summative final assessment and positioned teachers in the context of a nationwide Finnish assessment reform. The reform introduced detailed national assessment criteria for final grades at the end of lower-secondary education, aiming to make teachers’ grading more consistent while preserving classroom-based assessment practices.

A clear understanding of final assessment is essential because of its profound impact on students’ futures and the functioning of society. In this context, teachers are users of public power and may need to justify their grading decisions. Clarity in grading principles is critical: Should professional judgment count as valid evidence, or should grades rely solely on external measures? Do we want teachers to act as assessment machines? Or do we want to externalise assessment to tests, viewing teachers as technicians without power over assessment?

We analyzed two complementary datasets: 56 official and media documents saping public discourse during the reform, and 66 semi-structured interviews with lower-secondary subject teachers across six disciplines.

Our analysis of the public and official documents showed that the measurement and judgment paradigms were intertwined in discourse, and so tightly that it was difficult to disentangle them. Moreover, teachers were positioned in somewhat confusing ways both 1) as parts of the assessment machine (conductors of measurement) and 2) as assessment machines (professionals using evaluative judgment).

In the interview data, we identified three distinct ways in which teachers referred to assessment and their positionings within it: 1) as designers of the assessment machine, who constructed instruments to ensure reliability (referring to measurement), 2) as assessment machines, who applied holistic, criteria-based judgments that resisted reduction to mere metrics (referring to judgment); and 3) as calibrators who adjusted quantitative indicators intuitively with the criteria (referring to intuitive judgment).

Our findings indicate that these paradigms of assessment are intertwined in societal discourse, simultaneously positioning teachers as agentic professionals and sources of error – an inherently contradictory stance. This underscores the need for clearer conceptualisations of assessment to inform both policy decisions and practical implementation, clarifying whether teachers are expected to act as objective technicians or professional interpreters.

In Lithuania, a new competency-based curriculum has recently been introduced alongside a two-stage external state examination system. The intention was to replace high-stakes examinations, predominantly composed of closed- or short-answer items, with competency-based assessments that give students an opportunity to demonstrate their unique skills and performance. However, the newly developed examination system did not deliver the expected results. The number of examinations increased when examination parts were transferred to a digital format, the test design remained unchanged, and no performance assessment was introduced (Raudiene et al., 2025).

After the first year of implementing the new exam, marked by a high number of issues related to poor item quality, difficulty-level mismatches, and technical implementation, which raised public dissatisfaction and anger, we decided to analyze how the decision-making process was carried out and how far the current assessment system is from the competency-based assessment.

Focusing on competencies as learning outcomes assumes that checking whether students recall information is no longer the highest priority, while assessing what students can do with knowledge is (Darling-Hammond & Adamson, 2010; Elmer et al., 2019; Black & Wiliam, 1998). Assessing how students apply knowledge in new situations goes beyond traditional paper-and-pencil tests or their digital alternatives and involves direct interaction between the assessor and the assessed, which may not consistently yield quantitative feedback. This does not satisfy the bureaucratic apparatus that uses exam scores, indicators, and rankings for governance and accountability at all levels, from students and teachers to municipalities and states (Ball, 2003). Little is questioned about what this culture of measurement brings to education apart from competition, stratification, the distortion of basic pedagogical principles, and distrust in education (Biesta, 2010; Koretz, 2017; Au, 2009).

Because of the lack of clear rules and a diverse array of actors, the development of education policy is complex (Shaw, 2019). Ball (1998) described it as a process of bricolage, borrowing and copying bits of ideas from elsewhere, interpreting theories and research, and reusing them as though they might work. Due to the high stakes associated with school examinations, university enrollment, school rankings, and follow-up consequences, assessment policy-making is considered extremely complex and multilayered, thus requiring a well-established theoretical approach.

The Multiple Streams Framework (MSF) (Kingdon, 2014) offers a lens for understanding how certain proposals reach the political agenda and why some issues receive political attention at certain times while others do not. The MSF is a tool for analyzing nonlinear and non-rational political processes (Shaw, 2019; Stout & Stevens, 2000), including policy agenda-setting, identifying the necessary conditions for translating political agendas into concrete decisions, determining the timing of reform, and diagnosing policy failures.

Kingdon (2014) suggests that policy development comprises three streams:

• The problem stream explains “how problems come to be recognized and how conditions come to be defined as problems” (Kingdon, 2014, p. 19), bringing stakeholders’ concerns to light.
• In the policy stream, people with specialized knowledge generate alternative proposals to address the problem. The proposals that win should meet criteria such as value alignment, societal support, financial capability, and political support.
• The political stream represents “swings of national mood” (Kingdon, 2014, p. 20) or the dominant view on the issue and the proposed responses to it. This stream is characterized by stakeholders' involvement in solving the problem.

These streams develop independently, but at critical moments all three conflate, opening a window of opportunity to introduce a new policy. Policy-making is enhanced by policy entrepreneurs who invest resources, such as expertise, political influence, and determination, to bring together all three streams to achieve policy change.

Assessment has become an increasingly central aspect of teachers’ professional work, reshaping how teaching, learning, and evaluation are understood in everyday school practice. Scholars in education around the world suggest how teachers have to respond to these accountability claims in order to show effectiveness and quality, ultimately making school results the focus of education (Biesta, 2009; Irisdotter Aldenmyr et al., 2025). This has affected the teaching profession in several regards, changing core aspects of the institutional structures for teaching and even what it means to be a teacher (Ball, 2003).

This paper explores how teachers in Sweden navigate professionalism in relation to assessment in a school system characterized by a focus on school results. Drawing on focus group interviews with teachers across primary, lower secondary, and upper secondary education, the study aims at exploring how teachers in Sweden understand and enact professionalism in assessment, with particular attention to how their professional judgement is negotiated within institutional conditions shaping teaching and learning.

When the quality of schools, and even teachers, is defined in terms of measurable educational outcomes, there is a risk of teaching focusing on testing and grading, or what has been labelled “teaching to the test”, which, in turn, leads to tests gaining an ever-increasing role (Vogt, 2022), constructing a testing culture in schools (Birenbaum, 2016). Additionally, a school characterized by performance activities and excessive summative assessments is problematic as it often causes teacher stress (Lundahl, et al., 2022; Saeki et al., 2018; von der Embse et al., 2015) and put students learning and well-being at risk (Brolin Låftman et al., 2013; Hirsh, 2020; Luthar et al., 2020).

The study is grounded in sociological theories of professionalism and professional work, combined with research on teachers’ assessment practices and assessment literacy. Professionalism is understood not merely as the possession of formal knowledge or skills, but as a socially and institutionally situated practice shaped by norms, values, and expectations regarding appropriate professional behaviour (Freidson, 2001; Evetts, 2006, 2013). From this perspective, what counts as “professional” teaching is continuously negotiated within specific organizational and policy contexts. A key distinction is made between professionalism as enacted from within the profession and organizational professionalism which is defined from outside of the profession and based on bureaucratic logics and accountability activities (Evetts, 2013). This distinction is used to analyse how teachers’ assessment practices are influenced by institutional demands while still relying on teachers' professional judgement.

Teacher agency is conceptualized as relational and context-dependent, drawing on an ecological understanding of agency (Priestley et al., 2015). Teachers’ responses to assessment practices are thus seen as negotiated rather than uniform. To capture the complexity of assessment work, the analysis is further informed by the concept of assessment literacy (Pastore & Andrade, 2019), which distinguishes between conceptual, praxeological, and socio-emotional dimensions of assessment. This framework enables an examination of how teachers balance formal requirements with pedagogical concerns not forgetting ethical considerations and the pupils’ socio-emotional development. Together, these perspectives frame teacher professionalism in assessment as a dynamic process shaped by institutional conditions and pedagogical values.

Performance Level Descriptors (PLDs) are embedded in curriculum frameworks internationally to describe the quality of student achievement (Cizek & Bunch, 2007; Sadler, 1987). These frameworks respond to legitimate concerns about the limitations of numerical scores and provide a common language for describing achievement. However, existing approaches to PLDs face a persistent dilemma: they tend to be either too generic to guide classroom assessment or too holistic to provide diagnostic information (Sadler, 2009; Wyatt-Smith, Klenowski, & Colbert, 2014).
In Australia's New South Wales curriculum, for example, the Common Grade Scale describes achievement from A to E using broad descriptors such as "the student has an extensive knowledge and understanding of the content and can readily apply this knowledge" (Level A) or "a sound knowledge and understanding of the main areas of content" (Level C). While these descriptors provide consistent language for reporting, they do not specify what "extensive knowledge" or "sound understanding" looks like for particular subjects or curriculum outcomes. Subject-specific Course Performance Descriptors exist for some stages, but these describe holistic profiles of achievement - what an A student in English looks like overall - rather than differentiating components of performance or connecting to specific curriculum outcomes. Teachers are required to make holistic judgements without explicit inferential warrants (Wyatt-Smith, Klenowski, & Colbert, 2014). This challenge is common across standards-referenced systems internationally; the translation from generic descriptors to specific assessment judgements remains under-theorised (Dolin & Evans, 2017; Harlen, 2005). The tension between system-level consistency and classroom-level specificity is a global problem.
This paper addresses the question: How can Performance Level Descriptors be operationalised to support warranted teacher inference at the classroom level?
We propose a framework that addresses this gap by synthesising three research traditions. First, research on disciplinary competency dimensions identifies what constitutes competent performance across domains - factual knowledge, core concepts, disciplinary skills and practices, and metacognition (e.g., Liu, Rogat, & Bertling, 2013). Second, research on transfer and assessment of learning outcomes distinguishes levels of cognitive demand - from recall of presented material through application in new situations to solving novel problems (e.g., Mayer, 2011; Barnett & Ceci, 2002). Third, research on expressions of quality articulates intellectual standards that characterise quality thinking - clarity, precision, logic, accuracy, relevance, and depth (e.g., Paul & Elder, 2019). From this synthesis, we derive six components of performance level descriptions - knowledge integration and use, conceptual understanding, reasoning and metacognition, disciplinary practice, skill fluency, and communication - that apply across subjects while manifesting differently for different content.
We argue that valid inference from student performance to achievement level requires operationalisation at the intersection of these components and specific curriculum outcomes. Drawing on argument-based validity frameworks (Kane, 2013; Toulmin, 2003), we present a specification structure that makes explicit: (a) which curriculum outcomes are being assessed, (b) what each component means for those specific outcomes, (c) what observable evidence warrants each level of placement, and (d) the limits of generalisability from any single task. The specification documents the inferential argument, making visible the warrants that are typically implicit in teacher judgement.
The framework is demonstrated through an assessment task developed for Stage 2 (Years 3–4) students in NSW. The task integrates English outcomes (persuasive writing) with social studies curriculum outcomes (comparing conservation and sustainability; Aboriginal sustainable land management practices). The design process articulates what each PLD component looks like for these particular outcomes, and reveals how task conditions must be engineered to elicit evidence for specific component-outcome intersections and how readily a task can fail to create these conditions if designed generically.