Conference Agenda

Open Science especially recent endeavours to archive and share research data on a large scale provoked a discussion of how research, as a search for knowledge – if not: truth – deals with data as an offset of this knowledge. The Symposium reflects on practices of sharing and reusing data and asks, first, exactly what knowledge it generates and, second, where this knowledge comes from in the process of scientific work.

The first contribution discusses Open Science as a collection of related practices concerning access to data and resources as well as results and knowledge, methodologies and participatory research practices (Reichmann, 2017). This complexity evokes an epistemic discussion of the concept of open knowledge (Rubin 2021) and its implications for education and educational science against the background of a new practice of science through Open Science and its involvement in certainty and uncertainty as an epistemic question of research culture(s).

The second contribution takes on a position of quantitative methods and methodology and discusses replication crisis versus opportunities of Open Research practices for quantitative analysis. While a re-use of data opens up great and economical opportunities for the generation of reliable knowledge (Krammer & Svencik, 2021), a light is shed on methodological and scientific-theoretical challenges in the re-use of data, like comparability and consistency of the constructs recorded. At the example of pracitices like HARKing (hypothesizing after the results are known; Kerr, 1998) possible threats to both value and validity of statistical hypothesis tests and thus of scientific findings are discussed.

The third contribution takes the position for qualitative research and shows how formal data sharing standards of for instance findability, accessibility, interoperability and re-usabilty like the European commission framework FAIR in Horizon Europe (European Commission, n.d.) meet challenges concerning the distribution of the way, data was collected (Jesser, 2011) as well as processed and what role participants played in making sense of it. Data sharing will therefore be regarded in the light of standards for qualitative research (Strübing et al., 2018), opening the discussion for considering the whole process of knowledge construction in Open Science practices.

In the fourth contribution Open Research practice in educational science is discussed against the background of data archiving, sharing, and re-use. Quantitative and qualitative data more and more has to meet requirements of scientific funders and journals (Logan, Hart, & Schatschneider, 2021). Data curators are introduced as players in the Open Research community supporting researchers in overcoming the discussed challenges of sharing data and in meeting Open Science standards.

Open science can be understood as a collective term for various movements (Fecher & Friesike, 2014) that advocate for a cultural shift toward openness within the scientific system (Reichmann 2017). Practices related to openness such as open access, open data, open methodology, open peer review and open educational resources not only affect the dissemination of knowledge but also the production of knowledge (see Grabensteiner and Svecnik, Grabensteiner and Heers in this symposium) and the related establishment of insights and truth. These movements in the sciences are taking place in the context of a society that is more dependent than ever on robust scientific insights to deal with the uncertainty of today's world and the crisis of truth. Against this background, questions arise such as: - What concept of knowledge do open science practices presuppose? - How important are openness and pluralism as epistemological principles in open science? (Leonelli 2022; Rubin, 2023)? - What is the relationship between openness and uncertainty? - What significance does an open view of knowledge have for education and educational science, which has the transfer of knowledge as its concern? In light of this, the concept of knowledge in the context of open science and its implications for education and educational science will be discussed from a social perspective on knowledge.

Fecher, B.; Friesike, S. (2014). Open Science: One Term, Five Schools of Thought. In Opening Science by Sönke Bartling and Sascha Friesike. Springer, https://doi.org/10.1007/978-3-319-00026-8_2. Leonelli, S (2022): Open Science and Epistemic Diversity: Friends or Foes? In: Philos. sci. 89 (5), S. 991–1001. DOI: 10.1017/psa.2022.45. Reichmann, W. (2017). open Science between social structures and epistemic cultures. A Conceptual Complement from a Science Studies Perspective. TATuP, https://doi.org/10.14512/tatup.26.1-2.43 Rubin, M. (2023). Opening up open science to epistemic pluralism: Comment on Bazzoli (2022) and some additional thoughts. Critical Metascience. https://doi.org/10.31222/osf.io/dgzxa

The so-called "replication crisis" in (social) psychology a good 10 years ago showed how uncertain scientific findings can sometimes be. In many cases, it was not possible to replicate seemingly undisputed effects that had been published in high-ranking journals following peer review and taught in university studies (Open Science Collaboration, 2015). This phenomenon is not limited to psychology and resulted in the dictum ‘Why Most Published Research Findings Are False’ (Ioannidis, 2005) what can also be expected for educational research (Makel et al., 2021). This raises the question of how scientific knowledge can be improved and made more reliable. There are indications that Open Science, or more precisely its components Open Materials and Open Data, can make a significant contribution (e.g. Krammer & Svecnik, 2021). Open data in particular can be seen as an opportunity to generate stable findings in educational research, but it also raises a number of related questions. For example, the sequence of theory - hypotheses - data collection - analysis and conclusion required as good practice in the classic NHST paradigm (Neyman & Pearson, 1928) is disrupted by the data basis already available. On the one hand, this threatens the validity of statistical hypothesis tests and, on the other hand, encourages HARKing (hypothesizing after the results are known; Kerr, 1998). Both endanger the value and validity of scientific findings. Furthermore, the re-use of data, among others, raises the question of comparability and consistency of the constructs recorded. These and other questions of gaining knowledge through empirical research are discussed in the contribution.

Ioannidis, J.P.A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. Kerr, N.L. (1998). HARKing: hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. Krammer, G. & Svecnik, E. (2021). Open Science als Beitrag zur Qualität in der Bildungsforschung. Zeitschrift für Bildungsforschung, 10(3), 263-278. https://doi.org/10.1007/s35834-020-00286-z Makel, M. C., Hodges, J., Cook, B. G., & Plucker, J. A. (2021). Both questionable and open research practices are prevalent in education research. Educational Researcher, 50(8), 493-504. https://doi.org/10.3102/0013189X211001356. Neyman, J. & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference: part I. Biometrika 20A:1/2, 175-240. https://doi.org/10.2307/2331945 Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science. https://doi.org/10.1126/science.aac4716.

Open Science will be discussed along the methodological principles of Constructivist Grounded Theory (Charmaz, 2006; Grabensteiner, 2023). Research processes as communicative endeavor will be distinguished from methodically guided knowledge construction through an interlinkage of theoretical sensitivity and data collection. Data sharing practices of Open Science ask for standards, focusing on research data to be findable, accessible, interoperable and re-usable, as for instance stated in the European commission framework FAIR in Horizon Europe (European Commission, n.d.). Beyond that there are criteria for scientific practices to meet standards. Strübing et al. (2018) propose appropriateness towards a specific subject matter, empirical saturation, theoretical depth, writing performance and originality (Strübing et al., 2018, p. 85f) as quality criteria. Discussing frameworks, both for data sharing and for data collection, the question arises, how data and knowledge are intertwined and in what way qualitative research practice challenges and enables Open Science simultaneously by meeting its own quality criteria. Jesser (Jesser, 2011) proposes two forms of data information to be shared and archived along with the data. First, meta-information “necessary to understand the content and structure of the dataset” (Jesser, 2011, p. 8) and second “context information”, meaning “institutional, theoretical and methodological background” (Jesser, 2011, p. 8). This enables insight into ways of data collection, data processing as well as reflections by researchers in the course of dealing with the dataset. Writing memos is already an established practice in qualitative research whereas haring them in order to make data accessible for secondary analysis is still in progress of becoming a standard. New forms of Open Science shed a light on data documentation practices, making way for qualitative research to contribute to customs of “openness” in qualitative and quantitative research. Up to the point where research participants are not only “voices” heard in the research process, but also contributors to knowledge construction. Borg et al. (Borg et al., 2012) show at the example of Co-Operative Inquiry, they develop different criteria of openness, being consensus, historicity (process of knowledge production), reflexivity (especially on asymmetries) and knowledge co-production (interaction with participants, giving something back) (Borg et al., 2012, p. 10ff). Applying those as standards in the process of data construction, shared data gain a further dimension of depth and saturation. Synopsis of standards for data sharing and documentation of knowledge construction processes shall inspire reflections on future Open Science practices considering the whole research process.

Borg, M., Karlsson, B., Kim, H. S., & McCormack, B. (2012). Opening up for Many Voices in Knowledge Construction. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 13(1), Article 1. https://doi.org/10.17169/fqs-13.1.1793 Charmaz, K. C. (2006). Constructing grounded theory: A practical guide through qualitative analysis. SAGE Publications Ltd. European Commission. (n.d.). Open science. Retrieved 22 January 2024, from https://rea.ec.europa.eu/open-science_en Grabensteiner, C. (2023). Medienbildung im Medienhandeln. Rekonstruktion relationaler Bildungsprozesse am Beispiel von Instant Messaging in Schulklassen. Springer VS. https://doi.org/10.1007/978-3-658-40699-8 Jesser, A. C. (2011). Archiving Qualitative Data: Infrastructure, Acquisition, Documentation, Distribution. Experiences from WISDOM, the Austrian Data Archive. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 12(3), Article 3. https://doi.org/10.17169/fqs-12.3.1734 Strübing, J., Hirschauer, S., Ayaß, R., Krähnke, U., & Scheffer, T. (2018). Gütekriterien qualitativer Sozialforschung. Ein Diskussionsanstoß. Zeitschrift Für Soziologie, 47(2), 83–100. https://doi.org/10.1515/zfsoz-2018-1006

As in other social science disciplines, in educational research, there is a growing demand for more transparency throughout the research cycle (van der Zee & Reich, 2018). Data archiving, sharing, and re-use are at the center of these discussions. Against this background, more and more educational data are made available for secondary analyses. This holds for quantitative but also more and more for qualitative data. Data sharing is also increasingly important to meet the requirements of scientific funders and journals (Logan, Hart, & Schatschneider, 2021). In order to provide high-quality data with re-use potential, data curators play a crucial role. This contribution will outline specific challenges that researchers face when sharing their data. It will elaborate on how data curators can support them in overcoming these challenges. In a final part, the benefits for researchers of sharing and having data professionally curated are outlined.

Logan, J. A. R., Hart, S. A., & Schatschneider, C. (2021). Data Sharing in Education Science. AERA Open, 7, 23328584211006475. https://doi.org/10.1177/23328584211006475 van der Zee, T., & Reich, J. (2018). Open Education Science. AERA Open, 4(3), 2332858418787466. https://doi.org/10.1177/2332858418787466