## **Evidence and Expertise in Nordic Education Policy**

## A Comparative Network Analysis

*Edited by*  Berit Karseth · Kirsten Sivesind Gita Steiner-Khamsi

## Evidence and Expertise in Nordic Education Policy

Berit Karseth • Kirsten Sivesind Gita Steiner-Khamsi Editors

## Evidence and Expertise in Nordic Education Policy

A Comparative Network Analysis

*Editors* Berit Karseth Department of Education University of Oslo Oslo, Norway

Gita Steiner-Khamsi Teachers College Columbia University New York, NY, USA

Kirsten Sivesind Department of Education University of Oslo Oslo, Norway

ISBN 978-3-030-91958-0 ISBN 978-3-030-91959-7 (eBook) https://doi.org/10.1007/978-3-030-91959-7

© Te Editor(s) (if applicable) and Te Author(s) 2022.Tis book is an open access publication.

**Open Access** Tis book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this book are included in the book's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Te use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specifc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Te publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. Te publisher remains neutral with regard to jurisdictional claims in published maps and institutional afliations.

Cover illustration: Orbon Alija

Tis Palgrave Macmillan imprint is published by the registered company Springer Nature Switzerland AG. Te registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

## **Acknowledgments**

Te idea of applying comparative methods of inquiry to understand reforms that travel, which is presented in this volume, was conceived in 2017 within the context of the Policy Transfer Project (POLTRANS). In this project Gita Steiner-Khamsi, Teachers College (TC), Columbia University, was appointed as a part-time adjunct visiting professor at the Department of Education, University of Oslo. As part of the POLTRANS project (NRC 271314), which was funded by the Norwegian Research Council for the period of 2017–2021, we were able to collaboratively develop research proposals that enabled us to empirically examine why transnational policy and research resonate in a national context and how they are subsequently translated and contextualized in the form of national school reforms. Te project brought together senior and junior researchers from the University of Oslo and TC, Columbia University in New York.

In a remarkable team efort, we elaborated the contours of a larger research project that draws on social network analysis and bibliometric analysis to investigate evidence usage and knowledge transfer in education policy across the fve countries of the Nordic region: Denmark, Finland, Iceland, Norway, and Sweden. Tis larger, comparative study was approved, and the University of Oslo was awarded a research grant by the Norwegian Research Council in 2018 (NRC 283467). Tis research project, Policy Knowledge and Lesson Drawing in Nordic School Reform in an Era of International Comparison (POLNET), is chaired by Kirsten Sivesind, University of Oslo in collaboration with Berit Karseth, University of Oslo and Gita Steiner-Khamsi, TC, Columbia University. It will last until August 2023.

One of the recurring themes in comparative research is how to do justice to the vast diferences that exist in political systems. For our study, identifying the diferences in the policy process of the fve Nordic countries became key for sense making and interpreting the fndings. Unsurprisingly, acknowledging the diferences and similarities in how national authorities produce policy papers and authorize national school reforms in the same geographical region of Europe soon became a key research question of the project and shaped the thinking of the members of the POLNET research group and the authors of this book. By collecting and systematizing bibliographic references in a large sample of policy documents, the work fulflled our aspiration to acquire deep insights into areas that have been both theoretically and empirically underexplored.

We would like to thank all individuals and institutions that spent time and resources designing and supporting the research project. We are extremely pleased that we attracted leading scholars in education policy studies and several of their doctoral students to the project. Teir names may be found in the table of contents of the book. In the same vein, we are humbled that extraordinary scholars with international stature— Kerstin Martens, University of Bremen, and Antoni Verger, Autonomous University of Barcelona—agreed to read the edited volume and provide their comments for consideration.

In addition to acknowledging the great commitment of the scholars contributing to the research project and this book, we would like to extend our gratitude to their institutions. Te following fve Nordic universities allocated supplemental fnancial resources to ensure solid data collection and analysis at the national level and for cross-national comparison: Linnaeus University, the University of Iceland, Tampere University, Aalborg University, and the University of Oslo. We would also like to thank all of our research partners, including their research assistants, who contributed either directly by participating in meetings or indirectly by collecting, entering, and categorizing data. Te administrative leadership team and staf at the Department of Education and the Faculty of Education at the University of Oslo provided valuable support throughout the study period. We would like to express our deep appreciation to Kari-Anne Ulfsnes. Additionally, we want to thank Simona Bernotaite for her invaluable help in copy editing and formatting the manuscript.

We are especially grateful to Bernadette Hörmann, who assisted the team in preparing the POLNET research proposal and systematizing the Norwegian data set over several months during her postdoctoral position at the University of Oslo, and to Oren Pizmony-Levy, who developed the research protocol with the team from TC, Columbia University and served as a senior research advisor. Chanwoong Baek, who was a PhD student from TC, Columbia University at the start of the research project and is now a postdoctoral fellow at the University of Oslo, deserves our deepest appreciation for creating and cleaning the international data set and producing bibliometric analyses while he served as an associated research fellow as well as for producing and editing nearly all of the tables and fgures presented in this volume throughout the research study.

Finally, special thanks goes to our peer reviewers—Daniel Petterson, University of Gävle, Guri Skedsmo, the Schwyz University of Teacher Education, Justin W. Powell, University of Luxembourg, Robin A. Shields, University of Bristol, Tone Kvernbekk, University of Oslo, and Florian Waldow, Humboldt University—for their valuable comments on the papers we presented at international conferences, panels, and meetings.


## **Contents**




*Berit Karseth and Kirsten Sivesind*

## **Notes on Contributors**

**Juho Anturaniemi** is a Master's degree student in the Faculty of Education and Culture, specialization in governance and politics in education and a Research Assistant in POLNET project, Tampere University, Finland. His research interest is on the language of politics and policy.

**Chanwoong Baek, PhD** is a Postdoctoral Fellow, University of Oslo. His areas of expertise include comparative policy studies, global governance, sociology of knowledge, and social network analysis. He holds a PhD in comparative and international education with a specialization in sociology from Columbia University and a Master's degree from Stanford University.

**Bernadette Hörmann, PhD** is a Postdoctoral Fellow, University of Oslo. Her research focuses on school reforms from an international and comparative perspective, education policy, and school and curriculum theory. Her latest articles are published in the *Nordic Journal of Studies in Educational Policy* (2018) and in the *Journal of Curriculum Studies* (2021).

**Jón Torf Jónasson, PhD** is Professor Emeritus, School of Education, University of Iceland. He was previously the dean of the school and, prior to that, dean of the Faculty of Social Science at the University of Iceland. His research emphasis has been on the development of the education system from pre-primary to tertiary levels. Jónasson's recent works on Nordic education policy include a co-authored article in NordSTEP, coauthored with L. Lundahl and A. L. Arnesen.

**Berit Karseth, PhD** is a Professor at the University of Oslo, Department of Education. Karseth's research and publications are in the felds of education policy, reforms, and curriculum studies. She is a former dean (2013–2016) and is the leader of the research group CLEG at the Faculty of Educational Sciences, University of Oslo.

**Jaakko Kauko, PhD** is Professor of Education Policy, Faculty of Education and Culture, Tampere University, Finland. His research focuses on politics of education and comparative education. Kauko's publications include *Politics of Quality in Education: A Comparative Study of Brazil, China, and Russia* (edited with R. Rinne and T. Takala; 2018).

**Berglind Rós Magnúsdóttir, PhD,** is Professor University of Iceland. She is leading several research projects on social justice in education. Recent publications concern education policy for refugee youth, emerging ethnic and class segregation in the Icelandic education feld, and systematic review on educational research on inclusion in Finland, Netherlands, and Iceland.

**Kerstin Martens, PhD** is Professor of International Relations at the University of Bremen, Germany. Her research interests include education policy, NGOs, and international organizations. She heads the research project "Te Global Development, Difusion and Transformation of Education Systems" of the Research Centre 1342 "Global Dynamics of Social Policy," University of Bremen.

**Andreas Nordin, PhD** is Associate Professor of Education, Linnaeus University, Sweden. His areas of expertise include curriculum theory, education policy, and comparative and international education. He is a co-author of the edited volume *Transnational Policy-fows in European Education—Te Making and Governing of Knowledge in the Education Policy Field* (2014).

**Oren Pizmony-Levy, PhD** is an Associate Professor, Department of International and Transcultural Studies, Teachers College, Columbia University. He holds a PhD in sociology and comparative and international education from Indiana University-Bloomington. His research and teaching focus on the intersection between education and social movements, such as the accountability movement and its role in the emergence of international assessments of student achievement (e.g., TIMSS and PISA) and the environmental and sustainable education movement.

**Trine Juul Reder** is a PhD candidate, Department of Culture and Learning, Aalborg University, Denmark. Her main research interests lie within the feld of global education and the historical role of international organizations. She is part of the research project "Te Global History of the OECD in Education" led by Professor (WSR) Christian Ydesen.

**Íris Santos** is a PhD candidate at the Faculty of Education and Culture at Tampere University, Finland, and a PhD student at the Institute of Education at the University of Lisbon, Portugal, under a Cotutelle agreement. Her research interests concern the dynamics of global education governance and policy, and her recently published work features in the *Journal of Education Policy*, the *European Educational research Journal,* and in *Compare: A Journal of Comparative and International Education*.

**Kirsten Sivesind, PhD** is an Associate Professor in the Department of Education at the University of Oslo. Beyond her academic discipline of curriculum theory, her research focuses on comparative analysis and school reform. She serves as the Principal Investigator for the fve-country study, "Policy Knowledge and Lesson Drawing in Nordic School Reform in an Era of International Comparison," co-headed with B. Karseth and G. Steiner-Khamsi.

**Gita Steiner-Khamsi, PhD** is Professor of Comparative and International Education, Teachers College, Columbia University. She is a past president of the Comparative and International Education Society, an editor of two series related to comparative policy studies and development (Teachers College Press, E. Elgar Publisher), and an author or (co-) editor of 12 books related to comparative education, policy studies in education, and globalization research.

**Dijana Tiplic, PhD** is a Professor, Department of Teacher Education and School Research, University of Oslo. In her research, Tiplic pursues several lines of work related to studies of organizational phenomena, with a specifc focus on educational organizations and comparative education research. Her work has been published in the *Cambridge Journal of Education*, in *Higher Education*, and in *Compare: A Journal of Comparative and International Education*.

**Antoni Verger, PhD** is a Professor at the Department of Sociology of the Universitat Autònoma de Barcelona (UAB) and research fellow at the Catalan Institution for Research and Advanced Studies (ICREA). With a cross-disciplinary training in sociology and education studies, his research examines the relationship between globalization, governance institutions, and education policy.

**Saija Volmari** is a PhD candidate at the Doctoral Program in School, Education, Society and Culture, University of Helsinki and she has a M.A. from the University of Amsterdam. Her areas of expertise include globalization and education, education politics and policymaking in the Nordic countries, social justice in education, and comparative and spatio-temporal policy analysis.

**Ninni Wahlström, PhD** is Professor of Education at Linnaeus University, Sweden. Her current research focuses on transnational and national policy discourses and their implications for national curriculum and classroom teaching from a perspective of educational philosophy and curriculum theory. Her recent publications include the book *Transnational Curriculum Standards and Classroom Practices: Te New Meaning of Teaching* (2018), co-edited with D. Sundberg.

**Christian Ydesen, PhD** is a Professor at the Department of Culture and Learning, Aalborg University. He has published several chapters and articles on educational testing, international organizations, accountability, educational psychology, and diversity in education from historical and international perspectives. He serves as an executive editor of the European Educational Research Journal.

## **Acronyms**




## **List of Figures**


#### **xxii List of Figures**


## **List of Tables**



# **1**

## **Introduction: A Comparative Network Analysis of Knowledge Use in Nordic Education Policies**

**Kirsten Sivesind and Berit Karseth**

Tere is wide consensus in public policy today that globalization urges countries and their state authorities to bring scientifc truth into policymaking processes. However, many scholars and policy analysts have also addressed ambiguity and complexity in regard to how scientifc knowledge is used in policymaking processes (Cairney, 2016; Smith et al., 2020; Steiner-Khamsi et al., 2020; Weingart & Guenther, 2016). Even though evidence is highly integral in terms of advanced, specialized knowledge, few scholars expect science and politics to serve the same roles and rationales. Moreover, the status and role of empirical evidence can be diferent between policy projects depending on the realms and knowledge these projects address, as well as their political contexts. Terefore, researchers have long expressed great interest in investigating how experts pursue their responsibilities and how they provide evidence in education policy (Heggen et al., 2010; Krejsler, 2013; Steiner-Khamsi, 2013).

K. Sivesind (\*) • B. Karseth

Department of Education, University of Oslo, Oslo, Norway e-mail: kirsten.sivesind@iped.uio.no; berit.karseth@iped.uio.no

<sup>©</sup> Te Author(s) 2022 **1**

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_1

#### **2 K. Sivesind and B. Karseth**

In order to gain deeper insights into education policy and the way policymakers and experts deploy knowledge to legitimize school reforms, the authors in this edited volume examine citation patterns in corpuses of governmental papers written under the auspices of state authorities in the Nordic region of Europe. All empirical chapters are end results from the research project Policy Knowledge and Lesson Drawing in Nordic School Reform in an Era of International Comparisons (POLNET), funded by the Norwegian Research Council (NRC 283467). In this project, fve Nordic research teams and a research team at Teacher College, Columbia University, New York, developed a shared database by systematizing references in policy documents. By collecting and analyzing bibliographic meta-data extracted from sources referenced in policy documents, the authors examine how these sources are tied together into networks of references in and across reform-making processes in fve Nordic countries (Denmark, Iceland, Finland, Norway, and Sweden). Such sources include governmental documents and other types of work published by nonacademic authors alongside research-based evidence produced at research institutions. In addition, we have interviewed key experts who have experience from public inquiry bodies and ministry or agency ofcials with a particular responsibility for assisting leaders and members of the inquiry bodies. Some interviewees had also experience with referencing the inquiries in their work of authoring white papers for the government.

In the POLNET study, our main interest is to examine how scientifc knowledge and research-based evidence become afliated with other knowledge sources in reference networks that legitimize policymaking processes. Our fndings and interpretations result from a systematic analysis of white and green papers. In the Nordic contexts, white papers are documents produced by governments to legitimize political recommendations that are reviewed and debated in parliamentary processes. Tese papers may have a profound impact on how discussions proceed in parliamentary processes and how the bureaucracy and institutions within various sectors, such as education, eventually handle political problems. Due to the complexity of these problems as well as institutional demands, both politicians and governments are in a need of catalyzing changes and surveying people and practices to make sure that they adopt solutions that beneft society. Terefore, these policymakers also ask for public inquiry reports, or what we in the volume call green papers. Tese green papers are most often written before a white paper is produced to evaluate issues and clarify the pros and cons of various options and solutions.

In some countries, public inquiries are conducted by one senior ofcer within a ministry or by an external expert formally mandated by the ministry. Tis person reviews documents, collects data, and draws on various types of expertise to write a report about a particular topic. In the Nordic region of Europe, such inquiries are also made by advisory panels consisting of stakeholders, representatives from the sector, and/or experts who have the mandate to independently author a green paper to make recommendations for how to reform or renew policies in a particular area. Tese inquiries are organized to enlighten the work of policymakers and politicians and to inform the public, since they target political, social, cultural, and economic conditions of relevance for large parts of the population in a country. Besides these inquires, ministries may ask experts to review a particular knowledge feld and write a report without a government-issued mandate. In any case, debates based on these inquiries and expert reports can have various consequences. Formally, they may lead to political decisions within the parliament, which has the formal duty to represent the interests of the citizens in the country by making laws, legitimizing reforms, and overseeing the work of the government in terms of hearings, inquiries, and evaluations. Informally, discussions upon these reports can engage stakeholders and others in dialogues that make impact on education policy.

A core aim of our analysis has been to discover diferences between reference patterns in the countries' white and green papers for the latest school reform related to compulsory education (grade 1–9/10), and to explore the role of knowledge usage in the policy processes that resulted in these papers. By drawing on Paul Cairney's (2016, p. 3) defnition, "'evidence' is assertion backed by information" (p. 3), we regard the knowledge sources that are referenced as the actual evidence. Nutley et al. (2019) have provided an overview of evidence-promoting organizations and their ways of identifying, labeling, and ranking "good evidence," on which the organizations have not reached any consensus yet. Tereby, we consider the relative merits of various types of evidence as an open question for academic institutions and public branches, such as ministries and agencies, to have an ongoing debate about.

A common view is that evidence-based policies draw on knowledge from outside of public policies, including from universities and researchbased institutes where dedicated researchers produce scientifc knowledge of relevance for policymakers. However, evidence and expertise are not only produced at universities and research institutes but also result from various activities and innovations across a variety of branches. Tese activities are arranged through networks of academic and non-academic partners that consist of public ministries, state agencies, non-governmental organizations, and even grassroots initiatives (Nelson & Campbell, 2019). Various types of meetings, projects, and networks bring together partners in joint eforts to advance expertise that enlightens and legitimizes political actions and solutions. Researchers have ofered good reasons to think of policy-relevant knowledge as developed across traditional division lines between science and politics (Godin & Shauz, 2016). In this context, policymakers and experts are expected to assess the evidence to determine "whether the source providing us with information is trustworthy" (Eyal, 2019, p. 34) and relevant. Although evidence is expected to be scientifc in a narrow sense, they may anyhow apply a broad concept of knowledge, including information, ideas, and arguments; well-tested beliefs; and lay, professional, and academic knowledge (Radaelli, 1995). Against this backdrop, relevant knowledge in public policy, authorized as a trustworthy source, can be broadly defned to include both academic work developed by scholars at universities or research institutes and knowledge that is regarded as less scientifc because it is produced in nonacademic contexts. Consequently, evidence can be based on or afliated with various knowledge sources depending on where and how policymakers and experts interact as well as how institutional reputation and legitimization serve as the modus operandi for the way policy is governed (Ball & Junemann, 2012). A key point in Steiner-Khamsi's (2013) work is that the production of knowledge, especially the design of comparative studies, optimizes evidence for executing at least three options of diferent types of knowledge use. Tat is, references can be used either (a) as evidence that informs policy planning within particular contexts, (b) as normative guidelines for how to change educational processes concerning global problems, or (c) as projecting best practices that are evaluated against a set of international performance standards. Tere are good reasons for regarding the production of knowledge, especially the design of comparative research projects, as optimizing evidence for executing one or more of the three alternatives. Te chapters that follow elaborate on these and related purposes by drawing on various theories as well as empirical data collected within the Nordic research project.

## **Topics and Perspectives**

Te POLNET study pays special attention to policy processes, in particular to the nexus of national, regional, and international policy brokers and their knowledge provision and usage (Steiner-Khamsi, 2013). Te combination of policy learning and borrowing (or reception) and the sociology of policy transfer serves as an analytical lens to study these topics. Drawing on globalization studies (Verger et al., 2018) and research on policy borrowing and lending (see, for instance, Steiner-Khamsi, 2012; Waldow, 2012), the book's overall purpose is to enlighten a broad discussion on what counts as evidence for policymakers and which roles various types of expertise play in policymaking processes. Following chapters address one or several of the following research questions:


Our primary concern has been to explore the signifcance of evidence as a lever in policy formulation processes, notably by the prominence and role of *single* or individual references in terms of citations to specifc articles, book chapters, books, reports, and other documents. When source documents refer to clusters of references that share particular features, we can speak of spatial reference types. A central question that begs analysis is how policy processes provide reference patterns of both direct and spatial types within collections of governmental documents in areas of interest.

It is our assumption that both single references and spatial reference patterns can project systems as well as outcomes. Tey can, for example, locate knowledge in diferent geographical entities (e.g., locally, nationally, globally) or refer to particular types of knowledge (e.g., books, journal articles, governmental papers, reports) or to particular realms that are characterized by a set of thematic areas of interest. Tey can also project preferences for and prominence of certain types of policy analysis, such as systematic reviews, evaluations, sector analyses, and comparative studies in terms of international large-scale assessments that project best practices in terms of evidence-based standards. By conducting comparative network analysis, there are many possibilities to identify both individual and spatial reference types that reveal insights into the way policymakers and experts co-construct knowledge that legitimize education policy.

Another feature we have investigated relates to semantic patterns that can be extracted from content analysis of documents. Te use of references and spatial patterns that evolve from policy processes results in thematic areas or policy realms that, in our case, relate to school reforms in the Nordic countries. Tese capture legislative reforms, curriculum and assessment policy, and policies addressing the development of literacy among children and adolescents. Terefore, we also look into the policy texts by describing the content of reforms (see Chaps. 4, 5, 6, 7, and 8) and by unraveling core themes, decision strategies, and conceptual schemes that can be more or less outcome-oriented (see Chap. 7 in this volume). Eventually, we examine key narratives to discern how policymakers and experts make sense of evidence provided by public inquiry bodies and other branches (see Chap. 12).

Te following three bodies of research have been important for designing sub-studies in the POLNET project that are included in this edited volume: (a) research on network governance; (b) studies of "travelling reform" (i.e., difusion vs. reception studies); and (c) research on evidence-based policy. In the next section, we will briefy present each of these bodies.

#### **Research on Network Governance**

First, this book draws on research about the role network governance plays in education policies and how changing conditions for state regulation can help with interpreting variations in reference patterns. Several researchers have pinpointed the recent transformation of network governance that extends new public management policies introduced in most European countries. Helgøy et al. (2007) have described the contemporary changes in education policy as a re-regulation of the society by ways networks of actors make use of new governance tools. In the era of comparative education analysis, Ball and Junemann (2012) have contributed to a renewed understanding of how policy networks are changing education policy by including philanthropy and business partners in public policymaking. Combining social network analysis and ethnographic research, the authors unraveled how new communities of social actors, such as think tanks and interest organizations, infuence public policy by actively engaging in conversations beyond institutional procedures. By referencing political scientists (Bevir & Rhodes, 2003), Ball and Junemann (2012) argued that there is a shift from government to governance, or from a unitary state where government is "carried out through hierarchies or specifcally within administrations and by bureaucratic methods" to governance, which is "accomplished through the 'informal authority' of diverse and fexible networks" (p. 168). Moreover, these networks move beyond the methods of new public management by being confgured in terms of tightly connected policy communities or more loosely connected issue networks. Such networks arise from the exchange of resources, such as money, information, or expertise that enable policymakers to accomplish their goals.

Our book contributes new insights about the development, use, and reception of expertise in public policy. Building upon Ball and Junemann's (2012) theorizing about network governance, we extend the idea of loosely coupled issue networks. By looking for spatial reference patterns in policy documents, we explore how network governance underpins agenda setting and policy formulation processes in reform areas such as curriculum and assessment policies. Certainly, both of these areas are emphasized in Nordic school policy today and have roots in reform trajectories, developed by professional communities from the early twentieth century and onward. Troughout the 1900s, both intellectuals and reformists developed knowledge and expertise about assessment that resulted in progressive reform movements in the Nordic countries (Ydesen et al., 2013). More recently, international comparisons have been conducted in all Nordic countries (Sivesind, 2019) as a key monitoring tool by the national governments to make policy decisions and assess the quality of teachers, schools, districts, and the education system itself (Prøitz et al., 2017; Camphuijsen et al., 2020; Skedsmo et al., 2020). Tese comparisons enable international organizations to serve as policy actors and knowledge providers, which have implications for how national assessment systems are designed (Tveit & Lundahl, 2018) and how curriculum policy is formulated and enacted under the auspices of national governments (Mølstad & Karseth, 2016; Nordin & Sundberg, 2016; Sivesind & Wahlström, 2016).

However, countries difer in the expansion of non-state actors in their educational policy processes. Tompson (2003) explained this diference by noting that networks result from interactions between various components, such as ideas or concepts, people, institutions, social practices, or bodies of knowledge, and thereby constitute a form of "assemblage." Moreover, networks are also characterized by a feedback loop mechanism that transforms policies by their own logics that more often than not result in confgurations that connect hierarchies with the market (p. 363). While researchers in both political and educational sciences frst assumed that network governance was a reform strategy that changed the way governments could govern public services, several scholars have highlighted the fact that this strategy does not necessarily transform democracies in all that matters. As a result, network governance does not necessarily change the hierarchies of bureaucracies into a totally new shape. Based upon this discovery, researchers have begun to consider networks as being characterized by hierarchies in some form. To what degree bureaucracies are shaped by network governance is thereby an empirical question.

#### **1 Introduction: A Comparative Network Analysis of Knowledge…**

Tis notion led to new insights about the role of the state in public administration and the idea of hybrid forms of governance, which is a highly contested concept as well. Renowned scholars in the feld of education and political science have questioned whether public policy has transformed into a post-bureaucratic mode of governance (Maroy, 2012). Tis notion motivated Philippopoulos-Mihalopoulos (2012, p. 12) to write, "Tere is no such thing as a hybrid (yet)" (p. 12), meaning that there can be hybrid governance modes present in public policies, but not necessarily everywhere. Te same author argued that the appellation *hybrid* more often than not refers to pre-existing connections with "a certain otherness" (p. 12). Tus, formal bureaucracies that are hierarchically organized and governed by formal and substantial rationales in the Weberian way can still survive within a network-based society; however, they are observed diferently within the context of post-bureaucratic governance (Hall & Sivesind, 2015). For example, Teisens et al. (2016, p. 467) argued from the perspective of New Public Governance that the state still plays a dominant role in the political context where public sector organizations operate as knowledge providers. Tis is certainly true for the Nordic countries, which in many ways moved beyond the era of new public management during the 1980s (Christensen & Lægreid, 2007).

Following the same argument, Greve et al. (2016, p. 192) argued that reform policy today seems to emphasize state-centered solutions while acknowledging the mix of governance mechanisms and institutional complexity that characterizes the public sector. Tey concluded that state bureaucracies are still operative institutions that govern reform processes in the Nordic countries, but they are "less driven by politicians, more planned, and less contested by the unions, and there is more public involvement" (p. 201). Consequently, public policy is still observed as shaped by hierarchical structures in terms of government; however, it is observed diferently than before, due to evolving networks that are reordering the environment of bureaucracies. Tis situation has generated renewed interest in collaborative governance. In this volume, authors argue that there are many more reform actors on the international stage who pursue their own strategies and reform activities besides national governments; furthermore, this situation does actually lead to deregulation in particular policy realms "as a process of removing or reducing state regulations" (Dovemark et al., 2018, p. 123). Against this backdrop, there are good reasons for looking at the orchestration of global governance and at international organizations as intermediary bodies that infuence national governments (Abbott et al., 2015).

## **Studies of Traveling Reform: Diffusion Versus Reception Studies**

Second, in part because of new relations and connections between local and global venues for policymaking, authors in this volume draw on academic research about the increased reliance on externalization in education policy. Tis theory is based on a sociological theory about how systems, such as education, politics, and science, interplay within a world characterized by increased complexity. Te theory pinpoints the importance of researching how education policy is legitimized (Steiner & Waldow, 2012), and therefore we are examining how policymakers reference international and regional sources and/or cross-sectorial works to justify reforms in their Nordic countries. Tis dimension, which focuses on the legitimization of reforms, helps us to explore how policy processes change character through policy transfer (i.e., traveling reforms) and, in particular, if and how policymakers and experts borrow solutions from abroad or draw lessons about how to reform education in specifc contexts (Steiner-Khamsi, 2012).

A group of institutional researchers studying traveling policy looks at the difusion and dissemination of international standards or global education regulations as the key mechanisms for researching the transfer of policy knowledge. Teir views enable them to understand why certain policies are expanded and promoted while others are not. Obviously, their views allow us to understand the active role of international organizations in lending or disseminating specifc policies and programs, which are referred to as "best practices" or "international standards" (Bromley & Meyer, 2015; Krücken & Drori, 2009). However, this "from above" view, which is often associated with the theory of world culture, provides only one of several perspectives for understanding the dissemination of global education policy. Another angle clarifes how, why, and when national or local policy actors selectively learn from global education policies based on local conditions (Baek et al., 2018). When borrowing global education policies, such as competency-based curriculum reforms, accountability reforms, or public–private partnership policies at the local or national level, this "bottom-up perspective" focuses on the process of both reception and translation (Steiner-Khamsi, 2015).

In the past few years, more studies have emerged that have advanced the "look from below" view in important ways. Te Scandinavian institutionalist approach belongs to this tradition, which difers from other institutional research traditions by being concerned with non-strategic approaches and involving researchers who proclaim that ideas and practices undergo profound changes when deployed in new organizational settings (Czarniawska & Sevón, 1996; Røvik, 1996; Sahlin-Andersson, 1996). In this theory, stories, images, narratives, and master ideas serve as lenses to think of policy transfer that is shaped globally without necessarily attracting everybody's attention or direct involvement across the globe.

Tere is also a long tradition in comparative education of studying the reasons for "transnational policy attraction" within local and regional contexts, that is, to study why the government borrowed, imitated, or transferred others' policies (Phillips, 2004). Similarly, researchers have studied the role of public–private partnerships and privatization in such contexts to examine the question: Why does global education policy resonate in a specifc context; in other words, why do policy participants "buy" the policy? Verger (2014) has ofered an explanatory framework to understand how reform plans are "sold" to the governments of lowincome countries (Verger et al., 2016). In this case, an economic theory helps to explain why projects and plans are bought by non-afuent countries. Recent studies have also asserted that global education policies resonate for diferent reasons in diferent contexts, however, not necessarily due to the logic of quasi-markets (Maroy & Pons, 2019). Te idea of evaluative state and international organizations as standard-setters makes sense for understanding why international large-scale assessments (ILSAs), such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS), resonate diferently in diferent countries (Addey et al., 2017; Martens et al., 2010; Pizmony-Levy, 2017; Ydesen, 2019). Nonetheless, the reception and use of ILSAs are most often dependent on how national policies project their own goals and conceptions of best practices (Waldow & Steiner-Khamsi, 2019).

Verger (2014) emphasized that policy ideas need to be placed into a context for understanding how global policies get transformed once they have penetrated a local arena. Political institutions, administrative and regulatory viability, government ideology, domestic political contention, and periods of crisis create a context for strategic selectivity regarding the reception of policy ideas. Maroy and Pons (2019) demonstrated the importance of context in understanding how transnational models of accountability are recontextualized and translated into national policies in various ways. Furthermore, Karseth and Solbrekke (2010) analyzed the Bologna policy and the translation of the European Qualifcations Framework in terms of how national qualifcation frameworks have been developed in diferent countries. All the powerful institutions within these countries examined, emphasized the importance of learning outcomes as a reference point; however, they interpreted the Bologna policy diferently, as they created various windows of opportunities for reform and change within their respective contexts.

In European countries and elsewhere, policy knowledge has initiated changes in various directions. On the national level, civil society organizations, companies, and local associations have expanded their infuence in public education. On the transnational level, international organizations have evaluated, compared, and ranked education systems, producing various difusion mechanisms that have made non-state actors into standard-setters (Peters et al., 2009) and by advocating what Steiner-Khamsi (2013) characterizes as "best practices." Moreover, scientifc communication itself, mediated by international journals and their media strategies, legitimizes reforms in the education sector (Mølstad et al., 2017). Although non-state-actor organizations have no formal mandate to govern education reform, they still execute informal power through new "soft" but efective knowledge products within policy networks that are often characterized by a particular language, where measurements in terms of calculations and numbers serve as the core instrument (Grek, 2008). As a result of new technology for governing education, policy processes have come to be confgured by the production and use of knowledge (Abbott et al., 2015; Brøgger, 2018; Fenwick et al., 2014; Littoz-Monnet, 2017; Peters et al., 2009).

One approach to grasping "travelling reforms" and the use of references to foreign educational systems is, as already mentioned, the externalization thesis. According to Schriewer (1988), references to external systems (i.e., externalization) are mobilized to add meaning, weight, or legitimacy to domestic reforms. Renowned scholars have used the systematic theoretical concept of externalization to explain the selective borrowing of global education policies, the popularization of international best practices, and the transfer of reforms from one country to another (Schriewer & Martinez, 2004; Steiner-Khamsi, 2003, 2009, 2012, 2021; Waldow, 2012). Likewise, Nordic researchers have contributed to research on policy transfer in the feld of education policy and applied semantic analysis that draws on sociological system theory (Luhmann, 1990, 1997). Tey show how national public inquiry reports create their own projections of the future school as a contemporary way to manage coexisting expectations (Hansen et al., 2021) and how national curricula reconfgure the scientifc logics of comparative assessment designs that characterize transnational policies (Sivesind et al., 2016).

#### **Research on Evidence-Based Policy**

Tird, we discuss how evidence-based policy is anchored in particular forms of knowledge that legitimize school reforms, education policies, and practices. As a feld in political science, the politics of knowledge focuses on the legitimacy of knowledge forms and institutional logics that are shaping, for example, the evaluation–knowledge nexus (Segerholm et al., 2019). Simultaneously, theories of the policy process more generally focus on various features of evidence-based policies and practices. Together, these felds refect a rather weak interest in knowledge use (Daviter, 2015). Moreover, several studies have examined what can be used as evidence, usually as the foundation of knowledge-based policies and regulations. Kvernbekk (2011) recommended a deep look into the question of the nature of evidence and how it is viewed and conceptualized in academic scholarships; however, both previous and recent literature has acknowledged the use of multiple types of evidence in the policy process beyond what is considered validated by scientifc procedures (Boaz et al., 2019; Weiss, 1979).

Despite doubts about how policy synchronizes its projected solutions and scientifc evidence accordingly, a considerable number of knowledge providers have written books, articles, reports, and policy briefs to aid the development of evidence-based policy. Some of these knowledge providers have sought to single out "paths forward, toward bringing about more/better evidence use in education" (Malin et al., 2020, p. 11); as a result, evidence-based policy is regarded as a campaign-like adventure, or at least a master idea or a trend (Røvik, 2016). By pointing to various challenges that policy experts often experience, researchers have also raised doubts about the scientifc validity of the knowledge sources produced and used throughout policy processes (Holst & Molander, 2018). Although academic scholars are more frequently involved in public inquiries (Christensen & Hesstvedt, 2019), the institutional logics and contextual conditions that regulate policymaking processes are not necessarily resulting in scientifc practices as they normally conducted in universities or at research institutes. Policymaking processes follow their own institutional logics and can be seen as more pragmatic in terms of how expertise apply scientifc procedures and standards. Moreover, as Cairney (2016) emphasized, the environments of decision processes are characterized by complex systems where connections between actors and agencies are many and varied. Due to a lack of centralized control that follows from the way society develops, there is no guarantee that evidence will actually be used even when it is provided by experts (Cairney, 2016).

Tus, the defnition of *evidence* is broad and highly contested; consequently, researchers and policy analysts frequently discuss what constitutes "good" evidence. Moreover, evidence cannot be researched without insights into the contexts where it is used. In Chap. 2, Steiner-Khamsi presents a theoretical overview of the relevant defnitions and literature that enlighten the various features of evidence-based policy in the education sector. In particular, she outlines how the defnition of evidence (depending on the situation and the stage of the policy process in our case) relates to the kind of reform policies we have explored in the POLNET project. In the next sections of this chapter, we will provide a brief overview of the data and methods we draw on in the following national and comparative chapters and give a brief overview of the Chaps. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, and 13.

## **Signifcance of the Data and Methods**

To examine evidence-based policy, the POLNET study applies bibliometric and network analysis to trace the reception of knowledge sources and their formation into reference networks. A core purpose has been to explore diferences in reference patterns extracted for a collection of source documents that legitimized the most recent school reforms for compulsory education in fve Nordic countries. In this method section, we will provide an overview of the database we have searched, including the search strategy and the date we collected the data, the selection process, and the number of records retrieved in order to make our visualizations reproducible. Against this backdrop, each empirical chapter covers the analytical strategies we have pursued in more depth.

We consider both bibliometric and network analysis to be useful and versatile tools that provide insights into what Eyal (2019, p. 33) labeled "distributed cognitions of expertise" (p. 33); in our case, this refers to how expertise outside individuals is visualized through bibliometric reference patterns. In order to interpret and make sense of the patterns visualized in terms of quantitative measures and graphs, we have also interviewed policymakers and experts as well as synthesized and examined the content of core documents. Moreover, in all chapters, we draw on contextual information to analyze the reception and translation of knowledge sources within our source documents. We combine quantitative and qualitative approaches to provide a rich and comprehensive understanding of the cases we look into and to explore diferences in citation patterns.

Te origins of this book lie in the joint research project POLNET, which enabled six research teams to systematize references in policy documents to create a shared bibliometric network database. Based on a joint protocol developed by the research team at Teacher College, Columbia University, New York, each team selected a comparable set of policy documents and associated sources that were produced in conjunction with the last national school reforms in the fve Nordic countries. Tese reforms addressed broad themes, such as renewing the national curricula, redesigning assessment systems, and amending the legislative system.

An overall goal was to measure the same phenomenon, specifcally ofcial/state policy knowledge used to set reform agendas, develop new or modifed policy options, or issue reforms for basic education (Years 1–9/10), and to focus on the same variables, that is to enter relevant information on the references (e.g., year of publication, location of publication). We used sampling plans and a shared template for how to enter data to guarantee consistency. In the Nordic group meetings, we agreed that the comparative study would use a sampling strategy that approximates the one used for the Norwegian study (see Baek et al., 2018). In particular, the selected source documents should refect ofcial/state policy knowledge that the government had used in preparation for a particular school reform; in other words, these were white papers, green papers, or functional equivalents of these Norwegian documents. Once all the data from the fve research teams were entered, the Teacher College, Columbia University-based research team cross-checked the entered data in collaboration with the researchers. In addition, all the researchers participated in joint discussions during Nordic research meetings about challenges and solutions related to data entry, cleaning, and coding. In this way, we ensured that the data were comparable. For an overview of the source documents, see Chap. 3 (Table 3.2).

Te joint protocol outlined the sampling method (i.e., the selection of ofcial/state policy knowledge documents) and the data entry protocol (i.e., the identifcation of variables used in the databases). We regard the documents as refecting both the artifacts and discourses produced by policy actors, whereas the references in those documents refect the evidence base for these artifacts and discourses. Additionally, the references in policy documents show the linkages between policy documents; for example, if two policy documents draw on similar evidence, they share some kind of afnity. By applying network analysis, we analyzed how policymakers and experts recognized the same citations and references across thematic areas and contexts. Tis view helps us to interpret the various knowledge networks they build, being based on both proximity and distance, respectively. Furthermore, since our cases are located in the same geographical region of Europe, we explore to what extent school reforms in these countries draw on shared knowledge sources and, more generally, whether authorization of knowledge sources difers across contexts. Table 1.1 summarizes the reforms examined in the POLNET study, the number of source documents, and their references.

After extracting the bibliometric network data on the relationship between source documents and references from each source document, we entered a series of attributes for all documents in the database to allow for interpretation. Tese attributes are as follows: (a) year of publication;


**Table 1.1** Reform titles and data

*Note*: The number of references refers to the sum of every reference in all source documents. It counts the references that are cited by multiple sources separately. WP: White Papers. GP: Green Papers

(b) publisher or institutional afliation of the author/authoring organization; (c) location of publication (i.e., domestic, regional, and international); and (d) type of publication (i.e., report, book, journal article, government-published document, and other). We used the attribute information to comprehend the networks and reference utilization patterns within and across the countries.

In this project, we focused on one of the major network measures (i.e., in-degree centrality, which equals the total of incoming citations for a given document) to assess the importance of a reference in a bibliometric network. We have also used other measures, such as co-citations, to identify policy networks (i.e., authors within the ministries or public inquiry bodies that cite each other or that cite the same texts) and peripheral texts/discourses that bridge two or more networks. Te software programs UCINET 6.289 and NetDraw 2.097 were used for network analysis and visualization, respectively (Borgatti et al., 2002).

In addition to bibliometric network analysis, several chapters in this book include additional data. Text analysis is essential for interpreting the quality of the relations between referenced knowledge sources at levels beyond what can be recognized by individuals within the policy context and what can be perceived and potentially developed. Te content analysis of the white and green papers has enabled the research teams to trace and compare reception and translation processes—that is, to explore which concepts and statements of a national, regional, or international text resonate and how they are reframed or translated in a national setting.

Te semi-structured interviews we have added to complement the bibliometric network analysis helped us make sense of the reference patterns. During in-person and Zoom interviews, a group of 20 policymakers and experts involved in reform-making processes in 4 of the given countries shared their experiences and knowledge about reference practices in and across policy areas and realms. Tese interviews covered the same themes for the purpose of synthesizing and analyzing fndings based on the interview transcriptions. Examining their responses has allowed us to explore why knowledge is used diferently between the Nordic countries. We collected data according to the General Data Protection Regulation, set by the European Union, and followed ethical guidelines.

#### **1 Introduction: A Comparative Network Analysis of Knowledge…**

Troughout our POLNET study, we have applied bibliometric analysis, which have been primarily utilized in the feld of information science (Gingras, 2016). Policy analysts have used these instruments to evaluate education research and its impact, which is particularly known from research policy in the United Kingdom (Smith et al., 2020). However, few researchers have applied bibliometric analysis to conduct comparative and international research studies of policy knowledge in education reform projects and critically examined what these methods can be used for. Tis book contributes to the scholarship by applying bibliometric analysis to examine citation patterns in policy documents and theorize the policy process from a comparative perspective.

## **Chapter Overview**

In Chap. 2, Gita Steiner-Khamsi provides a theoretical framework for how to examine knowledge use to understand how evidence-based policy evolves and changes within contemporary practices. She argues that the unit of analysis for bibliometric analyses is the reference, which may be listed conveniently at the end of a document in a separate bibliographical section, in a footnote or endnote, or, more inconveniently, only vaguely alluded to in a text. In either case, the reference communicates or conveys something to the reader. Te question then becomes what exactly it communicates; in the context of the POLNET study, this raises the following question: How have we interpreted the reference in the larger corpus of the policy documents that we collected in Denmark, Iceland, Finland, Norway, and Sweden in order to understand the policy process?

For our bibliometric analysis of evidence-based policymaking in the Nordic region, Steiner-Khamsi found Paul Cairney's (2016, p. 3) defnition useful: "'evidence' is assertion backed by information" (p. 3). In concert with Cairney's defnition, Steiner-Khamsi argues that references are a construct or an aggregate of several pieces of information (e.g., authorship, year of publication, topic, or theme) that help position the author in a larger semantic space. According to Steiner-Khamsi, all these constitutive elements are essential in bibliometric analysis, as they are utilized as epistemological cues for understanding not only whose texts or whose knowledge the authors have selected to substantiate their points but also whose knowledge they cite as sources of expertise to reduce uncertainty or generate legitimacy about the validity of their own claims or assertions. Steiner-Khamsi recommends drawing on the systems-theoretical deliberations presented in Gil Eyal's (2019) book *Te Crisis of Expertise*, and what Baek (2020) labeled the "expertise-seeking arrangements" in the fve Nordic countries. Finally, Steiner-Khamsi invites further research that helps to refne the framework and the method of inquiry, including investigating the hierarchization of evidence as refected in the choice of references.

In Chap. 3, Oren Pizmony-Levy and Chanwoong Baek describe the methodological approach behind the project. Te authors discuss how the research design is theoretically and methodologically inspired by existing literature on social network analysis and sociology of knowledge. Tey demonstrate how bibliometric network analysis that is grounded in these two lines of research allows us to examine the architecture of policy knowledge in the fve Nordic countries. Te authors also detail the procedures for collecting, analyzing, and presenting data for this project and provide the rationale for drawing conclusions based on bibliometric analysis.

Chapters 4, 5, 6, 7, and 8 represent national case studies from the Nordic countries where bibliometric data are used to shed light on the policy processes behind recent school reforms. Tey all examine the use of references in key documents, such as white papers and green papers, and address how school reforms are being legitimated in certain ways through alignments of evidence produced on domestic, regional, and international levels. To contextualize the reform, several of the studies combine data.

In Chap. 4, Trine Juul Reder and Christian Ydesen analyze the evidence base and policy context of the Danish public school reform of 2013. In addition to the bibliometric data, the study draws on a contextual reading of policy documents and interviews with informants who played a key role in the policymaking process. Te fndings indicate that the reform was based on a quite limited number of written sources and left a great role to be played by more informal knowledge. Overall, the chapter categorizes the evidence base into academic knowledge, strategic knowledge, and knowledge produced by international organizations, such as the OECD. Furthermore, the authors also identify stakeholder knowledge and practice-based evidence as additional categories.

In Chap. 5, Saija Volmari, Jaakko Kauko, Juho Anturaniemi, and Íris Santos examine what kind of evidence Finland draws on in the policymaking process of the 2014 national core curriculum. Based on bibliometric and content analysis, the authors show that the most infuential international policy documents in terms of policy design came from the OECD; however, the references used were mostly domestic. Hence, while the global level has the power to produce evidence, the national or local level experts, who build their infuence through networks, have the power to select the evidence and adjust it to meet national needs. Te authors conclude that the type of evidence, merely consisting of empirical evaluation data, appeared to be more important than where the evidence originated from (local or global level). Tis means that a global policy space intervened with the national.

In Chap. 6, Magnúsdóttir and Jónasson explore the formation of policy documents that were issued after the school reform the national school authorities in Iceland organized during 2008–2013. Te main emphasis is on the frst document issued as a white paper (2014) by Icelandic state educational authorities, while the two other policy documents were framed by international organizations. In addition to bibliometric analysis, a content and a interview study with with fve experts, confrm a minimal use of academic references and unsystematic development of green papers. Apart from a shared focus on enhancing quality in education, there is a low interconnection between the documents in terms of content, bibliography, and semantics. Only one of these three source documents was written in Icelandic, and therefore the Icelandic case is exemplary in terms of externalization with robust references from the OECD. Te reform's short timeframe shaped by the aim of the Minister of Education was to react promptly to declining results in PISA. Te different formulations manifested by these documents reveal an eclectic approach to knowledge usage shaped by ministerial governance.

In Chap. 7, Bernadette Hörmann and Kirsten Sivesind present a bibliometric and semantic analysis of the use of knowledge sources referenced in two white and eight green papers about the recent school reform for primary and secondary education in Norway (2016/2020). Te authors identify the most often co-cited texts in the bibliographies and the most frequent in-text references in the two white papers. Based on Luhmann's (2000) distinction between conditional and purposive programs, the authors examine how policymakers use the prominent references, classifed into four types, as evidence to propose a set of policy options. Te four groups are: (a) formal documents that are not primarily a result of research but serve as one type of evidence, (b) meta-analysis, (c) confgurational reviews, and (d) empirical research studies. Te chapter concludes that meta-analysis and confgurational reviews support purposive arguments along with means-end reasoning. In addition, the authors fnd that formal documents, with some exceptions, serve a regulatory role in terms of their conditional orientation, while empirical research reports, originally developed for evaluative purposes, help policymakers to align systems of reasoning depending on the issues they address.

In Chap. 8, Andreas Nordin and Ninni Wahlström examine the selection and use of evidence in the most recent school reform in Sweden, the Knowledge Achievement Reform (2015/2018). Te analyses of citation frequency show that, although the OECD had an important role in initiating the Swedish reform, the highest percentage of references was domestic and mainly governmental references. Tis shows the possibility for national politics to uphold a high level of self-referentiality even when, to a large extent, international organizations such as the OECD suggest the national political agenda. Another distinct feature the authors emphasize is the low number of academic references.

Unlike the individual country studies in the previous chapters, Chaps. 9, 10, 11, and 12 address four comparative topics and bring together researchers from the diferent national teams. Chapter 9 compares the knowledge governments use to inform their policy decisions. Chanwoong Baek, Dijana Tiplic, and Íris Santos show that all fve Nordic countries actively utilized knowledge to support and legitimate their policy proposals. However, they did so in diferent ways and in diferent settings. Some Nordic countries sought evidence for policy proposals mainly through the policy advisory system within the bureaucracy, while others outsourced the production of policy advice. Furthermore, the analysis showed that reference utilization depends on the extent to which the policy system is self-referential or receptive to externalization. Te authors also highlight the diferences regarding whether the reform can be described as an incremental or a fundamental reform. A fundamental or controversial reform often utilizes more international references than an incremental or non-controversial reform.

In Chap. 10, Gita Steiner-Khamsi, Chanwoong Baek, Berit Karseth, and Andreas Nordin compare reference patterns between Norwegian and Swedish green papers (cross-national comparison), as well as between green papers and white papers (political translation). Te authors conclude that the advisory commissions have been repurposed in ways that place greater emphasis on expertise rather than accountability and representation. Second, the multi-level analysis shows that the commissions merely represent one stage in a long sequence of evidence-based policymaking. Finally, by following a transnational perspective, the chapter shows how advisory commissions today are used as bridges between the global and the national.

Chapter 11 draws attention to the OECD's role in the policy process in Nordic countries. Christian Ydesen, Jaakko Kauko, and Berglind Rós Magnúsdóttir investigate the extent to which the OECD and national institutions function as data-driven knowledge brokers in the shaping of education in the Nordic region. Te chapter ofers an in-depth analysis of Denmark, Finland, and Iceland to understand the feld of knowledge brokers in general and the role of the OECD in particular. Te chapter shows how policy fows via its illumination of the confguration and workings of the OECD-centered epistemic community, forming the modes of knowledge and governance woven into the fabric of Nordic education.

Chapter 12 considers how the regional level works as a reference in national education policy planning. By drawing on bibliometric analyses, content analysis, and interviews, Saija Volmari, Kirsten Sivesind, and Jón Torf Jónasson examine the actual role and infuence of Nordic knowledge and cooperation in recent school reforms. Te bibliometric analysis uncovers an extremely low number of regional references in the actual policy documents. However, drawing on the interview data, the authors conclude that both global policy spaces and policy places in local settings provide regional evidence of relevance for policymaking processes. Nordic interrelations and interactions are experienced, meaningful, and important; however, not made visible using specifc bibliometric sources but rather more indirectly by translating ideas about the "Nordic" or other Nordic countries.

In Chaps. 13 and 14, two renowned scholars, Kerstin Martens and Antoni Verger, contribute by commenting on the book's overall theme and proposing what they regard as signifcant problems that research on evidence-based policy should address in the future. Finally, in the concluding chapter, Berit Karseth and Kirsten Sivesind synthesize and interpret fndings from the Nordic POLNET study.

## **References**


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **2**

## **What Is in a Reference? Theoretically Understanding the Uses of Evidence in Education Policy**

**Gita Steiner-Khamsi**

Arguably, the main unit of analysis for all bibliometric analyses is the reference. It may be listed conveniently at the end of a document in a separate bibliographical section, in a footnote, in an endnote or, more inconveniently, only vaguely alluded to in the main text. In either case, the reference communicates or conveys "something" to the reader. One question arises: What exactly does it communicate, what is embedded in a reference, or, to place the question into the context of our study, how have we interpreted the reference in the larger corpus of policy documents that we collected in Denmark, Iceland, Finland, Norway, and Sweden in an attempt to understand the policy process?

References have several constitutive elements: an indication of authorship, year of publication, topic or theme, location of publisher, and type of publisher. Te authorship may be further diferentiated by gender,

G. Steiner-Khamsi (\*)

Teachers College, Columbia University, New York, NY, USA e-mail: gs174@tc.columbia.edu

nationality, institutional afliation, and, if several coauthors are involved, the network structure within the group of authors. All these constitutive elements are essential in a bibliometric analysis because they are utilized as epistemological cues for understanding not only whose texts or whose knowledge the authors have selected to substantiate their points, but also whose knowledge they cite as sources of expertise to reduce uncertainty or generate legitimacy about the validity of their own claims or assertions.

According to political scientist Paul Cairney (2015), "'[e]vidence' is assertion backed by information." In our bibliometric network analysis, we treat references as a construct or aggregate of several pieces of information (authorship, year of publication, topic or theme, etc.), helping position the author in a larger semantic space. Referencing sources in a policy document—whether it is a Green Paper (in the Nordic context: prepared by government-appointed expert panels) or a White Paper (issued by the government)—is fundamentally diferent from the references in, for example, an encyclopedia, where the expectation is that the researcher at least pretends to identify the universe of relevant research literature on the topic with which they can situate their own framework. Policy documents are much more evaluative and controversial in nature; therefore, they tend to openly take a stance for or against existing assertions. Unsurprisingly, a bibliometric analysis of policy documents will often surface clusters of like-minded authors/documents and set them apart from those holding viewpoints that are seen as diferent or distant.

## **References from the Perspective of Sociological Systems Theory**

In the Nordic POLNET (Policy Knowledge and Lesson Drawing in an Era of International Comparison) study, we interpret a reference functionally, that is, in its larger context of evidence-based policy planning. We ask the following: What does a reference stand for—or rather do—in a policy document? From the perspective of sociological systems theory, references are meant to reduce uncertainty for the reader. Tey do this by making transparent the positionality of the author in the larger, discursive policy space and by documenting the credibility of the sources used for the assertions the author is making. As Jenny Ozga (2019) has astutely pointed out, the discursive space in policy documents is by default a political space in which authors position themselves in terms of political orientation and alliances.

To reiterate, references help validate or provide legitimacy to the evidence that the author (e.g., the government-appointed expert commissions or the government) has presented in the document. Tus, if "evidence is assertion backed by information" (Cairney, 2015), then *a reference is validation of evidence*. Said diferently, references are used to provide authoritative status to the evidence presented in policy documents. Naturally, a host of questions surface with this particular conceptualization of references: Which texts are infuential, that is, referenced frequently or referenced by two or more diferent knowledge networks? Are international references, that is, texts published outside the Nordic region, more infuential than national or regional references or vice versa? Does the institutional afliation (government, academe, "institute sector," private think-tank, civil society organization) of the author matter? Finally, the cross-national dimension enables us to examine the varied legitimization or authorization strategies in the fve countries in depth and, by means of comparison, identify nation-specifc patterns in the policy process.

Eyal (2019) explains Luhmann's conceptualization of authority, validity, and legitimacy of expert knowledge, juxtaposing the systemstheoretical approach against Jürgen Habermas' work on the legitimacy crisis. In concert with Luhmann, Eyal uses "validation" in the sense of defensibility to "reassure people that if they would bother to check, they would fnd that the particular decision was rationally taken and justifed, so no need to bother!" (Eyal, 2019, p. 88). For Luhmann, the insistence on validation is ultimately a "functionally necessary deception" that saves time and prevents discord, here given that every fact may also be interpreted diferently.

Strikingly, the validity issue has become key at a time when information is openly and abundantly available and when expertise has become democratized, enabling users and other lay persons to participate in the production of evidence and render each and every piece of evidence contestable. In fact, the two prominent trends in "modern governance practices" are, according to Krick et al. (2019, p. 927), a trend toward scientization and a participatory turn. Te frst trend is discernible in the composition of government-appointed advisory committees (a growing number of researchers), citation patterns (reference to studies, technical reports, and academic publications), and epistemic language (reference to "evidence," "knowledge," "data," and "research"). Te second trend implies greater public engagement in agenda setting as a result of open access to information, calling into question the corporatist or representative model of democracy (see Rommetvedt, 2017; Rommetvedt et al., 2012). Other scholars (Stehr & Grundmann, 2011) have labeled the second trend as a pluralization of expertise. Tese two trends—scientization and pluralization of expertise—have triggered a third trend that has only recently been discussed (see Lubienski, 2019): a surplus of evidence. Taken together, all three trends account for the fact that references, that is, the sources of information used to validate the evidence, have gained authoritative status.

Arguably, referencing other texts as an instrument of validation of one's assertions has become an object of intense scrutiny, including in bibliometric network analyses. Te pressure to disclose the sources of information that were used to produce evidence is discernible in the everincreasing number of references listed in Green Papers. Te Green Papers of the Norwegian Ofcial Commissions (NOUs; *Norges ofentlige utredninger*) and the White Papers of the Ministry of Education and Research of Norway make for good cases for demonstrating the trend over time. Te relevant papers of the 1996 School Reform in Norway made only sparse use of references, many of which were either embedded in the text or listed as footnotes. Twenty years later, however, there were 246 references on average per relevant Green or White Paper for the 2020 Curriculum Renewal Reform (Baek et al., 2018).

Paradoxically, the proliferation of evidence-based policy planning has added fuel to the crisis of expertise (see Eyal, 2019). Not only has science become politicized and politics scientized, but science has also become demystifed in front of everyone's eyes:

[T]he very discourse on expertise increases uncertainty and threatens legitimacy because now the public is witness to controversies between scientists. (Eyal, 2019, p. 102)

In efect, the proliferation of evidence-based policy planning has brought to light that evidence is considered not more, and not less, than a subjective assertion backed by information. Te boom has generated a surplus of evidence to the extent that there is now the challenge of how to weed out evidence based on relevance and credibility criteria. Concretely, in the wake of complexity reduction, we are witnessing a hierarchization of information (very often with randomized controlled trials on the top and qualitative data on the bottom), rendering some types of evidence more relevant than others. At the same time, the disclosure of the source of information to make a case for the credibility of the evidence, that is, the reference, has become as important, if not more so, than the information itself. In fact, the legitimacy of the assertion rests in great part on the source of the information itself. For example, a reference here and there to OECD studies has become a sine qua non for policy analysts in Europe because the OECD is seen, in the Foucauldian sense, as the founder of discursiveness for a very special kind of policy knowledge that ranks at the top in the hierarchy of evidence, one that operates with numbers and draws on international comparisons to enforce a political program of accountability. Ydesen (2019) has convincingly documented the rise of the OECD as a global education governing complex that uses a range of policy instruments (PISA, Education at a Glance, country reports, etc.) to diagnose and monitor national developments and advance the global solutions of a particular kind for national reforms.

Tere are many reasons why OECD studies are attractive for government ofcials (see Martens & Jakobi, 2010; Niemann & Martens, 2018). Espeland (2015) and Gorur (2015) masterfully observe the advantages of numbers over complex narratives because one may attach one's own narratives to numbers. What is especially appealing to policy actors are OECD-type studies, that is, statistics, scores, ranking, and benchmarks based on international comparisons or on comparisons over time. Novoa and Yariv-Mashal dissect the politics of international comparison and examine how:

[T]his ongoing collection, production and publication of surveys leads to an 'instant democracy', a regime of urgency that provokes a permanent need for self-justifcation. (2003, p. 427)

Espeland (2015, p. 56) explains the dual process of *simplifcation* and *elaboration* involved in using numbers. In the frst step, numbers "erase narratives" by systematically removing the persons, institutions, or systems being evaluated by the indicator and the researcher doing the evaluation. Tis technology of simplifcation stimulates narratives, or as Espeland astutely observes:

If the main job of indicators is to classify, reduce, simplify, and make visible certain kinds of knowledge, indicators are also generative in ways we sometimes ignore: the evoke narratives, stories about what the indicators mean, what their virtues or limitations are, who should use them to what efect, their promises, and their failings. (2015, p. 65)

Scholars in comparative policy studies have started to explore why PISA and other international large-scale student assessments are so attractive to policy actors and politicians (Addey et al., 2017; Pizmony-Levy, 2018). A few studies focused on the "narrative evoking" phase (Espeland, 2015, p. 65) of such studies have dissected what national governments interpret or project onto OECD reports or other international comparative studies based on their own policy context and agenda (Waldow & Steiner-Khamsi, 2019).

## **Reference Societies in Comparative Education Research**

In addition to the governance-by-numbers argument presented above, for many countries, the OECD represents an attractive geo-political space inhabited by people in 36 high-income economies. Terefore, a recourse to OECD publications may be seen both as an afrmation of the afliation and an acknowledgment of the OECD as a "reference society" (Bendix, 1978, p. 292) or rather "[transnational] reference space" toward which national governments orient themselves or aspire to belong. In fact, in *all* fve Nordic countries of the POLNET study, OECD publications represent the most cited international texts (see Chap. 11). Tis may come as a surprise for a non-Nordic audience because one would expect competition between two dominant policy discourses in the fve countries of the POLNET study: the Nordic reference space, which traditionally has had a strong commitment to equity, and the OECD reference space, which has a mission to advance economic growth.

Tus, in comparative policy studies, the term "reference" also carries a spatial, geo-political, or epistemological connotation. It is used in connection with "reference society" or "reference space." Te reference as a validation instrument and the reference as a point of epistemological orientation both share a common feature: they position the author (in the case of references) or the state (in the case of reference society) in its larger discursive space.

By now, there is a well-established tradition in comparative policy studies to draw on references as an analytical tool to situate the positionality of an actor (author, institution, government) in a broader transnational, geo-political space. Tis body of scholarship is closely associated with studies on the "reference society" presented by sociologist Reinhard Bendix (1978, p. 292). Bendix uses the term to denote how governments used economic competitors and military rivals as reference societies for their own development. One of the examples discussed by Bendix is the fascination of Meiji-era Japan with the West.

In comparative education, the term was—according to Waldow (2019)—frst introduced by Butts (1973), associate dean and professor of Teachers College, Columbia University. Butts observes that the governments of developing countries frequently used a specifc educational system in the Global North as a model for emulation. Tat country's path to "modernization" served government ofcials in the Global South as a reference for educational reforms in their country. It is important to bear in mind here that during Butts' time, transnational networks and dependencies established during colonial times had endured into the present and determined in great part the choice of reference societies. Another noted historian and comparativist, David Phillips, frst coined the term "cross-national policy attraction" to denote the keen interest of nineteenth-century British government ofcials in the educational reforms of Germany (see Ochs & Phillips, 2002). Both Butts and Phillips use records of study visits and government reports as sources for their analyses of cross-national attraction or policy borrowing.

Similar in the conceptual framework but diferent in terms of unit analysis, Schriewer and Martinez (2004) use bibliometric data to examine the use of reference societies in "educational knowledge," as refected in the publications of educational research journals; they wonder whether educational researchers in their sample of three countries draw on similar or diferent bodies of knowledge or texts. Tey purposefully use a time period of 70 years to see whether a convergence toward a single international canon of scientifc educational knowledge, here interpreted as internationalization or globalization, has occurred. Concretely, they examine the references listed in fagship educational research journals in three countries (Spain, Russia/Soviet Union, PR China) and code them in terms of the national origin of the referenced authors. Rather than detecting a pattern of steady internationalization toward a single body of internationally acclaimed authors, they notice considerable fuctuation regarding the space allocated to international scholarship, as measured in the number and type of foreign bibliographical references made in the journal articles of the three countries. Tey fnd that the "socio-logic" (particularly political developments in a given country) was a better predictor of receptiveness toward international scholarship than an external logic as manifested in the ever-expanding transnational network of educational researchers.

In fact, the era of the greatest convergence regarding educational knowledge was in the 1920s and 1930s, when educational researchers in Spain, the Soviet Union, and China were drawn to the work of John Dewey. Once that brief period was over, Dewey was dropped from the reference list in Soviet educational journals and replaced by Nadezhda Krupskaya (Lenin's wife). It is striking that against all expectations of globalization or international convergence theorists, educational knowledge in these three countries did not become more internationalized until after the mid-1980s, when all three opened their ideological boundaries and increased international cooperation. Even though Schriewer and Martinez's (2004) justifcation for their case selection leans on a problematic notion of culture and "civilization," the design and methodology of the study is compelling and well-suited for analyzing international convergence/divergence processes in educational research.

Te link between the reference society and political change has also been well documented in comparative education research. Examples include two cases of a radical *change* in reference societies as a result of fundamental political changes in post-Soviet Latvia and post-socialist Mongolia, respectively. Silova (2006) examines the erasure of Soviet references and their subsequent replacement with Western European references. She interprets the shift from the Soviet to the Western European reference system as a marker for the new geo-political educational space that Latvia politically and economically had been aspiring to inhabit at the turn of the millennium. What is fascinating about this particular change of political allies is that it has merely afected the discursive level, not the practice of separate schooling. Te separation of school systems, one for Latvian speakers and another for Russian and other ethnic speakers, continues to exist; however, segregated schools are no longer seen as "sites of occupation" but are now being reframed as "symbols of multiculturalism." Te list of comparative policy studies on reference societies is too long to present in an exhaustive manner. In our own study on Mongolia (Steiner-Khamsi & Stolpe, 2006), we observe the discursive ruptures and reorientation in terms of reference spaces that accompanied the political changes, notably the replacement of the Communist Council for Mutual Economic Assistance with the Asian Development, World Bank, and other post-communist international aid agencies.

Strikingly, studies on reference societies and cross-national policy attraction have experienced a revitalization of a special sort in recent years with the fast advance of international large-scale student assessments (ILSAs) used in many countries as a policy tool for governance by numbers (see Carvalho & Costa, 2015; Volante, 2018; Waldow & Steiner-Khamsi, 2019). Preoccupation with what league leaders (Finland, Shanghai, Singapore, etc.) have "done right" has generated new momentum for policy borrowing research. Precisely at a stage in policy borrowing research when scholars have put the study of cross-national policy attraction to rest and instead directed their attention at the ubiquitous difusion processes of global education policies in the form of "best practices" or "international standards" vaguely defned, the cross-national dimension—and by implication the focus on the nation-state and its national policy actors—has regained importance in ILSA policy research.

In the case of PISA (Programme of International Student Assessment), the preoccupation of national policy actors is, at least rhetorically, on how their own system scores compared with others and what there is to "learn" from the league winners, league-slippers, and league-losers, in terms of PISA's twenty-frst-century skills. Because policy actors often attribute "best practices" to particular national educational systems, the national level regained importance as a unit of analysis. Terefore, ILSA policy researchers found themselves in a position of having to bring back the focus on national systems, a unit of analysis criticized as "methodological nationalism," which, if used naively, is a cause for concern because of its homogenizing efects (see Giddens, 1995; Wimmer & Glick Schiller, 2003; Robertson & Dale, 2008).

Research on reference societies has also been refned over the past few years in other ways. For example, intrigued by negative media accounts in Germany about the PISA league leader Shanghai (during the 2012 PISA round), Waldow scrutinizes the policy usage of "negative reference societies" (2016) or "counter-reference societies," respectively. Te concept of a reference or counter-reference society is based on commensurability. How do national policy actors *make* the educational systems of league winners *appear* to be comparable to their own educational system in a way that can suggest that lessons could be drawn? Vice versa, how do they manage to make two educational systems incommensurable and incomparable to *avoid* lesson-drawing? Te disbelief or the downplaying, respectively, of Chinese success in ILSAs, notably in the PISA rounds of 2012 and 2018, is comparable to earlier stereotypical accounts of Japanese or pan-Asian education.

Similar to the US media accounts of *A Nation at Risk* (1983) in which American policy analyses attempted but ultimately failed to persuade Americans of the great benefts of the German and Japanese educational systems, the education systems of Beijing, Shanghai, Jiangsu, and Zhejian are, despite "PISA success," hardly used as models for emulation in Western countries. As with the *A Nation at Risk* report, the common reaction to Chinese success refects a "yes, but …" attitude (see Cummings, 1989, p. 296): even though there is a general agreement about the outstanding student performance in ILSAs in Hong Kong, Japan, Korea, Macao, Singapore, and select cities of PR China, there are too many negative stereotypes associated with education in these locations to assign them reference or emulation status. In fact, the exaggerated statements or myths about "Asian education" include images of overly ambitious mothers ("tiger mothers"), excessive use of cram schools, competition and suicide among students, elitist higher education, and social inequality. More often than not, the educational systems in Asia are politically instrumentalized as a counter-reference, that is, examples of how educational systems should *not* be developed.

Let us now circle back to the fve-country study at hand. In the Nordic region, there is an elephant in the room in the broader ILSA space, making one wonder the following: Do the other countries of the region (Denmark, Iceland, Norway, and Sweden) consider the educational system of Finland (a PISA league leader) as a reference or a counter-reference for educational reform in their own system, or are they indiferent toward lesson-drawing from Finland? Te coding of the references by their country of publication and the qualitative analysis of thematic cross references make it possible to examine the fascinating question of a reference society *within* the Nordic education space (see Sivesind, 2019; Chap. 12).

Clearly, the bibliometric analyses presented in this volume demonstrate that OECD publications eclipse studies from Finland. Two possible yet inconclusive interpretations lend themselves to further investigation: either "Finnish success" is acknowledged but rendered irrelevant for one's own national context (the "yes, but …" attitude explained earlier), or Finnish success is, for a variety of reasons, including linguistic ones, referenced via an authoritative source of information: OECD publications.

## **Expertise-Seeking Arrangements in Policy Making**

In his dissertation research, Baek (2020) coins the term "expertise-seeking arrangements," which captures very well the dilemma of governments in an era of evidence-based policy making: Where and how do they seek advice for policy analysis, evaluation, and formulation? Given that "the authority of experts is destabilized" (Eyal, 2019, p. 102) in an era in which scientifc evidence production is easily demystifed and an everincreasing number of individuals, including concerned citizens and other laypersons, lay claim to expertise, the question of governments' expertiseseeking arrangements is taking center stage.

Eyal presents a typology of responses to the legitimation crisis, which is reproduced in Table 2.1 below. His focus is on "regulatory science" or the "interface between scientifc research, law and policy" (see Eyal, 2019, p. 7f.).

Te frst strategy of the state is to pretend that science is purifed from politics by pursuing "mechanical *objectivity*" (Eyal, 2019, p. 115), as refected in references to scores, rankings, numbers, impact evaluations, and quantifable comparisons. In our case, references to OECD studies, evaluations, and ILSAs belong to this category.

Te second strategy of *inclusion* is to acknowledge that science is politicized—or as Latour has eloquently put it, "Science is not politics. It is politics by other means" (1984, p. 229)—and, therefore, includes


**Table 2.1** Typology of responses to the legitimation crisis

*Source*: Eyal (2019, p. 103)

laypersons, interest groups, and other engaged citizens into the advisory bodies of the government.

Te third strategy of *exclusion* is to decouple science from politics and generate "gate-keeping mechanisms designed to maintain an artifcial scarcity of expertise" (Eyal, 2019, p. 105; see also Weingart, 2003); these are typically academic associations (academy of sciences) or professional associations that claim exclusive expertise and advise the government.

Te fourth and fnal strategy of *outsourcing* represents another functional diferentiation process. It decouples science from politics to the extent that it delegates research *and* policy formulation to outside groups, think-tanks, or semiprivate entities. Eyal contends that the fourth strategy is oftentimes a reaction to failed attempts of the state to generate trust or being inclusive, which would be pursued in the frst three strategies. In particular, the fourth strategy is often a response to the critique that the second strategy—the appointment of ad hoc advisory commissions or government-appointed expert commissions—are merely meant for window-dressing and rarely impact the ultimate policy decisions and formulations prepared by government ofcials.

Another useful typology of "advisory system activity" has been developed by Craft and Howlett (2013, p. 193f.). In general, they fnd a trend toward the inclusion of nonstate actors (think-tanks, open data citizen engagement driven policy initiatives/web 2.0, etc.) and international actors (e.g., OECD, ILO, UN organizations) as policy advisors. Tis is in stark contrast to traditional advisory systems, which mainly have drawn on national advisory bodies, including statistical ofces, strategic policy units within the government, or government-appointed ad hoc commissions.

Naturally, the changing nature of the relationship between politics and science has preoccupied comparative policy studies for a while. One of the early, more important comparative studies exploring the interpenetration of the two function systems is the research project "Te role of knowledge in the construction and regulation of health and education policy in Europe: convergences and specifcities among nations and sectors," abbreviated as Know&Pol and funded in the Sixth Framework Programme of the European Commission (see, e.g., Fenwick et al., 2014). Knowledge-based regulation, which is analyzed in the Know&Pol research project, has fundamentally changed the role of the state from one that runs schools to one that establishes learning standards and monitors learning outcomes, thereby enabling a multitude of providers, including businesses, to enter the school market. Over the past 20 years or so, the private sector has become not only a major provider of education, but also a key policy actor, lobbying for reforms that further restrict the role of the state in the education sector (Verger et al., 2017).

As mentioned above, knowledge-based regulation has also enlarged the radius of individuals contributing to policy-relevant educational knowledge. An early indication of changes in knowledge production and sharing is the open-access policies that both governments and research councils have put in place recently. System theorists Peter Weingart and Justus Lentsch (2008) consider such open-access policies to be part and parcel of a democratization of expertise; here, the relationship between science and politics has experienced three distinct shifts over the past 70 years (2008, p. 207 f.). During the early period of scientifc policy advice (1950s–1970s), the ad hoc expert commissions insisted on being autonomous and independent from governments. As a corollary, their reports amassed foundational scientifc knowledge that policy actors could or could not use, respectively. In a second phase, the commissions became increasingly politicized (1970s to 1990s) because they were charged with the task of producing policy-relevant scientifc knowledge. In the current third phase, the governments in many countries have experienced a shift from "knowledge-based legitimacy" to "participation-based legitimacy." Tis also applies to government-appointed ad hoc expert commissions. Governments are under pressure to "democratize" scientifc policy advice by (i) providing open access to reviews and expertise, (ii) expanding the defnition of "experts" (including nowadays, both producers and consumers), and (iii) insisting that the knowledge products are useful, that is, provide a clear foundation for stop/go policy decisions.

In the fve participating countries of the Nordic region, there is a wide array of expertise-seeking arrangements that these countries' governments have put in place (see Chap. 10). Te Eurydice Report (2017), *Support Mechanisms for Evidence-Based Policy-Making in Education*, is incomplete (data on Iceland is missing) and too imprecise to provide any useful clues for a categorization of expertise-seeking arrangements. For example, the government-appointed expert commissions in Norway (NOUs) and Sweden (SOUs) that amass evidence to substantiate their evaluation of past reforms and their recommendations for new directions are not mentioned. As documented in the OECD study on policy advisory systems (OECD, 2017), in all fve countries, there is a commitment to evidencebased policy planning (which in some countries is inscribed in law), an extensive stakeholder review, or a "hearing" process in which draft versions of new policy are opened up for public consultation.

Te type of expertise-seeking arrangement in each of the fve countries needs to be kept in mind when interpreting the role of experts in producing evidence for policy making. It may be useful to draw on an existing typology of such arrangements. For example, Weingart and Lentsch identify six types of commissions that provide scientifc advice for policy making (2008, Chap. 2): (i) policy-domain-specifc advisory councils, (ii) expert commissions for risk management, (iii) policy-specifc expert commissions, (iv) ad hoc commissions, (v) enquete commissions, and (vi) sector research.

Te typology may be used to categorize the fve types of expertiseseeking arrangements in the countries of the Nordic region. According to the typology of Weingart and Lentsch (2008), the government-appointed "ofcial commissions" in Norway and Sweden (NOUs and SOUs, respectively), which prepare and help legitimize policy decisions, fall into the categorization of "ad hoc commissions." In Denmark, the School Council, which was established in 2006, serves to advise the ministry on topics related to elementary school (see Chap. 4). In Finland, the expertiseseeking arrangement is multisited or hybridized, according to Holli and Turkka (2021). Te government-appointed ad hoc commissions, which exist in Norway and Sweden, were abolished in 2003 and replaced with broad-based working groups. Te representation of academics declined over time and constituted only 4.7% of all working group members in 2015. At the same time, the Government of Finland pluralized the policy advisory system:

[T]he policy advisory system of Finland shows signs of hybridisation, as the channels and organization of policy advice have pluralised and advice has taken new forms. (Holli & Turkka, 2021, p. 58 [translation by the authors])

Te partial externalization of policy advice is manifested in the rise of "state investigators" or consultants who are hired to produce governmentcommissioned reports to a general outsourcing of policy research. According to the authors, Finland has created a "research market," where the state buys the research it needs for policy preparation. In Iceland, fnally, the composition of advisory bodies is strictly regulated in terms of gender and political parties to ensure an inclusive consultative process. According to the OECD survey on policy advisory systems, which is carried out in 17 countries, including the 5 Nordic countries studied here, the policy advisory system in Iceland requires that at least 40 of the members of ad hoc advisory commissions are female and that all political parties are represented (OECD, 2017).

Te OECD has formulated fve quality standards for policy advisory systems: adaptability, transparency, autonomy, inclusiveness, and efectiveness. Te OECD 17-country study (OECD, 2017) presents a positive assessment of the ad hoc advisory committees found in the Nordic region:

Ad hoc advisory bodies […] are often used by governments to gather evidence-based answers to particular questions relatively quickly. Tey often serve as a "fast track" and specialized option for governments to obtain advice. Te Nordic countries have well-established traditions of creating ad hoc bodies to enhance the adaptability of the system. (OECD, 2017, p. 17)

## **Implications for the Five-Country Bibliometric Network Analyses**

In the parallel universe of "gray literature" or technical reports, which are often commissioned by international organizations, there is an interesting discussion unfolding on the rapid spread of "global public goods" (GPGs) or global knowledge banks. GPGs include, for example, openly accessible international toolkits, documents, studies and databanks, training modules, good practices, and global monitoring reports. Clearly, the GPGs are openly accessible information, nowadays often backed up with numbers that any local, national, or international organization may use to back up their production of evidence. Te rapid spread of GPGs was addressed by a few scholars around the turn of the millennium (notably Stone, 2000), but curiously, the discussion has not yet gained traction in academic debates. Hence, a brief summary of the debates, carried out in the context of development studies, may be in order here.

Within development studies, the discussion is now bifurcating in at least two diferent directions. One group of authors makes the argument for more funding for GPGs produced by a more diverse body of researchers (based in the Global North *and* the Global South) and another group critically examines the uptake of GPGs for national policy making at the national level (see Vasquez Cuevas, 2020).

In some countries, a wide range of propositions have been made about how to remedy the shortfalls related to global agenda setting, channeling of aid, and GPGs (see Schäferhof & Burnett, 2016). Some suggestions entail more funding at the global level, whereas others notice that the production of GPGs is mostly done by a few global actors (OECD, the World Bank, the UN system) at the expense of national research institutions outside of North America, Europe, and Australia. Tis applies in particular to think-tanks, research institutions, and universities in the Global South, whose knowledge products are rarely taken up at the global level. Examples of more funding to the Global South include Oxfam's early suggestion to eliminate one-size-fts-all benchmarking processes and dedicate three grants for capacity building to recipient governments and civil society organizations of low- and lower-middle-income countries (Oxfam, 2010).

For a while, the question arose whether the World Bank, UNESCO institutions, UNICEF, Global Partnership for Education, or other international organizations should earmark funds for research capacity building and policy analysis. One of the early suggestions was to increase funding for the global and regional agencies of UNESCO and UNICEF to advance cross-country sharing of knowledge on education and development. In addition to statistics, the UN organizations would use the funds to disseminate knowledge derived from research and from global sharing of experience. Others found the World Bank to be ideally suited for helping expand research funding and activities given its commitment to evidence-based policy making; they recommended that researchers at the World Bank would work more closely with other staf for countrylevel policy advice (Clemens & Kremer, 2016). Unsurprisingly, the lively debate in development studies is on whose knowledge is made publicly available at the global level and whose knowledge is confned to the national boundaries of its producers.

In OECD countries, the debate over the asymmetry of global knowledge production and update seems to center more on the language of publication (English vs. all other languages) rather than on the center/ periphery diferentiation in an unequal world system. Some countries in the Nordic region (Sweden and Norway in particular) require that all public documents be made openly accessible. Te transparency standard of policy advisory systems, which are forcefully promoted by the OECD (2015, 2017), is also practiced in the other three Nordic countries.

Te open-access policies and practices have both enabled and exacerbated the "participatory turn" or the "pluralization of expertise" (see Krick et al., 2019), which has been explained above. Te decline in corporatism or interest group representation in policy advisory systems is but one of the manifestations of this trend. Another manifestation is the surplus of evidence.

In the US context, Lubienski (2019) contends that there is not a scarcity but rather a "surplus of evidence." In such a "marketplace of ideas," there is ample opportunity for new, nonstate actors—specifcally the private sector—to serve as intermediaries between research production and policy making:

Into the chasm between research production and policy-making, we are seeing the entrance of new actors—networks of intermediaries—that seek to collect, interpret, package, and promote evidence for policymakers to use in forming their decisions. (Lubienski, 2019, p. 70)

Indeed, two decades after neoliberal calls for less politics and more scientifc rationality in the policy process, we are now entering a new phase in policy making: the stage of surplus of evidence in which calls for "actionable research," "policy-relevant research," or "what works" studies are being heard. At the 2015 Public Governance Ministerial Meeting in Helsinki, the ministers from OECD countries agreed (OECD, 2015, p. 3) that evidence alone is not sufcient. Evidence needs to be policy relevant, robust, and comparable:

We acknowledge the importance of evidence as a critical underpinning of public policies and recognize the need for a continuous efort to develop policy-relevant evidence on government performance that is robust and comparable. "What Works" initiatives are an example of how to ensure systematic assessment and leverage the stock of information on good practices available at the international level on policy impact. (OECD, 2015, p. 3)

In the Nordic policy context, the OECD plays a major role as a transnational policy advisor and standard setter. During the frst phase of evidence-based policy planning, the OECD advanced the notion of autonomous policy advisor systems that produce, independent of the state, evidence. In today's stage of evidence overproduction, the OECD ofers itself as an interpreter of evidence by selecting from the marketplace of ideas those that are actionable and in line with the broader political program of accountability.

Te proliferation of GPGs and the overproduction of evidence make it pressing to investigate the changing role of government-appointed advisory commissions in the production and interpretation of evidence. A host of research questions open up once we acknowledge that the presentation of "facts" (information) and transformation of these facts into evidence rests on a subjective selection process that refects the frame of reference or broader discursive orientation of the author.

Based on the elaborations presented above, we have several research questions that lend themselves to a systems-theoretical preoccupation with evidence-based policy planning: First, what sources of information do government-appointed advisory panels consider credible and, therefore, select to reduce uncertainty and generate trust? Second, what kind of hierarchization of evidence do government-appointed advisory panels generate to reduce complexity? Tird, bearing in mind the three most common types of externalization, what type of externalization do the cited texts represent: reference to (i) tradition or values (e.g., Nordic value of equity), (ii) organization (e.g., reference to laws and regulations), or (iii) scientifc rationality (e.g., studies and evaluations). Fourth, in which broader epistemic community, or rather political reference space, do the authors situate themselves? Finally, given the academization of government-appointed advisory commissions in some countries (Christensen & Hesstvedt, 2018), as a result of which the number of academics serving as members increased at the expense of interest group representatives, the phenomenon of structural coupling between science and politics (Steiner-Khamsi et al., 2019; Weingart, 2003; Stehr & Grundmann, 2011; see also Chap. 10) ofers itself as an object of empirical scrutiny.

Tis chapter has attempted to demonstrate how a bibliometric network analysis may be used as a method of inquiry for understanding legitimization processes in evidence-based policy making. Drawing our attention to how uncertainty is reduced and trust in evidence is created is essential in an era in which there is a surplus and, by implication, competing notions of evidence. A bibliometric investigation of the references in a text provides important clues about the (i) selection of sources of information, (ii) hierarchization of evidence, (iii) and type of externalization made by an author to leverage authority for the claims made in the text. A network analysis of the references further helps complicate the fndings. In fact, the focus on relations brings to light that diferent authors use the same references for diferent purposes, that is, one and the same reference may show up in, and bridge, two diferent knowledge networks. Perhaps needless to reiterate, the long list of fascinating research questions, which are presented in a nonexhaustive manner in this chapter, gains additional attraction when investigated across the fve diferent national contexts within the Nordic region.

## **References**


Eyal, G. (2019). *Te crisis of expertise*. Polity Press.


Verger, A., Steiner-Khamsi, G., & Lubienski, C. (2017). Te emerging global education industry: Analyzing market-making in education through market sociology. *Globalisation, Societies and Education, 15*(3), 325–340.

Volante, L. (2018). *Te PISA efect on global educational governance*. Routledge.


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **3**

## **Exploring the Architecture of Policy Knowledge: A Methodological Note**

**Oren Pizmony-Levy and Chanwoong Baek**

Te intention of this research project has been to examine how policymakers mobilize evidence to advance educational reforms. Specifcally, we sought to inspect how policymakers in fve countries—Denmark, Finland, Iceland, Norway, and Sweden—link diferent types of evidence to policy, and to explore diferences and commonalities between these countries. Tis set of objectives required a careful look into the *architecture of policy knowledge*, which includes the visible links between policy documents and other knowledge artifacts, such as articles, chapters, books, reports, and statistical analyses. In what follows, we describe the method behind this research project and the procedures we employed in each of the chapters. We detail important decisions about our methodological approach for analyzing and presenting data from policy documents.

C. Baek University of Oslo, Oslo, Norway e-mail: chanwoong.baek@iped.uio.no

O. Pizmony-Levy (\*)

Teachers College, Columbia University, New York, NY, USA e-mail: op2183@tc.columbia.edu

## **Theoretical and Methodological Inspirations**

Two lines of research inspired the research design: (a) social network analysis and (b) sociology of knowledge. In this section, we discuss the premise of each literature and the main concepts. We posit that a synthesis of these two literatures could ofer a new way to examine how policymakers mobilize evidence to advance educational reforms.

Te frst is social network analysis (SNA), which is a broad research paradigm that includes theory, substance, and methodology. Te basic defnition of a social network is "a fnite set or sets of actors and the relation or relations defned on them" (Wasserman & Faust, 1994, p. 20). In other words, a network is a set of socially relevant nodes that are *possibly* connected by one or more relations (Marin & Wellman, 2011). While some actors are connected, other actors might be disconcerted or isolated from each other. Te key premise of SNA is that relationships between actors infuence outcomes (e.g., attitudes and behaviors) beyond the actor's characteristics alone (Valente, 2010). Further, relationships between actors determine in part what happens to a group of actors as a whole. Terefore, social network scholars examine the structure that emerges from these social patterns with the objective of understanding the ways in which this structure contributes to specifc outcomes. Because actors and relationships are fundamental components in SNA, we now turn to defning these concepts.

*Actors* (or nodes) are discrete units or groups. Examples of actors are students in a classroom, schools within an educational system, nongovernmental organizations (NGOs) in a given feld, or nation-states in the world system. Most social network studies focus on collections of actors that are all the same type (e.g., schools in the same education system). However, some studies include actors from diferent sets (e.g., schools and NGOs that support them). Te former is often called a onemode network, whereas the latter is often called a two-mode network. Actors have attributes—or characteristics—that describe and distinguish them. For example, students could be described by their sex/gender, age, grade, and socio-economic background; schools could be described by their location (rural/urban), selectivity, and extent of important resources.

#### **3 Exploring the Architecture of Policy Knowledge…**

*Relationships* (or edges) are the ties that connect actors. Te defning feature of a relationship is that it establishes a linkage between two actors. Relationships among actors can be of diferent kinds, and each type facilitates a corresponding network. Borgatti and Ofem (2010) ofer a useful typology of relations studied in SNA that divides relations into fve types: (a) similarities include spatial and temporal proximities, co-membership in groups or events, and sharing an attribute; (b) social relations are ties such as kinship and friendship; (c) mental relations are perceptions of and attitudes toward others; (d) interactions are discrete events that can be tallied over a period; and (e) fows are interactions that are transmitted.

SNA posits that an actor's position in each network shapes the opportunities and constraints that the actor will encounter. Tis perspective is distinctive from traditional social science, which focuses on the characteristics of actors as predictors of diferent outcomes. In traditional social science, we might explain diferences in the performance of individuals or groups by certain qualities or characteristics. In contrast, SNA considers the web of relationships in which individuals or groups are embedded.

In this research project, actors/nodes are documents produced and used in a given policy space.1 Specifcally, we operationalize the architecture of policy knowledge as a two-mode network that consists of source documents and reference documents. Source documents include a set of white papers (WPs) and green papers (GPs); reference documents include other artifacts referenced in the source documents. We posit that policy documents lend themselves as a strategic site to examine how policymakers draw on evidence and justify their political decisions. We describe each document with the following attributes: year of publication, type of publication (e.g., articles, chapters, books, reports, and statistical analyses), author (either individual or institutional), publisher, and place of origin (e.g., domestic, regional, international). We focus on relationships/ edges between policy documents and other knowledge artifacts. Specifcally, we examine citations as particularly important connections between policy documents and evidence. We assume that authors of policy documents mobilize evidence—and thus cite knowledge artifacts—to persuade audiences of the legitimacy of a policy statement. In other words, references/citations are interactions through which authority fows from policy documents to evidence.

Te second line of research that informs this research project is the sociology of knowledge, which explores the production of knowledge (any knowledge) as a social activity. For example, recent work in the sociology of knowledge demonstrates a direct link between scientifc collaboration networks and the structure of ideas. Moody (2004) analyzes patterns of co-authorship in all English journal articles listed in sociological abstracts that were published between 1963 and 1999. Te results show that research specialty and methodology shape participation in the sociology collaboration network. Pizmony-Levy (2016) analyzed joint membership patterns in the Comparative and International Education Society (CIES) Special Interest Groups (SIGs), arguing that membership in these groups indicates a commitment to the felds of interest. Te research suggests that thematic SIGs (e.g., Globalization and Education and Higher Education) are more central than regional SIGs.

Specifcally, we draw on bibliometric analysis, which identifes prominent authors, documents, and journals within a scientifc community (Börner et al., 2003). Basic bibliometric analysis uses descriptive statistics to document trends in topics and research approaches used by scholars. More advanced bibliometric analysis uses network analysis to provide a deeper and more comprehensive view of relational and structural features of a given corpus of knowledge. For example, Menashy and Read (2016) examined the references in World Bank publications to identify the disciplinary foundation and the geographic representation of Bank knowledge on the theme of private sector engagement in education. Verger et al. (2019) looked at the references in the education privatization literature to explore the bibliographic coupling of academic and international agency's body of knowledge on education privatization.

Citations—also known as bibliographic references—are the building blocks of any bibliometric analysis. Martyn (1975) argues that bibliographic references "expressly state a connection between two documents, one which cites and the other which is cited" (p. 290). Merton (1973) asserts that citations are designed to "prove the historical lineage of knowledge and to guide readers of new work to sources they may want to check or draw upon themselves" (p. VI). Indeed, scientifc tradition requires that scientists, when reporting their own research, refer to earlier works that relate to their research (Nicolaisen, 2007). Each bibliographic reference is an inscription (Latour & Woolgar, 1986, pp. 45–53) describing a certain text by a standardized code that includes author name, title, journal name, publisher, year of publication, and page numbers. Te impact of a publication is often gauged by the number of times it has been cited by other authors.

In this research project, we combine SNA and bibliometric analysis to examine the architecture of policy knowledge in fve countries. We investigate the extent to which policy documents cite reference documents included in our database. Citation networks, such as the one we study, are more of a sociocultural network in that authors of policy documents may cite other authors they have never met or could not possibly have met (White, 2011). Whereas bibliometric analysis of scientifc papers often uses existing databases of scientifc and scholarly research (e.g., Web of Science), there are no similar databases for policy documents. Terefore, in the next section, we describe the data and methods we adopt for this research project.

## **Constructing Networks**

Researchers have been creative in obtaining data on social networks from diverse sources. In addition to using surveys and questionnaires, scholars have used archival sources and other documents extensively (Marsden, 1990). Interested in international student mobility, for example, Shields (2013) assembled information from country reports to the UNESCO Institute of Statistics on incoming students and their country of origin. Addressing questions about the system of "reference societies" in the context of education reform, Kessler and Pizmony-Levy (2020) extracted information from news stories published following the release of OECD/ PISA results in 23 countries.

To understand the architecture of policy knowledge, we examined a sample of ofcial policy documents from each of the fve countries participating in the study. Each national team has identifed a set of key


**Table 3.1** List of reforms by country

policy documents that refected ofcial/state policy knowledge that the government had used in preparation for the most recent school reform in each country. Table 3.1 presents the titles of the reforms in all fve countries. We frst selected WPs that outlined the policy proposal in each country. Te purpose of WPs is to launch a debate with stakeholders, including the public, unions, civil society, parliament, and the government. We then selected GPs that were explicitly cited in the WPs. GPs are written by government-appointed expert commissions to stimulate discussion on given topics; they often refect insights from multiple sources and relevant parties. In countries where WPs and GPs are not a compulsory part of the institutionalized education policy process, each national team carefully identifed ofcial policy documents that are functionally equivalent to the WPs and GPs, respectively, in their policymaking contexts (see Chap. 9 for comparative discussion on the reform and policymaking contexts of the fve Nordic countries). In this book, we use the terms "WP" and "GP" to refer to white papers and green papers as well as their functional equivalents. Our fnal sample of policy documents includes 8 WPs and 30 GPs (see Table 3.2).

Tere are strengths and weaknesses to this sampling strategy. On the one hand, this strategy ensures the comparability of concepts and results in the project. By drawing on policy documents published in the context of a recent education reform, we can also assess the prominence of similar international knowledge artifacts, such as OECD/ PISA reports. On the other hand, our focus on ofcial policy documents excludes texts produced by other stakeholders that participate


**Table 3.2** Policy documents (source), by country



(*continued*)

**Table 3.2** (continued)


in the policymaking process. Terefore, we have a limited perspective on the kind of evidence that is mobilized in the policymaking process. Future research could address this limitation by incorporating policy documents produced by stakeholders such as labor unions, think tanks, and civil society organizations.

Once we settled on the sample of policy documents for each country, we implemented a standardized procedure for coding individual references from each document. We trained and supervised national research teams from the fve countries participating in the study. Each team included two to three members; all of them had sufcient fuency and familiarity with education politics and policy to read and code policy documents. Te research team from each participating country was responsible for their country's data entry. All research teams followed a detailed protocol and used an Excel spreadsheet to enter the data. Data entry began by extracting all the items in the bibliography or reference list. Tat is, we coded references and not in-text citations. Research teams coded every reference in each source document (WPs and GPs) as the unit of analysis; they noted the content of the reference (e.g., author, year, title, publisher, type, and location). References were categorized as one of the fve document types (reports, books, journal articles, governmentpublished documents, and others) as well as one of the three location groups (domestic, regional/Nordic, and international). As international collaborations and multinational co-authorship increase, it has become more challenging to classify the location of a publication. In this project, the location was coded based on the location of the publisher. Figure 3.1 shows the frst page of the reference section for a GP in Norway: NOU 2015:8 Fremtidens skole (Te School of the Future). Te page includes 23 citations; each of them was entered as a unit/record in the database. In addition to coding the relationship between the GP and the references, we also coded the references' attributes. For example, we coded background information for the following citations:



**Fig. 3.1** Green paper from Norway, *NOU 2015:8 Fremtidens skole. Fornyelse av fag og kompetanser* [The School of the Future. Renewal of Subjects and Competences]

**70**

Te content of the reference was entered as it was listed in the source documents; however, when any errors were suspected or observed, national teams checked the cited document to address the issues. Troughout the data entry and coding process, we communicated closely with the national teams to ensure validity and reliability across the countries.

After the initial data entry, the national research teams prepared the data for network analysis. First, they assigned a unique identifcation number to each source document and reference document. Te identifcation number included three digits country (ISO-UN code) and threefour digits for the specifc document. Second, they combined variants of the same text under one identifcation number. For example, they merged under one identifcation number references to the original and translated versions of John Hattie's book *Visible Learning: A Synthesis of Over 800 Meta-analyses Relating to Achievement* (2008). Furthermore, one identifcation number was assigned for diferent editions of the same document if other contents of the reference, such as authors and publishers, remained the same. For serial publications such as OECD's *Education at a Glance*, only publications with the same title and author(s), published in the same year, were assigned the same identifcation number.

Te national research teams sent the data to the technical team at Teachers College, Columbia University for further review and cleaning. Te main purpose of this cleaning was to ensure that all information in the database was ready for SNA. After multiple rounds of data cleaning, we fnalized the database and constructed the network matrices for the analysis.

Finally, we produced fve sets (one for each country) of a social network fle and an attributes fle. Te network fle includes a two-mode matrix, with source documents (WPs or GPs) in columns and reference documents in rows. A cell in the matrix is coded one (1) if the source document cites the reference document and coded zero (0) otherwise. Te attributes fle includes background variables describing the documents (see above). Figure 3.2 illustrates this process.

In the fgure, circles mark nodes/actors and lines mark reference relationships. White Paper #1 cites two Green Papers (#1 and #2) and one reference document. Green Paper #1 cites two reference documents (#1 and #2). Green Paper #2 cites three reference documents (#1, #3, and #4). Reference document #1 has an in-degree of three; that is, three source documents cite this reference document. All other reference

**Fig. 3.2** Illustrative example of a policy knowledge network

documents have an in-degree of one. Te following matrix represents the relationships in the fgure as a two-mode network:


## **Analyzing Networks: Exploring the Architecture of Policy Knowledge**

In this book, the authors report data from policy knowledge networks in fve Nordic countries: Denmark, Finland, Iceland, Norway, and Sweden. Our analytical strategy included two steps. First, we calculated measures of centrality (degree) and centralization (density). Second, we produced visuals or maps of the networks. We conducted all analyses with UCINET 6.708 (Borgatti et al., 2002). UCINET is a comprehensive package for the analysis of social network data; most importantly, it can handle large networks.

At the micro-level, we calculated the **degree centrality** of reference documents. Degree centrality is the simplest centrality measure to compute; it is simply a count of how many connections an actor has with other actors (Wasserman & Faust, 1994). In this research, the degree centrality is equal to the number of source documents that cite a given reference document. For example, if WP 1 and GP 2 mention the same reference document, then the degree centrality for that reference document is equal to two. Following past research, we assume that documents with a higher number of citations are prominent or important in the context of the policy process.

At the meso-level, we examined **descriptive statistics of the degree centrality** (i.e., minimum and maximum, average, and standard deviation). Using these simple indicators, we assessed variability across the reference documents (Wasserman & Faust, 1994). Low variability suggests that reference documents are homogeneous in their degree centrality and structural position in the network. High variability suggests that reference documents are more heterogeneous in their degree centrality and structural position in the network.

At the macro-level, we examined the **density of the entire network**. Density is the ratio of the number of links to the maximum possible number of links. Higher density means that policy documents (i.e., WPs and GPs) draw on similar sources for evidence.

Finally, we used NetDraw to visualize the relationships between source documents and reference documents in the dataset. All fgures use a Multidimensional Scaling (MDS) layout with node repulsion and equal edge length bias (Borgatti et al., 2002). Tis approach puts two documents (nodes) closer together if they are more similar (in terms of their connections to other nodes). Te distances between documents and the direction (or location) are interpretable.

## **Limitations**

Our study has three limitations that readers should consider. First, we opted to code whether a source document cites a reference document (binary variable yes/no) and to ignore the number of times a source document cites a reference document. Tis means that our analysis does not distinguish between documents cited only once and documents cited in a variety of diferent ways and places over the course of the source document. Second, our coding protocol did not address the context in which source documents cited references. Like bibliographical analysis of books and articles, we expect that authors of policy documents can use references constructively or critically (Mayrl & Wilson, 2020). Tird, our coding protocol focuses on explicit references. Tus, it overlooks implicit references to culture, norms, and values (Steiner-Khamsi & Waldow, 2018; Waldow, 2017) that might be included in the text. Lastly, we did not carry out any multivariate analyses or statistical network modeling at this stage of the project. Despite these limitations, the database and analytic approaches utilized in this research project provide a unique opportunity to examine and map how policymakers mobilize evidence to advance educational reforms.

## **Conclusion**

In this methodological note, we presented the theoretical and methodological inspirations behind this research project. We defned key concepts—network, actors/nodes, relationships/edges, and bibliometric analysis—and described the process that led to the database. Te following chapters draw on the database and analysis we described above. Some chapters also draw on additional methodological approaches such as semantic and content analysis of policy documents and interviews with policy actors to address some limitations discussed above; the authors of these chapters provide additional information about their methodology.

Our approach to the study of the architecture of policy knowledge could be applied to other cases and domains. We hope scholars will fnd this methodological note useful as they extend this research. Te database we generated through this research project provides an opportunity to explore many more questions about policy knowledge. For example, scholars could explore the selection process of references into WPs (i.e., why are some references, but not others, included in WPs?) Scholars could contribute to the mapping of the policy knowledge domain by pointing out the relationships of co-cited authors (i.e., which authors are co-cited frequently?) Also, scholars could examine the titles/abstracts of documents in the database to examine topical patterns and frequent labels/words.

## **Note**

1. Following SNA literature we conceptualize documents as actors/nodes. However, it is important to note that we do not make any assumption about documents having agency or ability to form relationships with others.

## **References**


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **4**

## **Policy Borrowing and Evidence in Danish Education Policy Preparation: The Case of the Public School Reform of 2013**

**Trine Juul Reder and Christian Ydesen**

Since the beginning of the 1990s, Danish education policy has increasingly aligned with transnational trends. Danish student performance in international large-scale assessments (ILSAs) has sparked public and political debates concerning the Danish education system (Holm-Larsen, 2010). Given the students' mediocre performance and relatively high public education spending, Danish policymakers have focused on reforming the education system (Imsen et al., 2017), resulting in the initiation of several educational reforms in Denmark since 2000. Tese reforms include the 2006 public school reform, several reforms addressing primary school curriculum objectives (2003, 2009 and 2015), the 2007 introduction of national assessment tests in primary and lower secondary schools, and fnally the 2013 public school reform (Danish Evaluation Institute, 2012, 2015).

Tis chapter focuses on the 20131 public school reform, which is the latest major reform of the Danish public school system and one of the

T. J. Reder (\*) • C. Ydesen

Aalborg University, Aalborg, Denmark e-mail: cy@learning.aau.dk

<sup>©</sup> Te Author(s) 2022 **77**

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_4

most controversial and hotly debated political reforms in recent years. Te reform had three overall goals (Danish Government, 2012; Ministry of Education, 2014) 2 :


Tese goals were to be accomplished through measures such as longer school days, earlier foreign language learning, 45 minutes of exercise daily, homework assistance in "homework cafés," and more lessons in Danish and math (Ministry of Education, 2014).

Te reform proposal sparked heated public debate concerning the reform's content and the scientifc evidence behind its elements. Longer school hours were a central topic because the change interfered in the balance between school and leisure activities. A main argument for longer school hours was to improve students' academic level; however, Danish reports on the Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) results published a mere week after the reform's presentation stated there was no scientifc evidence indicating a relation between school hours and academic achievement (Allerup, 2011; Mejding & Rønberg, 2012). Merete Riisager, of the right-wing opposition, consequently called for a consultation concerning the knowledge sources underpinning the reform, and the debates continued in the media and in Parliament during the spring of 2013.

Another striking point in the reform's presentation was reference to ILSAs and foreign school systems' experiences, particularly the reform of Ontario's school system, which markedly improved its PISA rankings (Levin et al., 2008; Coninck-Smith et al., 2015). In 2012, Christine Antorini, then Minister of Education, expressed that the upcoming reform was largely based on an "inspirational" trip made by Danish Parliament representatives to Ontario (Aisinger, 2012) and later repeated that Denmark could learn much from Ontario's success (Larsen, 2012; Møller, 2012). Te reform was introduced following over a decade of discussion and negotiations around public school reform, justifed mainly by the Danish students' mediocre results in the PISA surveys from 2000 to 2011 (Cort & Larson, 2015; Egelund, 2008). Regarding ILSAs, only 5% of Danish pupils were "strong readers," and 17% lacked both functional math and science skills (Danish Government, 2012). When the Social Democrats won the October 2011 election, the new government committed to reforming public school. In December 2012, the government presented a proposal for public school reform, entitled *Make a Good School Better—Improving the Academic Level of the Public School* (Danish Government, 2012).

Given this background, this chapter connects with the theme of this volume and contributes to understanding the interplay between international policy trends and national education policy development in general and the role of evidence in the process behind Denmark's 2013 public school reform in particular. However, such an analysis calls for initial conceptual refections because discussion about what constitutes *real* evidence often steals the focus (Christensen & Krejsler, 2015).

As Steiner-Khamsi (Chap. 2, this volume) argues, a reference can be understood as "validation of evidence." Tus, a clear connection is established between references and evidence. Following Steiner-Khamsi, we employ a pragmatic approach to evidence, meaning we consider all kinds of knowledge sources and information used to inform policy processes and create a basis for decision making to be evidence (Cairney, 2015). Tese initial refections allow for the following overall research questions:

1. What evidence base underpins the 2013 Danish public school reform?

2. In what policy context was the evidence base formed and used?

## **Methodological Considerations and Chapter Structure**

Steiner-Khamsi et al. (2020) emphasized that evidence behind education reforms difers vastly. Terefore, we need to remain exploratory and open in terms of investigating "which evidence governments actually use when they formulate the policy" (p. 138). We also connect with Ozga's (2019) argument about taking the political dimension of policy seriously: "Seeing policy as politics focuses on how state policy in its design and attempted delivery involves politics, through interests, conficts, power and control—so that politics is an essential element of policy" (p. 21).

To realize these insights, we employ a mixed-methods design that creates an abductive platform between sources of ofcial policy knowledge found in a bibliometric analysis of the policy documents and sources of unofcial policy knowledge emerging from research, media articles, and a contextual reading of policy documents. Additionally, we conducted four qualitative interviews with key informants in the spring of 2019 (Appendix 1) to uncover other sources of knowledge than those explicitly cited in the policy documents.

We start with a bibliometric analysis based on 231 references in the preparatory school reform policy documents. Tis quantitative analysis sheds light on the types of policy evidence used in the proposed reform and patterns emerging from the reference network of knowledge sources. Te second part analyzes the policy context framing the formation and use of the evidence base. Te conclusion summarizes the main fndings and connects to the overall research questions.

## **Source Documents**

For the basis of the bibliometric analysis, we include the reform proposal itself (Source Document 1) and the four references of the reform proposal authored by a government institution: two reports by the Agency for the School Council (Source Documents 2 and 3)3 and two reports by the national sector research agencies Te Danish Evaluation Institute (EVA) and Te Danish National Centre for Social Research (SFI) (Source Documents 4 and 5). Unlike the *Ofcial Norwegian Reports* (Steiner-Khamsi et al., 2020), these four reports were not specifcally prepared as background papers to formulate the school reform policy; however, we argue they constitute the best possible data for our bibliometric analysis. Table 4.1 displays the ten references of the school reform, where the frst four constitute the source documents of this analysis.

Source Documents 2 and 3 were authored by the Agency for the School Council, which was established in 2006 to "follow and assess the quality of the primary and lower secondary school and advise the Minister of Education" (School Council, 2007, p. 3). It was independent of the Minister of Education, and its members were representatives of diferent interests in the elementary school area. Although the council reports were produced by a secretariat at the Ministry, its content was decided by the council.

Te two sector research reports were authored by SFI (Source Document 4) and EVA (Source Document 5), and both institutions were partly funded by the government. An independent state institution established under the Ministry of Education in 1999, EVA, is one of the central research institutions working in education policy (Bjerre & Reimer, 2014). EVA evaluates and researches within the education feld at the request of other branches (e.g., ministries, local authorities), as well as on their own initiative (Danish Evaluation Institute, 2019). SFI was a sector research institution under the Ministry for Economic Afairs and the Interior. Tis institution mainly conducted research in welfare state policies (e.g., concerning the labor market and family-related issues). In January 2017 SFI merged with another public research institution and was renamed Te Danish Center for Social Science Research (VIVE).

## **Qualitative Data**

Te qualitative analysis relies on four semi-structured interviews with key policymakers behind the reform: a former ministerial ofcial who worked on the reform proposal (INF1), a former Parliament member (INF2), and two former members of the School Council (INF3 and INF4). Te informants were directly involved in the reform preparation or in the work of the School Council, and the interviews were coded in a datadriven approach (Schreier, 2014). Te codes are presented in Appendix 1. Te interviews ofer insights into the preparatory work in meetings and seminars with stakeholders for which no records or summaries are publicly available. Te informants were anonymized, since the reform sparked heavy public debate regarding the evidence behind the reform. Terefore, it was an ethical and strategic choice to give the informants the opportunity to speak freely without fearing consequences in the form of media critique or otherwise.

To verify the reliability of the interview data, we cross-checked the interview information, arguments, and claims through other interviews and weighed it against open-source information using source criticism procedures found in historical research (Ifversen, 2003). We therefore include public statements from key policy ofcials to connect to the ways the reform was presented and justifed to the public. We located the quotes and articles using Infomedia, a Danish media surveillance company that covers all Danish newspapers.

## **Mapping the Evidence Base of the School Reform**

In this section, we present the main fndings of the bibliometric analysis in terms of the reference locations, document types, network of references, and publishers. Table 4.2 presents the reference distribution between the fve Danish source documents and the distribution between the document types and location.

Te School Council reports and SFI reports have the most extensive citation practices, explained by the nature of these institutions: Te School Council's (2007) purpose is to "provide documentation for the initiatives and actions that contribute to increasing the quality of the primary and lower secondary school" (p. 14). Tus, its purpose is to provide evidence and disseminate research to the Ministry of Education in yearly reports, focusing on specifc topics. SFI is generally considered a research-heavy institution compared to EVA, which mainly conducts empirical research and evaluates specifc national initiatives in education.

We split the references into fve document types: reports, books, academic, government, and others. Te reference distribution reveals that the source documents mainly cite government documents (33.77%) or reports (29.87%; see Table 4.2). "Government" evidence includes documents and reports produced by the government, a ministry, or an agency


**Table 4.2** Reference distribution of a ministry. "Reports" can be produced by research agencies, international organizations (IOs), or private consultancy frms. Te "academic" literature accounts for 16.02% of all references, mainly distributed in the 2012 School Council report (Source Document 3) and the 2011 SFI report (source Document 5), both of which concern school leadership. Furthermore, the SFI report was commissioned by the School Council (Danish National Centre for Social Research, 2011, p. 7). It is therefore not surprising that the reports are similar in terms of references.

#### **Network of References**

Figure 4.1 illustrates the complete network structure and co-citations between the fve source documents. Te references are displayed as squares, and Source Documents 1–5 as circles. Te size of each note refects its in-degree centrality; the larger the note, the more source documents have cited it. Te bibliometric analysis shows a relatively low frequency of co-citations between the source documents: only 16 references

**Fig. 4.1** Complete network structure. (*Note*: Circles represent source documents. Squares represent citations. The colors determine whether the citation is regional (gray), domestic (white), or international (black). Node size refects in-degree centrality)

are shared by a minimum of 2 source documents, and no reference is shared by more than 3. Te report cited by three source documents and thus at the center of Fig. 4.1 is *Te High-Performing School—How Can Schools Improve the Profciency Level of Students with a Weak Social Background?* (Mehlbye, 2010). It was published by the Danish Institute of Government Research (AKF) and commissioned by the School Council to examine so-called high-performing schools, focusing on schools that improved the academic performance of students with weak social backgrounds. Te report is thus a classic example of a "what went right" approach, where policymakers attempt to fnd the best practice by studying successful examples. Qualitatively analyzing 12 schools, the report seeks to deduce what characterizes high-performing schools and concludes that, for instance, clear goals, academically competent teachers, and homework assistance were important for the pupils to succeed (Mehlbye, 2010). Since reducing the infuence of students' social background on academic results was one of the three goals of the school reform, it is not surprising that a report addressing these issues appears signifcant.

Te shared references in the source documents mainly comprise research by the government or connected institutions4 (11 of 16 references) or international evidence from either IOs or international edubusinesses (3 of 16 references; see Appendix 2 for a list of all 16 co-citations). Te low frequency of co-citations in Source Documents 2–5 reveals that the 4 reports behind the school proposal represent specialized knowledge: they each focus on specifc subjects used to substantiate the diferent reform elements. Te 2011 School Council report focuses on transfer between primary and secondary education, schools' collaboration with municipalities, and after-school classes, whereas the 2012 School Council report is concerned with school management. Te EVA report (commissioned by the School Council) concentrates on differentiated teaching, evaluation, and teacher professionalism, and the SFI report (commissioned by the Ministry of Education) centers on school leadership. Tis picture of specialized knowledge is similar to Norway's case, in which the Norwegian Ofcial Commissions (NOUs) generate "highly specialized knowledge" that allows the government to "selectively transfer what was produced at the commission level to the political level" (Steiner-Khamsi et al., 2020, p. 128).

## **International Inspirations: Academic Research and Large-Scale Assessments**

A remarkable pattern appears when we look at the international and domestic references individually, as in Table 4.3. Namely, the academic references consist almost exclusively of international research. Tis fnding is in line with a case study of Norwegian policy advisory commissions by Christensen and Holst (2017), who found growing reliance on what can be categorized as *academic knowledge* in general and (in the Norwegian context) on international academic knowledge in particular. Although Denmark has a long tradition of educational research going back to World War I (Gjerløf & Jacobsen, 2014), this research surprisingly does not seem to appear in the preparation of the 2013 school reform. Te international academic references are mainly centered on the topic of educational leadership, and the majority are referenced in Source Documents 3 and 5, both reports concentrating on school leadership at diferent levels (school, municipal, and national). Te most cited international journals are *Educational Administration Quarterly* and *Journal of Public Administration Research and Teory*, both of which focus on educational leadership and management, as well as public administration (Oxford Academic, 2020; Sage, 2020).

Te remaining international references are distributed mainly as reports (21.43%) and books (25%). Of all international reports (*n* = 23), the majority come from IOs (14) and consultancy frms (4).5 Te OECD

**Table 4.3** International, regional, and domestic references distributed by types of knowledge


(2010b, 2010c, 2010d, 2011a, 2011c) is the most cited IO, and the citations mainly refer to PISA.6 Tis emphasizes the importance of IO evidence, especially concerning PISA, in Danish school policymaking in preparing and legitimizing the reform. We detail this in the subsequent qualitative analysis, focusing on the use of international experience and lessons in the 2013 school reform.

## **Reliance on Government-Commissioned Evidence**

While the international citations mainly refer to academic and IO evidence, domestic citations largely refer to government and public institution reports. Figure 4.2 shows the 11 most cited publishers, 6 of which are the following public institutions: Ministry of Education, EVA, SFI, the Danish Government, the School Council, and AKF.7 Te Ministry of Education is by far the most cited.

As seen, the feld of social policy (including education) in Denmark contains a range of institutions aimed at providing evidence for policymakers. Te bibliometric analysis reveals three of these institutions (EVA, SFI, and AKF) are among the most commonly cited in the preparations for the public school reform. One explanation for this fnding is that it is common practice for institutions to cite themselves where possible. SFI and EVA are the authors of Source Documents 4 and 5, respectively, and

**Fig. 4.2** Most cited publishers

the EVA evaluations on compulsory education were all assigned by the School Council, author of Source Documents 2 and 3 (OECD, 2011b; School Council, 2007).

Research from public agencies thus constitutes a large part of the references.8 Te evidence produced by these institutes is mostly commissioned by public institutions, such as ministries and municipalities. Bjerre and Reimer (2014) explained,

the research institutes have established their own system of peer review and quality check […] and thus the reports move directly from the research agencies into the political, administrative, and public process, without the intermediation of a scientifc community of critical readers and the general peer review system […]. (p. 85)

Te majority of the evidence produced by these institutions can be categorized as *strategic* evidence, which is evidence commissioned by the government (or a government institution) or carried out by a public institution. However, there are two reservations. First, not all research by public agencies is strategic; for example, institutions such as EVA and SFI also conduct research for external parties, such as private funds (e.g., Danish Evaluation Institute, 2020). Tese agencies also conduct what they call independent9 research, which is research funded by their annual budget. Second, strategic evidence is sometimes produced by private organizations, companies, or universities. To establish the extent of strategic evidence in the 2013 public school reform, we therefore take a closer look at the references.

To identify how strongly the school proposal relies on strategic evidence*,* we thoroughly examine the references. First, we exclude all references to academic articles. We then select all references that could be categorized as strategic evidence based on the publisher. Finally, we examine each reference individually to establish the report's funding and commissioning (see Appendix 3 for selection criteria).

Our results show that 63 of the 231 references count as strategic evidence produced in Denmark.10 Te reports were mainly commissioned by the Ministry of Education or the School Council and carried out by various actors and organizations, mostly EVA, the Agency for the School Council, or SFI. A picture can be drawn of the remarkable evidence composition where strategic and politically infuenced evidence accounts for as much as 27.27%.

## **Summary: Types of Policy Evidence**

According to the reference analysis, we found that the Danish public school reform relies on at least three types of evidence, which we categorize as *academic*, *strategic*, and *IO evidence.* Te frst two categories are inspired by Bjerre and Reimer (2014), who distinguish between *strategic* and *academic evidence* in their analysis of Danish teacher training programs.

Our analysis of the references in the fve source documents shows that strategic evidence constitutes a large part of the references. Research agencies such as EVA and SFI are behind many of them. Academic evidence constitutes a signifcant percentage (16.02%) of the references, almost all of which are published in international journals. Te international academic references focus mainly on leadership, a central part of the reform. Te IO evidence has been a large source of inspiration and legitimization of the school reform. Te analysis shows the OECD is the fourth most cited publisher. Furthermore, Aarhus University, which produces the Danish TIMSS, PIRLS, and OECD reports on education, is also among the most frequently cited publishers. Te reform proposal references OECD data to justify the need for reform: Danish students do not have profcient reading skills, "have problems in mathematics," and are not "good enough" in science (Danish Government, 2012, p. 7).

Te bibliometric analysis reveals reference patterns that we explore further in the following two qualitative sections. Te frst research question (RQ1) relates to the knowledge sources underpinning the reform. We have shown that international academic evidence, IO evidence, and strategic evidence constitute a large part of the references. However, although the analysis enables us to identify reference patterns, it does not tell us much about the context or reasons for the patterns. Moreover, it reveals only evidence explicitly referenced in the policy papers. As shown, in Denmark's case, the number of explicit references is signifcantly lower than for other Nordic countries, which raises the question of whether the 2013 Danish public school reform relied on other types of evidence than what appears in the policy paper references.

Our analysis also shows that international references constitute a large part of the references (36.36%) and that these are partly academic and partly stem from IOs, particularly the OECD. In the second qualitative section, we analyze the policy context in which the evidence base was formed and used (RQ2).

## **Evidence Beyond the References**

Scholars have argued that commissions play an important role in policy preparation in Scandinavia today, especially in Norway and Sweden (Christensen & Holst, 2017; Holst & Molander, 2018; Steiner-Khamsi et al., 2020). In contrast, such a practice is seldom seen in Denmark (Christensen et al., 2009). With the 2013 public school reform, no commission was established to prepare the bill, and there were no reports or documentation of the evidence, except for the references in the proposal itself (INF1, INF2, INF4).

It is worth noting that the new government's memorandum of understanding between the three participating parties expressed the need for public school reform some two years before the reform (Danish Government, 2011). Tis indicates that, even before the government won the election, the reform's main contents had already been negotiated between the Social Democratic Party, the Socialist People's Party, Te Red-Green Alliance, and the Danish Social-Liberal Party. A former Parliament member explained that preparing the reform proposal "has a long history *before* the government was established. In the years leading up to the election in 2011 […] there was close coordination between the spokespeople of the opposition parties" (INF2) and discussions about the reform's content, particularly the idea of a comprehensive school,11 inclusive of all children, regardless of wealth, social background, or abilities (Imsen et al., 2017). If anything, these preparatory tracks indicate the ideological priorities of the new government. Tese priorities are not rooted in evidence as such but testify to the politics of policymaking. One informant went so far as to say,

In my opinion, an important point to make is that the Danish school reform is not characterized so much by research evidence. Its main elements are based on political and ideological thinking rather than a scientifc foundation. And that is probably one of the reasons there are so few references. In addition, and especially in the pedagogical area, there has not been a strong tradition of assembling a common platform of research evidence. […]. So that is also part of the explanation. It is diferent in Norway and Sweden. (INF4)

Te quantitative analysis of references is based on ofcial policy evidence and an assumption that evidence constitutes the core building blocks of policy development, that is, that policy development is a fairly rational process. Instead, we argue that, in Denmark's case, we need to adopt a more nuanced view of what constitutes policy evidence and to refect on the role of ideology (i.e., politics) in the processes surrounding all applications of evidence in the preparation of the 2013 reform. To start, we argue that, in addition to academic, strategic, and IO evidence, two other types of evidence are fundamental in the preparations of the public school reform: *stakeholder evidence* and *practice-based evidence*.

## **Stakeholder Evidence**

After the 2011 election, the school reform proposal work took three different paths. Te frst was a partnership with the stakeholders of public school, who met in three seminars between October 2011 and December 2012 to discuss the main topics of the reform:

Te public school reform was described quite in detail in the government program, but it had to be translated into an actual bill. And […] a partnership with the main stakeholders was established. […] Tere were three seminars which were theme-based on the main topics […]—it was like inspiration for us to see if we could create—well, we didn't call it a consensus conference, but it was building on the idea: If everybody was there and we were talking about the content [of the reform]—if we could say, "there are these fve points that we agree on." (INF2)

Te seminars were organized by the Ministry of Education, and the six stakeholders were the teachers' union, the school principals' union, the union for early childhood teachers and youth education, the interest organization of the municipalities called Local Government Denmark, the students' union, and the parents' organization (INF2). Second, there was a working group across several ministries, including the Ministry of Education, the Ministry of Higher Education and Science, the Ministry of Economy and the Interior, and the Ministry of Finance. Tird, an ofce dedicated to the reform work was established in the Ministry of Education (INF1).

According to an informant from the Ministry of Education, the culture of negotiation is typical for Danish education policymaking:

It's funny with Denmark and Sweden—we [the Danish people] have a culture of being merchants and traders and adjusting things along the way, and we have fexibility and close collaborations, whereas Sweden is like a big industrial nation—they produce cars—they take the commission work and then they lead it into the government and decide something. … So, it's like an industrial nation way of thinking, which they also adopt in their policy development. And in that case, we are more like merchants … we negotiate something that everybody has a share in. And during the process, we collect some knowledge and try to establish a broad ownership. (INF1)

On a more objective note, a recurring observation in the Danish history of education is that reforms of the public school system have involved broad cross-party compromises to secure political continuity and broad support from stakeholders to secure involvement (Coninck-Smith et al., 2015; Gjerløf & Jacobsen, 2014). Te observations in this section are indicative of the political culture surrounding the reform process. Stakeholder evidence is found to play a role, even if the framing of stakeholder evidence can be considered an expression of political and ideological priorities. Tese observations gain currency when we consider the role of practice-based evidence.

## **Practice-Based Evidence: Bottom-Up and Experimental Projects**

In August 2012, the Minister of Education Antorini launched the project called New Nordic School, inviting institutions from the educational sector (from pre-primary education to upper secondary education) to participate. Te New Nordic School project was part of the ideological basis of the school reform and constitutes another type of evidence groundwork for the reform, which we denote practice-based evidence. Te project's three overall objectives—identical to those of the school reform—had been determined beforehand. In an annual meeting held in the town of Sorø, practitioners and experts discussed the challenges for public school, focusing on diferent themes, and determined ten objectives. Te 2012 Sorø meeting centered around the New Nordic School project; later, a manifest was released with these ten points, as well as three overall objectives (Ministry of Education, 2012). By November 2012, over 350 institutions had applied to participate in the project (Olsen, 2012). Te institutions committed themselves to initiate change processes inspired by the project's manifest and objectives. Across the country, "networking days" were held where institutions discussed and developed their individual projects.

Another example of practice-based evidence involves the tradition for experimental project work in Danish public schools (Andreasen & Ydesen, 2015; Coninck-Smith et al., 2015; Gjerløf & Jacobsen, 2014). Since the 1920s, Danish school policy development has been characterized by experimental projects in schools and teacher training colleges, a practice later known as the Danish Model (Coninck-Smith et al., 2015, pp. 112–115). In the case of the 2013 public school reform, an informant explained that "in the content of the school reform, there wasn't a single new thing which hadn't been experimented in schools" (INF2). Examples of experimental projects with elements of the reform include a 2005 project about homework cafés in 15 Copenhagen schools (Vogt-Nielsen & Hansen, 2005) and a project about extended school hours (*heldagsskole*), tested in 12 schools across the country and evaluated in October 2012 (Rambøll, 2012).

#### **4 Policy Borrowing and Evidence in Danish Education Policy…**

As seen, increasing schooling hours was one of the most radical and problematized elements of the reform. Interestingly, extending the school day is not referred to as *heldagsskole*, and Rambøll's (2012) evaluation is not mentioned in either of the source documents. Te reason could be that the report concludes there is no signifcant evidence supporting the statement that extended school hours improve academic performance. Furthermore, a former member of the School Council was skeptical about Rambøll's evaluation (INF4), suggesting the reform was based on the idea that

[…] if you wanted to help students with a weak social background, then the school would have to—to a larger extent—deal with all learning activities. And you had to leave fewer things up to the parents—less homework …. […] Tis is not based on any scientifc evidence—this was defnitely driven by ideology and visions […]. (INF4)

So far, our analysis has shown that practice-based evidence is an important part of the evidence base of the school reform. Other research has demonstrated that the role and involvement of practice-based evidence in education reforms has a long history in Denmark. However, our analysis has also revealed a strong tendency toward cherry-picking. One informant said, "Te politicians are very interested in evidence—if it matches their opinions" (INF4). Tis point takes us back to the ideological and political elements in the reform process.

## **Using International Evidence in the Preparation of the School Reform**

More than in the other Nordic countries, Denmark's public school reform draws on international evidence sources, namely international academic articles and IO reports. Te bibliometric analysis concludes that, in spite of a long tradition of Danish educational research, the source documents cite mainly international academic research. Tis could be because Danish researchers often publish in international journals, in which case the data are still considered international research. While this observation may hint at an error in the analysis, the qualitative interviews indicated there could be another reason for the "missing" Danish education research:

I still think there is something missing [in Danish education research]. Tere are a lot of things that have not been properly examined. … I thought it was a big challenge that we didn't have any study saying, "Ok, what will the efect be if we do A or B in some area?" … And some things are just not properly scientifcally examined and others are but […] in studies where it is difcult to conclude anything about causality and efect. (INF3)

Another informant expressed that there was just not enough Danish education research (INF1), or at least not the kind requested by decision-makers.

Te evidence provided by IOs has impacted the preparation of the public school reform both directly and indirectly. Since the frst ILSA in which Denmark took part was published, international experience has played an important role in Danish education policymaking (Andreasen, 2019). First, as Addey and Sellar (2018) noted, ILSAs often serve to legitimize the need for reform, as was also the case of the public school reform (INF2), where PISA was used to "diagnose" the problems (INF1). Te public school reform came after more than a decade of debates about the state of the public school system and even an earlier attempt to propose school reform by the former Danish Government (2010). Te ground was thus laid for the reform, a point that lends support to what Dobbin et al. (2007) refer to as the constructivist mechanism of policy difusion, emphasizing the importance of public policies becoming socially accepted for their difusion ability. However, the ILSAs themselves usually do not indicate any specifc solution:

As soon as you need to fnd solutions, then you have to render it probable that you causally will get the efect that you claim that you will get. And how can you get that? You need to have some studies where you have looked at some changes, done some intervention, and documented that this intervention had the planned efect. Tat is causality. And [those studies] are typically carried out in a diferent context [than the Danish one]. Ten you look at Ontario—they did this reform and had that result. And then we add this idea about causality—but you have to consider whether you can have the same efect in a Danish context. And you have to be extremely careful. (INF1)

Nevertheless, the OECD's PISA program set the agenda for which countries to look to when preparing school reforms. Tis holds true not only for the 2013 public reform, but also for the school policy reforms and development in the years leading up to the reform:

Under the Anders Fogh12 government, there was a lot of focus on Singapore—that was where everybody was going, and you had to learn from the miracle of Singapore. Ten the wind was blowing towards west, and it was Ontario that we all were looking towards. (INF2)

Tus, although Singapore functioned as a positive reference society during the right-wing government, it was replaced by Ontario after the election, with some criticism of the past focus on Asia. As explained by a former Parliament member,

Te Asian school system is very far from the Danish one, which makes it really difcult to translate. With the Canadian model … it was easier to be inspired by the way they have built their school reform—and transfer it into the Danish context. (INF2)

## **Policy Borrowing: Inspiration from Ontario**

Te inspiration from the reform of Ontario's school system in 2003 is refected in media reports, as well as in the interviews (Fuglsang, 2012; Møller, 2012; Søndergaard, 2012). In September 2011, the School Council (2012) visited Ontario to study the reasons behind its successful school reform in 2003. One year later, members of the Danish Parliament's Committee for Children and Education (consisting of the spokespersons for education from each political party) went on a similar feld trip to Ontario to meet with politicians, researchers, and teachers (Ministry of Foreign Afairs, 2012). Furthermore, Mary Jean Gallagher, Chief Student Achievement Ofcer in Ontario, was invited to the Danish Ministry of Education during the reform's preparation, as well as to a 2012 New Nordic School meeting (INF2; INF4). Because the representatives of all political parties had visited Ontario, it became a common reference point for the negotiations, with politicians referring to examples from Ontario when arguing (INF2). Not only did Parliament members and the School Council go to Ontario, but individual schools and municipalities interested in the "miracle of Ontario" also went on study trips to discover the factors for success (Fuglsang, 2012; Jacobsen, 2012). Trough this engagement with stakeholders, municipalities, and the public, general social acceptance of the need for school reform was established.

Te 2013 public school reform is a classic example of *policy borrowing*, in which reference (to Ontario) and transfer (of educational policies) occur together, lending authority to the reform as a package of best practice policies (Phillips & Ochs, 2004; Waldow, 2017). Te three goals of the Danish public school reform were nearly identical to the goals of the 2003 education strategy of Ontario: "raising the bar for all students, reducing achievement gaps and restoring public confdence in the publicly-funded school system" (Ontario Ministry of Education, 2010). However, while policy borrowing, in the terminology of Phillips and Ochs (2004, p. 778), begins with the stage of "cross-national attraction," how such "attraction" comes into play must be addressed. As Simons and Voß (2018) have argued, policy solutions do not always follow policy problems. Te concept of instrument constituencies accounts for the fact that "instrumental options and their consideration as viable solutions to certain problems does not necessarily […] follow the diagnosis of problems—neither chronologically, nor in terms of stages in a sequenced process of rational analysis and problem-solving" (p. 15)*.* While the qualitative interviews pointed to a common understanding of the policy solution (the school reform) following a problem (Danish students' poor performance) (INF1; INF2), the role of IOs, especially the OECD, has clearly had a large impact on both defning the problems and providing possible solutions. Hence, it was the performance of students in PISA that sparked the need for education reform in both Ontario and Denmark.

## **Concluding Discussion**

In this chapter, we have explored the landscape of evidence behind the 2013 Danish school reform by combining bibliometric and qualitative analysis. Tis particular reform has been the subject of intense debates concerning whether it was based on any evidence or whether it was a purely political project, with no documentation of the efect of the major changes it induced in Danish public school. Asking what evidence underpinned the reform could be interpreted as support for one side of this debate. However, as we argued, we have chosen to employ a rather broad and inclusive defnition of the concept of evidence. We therefore reviewed all types of knowledge sources as part of a landscape of evidence in the reform process.

Combining the bibliometric and qualitative analysis has enabled us to unravel multiple dimensions of the preparatory work behind the 2013 Danish school reform. Te bibliometric analysis allowed us to examine the reference patterns of the ofcial policy knowledge behind the school reform. A key fnding pertains to the academic evidence, which played a signifcant role in the bibliometric composition of evidence. Our analysis clearly reveals a particular clinical type of academic evidence considered by decision-makers to be useful to education reform. Te prevalent type of academic evidence can be characterized as evidence-based and "whatworks." Tis evidence provides hard-core data and/or is concerned with revealing best practices. Moreover, we saw how academic evidence does not stem from Danish education research in general, but from a very particular string of research. Teoretically, this valorization of a particular kind of academic evidence could be considered an expression of competition in policy borrowing (Dobbin et al., 2007). Te 2013 school reform seems to revolve around an idea of international competition, living up to international standards, and importing best practices to quench a fear of falling behind (Krejsler, 2019). From a broader perspective, the thesis about the competitive state could help further understand the implications. According to Pedersen (2010), the competitive state mostly views education as an investment in the state's competitive performance on the global level.

Furthermore, the analysis reveals that the reform largely references international sources and strategic evidence. Categories such as domestic academic literature or references to documents and data from Nordic countries are almost non-existent in the policy evidence explicitly referenced in the policy papers behind the school reform. Te strategic evidence is provided by public research institutes and ministry units (e.g., EVA, SFI, AKF, the School Council) and private actors and universities (e.g., Rambøll and Aarhus University).

Te number of explicit references in the policy papers behind the reform is signifcantly lower than in the other Nordic countries. However, as our analysis shows, this does not necessarily mean that the reform did not rely on *any* type of evidence. In the qualitative analysis, we constructed additional analytical categories to account for two types of policy evidence (i.e., stakeholder evidence and practice-based evidence) not apparent in the bibliometric analysis, but important to the reform's preparation. Te New Nordic School project allowed more than 350 institutions (e.g., schools, kindergartens) to work individually with the reform's goals. Meanwhile, meetings with the key public school stakeholders took place at the Ministry of Education, where the reform was discussed and developed. However, the reform elements were also largely based on ideology. Work on the reform began even before the government took ofce in 2011, and the key elements were determined before the ofcial work began that same year. Tis suggests that the ofcial portrait of evidence behind the reform has a distinct cherry-picking favor, what Pawson (2006, p. 7) called "policy-based evidence" to describe research that "travels straight from ideology to policy recommendations via the cherrypicking of evidence."

Te case of the 2013 Danish school reform is also a clear example of policy borrowing (Phillips & Ochs, 2004), and we saw how a constructivist mechanism of policy difusion supports the strong inspiration and lessons drawn from Ontario. In defning the constructivist mechanism, Dobbin et al. (2007) contends how "policy makers play follow the leader by mimicking the countries that appear to be doing best" (p. 452). It is remarkable how politicians, civil servants (ofcials), and researchers associated with the general reform process and the specifc bodies conducting the reform's preparatory work were all swayed by the results and methods of the Ontario education system. Teoretically, the reform was underpinned by a powerful narrative constructed by key agents about the wonders possible in education following the Ontario recipe. But the reform process also highlights the OECD's infuence in both legitimizing the need for reform (through PISA) and indirectly pointing to the solution in terms of Ontario as a positive reference society.

Tus, the evidence landscape emerging from this analysis is very uneven. It consists of distinct plateaus of evidence-based and what-works types of evidence underpinned by signifcant policy difusion from Ontario, OECD framings of what counts in education, strategic evidence, and a good portion of political ideology. Te absence of broader and often critical—Danish education research constitutes the valleys of the landscape. While stakeholder evidence and practice-based evidence were allowed voices in the reform process, the precise impact and signifcance of this type of evidence remain obscure.

## **Appendix 1: Coding of Qualitative Interviews**


**Table 4.4** Coding of qualitative interviews

#### **Table 4.5** Informants of qualitative interviews


*Note*: Conducted May–June 2019


**Table 4.6**16 references cited in at least two source documents


(*continued*)


**104**

## **Appendix 3: Methods for Counting**  *Strategic Evidence*

## **Step 1**

First, we selected all references identifed as *reports*, *books*, *governmentissued decrees or guidelines*, or *Other*. References categorized as *academic articles* were excluded since they cannot be strategic.

## **Step 2**

Of these, we identifed and isolated the references from publishers likely to have produced research commissioned by the government or a government institution.

Public research institutions, councils, or ministries


## Private research institutions


## Universities and higher education institutions


Unions and interest organizations

• Lederne

## Publishing houses


## Other


We thus have 138 references.

## **Step 3**

Ten we checked each of the 138 references individually to establish whether that particular report or book would be categorized as strategic evidence, for example, as follows:

1. All reports commissioned by the School Council are counted as strategic evidence*.* Tese are reports from EVA, SFI, and AKF.


## **Result**

Among the 231 references, we identifed 63 strategic evidence references. References cited by multiple sources were counted only once.

## **Notes**


## **References**


*tory vol 5. When the school became Everyone's. Te time after 1970]*. Aarhus University Press.


*Review of Sociology, 33*(1), 449–472. https://doi.org/10.1146/annurev. soc.33.090106.142507


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **5**

## **Evidence and Expert Power in Finnish Education Policy Making: The National Core Curriculum Reform**

**Saija Volmari, Jaakko Kauko, Juho Anturaniemi, and Íris Santos**

Since the 1990s, national policy reforms have been increasingly more infuenced by transnational actors and the global context (Rizvi & Lingard, 2010). International organizations have become particularly active players in setting policy direction on a national level (Morgan & Volante, 2016). For example, in the feld of education, organizations such as the World Bank, the United Nations Educational, Scientifc and Cultural Organization (UNESCO), and the Organisation for Economic

S. Volmari (\*)

University of Helsinki, Helsinki, Finland e-mail: saija.volmari@helsinki.f

J. Kauko • J. Anturaniemi Tampere University, Tampere, Finland e-mail: Jaakko.kauko@tuni.f

Í. Santos Tampere University, Tampere, Finland

University of Lisbon, Lisbon, Portugal e-mail: iris.santos@tuni.f

<sup>©</sup> Te Author(s) 2022 **115** B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_5

Co-operation and Development (OECD) have been prominent in engaging in national education policy making (Mundy et al., 2016). Teir instruments of assessment enable them to compare outcomes across countries, identify successful practices, and defne quality standards that can serve as references for subsequent policy adjustments. In this way they contribute to setting the rules of the game through "governing by comparison" (Martens & Niemann, 2013, p. 317). At the same time, expert advice and evidence is widely sought by national policy actors as support for framing and legitimizing complex decisions. Tis has led to critical voices claiming for instance that the unelected actors have gained power that challenges the conventional decision-making processes (Viber, 2007) and that high-level strategic thinking may be outsourced from the national to the supranational level (Lawn et al., 2011).

Since the frst Program for International Student Assessment (PISA) results were published in 2000, many have considered Finland a model of educational success. Indeed, the nation is widely used as a reference society (Bendix, 1978, as cited in Waldow, 2017; Waldow & Steiner-Khamsi, 2019) for standards of good practice by countries around the world, including other Nordic countries such as Norway (see, e.g., Sivesind, 2019), that are looking to improve their education performance. Although Finland has had a strong tradition of adopting state-led policies in education, the extent to which international organizations exercise their infuence on the national level is a subject of debate in the Finnish context as well.

In this chapter we address the recurring debate in comparative education on the extent to which education is shaped by national or international infuences and explore it from the point of view of expertise and evidence. Our focus is on *determining the kind of evidence Finland draws on when endeavoring to improve its schools and learning outcomes and identifying whose expertise is valued most as evidence in this process*. As an example of such a reform, we examine the policy-making process that produced the 2014 National Core Curriculum of Finland. We discuss our fndings in light of previous research on education policy making in Finland to elucidate the two opposing strands of the debate—one stressing the statecenteredness of education policy making in Finland and the other claiming that transnational organizations and international expertise and evidence are gradually gaining more leverage in the feld. To overcome the dichotomy between the two opposing forces—national and international—we view global as something that is constructed within the local (Massey, 2005; Sassen, 2007, 2013). We use Larsen and Beech's (2014) suggestion to focus on "networks, connections and fows" (p. 75) in researching educational transfer and Eyal's (2019) idea of expertise as located in relationships and networks between individuals. Our interpretative framework and our choice of data and methods of analysis stem from the defnition of evidence by Paul Cairney (2016, p. 3): "Evidence is an argument or assertion backed by information." Based on these theoretical starting points, we examine the bibliometric references as information that supports the arguments in the most prominent policy documents in the reform process.

Te chapter begins with a discussion on the meaning and role of evidence and expertise in current policy development, with a focus on the context of education policy and politics. We then elaborate on the context of Finnish education policy making and the argumentation in the relevant literature. Next, we present the research design and results. Finally, we conclude the chapter by discussing the fndings of our research in relation to our interpretative framework and presenting our conclusions.

## **Evidence and Expertise in Education Policy Making**

As the world has become increasingly interconnected and the issues facing policymakers more complex and global, government authorities worldwide have grown to rely extensively on expert advice to inform the decisions they must make. Seeking outside opinions has become second nature to policy making in modern democracies that strive to perform well (Holst & Molander, 2019; Maasen & Weingart, 2009; Moore, 2017). At the same time, the pace and scope of information fow is constantly escalating. It has become increasingly challenging, for instance, to maintain a working knowledge of the vast and increasing amount of scientifc output. Consequently, one key task of the experts and expert organizations today is to select the knowledge relevant to address the data and information needs of politicians and policymakers (Stanziola, 2012; Wolscheid et al., 2019).

Te role of knowledge in policy making and politics has become more visible due to the rise of the evidence agenda (Wolscheid et al., 2019, p. 273). Politicians and policymakers are expected to base their decisions not on intuition and beliefs but on objective and reliable information in other words, actual evidence. Te role of experts in providing this knowledge has become central. Contrary to popular belief about what constitutes expertise, Eyal (2019) argued that expertise is not a set of skills or knowledge an individual or a group possesses; instead, expertise depends on outside recognition, which qualifes what experts do as "expertise." To be perceived as an expert, one needs to master "the disciplinary knowledge system composed of abstractions and general rules" (Eyal, 2019, p. 31) and possess an ability to explicate in line with these rules and within this knowledge system on issues familiar to them and on new challenges and problems. In short, according to Eyal (2019), expertise is a "historically specifc way of talking" and "doubly external," meaning that the expert status and its disciplinary knowledge and practice are derived independently of the expert (p. 31), and it is constructed in networks and connections between individuals (Eyal, 2013).

In today's world, expert knowledge is often expected to be based on scientifc knowledge or, as Holst and Molander (2017, p. 238) noted, on knowledge that is validated by scientifc norms and procedures. Although to be recognized as an expert one must operate according to scientifc norms of knowledge production, knowledge and expertise for the use of policy making are not necessarily provided only by academics. One can gain an expert position through practical experience as well. For example, civil servants with extensive work experience in a certain feld may have gained knowledge and skills that are considered "expertise" in that feld (Holst & Molander, 2019; Krick et al., 2019).

Due to complexity and global interconnectivity, governments frequently refer to expertise to legitimize and frame political decisions (Moore, 2017, p. 3). Legitimization can be the most signifcant motive for choosing the evidence, particularly in the case of unpopular reform proposals (Maasen & Weingart, 2009; Steiner-Khamsi, 2003, 2004). As a legitimization instrument, numbers have become particularly seductive. Teir power lies in their appearing neutral, apolitical, and objective. Still, any political narrative can be attached to them (Stone, 2016). Another appealing feature of numbers and quantitative indicators is that they make complex realities and processes appear simple and comparable (Espeland, 2015, p. 61).

Te increasing demand for evidence-based policy making and expert knowledge based on numbers is a familiar phenomenon in education policy making as well. Wiseman (2010, p. 1) stated that evidence-based policy making in education has become particularly popular since education became closely tied with the economic, social, and political status of modern nation-states. Tis has led to raised expectations related to education system outcomes. Measurable results are expected as a revenue for the public expenditure invested in education. Evidence-based policy making, according to Wiseman (2010, p. 1), rests on two underlying assumptions: (1) education is abstract and universal and (2) empirical evidence is an efcient indicator of knowledge and learning. Tese two notions combined have created the belief that one can fnd what works well in one context and apply it to another (Wiseman, 2010).

Te popularity of international large-scale student assessments stems from this kind of thinking and supports its further development (Gorur, 2016). Te knowledge and evidence used in education policy making are frequently, in fact, globally comparative and quantitative. Prior research demonstrates the infuence international organizations, such as the OECD and its international assessment instruments (e.g., PISA), have on nation-states' policy making in the feld of education (e.g., Costa, 2011; Grek, 2009; Nieman & Martens, 2018; Sellar & Lingard, 2013; Takayama, 2008; Waldow & Steiner-Khamsi, 2019). Te views researchers take on this development vary. Tere are those who praise traveling reforms as proof of policy learning and implementation of best practices, and those who view this development as a sign of global players imposing their own standards on national governments (Steiner-Khamsi, 2012, pp. 3–4). Te most critical voices have raised concerns about high-level strategic thinking being outsourced from the national to the transnational level and to supranational expert organizations (Lawn et al., 2011, p. 18).

Both Doreen Massey (2005) and Saskia Sassen (2007, 2013) proposed that we view national or local and international or global not as separate or layered. Tey argued that both local and global are constantly under construction and that the space of global/international is, in fact, produced within the place of local/national. Massey (2005, p. 9) suggested thinking of space as something constituted in both local and global and in the interaction between these two and as something fuid and in a constant state of becoming. Applying these thoughts to the feld of comparative education, Larsen and Beech (2014, p. 85) claimed that much of the previous research on education transfer was based on a static view of transfer, "as if ideas are produced in one site and then received in another context." On the contrary, Larsen and Beech (2014) described education transfer as the "movement of educational knowledge across space" (p. 76) and suggested that comparative education research adopt a theoretical framework that focuses on researching networks and connections within which educational knowledge is constructed and fowing. Returning back to expertise as a historically specifc way of speaking and constructed in interactions and interrelations as proposed by Eyal (2013, 2019), he further proposed focusing on networks as well. He claimed that to understand the way expertise is constructed and functions, we need to shift our attention from individuals to networks. Inspired by these discussions, we built our analysis on interactions between local and global and on networks of knowledge, individuals, and organizations.

## **Curriculum Process in the National and International Context**

Our case in point, the curriculum reform of 2014, was framed by the national policy process. Te main actors of the process were the Finnish government and the National Agency for Education, a government agency working under the auspices of Finland's Ministry of Education and Culture. Te Finnish government politically steered the process, as it is responsible for the general objectives of the National Core Curriculum and the distribution of lesson hours. Tis is legislated in a government decree on objectives and distribution of hours, which also delegates the power to decide on the core curriculum to the National Agency for Education (Valtioneuvoston asetus 422/2012). Te decree was issued based on a white paper. Te political nature of the process became apparent when the last National Core Curriculum reform was rebooted after the draft of the general objectives and distribution of lesson hours was met with objections inside the center-right coalition government in 2010 (Siekkinen, 2017). Subsequently in 2012, the right-left coalition government that followed began work on National Core Curriculum reforms. Te main governance organ in the process, the National Agency for Education, is responsible for curriculum planning and works together with the Ministry of Education and Culture to determine specifc objectives and content for subject-specifc and cross-curricular themes (Kujala & Hakala, 2020). In this work, evidence was obtained from green papers and experts. Open consultations can also be part of the process. Te National Agency for Education was responsible for coordinating a cooperative process with a broad selection of stakeholders to draft the latest curriculum (Kujala & Hakala, 2020). For instance, the working group for green paper 1, the 2010 document "Basic Education 2020," organized fve seminars specifcally targeted to certain expert groups; received feedback from two-thirds of the Finnish municipalities; heard from researchers, experts, and stakeholders on diferent occasions; and considered the survey responses of over 60,000 children and young people in which they shared their thoughts on what was good about the school as it existed and what changes they would like to see in the future.

Following completion of the fnal draft of the latest curriculum, on 22 December 2014 the National Agency for Education issued the new curriculum. Since August 2016, grades 1–6 have followed the new curriculum; grades 7–9 began implementation in 2017–2019. Representatives for the National Agency for Education introduced the new curriculum as built around "competences needed in society and working life" that aimed to change "the content of teaching, pedagogy and school practices" (Halinen et al., 2014). According to Uljens and Rajakaltio (2017), the new curriculum follows the competence-based idea and key competencies promoted by the OECD. However, the competence orientation is less radical in comparison to curricula of other countries, such as Norway (Mølstad & Karseth, 2016; Sivesind et al., 2016).

Te policy development process for Finland's National Core Curriculum has been in fux. Te main structural change occurred in the 1990s. During a period of decentralization, legislation was changed to increase the autonomy of municipalities on education issues (Kuntalaki 365/1995, 1995; Laki peruskoululain muuttamisesta 707/1992, 1992). School inspections were gradually abolished (Varjo et al., 2016), and legislative restrictions on school choice were again tightened in 1998 (Ahonen, 2003, pp. 180–192; Seppänen, 2006, pp. 66–71). As a result, the curriculum has a dual character in steering. On one hand, the National Core Curriculum is the main content steering instrument in Finnish comprehensive education policy, and schools are legally bound to follow it. Te Basic Education Act (628/1998, §30) states that "an enrolled pupil shall be entitled to teaching according to the curriculum." In practice, the National Core Curriculum obligates the provider of education (in most cases, municipalities) to include its central aims as part of the educational program. On the other hand, broad degrees of freedom are at play in the implementation of the curriculum. Te providers of education and schools can draw on the National Core Curriculum to create their own curricula. Importantly, apart from formal complaints, no direct methods for monitoring the implementation of the curriculum exist, which leaves much autonomy for teachers and schools.

Te recurring debate in comparative education addressed in this volume is the extent to which education is national or international. Te degree of divergence or isomorphic convergence of national policies (e.g., Meyer et al., 1997; Steiner-Khamsi, 2010, p. 332) and the calls for reshaping or deconstructing the understanding of these nation-based categories (e.g., Kettunen, 2011; Robertson et al., 2002; Werner & Zimmermann, 2006) are examples of this debate. Te central questions circle around defning the role of the state and whether it is a central player in the globalizing world. Te corpus of Finnish research on infuences in education policy is characterized by the tension between these two arguments existing at the same time but pulling in opposite directions. Te analysis of these infuences in education policies has focused on the relation of Finland to international organizations. Te frst argument sees the international infuences important in shaping the national policies, fnding the OECD especially important in this role; however, no clear agreement has been reached on how and what infuence is channeled. Te second argument supports the state-centeredness of Finnish policy making.

In relation to the argument supporting the infuence of international organizations, interviews with top ofcials in the Ministry of Education and Culture indicate the close relationship the Ministry shares with the OECD (Niukko, 2006). In higher education, major reforms have frequently been preceded by an OECD investigation (Kallo, 2009). In some instances, the infuence is thought to be rather direct, as Rinne and Simola (2005, p. 16) pointed out that quality discourses are "directly from the arsenal of the EU and the OECD," and Kauko and Varjo (2008) have observed Finland riding in the OECD's slipstream. Ten again, Kallo (2009, p. 357) understood the infuence to be more epistemic: the OECD forms an epistemic community, the power of which is derived from deeply rooted networks. Moisio (2014) noted that in higher education policy making, Finland has resorted to a "policy spin," where national goals are fed back into the Finnish system via the EU. Naumanen and Rinne (2008) demonstrated that the national goals are not always far from European or OECD objectives. Terefore, the frst argument is characterized by the main deduction of a high-functioning and rather technically favored network, ending in infuences foating to national policy space.

In relation to the argument for state-centeredness, a historical overview reveals how intensively the state became involved in education. In simplifed terms, the grand change in nineteenth-century education in Finland was its secularization from the church-led system, while the primary change in the twentieth century was the nationalization and municipalization of education (Joutsivuo, 2010; Leino-Kaukiainen & Heikkinen, 2011). Te state-centered tradition in Finnish education politics is related to the strong state institutions created in the post-war period that are responsible for education and to the strong dependency that educational institutions and providers have to them. Statecenteredness is aligned with what has been recognized as the ideal of the universalistic Nordic welfare state (Esping-Andersen, 1990). Te buildup of the education system was a state-centered process in which comprehensive schools were planned as a large societal project to support equality and economic growth (e.g., Ahonen, 2003), while at the same time public higher education was subject to regionalization and massifcation (e.g., Lampinen, 2003). Simultaneously, the state institutions grew in importance. Te post-war "post-ofce size" (Kivinen et al., 1990, p. 39) Ministry of Education and Culture budget sector became the third largest among the ministries. Te strong role of the state was slightly reorganized at the time of the global and Nordic (Dovemark et al., 2018) management reforms, where public governance in Finland was reformed to a strategic and managerial style (Autio, 1997; Temmes, 1996). As part of the international trends, the 1990s brought a move toward decentralization and deregulation, which changed the steering system dramatically and gave responsibility to the municipalities (Simola et al., 2013). However, the Ministry of Education and Culture is still recognized as a central if not the most central power hub in Finnish education policy making. Research has documented its role as a bureaucratic-led and independent actor rather than a politically steered organization (Kivinen et al., 1990, p. 103; Lampinen, 2003, pp. 162–200; Lehtisalo & Raivola, 1999, pp. 122–123). However, research has also identifed that the general national policy steering tends to supersede the education-based policy steering signals (Kallunki et al., 2015; Seppänen et al., 2019), which does not diminish the argument of a state-centered system but, rather, supports it. In sum, the second argument sees the long-term growth of the state system.

Te analysis of the policy process in Finland has revealed the links between the state bodies and the international organizations. Te two partly opposing lines of argumentation provided by previous research on Finnish education policy and politics gives a rich picture of a system that traditionally has been strongly state led, yet has also become part of a global policy space and network from which infuences foat to national policy space. However, this picture is still rather dichotomous and at least partly based on the idea of ideas fowing from one level (international and global) to another (national or local), and previous research has revealed also more interactive processes (Centeno, 2017). Our aim is to surpass this dichotomy and to complement the picture by focusing on networks of evidence and expertise, as Larsen and Beech (2014) and Eyal (2013, 2019) proposed. In the process we aim to discover, along the line of Massey (2005) and Sassen (2007, 2013), how global is constructed in local and vice versa.

## **Research Design**

#### **Research Question and Methods of Analysis**

In this chapter we set out to explore how evidence was used in Finland's 2014 reform of the National Core Curriculum and whose evidence was most highly valued in this process. As the starting point for our methodological approach, we chose to apply Paul Cairney's (2016) defnition of evidence as "an argument or assertion backed by information (p. 3)". We share the view that Gita Steiner-Khamsi presented in Chap. 2 that the importance of examining bibliometric references is crucial since these references in the policy documents are used to "provide legitimacy to the evidence which the author has provided in the document." Hence, references can be seen as validation of evidence, so we applied a bibliometric network analysis as our frst method in this chapter. In the bibliometric network analysis, we focused on the 677 bibliometric references used in the ten core policy documents that constituted our database. Tese references were analyzed with the software programs UCINET and Netdraw, which generated descriptive statistics and an illustrative fgure of knowledge networks. Te statistics were used to examine what kinds of evidence (location of the publication; type of publication) and expertise (author of the publication) were used in the reform. Te visualization of the networks was used to illustrate the political process, the knowledge network, and evidence base of the reform.

We double-checked and complemented these fndings with a content analysis of the ten documents selected for investigation. We inspected the ten core documents, looking for any references to the OECD and PISA that appeared within the text but were not references to actual sources nor included in the reference list. Finally, we examined each of the most cited OECD documents and identifed the policy documents in which they were used.

To determine whose evidence was most highly valued in this reform, we additionally investigated the organizations that employed the most prominent authors at the time they authored these documents. Te top 18 of the most cited individual authors were selected and their literary works and the year of publishing that were found in the database were listed chronologically, after which their employers at the time that they wrote each text were researched using a variety of sources, such as other publications from the same year that had the author's position and organization cited, social media profles, publications and newsletters from the employing organizations, newspaper articles, worker profle pages of universities and other organizations, and even biographies in the case of some of the more experienced authors. In addition, we analyzed the titles of all 677 publications in our sample and singled out the titles of the publications of the most cited authors for closer analysis.

## **Selection of Data**

As there is not a universally agreed upon concept of a white paper (WP) or a green paper (GP), an interpretation was made that a WP constitutes a draft for new legislation, while GPs provide background information. Our focus of analysis for the main document was a WP produced by the ministry-assigned working group. Te WP "Future Basic Education" identifed the general national objectives, presented a proposal for renewed distribution of lesson hours, and suggested the necessary decree changes for the Government Statute on the National Education Objectives Referred to in the Basic Educational Act and the Distribution of Lesson Hours in Basic Education (Valtioneuvoston asetus 422/2012, 2012). From the in-text citations typical of the Ministry of Education and Culture documents, several documents were discovered, nine of which were identifed as GPs suitable for analysis.

Our criteria for determining relevance primarily consisted of three factors: (1) the document provided relevant information and suggestions for the curriculum reform, (2) the document contained a list of references or references in footnotes, and (3) the document was related to the WP. Te input of data was completed in two stages: the primary input stage, during which data from the selected sources were coded, and the secondary stage, during which the coded data were cleaned and corrected. Some subsequent data-cleaning was completed later when the need arose, but those changes were minor compared to the second stage and did not alter the original fndings.

Te coding process followed the original plans for creating a comparable database. In the Finnish coding process, some clarifcations had to be made, especially when categorizing the documents as "book," "report," "journal article," or "government-issued report." Te categorization of document types was problematic for two reasons: the National Agency for Education conducts plenty of scientifc research on its own and in conjunction with certain Finnish universities, most of which is then published under the agency's name, making it difcult to distinguish the level of government involvement at times. Second, many faculty in Finnish universities have their own publication series where studies conducted by the members of the faculty are presented, often in a book form. Many of these studies are peer-reviewed, making the exact categorization of the publication series difcult. Tese issues were solved as follows: any studies published by the National Agency for Education were coded as a "government-issued report," and the publication series were coded as either a "book" or a "report," depending on the form of the publication, as they are not academic journals in the strictest sense.

Listed in Table 5.1 are all the documents selected for the analysis. All the selected documents were written in Finnish; their English translations follow the document names in parentheses. Troughout the rest of this chapter, tables with Finnish words and names have their translations or explanations in parentheses for the ease of readability.



*Notes*: English translation of document names in parentheses. WP = White Paper; GP = Green Paper

## **Results**

## **Evidence Base of the Reform**

Te evidence base of the reform consisted of 677 referenced documents. Based on prior research, both international and Finnish, we expected to see signifcant use of international sources for policy evidence. Based on the prominence of the OECD in producing quantitative comparative data, the organization was anticipated to be especially prominent, and the evidence in this policy process was expected to be extensively drawn from PISA and other large-scale student assessments. On the contrary, however, the Finnish data indicate a strong state involvement and concentration of expertise in state organizations, namely, the Ministry of Education and Culture (that produced the WP), and especially, the National Agency of Education (that produced or commissioned the GPs in our sample). Table 5.2 depicts the distribution of references according to the location of publication (domestic, regional/Nordic, international) and the type of publication (report, book, journal article, governmental, other).

Of all references included in the ten policy documents, 76% are domestic, that is, published in Finland, and 22% are international, while only 1.6% were published in other Nordic countries, or regionally. In three of the ten policy documents, all references used are domestic. An analysis of the type of publications reveals that very little scientifc evidence was used in the process. Te percentage of journal articles is very low, only 9.45% in total. Almost no scientifc evidence in the strictest sense of the defnition was used in the WP (document ID 66), as the percentage of peer-reviewed academic journal articles is 0%. However, it is important to remember that, as explained previously (see "Selection of Data" section), the categorization of publications in the Finnish case was challenging. On one hand, the National Agency of Education publishes books that are written or co-authored by universities and researchers and comply with scientifc norms. On the other hand, universities publish book series that are not peer-reviewed in the strictest sense but, nevertheless, are academic publications. As we chose to follow the strictest possible interpretation of peer-reviewed academic publications (categorized as "journal articles"), any studies published by the National Agency for Education were coded as "government-issued report," and the publication series were coded as either a "book" or a "report," depending on the form of the publication, as they are not academic journals in the strictest sense.

Te network analysis reveals that the evidence base of the WP (in the bottom left hand corner of Fig. 5.1, document ID 66) to a great extent is based on the evidence base of GP 1 "Basic Education 2020," shown in the middle of the fgure. In fact, the evidence base of the GP appears to be more central and versatile in this reform than that of the WP itself. It can also be concluded that in this reform, GP 1 "Basic Education 2020" serves as an "intermediary" (Lubienski, 2019), as it connects the

**Table 5.2** Reference distribution


**Fig. 5.1** Complete network structure. (*Notes*: Regional: gray, domestic: white, international: black; *source*: Circle; node size=in-degree centrality)

knowledge network of the WP to that of the other GPs. Te link strengths are illustrated in Fig. 5.2.

Te WP (66) is linked only to two other documents, GP 1 "Basic Education 2020" (1) and the GP 6 (6) "Evaluation of the curriculum of pre-school and primary education." Te link between the curriculum evaluation GP and the WP is rather weak, indicating they share some of the same sources but not many. However, the link between the "Basic Education 2020" GP and the WP is strong, indicating they share largely the same sources and, hence, the same evidence base. To summarize, the evidence base of this reform appears to be predominantly domestic, and the WP and GP 1 share a signifcant portion of their evidence base. Furthermore, GP 1 serves as an intermediary and is the most central document in the network.

**Fig. 5.2** Source document network. (*Note*: Link strength is based on the number of references shared by two sources)



## **Whose Evidence Is Most Highly Valued?**

As discussed previously, the evidence base of this reform appears to be overwhelmingly domestic. A closer look at the publishers illustrates that the evidence base is not just domestic, but it is also exceedingly statecentered. Table 5.3 provides a list of the top ten most cited publishers.

#### **5 Evidence and Expert Power in Finnish Education Policy…**

Of the 395 publications on the most cited publishers list, 43% (170) were published by the National Agency of Education, and 14.5% (57) were published by the Ministry of Education and Culture. Terefore, of the documents published by the top ten most cited publishers, 57.5% (227) were published by one of these two government organizations. If the documents published by the Finnish government (9) and the State Printing Center (8) are added, the total amount published by a state/ government organization rises to 244 publications, which is 62% of the documents published by the top ten most cited publishers. Tis indicates a strong state involvement in the curriculum reform process and demonstrates that the evidence base of this reform was national, state-centered, and to a great extent, self-referential in nature.

On this list, the University of Jyväskylä ranks third and the University of Helsinki ranks fourth. As described in Chap. 11 (Ydesen, Kauko, Magnúsdóttir), these two universities hold a specifc and special position in Finnish education policy making. Te Finnish Institute of Educational Research at the University of Jyväskylä and the Centre for Educational Assessment at the University of Helsinki have been, in turns, contracted by the Ministry of Education and Culture to implement PISA. Te Finnish Institute of Educational Research has also been responsible for the OECD Teaching and Learning International Survey (TALIS). Tey are both main hubs for OECD data expertise in Finland. In this particular curriculum reform, they were also among the main publishers of the evidence used in the reform. Tey are the only universities in this top ten list, indicating that it is not so much the scientifc evidence but the evaluation expertise, and in particular, the OECD data expertise, that was valued in this reform. Hence, it seems that the evidence used in this reform is domestic and state-centered, and it is particularly assessment and evaluation results that were valued as evidence. A closer look at the most cited authors further strengthens this argument (see Table 5.4).

As on the list of the top ten most cited publishers, the National Agency of Education and the Ministry of Education and Culture occupy the two top positions on the list of the top ten most cited authors. Both are referenced as authors almost three times as often as, for example, the OECD. Even individual authors, like Jakku-Sihvonen, a former employee of the National Agency of Education, are referenced almost as many


*Note*: Cutoff point: minimum fve times

times as the OECD. Such individual authors can be seen as top experts in this particular reform but possibly also in the feld of Finnish education policy in general. A closer look at the background of these top experts reveals that many of them were, at the time they authored the documents that were referenced in this policy process, employed by the National Agency of Education (5) or by the two universities that traditionally work closely with the Agency—the University of Jyväskylä (6) and the University of Helsinki (6).

Tis demonstrates that the Ministry of Education and Culture and the National Agency of Education used, for the most part for this reform, evidence produced within their organizations or by the organizations they have a close relation with. Tis indicates that the policy evidence in this reform was largely self-referential and the policy process frmly steered by national government organizations. Tese fndings also highlight the strong expert position and power of the National Agency of Education in the feld of education policy and politics in Finland.

On the basis of these fndings, it would be easy to conclude that policy making in Finland is, indeed, state-centered, the evidence base predominantly domestic, and the infuence of international evidence and transnational organizations trivial. However, a closer look at the policy documents in our sample tells a slightly diferent story.

Firstly, although the OECD amounts to only 3% of the publications in the ranking of the top ten most cited publishers, and in the ranking of most cited authors it has 14 publications amounting to only 0.05%, it is important to note in which of the ten core policy documents these OECD documents were used.

Te OECD references appear mainly in the WP and GP 1 "Basic Education 2020." Te GP 1 was originally intended to be a WP, as explained in more detail earlier in "Curriculum Process in the National and International Context" in this chapter. Tough the OECD references are not used in all of the policy documents in our sample, and the OECD documents are cited far less frequently than domestic references, OECD references are used in the policy documents that were most infuential in terms of policy design and selection of the actual content of the curriculum reform. Tus, one can state that the OECD was, in fact, a signifcant component in the evidence base for this reform and that the OECD evidence was considered valuable.

Secondly, earlier research as referred to previously in this chapter has demonstrated the infuence of large-scale student assessments (e.g., PISA) and the transnational organizations behind them in the national level of education policy making. Tese organizations and their standardized tests promote, in particular, a focus on skills, student achievements, and evaluation of outcomes. In our sample, all GPs, apart from GP 1 "Basic Education 2020," are evaluations of some sort. Five are national assessments of learning outcomes. Moreover, 77 documents in our database (according to their titles) address topics related to evaluation or are evaluation reports of diferent subject areas and skills. It does seem that what was valued most as evidence in this reform are evaluations of skills and outcomes, indicating the evaluation and assessment culture promoted by the OECD does have an infuence in Finnish policy making as well.

Tirdly, a content analysis of these eight GPs reveals that references to PISA are sometimes used in the text, even though no actual source is given as a reference in the text or in the list of references. On the other hand, the references may be to publications of Finnish experts and not directly to the publications by the OECD. Tis can be seen in GP 3, an evaluation on mathematics skills. Paragraph 1.3 in this report is dedicated to PISA results. In the rest of the report, the OECD is mentioned fve times in the whole document, "OECD-countries" are mentioned fve times, and PISA receives 24 mentions—19 in the text and 5 in the references. PISA is used as evidence within the text throughout this national evaluation report, but the references used pertaining to PISA results or the OECD are domestic references. In other words, the references come from three domestically produced and published documents, one by the Ministry of Education and Culture and two by the PISA experts of the University of Jyväskylä.

In addition, more fndings must be considered when discussing the role and infuence of the OECD and the evaluation culture it promotes. A signifcant portion of the publications of the most cited authors are national evaluation reports or reports written based on either national or international assessments. For instance, authors Lappalainen and Mattila are very high up on the list of most published authors. Te six publications for which Lappalainen was the frst or only author concern the national evaluations of learning outcomes in Finnish language as the mother tongue and in literature. Te publications with Mattila as frst author discuss national evaluations of learning outcomes in mathematics. Tis indicates that evaluation expertise and evidence especially were valued in this reform. Finally, one specialized form of this kind of expertise is the ability to interpret and translate OECD data so that it fts the national policy context and needs. Among the most cited authors are for instance, Välijärvi, Kupari, and Arinen, each of whom belongs to this group of experts, their cited publications being predominantly Finnish language reports on PISA results.

## **Discussion**

Our analysis of the evidence used in the curriculum reform 2014 reveals a strong tendency to use not only domestic evidence, but evidence provided by the two most powerful state organizations in the landscape of Finnish education policy making: the Ministry of Education and Culture and the National Agency for Education. Tese two organizations hold a strong, legally established position as the most prominent expert organizations in Finnish policy making in terms of comprehensive education and the National Core Curriculum. In the curriculum reform of 2014, they produced most of the nine core policy documents according to their mandate; the remaining documents were commissioned by the National Agency for Education. Te working groups consisted, to a great extent, of the civil servants of either the Ministry of Education and Culture or the National Agency for Education, although the working group for "Basic Education 2020" from 2010 included representatives of main political parties and the most prominent stakeholder organizations as well. In addition, the bibliometric analysis of the cited references demonstrates that the working groups behind these policy papers chose predominantly to include evidence coming from within these two organizations as references in their reports, or alternatively, documents provided by organizations that have a strong and extended relationship with these two organizations as the contracted institutions for implementing PISA in Finland.

Our fndings suggest, however, that there is more to the story. Tough we found that the evidence used in the curriculum reform of 2014 was predominantly domestic and self-referential, this fnding does not paint a correct picture of the infuence of transnational actors, particularly the OECD. Te content analysis revealed that PISA was referred to in the key policy documents, but sometimes without a bibliometric reference to the OECD or PISA. In addition, a signifcant part of the references categorized as domestic in our sample were, in fact, reports on the PISA results written by Finnish authors and published in Finland. It seems the OECD data and, particularly, PISA indicators were, indeed, used as evidence in this reform, but this infuence was fltered through domestic experts and expertise so that in the bibliometric analysis it appears mostly as domestic evidence.

Te OECD data in this reform were especially used in the two most infuential papers in terms of actual policy design: the WP and GP 1, which, when drafted, was intended as a WP but was discarded after the political power changed from the center-right to right-left. Tis indicates that the importance of the OECD and its international assessment instruments cannot be measured merely by looking at the number of references used in the policy documents. In the case of the curriculum reform of 2014, it seems that the OECD data and references came into play particularly in the policy documents that carried the most weight in the ofcial policy design. Tis may indicate that the OECD evidence was used primarily to legitimize the policy recommendations and design, but this theory would need to be researched further. Our results, however, support the ideas of Deborah Stone (2016) and Wendy Espeland (2015) that numbers appear objective and apolitical, making them particularly appealing to policymakers. Sometimes these numbers from diferent evaluation reports come from within the local and national context and other times from the global and international context. Nevertheless, the hierarchization of evidence Gita Steiner-Khamsi discusses in Chap. 2 is undeniably evident in our example of an education reform as well.

In terms of expertise, what seemed to count in the main reform documents was, in fact, expertise in evaluation and assessment. Te evidence that was valued was not necessarily scientifc, but it was predominantly empirical. Tough we did not specifcally focus on narratives attached to numbers, it became clear to us that the preference in this reform was for a certain kind of evidence. Te information used was to a large extent empirical evaluation data, partly stemming from the OECD and its PISA indicators. Tis is very much in line with Wiseman's (2010) claim that the evidence agenda in education is based on an underlying assumption that empirical evidence is an efcient indicator of knowledge and learning. However, it was often only after the empirical data was fltered through the expertise of the core national actors that the data became policy evidence. Te international data were many times fltered through Finnish ofcials' publications or by Finnish institutions.

Holst and Molander (2019) claimed that expertise does not necessarily have to be scientifc. It can also be built on professional knowledge gained through long working experience in the feld. Te experts providing (most cited authors) and selecting (members of the working groups and the most cited authors) evidence in this reform were both scientists and civil servants with extensive experience in the feld. What they frequently had in common, however, was expertise in assessing learning outcomes or in interpreting the results of large-scale student assessments. Our fndings indicate that the ability to select, interpret, and translate this kind of data so that they ft the national context is what gives specifc national actors the authority to speak with the authority of an expert. Most had expertise in explicating evaluation data, either national or international. Hence, the historically specifc way of talking that Eyal (2019) referred to appears, in this case, to be the ability to speak numbers. Te double externality of expertise (Eyal, 2019) in this process is evident in the fact that this particular form of expert talk was combined with the already established expert position of the individual authors as representatives of the most prominent power hubs in Finnish education policy making and politics. In fact, our results illustrate that expertise in education policy in Finland is highly concentrated in a few organizations that work closely together. It is this expert position that appears to give these organizations and their most prominent experts rather sovereign power to determine what constitutes policy evidence.

We began our chapter with a discussion on the role of the state in Finland's education policy making and whether the infuence of international organizations has increased to the point that high-level expertise (and national decision-making along with it) is outsourced to the transnational level. Our research has demonstrated that education policy making, at least at the comprehensive education level in Finland, is primarily in the hands of two state organizations—the Ministry of Education and Culture and the National Agency for Education. We found no evidence of the transnational level overriding national expertise. In fact, the national players appeared to be a powerful flter between the global and local. Although part of the evidence was produced on the transnational level (particularly by the OECD), the selection of evidence for policy decisions was carried out on the national level and by domestic experts. Also, part of the expertise of the national players consisted of explicating international evidence so that it ft the national context. Te case we examined indicates that the international or global has the power to produce evidence but the national or local has the power to select the evidence and adjust it to meet the national needs. In fact, the type of evidence appeared to be more important than where the evidence originated from (local or global level). Both national and international evidence were used in this reform, but the key trait of the evidence was that it largely comprised empirical evaluation data. Similarly, the ability to "speak empirical evidence" seemed to be a key characteristic of the experts involved. Tis could include both domestic and international evidence, but the role of national experts in selecting, translating, and possibly even modifying the evidence produced by global players remained central in the process. Tis indicates that much like Larsen and Beech (2014) suggested, in the current era of global education transfer, the layered approach of researching comparative education no longer applies. Educational knowledge, as proposed by Larsen and Beech (2014), moves across the global space and through globally connected experts and expert organizations.

## **Conclusions**

In this chapter we set out to investigate what kind of evidence the reform of the Finnish 2014 National Core Curriculum drew on and whose evidence was most highly valued in the process. We aimed to answer this question with a bibliometric network analysis, complemented with content analyses of the ten policy documents in our sample. Our database consisted of 677 cited references in these documents. Our starting point was the recurring debate in comparative education regarding the extent to which education is "national" or "international" and the tension of the two main arguments related to this debate in the Finnish research corpus. We built our interpretative framework around the concept of expertise as a historically specifc way of talking, doubly external, and constituted in networks and connections between individuals, as theorized by Gil Eyal (2019), and around the discussion of the interplay between global and local in comparative education research (Larsen & Beech, 2014) inspired by Doreen Massey (2005) and Saskia Sassen (2007, 2013). Our main methods were bibliometric network and qualitative content analyses.

At frst glance, the state-centeredness appeared evident, as most evidence fell into the category of "domestic" in the bibliometric network analysis. A closer look revealed that reform was, in fact, based on both domestic and international evaluation evidence. Te expert power throughout the process was, nevertheless, frmly held in the hands of a rather small, domestic network of experts. Teir power appears to lie mainly in two areas: (1) the power to select the information relevant for policy evidence (as evidence is defned by Cairney, 2016) and the ability to explicate empirical evaluation data to cater domestic needs, as our fndings suggest that "the historically specifc way of talking" (Eyal, 2019, p. 31) that is needed for recognition of true expertise equates to speaking the language of evaluations and numbers; and (2) their relationship to either the Ministry of Education and Culture or the National Agency for Education, as these experts were either employed by these two most infuential organizations in Finnish education policy or were working for one of the institutions traditionally closely linked to these organizations through their special role administrating and interpreting OECD's PISA results.

Although international evidence was used in the process, our fndings do not support the most critical predictions that high-level expertise is being outsourced to the transnational level. At least in this reform process, the Ministry of Education and Culture stayed frmly in the steering wheel as a major hub of education policy expertise in Finland. Even the change in political power during the curriculum reform had little infuence on the reform led by the Ministry and its civil servants. With the new government, the curriculum reform stayed its course, and the work continued with the once discarded policy document as the basis, regardless of the political changes that surrounded it. Tough the process was state led, our fndings do not support unanimously the claim for state-centeredness, either. Much of the evidence, particularly in the most infuential policy documents in terms of policy design, came from the OECD. Additionally, the expert position in the process was frequently based on expertise in explicating transnational evidence.

All in all, our fndings indicate that the two layers of local and global are not separate or distinct but interconnected and intertwined. One gains its power from the other. Policy evidence may, in fact, function both as a legitimation tool for reform and as the source of strengthening expert power. In the process it may not be the state itself that is the central player in education policy, although it ofcially leads it. Much power may also be found working through and in the networks of experts. Te experts are often national but they draw their expertise from the networks based on both local and global knowledge and connections. In the globalized world these networks cannot be traced back to one place but instead play out their infuence in the global policy space that is constructed only partly within the national.

**Acknowledgments** Te authors would like to thank Elina Kuparinen (University of Tampere) for her extensive work in identifying and coding the bibliometric references, and Dr. Chanwoong Baek (University of Oslo) for providing us with the fgures and tables used in this chapter.

## **References**


*International Student Assessment (PISA) as regulation instrument in education policy]*. [Doctoral dissertation]. Instituto de Educação—Universidade de Lisboa.


H. Simola (Eds.), *Fabricating quality in education. Data and governance in Europe* (pp. 11–1e8). Routledge.


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **6**

## **The Irregular Formation of State Policy Documents in the Icelandic Field of Education 2013–2017**

**Berglind Rós Magnúsdóttir and Jón Torf Jónasson**

In recent decades, the role of the state in the governance of education has been changing, as program evaluations, best practices, and large-scale transnational student assessments have visibly shaped policy making in the feld. Tis development has been connected to such conceptual frameworks as governance by numbers, evidence-based policy, externalization of national policy, scientifcation, and neo-managerialism (Lingard, Martino et al., 2013; Lingard, Sellar et al., 2014; Robertson, 2016; Dovemark et al., 2018; Steiner-Khamsi, 2016). Te purpose of this chapter is to explore state policy formation in Iceland by analyzing three recent state documents, specifcally examining the following: (a) their antecedents, procedures, and follow-up; (b) their knowledge providers; and (c) the kind of knowledge that is interpreted and recontextualized into Icelandic documents.

Early in this process it became clear that the processes guiding education policy formation in Iceland difer from the Nordic protocol that

B. R. Magnúsdóttir (\*) • J. T. Jónasson

University of Iceland, Reykjavík, Iceland

e-mail: brm@hi.is; jtj@hi.is

<sup>©</sup> Te Author(s) 2022 **149**

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_6

relies on available green papers (GPs) and white papers (WPs) with reference lists as the main data sources (Steiner-Khamsi et al., 2020). Tus, the frst step in the process for this research was to locate documents that would fulfll the criteria of WPs and GPs as understood in the research project (POLNET). We found three documents that could be considered appropriate for the research protocol. In this comparative research project, references are understood as a policy tool to justify or authorize the content of reform (Steiner-Khamsi, Chap. 2 in this volume). Additionally, we obtained deeper phenomenological understanding of the government procedure by analyzing the content of the documents and of qualitative interviews with fve ofcials at the Ministry of Education, Science and Culture. Due to the scarcity of GPs to examine for the research, we felt it was important to conduct the interviews to elicit a clearer picture of the knowledge accumulation inside the Ministry.

Te main purpose of this chapter is to explore and compare two visionary policy documents and one background paper relevant to the reform of 2013–2017 to determine what counts as evidence at the stage of agenda-setting and policy formulation and identify the main knowledge providers according to reference lists. Governmental legal frameworks originate from the Ministry of Education, Science and Culture and gain legal status in the education system through acceptance from parliament. Visionary policy documents, on the other hand, outline what the government hopes to achieve and the knowledge, methods, and principles it will use to achieve them. Tey state the goals of the Ministry of Education, Science and Culture and local municipal governments.

Te two visionary policy documents under review in this chapter were both processed by the Minister of the Independence Party, Mr. Illugi Gunnarsson. Te earlier of the two (WP2014) was written and classifed formally as a WP and is the only document that largely fulflls all criteria for a WP (Ministry of Education, Science and Culture, 2014a). Te more recent policy document (WP2017) began as an audit on inclusive education, with background material and guidelines presented by the Ministry of Education, Science and Culture, but was adopted as a formal policy initiative (European Agency for Special Needs and Inclusive Education, 2017). Terefore, we also treat this as a WP. Te third document reviewed in this chapter is a background paper prepared under the auspices of the Ministry of Education, Science and Culture for an intended Organisation for Economic Co-Operation and Development (OECD) review on resources for the Icelandic schools, which never materialized. Still, the document is often quoted in policy discourse, and we treat it as a GP (Ministry of Education, Science and Culture, 2014b) and it was cited in WP2017.

Tese texts are important examples of the formal documentation that now governs or at least infuences Icelandic compulsory education, along with laws, regulations, and curricula that are considered of primary importance. Te document that receives the most attention in our analysis is the WP from 2014, which was written explicitly and exclusively as a policy document. Te other two are included to explore whether a very diferent picture is obtained with respect to the substantive underpinning of evidence, externalization, and governance procedure. Moreover, it was necessary to review these additional texts to obtain a more comprehensive understanding of what can count as evidence and of the formal governing practice for an issued policy document in the feld of education in Iceland than could have been gleaned from an analysis of any one single document.

Te chapter begins with an introduction to the Icelandic feld of education and governance, followed by methodological considerations for this investigation and then a presentation of the results. Te empirical portion of the chapter is divided into two sections. Te frst lays out the numerical pattern of the database. Its subsections go deeper into the texts through a content analysis of the WPs by exploring the actual use of references in the text. Te second section deals with the governing practice itself within the Ministry of Education, Science and Culture, focusing on the agenda setting, the power relations between the politics (Minister) and the administration (ofcials), the given time frame, and choices of structure, knowledge providers, and evidence. Te qualitative interviews on the governance practice were conducted based on the results from the frst empirical section, specifcally, the scarcity of written documents to produce a knowledge base for policy making and the numerical pattern in the evidence base.

## **Iceland: The Research Field**

To understand the present, we need to provide insight into the past in terms of educational reforms and governance concerning the themes of the two WPs analyzed—inclusion and literacy.

## **A Historical Account of Education Reforms in Iceland: Social Justice, Inclusion, and Literacy**

Te history of public schools in Iceland is relatively short compared to the public school histories of other Nordic countries. Te frst compulsory school reform in Iceland was led by Guðmundur Finnbogason, who was appointed by the Icelandic government to write a report, a background paper, to prepare for the passage of the country's frst bill on education. He visited progressive schools in Scandinavian countries to gather ideas and, in 1903, published his report as a book, as he felt it was important to inform not only the government of Iceland but the public as well about his ideas (Finnbogason, 1903/1994; Guttormsson, 2008). Te frst act on public schooling was passed in 1907. However, the legally unifed, compulsory, and comprehensive school did not exist in the law until 1946 (Lög um skólakerf og fræðsluskyldu, 1946). Te government viewed the educational system as "fundamental to the newly found independence of the nation from Denmark which required an educated public capable of running and administering a modern state" (Halldórsdóttir, Jónsson, & Magnúsdóttir, 2016, p. 438).

Te policy focus on literacy is in line with a strong emphasis on reading profciency in Icelandic educational history. Te major inspection eforts that were undertaken in the 1740s and later in the 1930s had their primary focus on reading (Guttormsson, 2008). From the beginning of public schooling in Iceland in the early twentieth century, reading scores in terms of speed were used to track and stream in the early grades (Garðarsdóttir, 2001). Indeed achievement in reading has been strongly related to the intersections of social class, residence, and gender in Iceland (Gísladóttir et al., 2019). As such, immigrant and refugee children in Iceland are the "new" disadvantaged group (Garðarsdóttir & Hauksson, 2011; Harðardóttir et al., 2020). Reading profciency has, for centuries, been the most critical reference point for educational quality, equity, and progress. What is new, however, is that the aim for higher achievement in reading is now put forward in a visionary policy document (Ministry of Education, 2014a).

Te 1974 Education Act (63/1974) was progressive in terms of highlighting democracy and inclusion as core purposes, when all children, including those with special needs and disabilities, were required to attend school (Marinósson & Bjarnason, 2014). Tis act represented a fundamental political change led by a former Minister of Education, Dr. Gylf Þ Gíslason, a member of the Social-Democratic Party (i. Alþýðufokkur), and his special advisor, Wolfgang Edelstein (Edelstein, 1988/2013). Jóhannesson (2006, p. 105) described the changes as based on "child-centred, humanistic, and egalitarian views … apparent in cooperative learning methods, integration of subject matter, evaluation as a process rather than a product, and many other 'progressive' views in education." Many of these progressive ideas and views that articulated shared notions of/preferences for certain ideas and values are still discursive themes in the current Act of 2008 (*Lög um grunnskóla*, 2008), despite its neoliberal orientation (Schriewer, 2003). Tis reform had a strong mandate and research-oriented focus within the Ministry itself (i. Skólarannsóknardeild) and involved interactive cooperation with teachers to develop new curriculum materials (Edelstein, 1988/2013; Halldórsdóttir et al., 2016).

Te last reform prior to the period under review in this paper was the Education Act 1995, followed by a new curriculum in 1999. Te emphasis then was on decentralization, neo-managerialism, and individualization in the diagnosis of learning disabilities (Jóhannesson, 2006). Te period under review was initiated with a new act on compulsory schools and process for development of a new curriculum, which was led by Ms. Þorgerður K. Gunnarsdóttir, a member of the Independence Party. Te only Minister who was not part of the Independence Party during 2008–2017 was Ms. Katrín Jakobsdóttir from the Left-Green Movement. She issued the curriculum for all school levels.

Te Education Act from 2008 and the 2011/2013 curriculum guides are still in force, and we note only one substantial addition—the Education Act (2015)—to establish a Directorate of Education.1

After the fnancial collapse in 2008, Iceland's two political parties on the left formed a coalition and became responsible for dealing with the aftermath. Table 6.1 gives an overview of these political turbulences after the fnancial collapse in the year 2008 and which political parties and ministers were responsible for laws, regulations, curriculum and other policy papers during the period under review in this chapter. Te frst part of the fundamental educational reform had already been issued in the 2008 Education Act of All School Levels, and drafts of the new curricula were available on the Ministry of Education, Science and Culture website. Te discourse in the legal documents was focused on the competitiveness and quality of the Icelandic education system with in-text citations to the European Commission (European Qualifcation Framework) and institutions like the OECD (Alþingi [Parliament], 2007). It was shaped with individualistic, market-oriented, managerial, and technological ideas (Dýrförð & Magnúsdóttir, 2016; Jónsson, 2018; Sigurðardóttir et al., 2014).

Te bill for the Compulsory School Act emphasized a competencebased curriculum, continuity and fexibility between school levels, quality through formative testing and evaluation asserted by the establishment of the Directorate of Education on market and parental infuence, and increased autonomy of schools and municipalities (Parliament, 2007) which was almost exactly the same emphasis found in the Norwegian reform several years prior (Karseth & Sivesind, 2010). Te emphasis on gender equity (Guðbjörnsdóttir, 2003), democracy, and social justice that had been a strong part of the Compulsory School Act since 1974 (Halldórsdóttir et al., 2016; Ólafur Páll Jónsson, 2014) was kept and further nuanced. Tese nuances focused on the emphasis on inclusion (i. skóli án aðgreiningar) as a policy that responded to the Salamanca Statement that Iceland signed in 1994 and had already impacted the whole education system. Iceland only had around 1% of students in special schools or units (Marinósson & Bjarnason, 2014).

A political shift occurred when Katrin Jakobsdottir assumed the position of Minister of Education, Science and Culture, substantially infuenced by an ethics report on the aftermath of the banking crisis (Árnason et al., 2010), emphasizing the strong democratic and critical role of the education system and the leeway for grounding that was needed to rewrite the drafts of the curriculum (Jónsson, 2018). She sought to revitalize core values for the entire educational system as a foundation for a democratic society by emphasizing six fundamental pillars for all school levels: (a) democracy and human rights, (b) literacy, (c) sustainability, (d) equality, (e) health and well-being, and (f) creativity. During an interview in 2018, Jakobsdóttir explained:

When our government came to power in May 2009, the work on the curriculum was in its initial stages and the approach was traditional, i.e., to focus on the subjects taught and list the felds that should be covered on diferent levels of education. But, from the very beginning, my main question was how do we put into practice the ideas of democracy that are the focus of our legislation? … My vision was that the school had to be a basic democratic institution, that we needed to describe in more detail what this involved. (Jónsson, 2018, p. 62)

Te exclusive access of the Independence Party to state governance of education is relevant to this discussion, as had been in control consistently for approximately 20 years until 2009 when Ms. Jakobsdóttir took ofce after the global economic downturn in the autumn of 2008. Te Independence Party then took the helm again from 2013–2017, which is the period of our study. Tese changes illustrate that the education governance landscape in Iceland during the fundamental and incremental reforms was disrupted by political shifts (Table 6.1) and economic crisis.

#### **Icelandic Governance**

Te fnancial meltdown of the Icelandic banks in 2008 created not only an economic crisis but also a democratic crisis that directed attention to the governing body of the nation. Te governmental and political foundation of the community was questioned. Te parliament organized a special research commission to investigate the failure of the banks. One of the issues uncovered concerned the lack of regulation and government supervision. "Te most important lessons to draw from these events are about weak social structures, political culture, and public institutions. It is the common responsibility of the Icelandic nation to work towards

#### **156 B. R. Magnúsdóttir and J. T. Jónasson**


**Table 6.1** List of Ministers of Education, their political parties, and the policy documents that formed the fundamental and incremental education reforms

*Note*: Documents in Italics are part of the bibliographic analysis

strengthening them and constructing a well-functioning democratic society" (Árnason et al., 2010, para. 4).

Governance operations in Iceland have been criticized for lacking professionalism and democratic practices (Árnason & Henrysson, 2018). According to research on government practices in 2010–2011, Iceland deviates from the other Nordic countries in preparing bills for legislative acts. Te premises on which policies are based are generally not as systematic in Iceland as in the other Nordic countries, especially Norway, Sweden, and Finland. In Iceland the Minister is given more autonomy to decide on procedures (Kristinsson, 2013). Compared to other countries, the independence of the Icelandic Minister has been enormous and the role of the position loosely defned (Kristinsson, 2009). In 2010 the Ministry of Education, Science and Culture issued a report commissioned by experts/scholars in administration and politics that proposed that the role of ministers be more clearly defned in legal frameworks. In the report, the authors concluded that a minister should "seek professional analysis from the ministerial administration before making decisions" (p. 3). Tis requirement is now part of the Act on Ministries' Ofce (Forsætisráðuneytið, 2010).

Te period under review is from 2013–2017 or a few years after the country went through a thorough discussion, reports, and stricter regulations on this process. Since the law (Lög um Stjórnarráð Íslands, 2011) was issued, three white papers2 have been published, one of which focused on education and is under review in this chapter. No research has been conducted to examine the governance practice that guided the policy document formulation in the Ministry of Education, Science and Culture; thus, this study is the frst to explore that by drawing on empirical data.

## **Methodology and Research Design**

Tis frame of research is attended to explore the use of evidence in the Icelandic nation's policy formulation, how the evidence was used in a specifc time frame, to what end, and by whom (authors, publishers). References are the primary defnition of evidence that are used to "provide legitimacy" for policy development decisions (Steiner-Khamsi, Chap. 2 in this volume).

#### **Methods**

Te following data from the three documents in focus in this study are explored within this analysis: the publishers/authoring organizations of the documents' references and citations and their geographical classifcation (domestic/Icelandic, Nordic/regional, and international); the type of knowledge (e.g., academic, reports, and laws/regulations) used; and the co-citations, which are studied through a bibliographical network analysis. For the selected documents, we look at in-degree centrality, that is, how many times a given reference is cited in these three papers, as a measure of its signifcance in the policy discourse.

Diferently from Finland and Norway, the WP and GP as concepts have not been directly used in Icelandic policy making. For the purpose of this analysis, the WP is understood as an ofcial document that formally suggests how to revise or develop new legislation or direct policy issued for state/municipality policy actors to implement.

A GP in this study constitutes a document that is commissioned for the Ministry to obtain relevant background information to justify amendments within legislation or create new directions in policy making. For this reason, a GP is often used as a resource/reference for development of a WP, including the reference list or formal citations.

In our case, we have examples of many documents that are beyond the scope of this study, as they were commissioned for institutions other than the Ministry, were not used as references, and/or lack a reference list. Terefore, the content analysis is important in determining how the GPs or other background documents are embedded within the texts. For this purpose, we used Atlas.ti word-counting and thematic analysis. Our analysis is designed to answer the research question: What kind of knowledge is used, interpreted, and recontextualized in the documents under examination?

In order to gain some insight into the policy development process within the Ministry of Education, Science and Culture, fve interviews were conducted with Ministry ofcials in order to inquire about Nordic and other multinational cooperation and to inquire about the modus operandi that underpins policy formation to clarify some aspects of the construction of the three documents. Te interviews were recorded and transcribed, and interview subjects were assured of their confdentiality and anonymity. Instead of names, the frst fve capital letters in the Icelandic alphabet (A, B, D, E, F) are used to represent the interviewees. Jónasson conducted the interviews in December 2019. Interviewees were asked to participate due to their involvement in the policy formation process at various times from 2013–2017.

### **Selection and Types of Source Documents**

Te only document that was an obvious choice for the analysis was the frst educational document issued with "white paper" in the title (WP2014). Te other document discovered as a WP that was part of the follow-up incremental reform is the audit report published in 2017 by the Ministry of Education, Science and Culture and the European Agency, originally written as a GP. It gained status as a WP after educational leaders in Iceland signed a declaration of cooperation based on the report (i. Samstarfsyfrlýsing).

In 2013, the Ministry of Education, Science and Culture decided to initiate work to examine how the general policy of school for all worked. Te European Agency for Special Education was designated to conduct an external evaluation of the situation concerning inclusive education, the policy on the issue, and its implementation. Tis task was undertaken in 2016 based on a special agreement among three ministries—the Ministry of Education, Science and Culture; the Ministry of Health; and the Ministry of Social Welfare—along with the Association of Municipalities, the Teachers' Union, Te Union of Upper Secondary Rectors, and the leading parental organization (i. Heimili og skóli). Te evaluation was based on input and consultation received domestically through surveys, focus group interviews with stakeholders and experts, and a background paper authorized by Ministry ofcials that was not published (according to an interviewee). Tus, the document was initially classifed as a GP. Te report was presented and discussed at a meeting on March 2, 2017. Te ministers and the representatives of these organizations that had agreed to this initiative in 2016 signed a declaration in which they agreed to follow through on the actions proposed in the report. Te status of the document was then changed from GP to WP, as it was brought into the policy arena by the main stakeholders, stating that "they will cooperate to follow up on the results of the audit report" (Ministry of Education, Science and Culture, 2017).

Te only GP that fulflled all requirements was Document ID 3 (see Table 6.2), which was written according to a structure from OECD, as the document's aim was to prepare for an OECD review for resource allocation and use in Icelandic schools. Te Icelandic government

#### **160 B. R. Magnúsdóttir and J. T. Jónasson**


**Table 6.2** List of documents for analysis

undertook the writing of a background report, intended to provide OECD with basic information, as is customary, which would be used as input for OECD's evaluation and recommendations. In the current context, it is important to keep in mind that its aim was to assemble information and provide an understanding of the system in preparation for an OECD evaluation. Te evaluation never took place, but the report was referred to in several subsequent documents, inter alia the WP2017 analyzed here. Te document, which contains an extensive description of the Icelandic system, was prepared by the Ministry of Education, Science and Culture for OECD and the EC (European Commission), and the committee that oversaw its writing was represented by important stakeholders.3 Even though the focus of the report was on the three school levels, the expert group placed emphasis on the compulsory level. Two local experts assisted in drafting the document. Document ID 4 (see Table 6.2) is understood as a source document. It is missing a reference list, so it is not part of the analysis.

## **Results**

Te documents analyzed came from fundamentally diferent directions, even though they were issued within a short time frame under the same Minister, which makes them particularly interesting for our analysis. Te interviews are used to clarify the nature of the documents and the diferences in the reference profles obtained.

## **The Knowledge Base in the Documents**

Tis section presents an overview of the fndings from the bibliometric analysis. Te analysis is grounded in the location of references, types of documents, the network of references, and the publishers.

A total of 203 references were included in these three documents, and more than half are referenced in the only GP that is mainly based on domestic publishers (Statistics Iceland and the state). Both the WPs contain relatively few references, and their reference lists are mostly based on international publishers in the form of reports, but similar to the GP, they include very few academic resources (Table 6.3).

Tere is little overlap in the resources cited. Te network structure (Fig. 6.1) shows how loosely related these documents are in terms of references, especially the two WPs: the only connection between them is that WP2017 refers to WP2014. Moreover, GP2014 is a reference in WP2017 and is mentioned as a work in progress in WP2014 without a formal reference.


**Table 6.3** Reference distribution in the source documents

*Note*: WP = White Paper; GP = Green Paper

**Fig. 6.1** The network structure of the three documents. (*Notes*: Regional = gray; domestic = white; international = black; node size = in-degree centrality. *Source*: circle)

Figure 6.1 shows that nine documents are referenced in more than one of the source documents. Seven are governmental documents, and two of these documents are reports on the PISA results from 2012 (OECD, 2013; Halldórsson et al., 2013). Accordingly, a joint focus in the two WPs concerns the Icelandic legal framework and PISA results from 2012.

#### **Emphasis on Quantitative Evidence from Reports**

In WP2014, the authors claim that the use of evidence is at the core of the proposed reform: "Work on the White Paper has been based on international studies of education reform, and attempts have been made to draw lessons from the experience of those nations that perform strongest in international comparisons" (p. 5).

Te policy discourse in the WP2014 is based on information about those countries that outperform other nations in the comparisons and the aim is to learn from their reforms. Te focus on best international performance indicates that "best practice" guides policy development. Table 6.4 provides an overview of the fgures that concern the compulsory education part of the WP2014 indicating sources and actual references used. Tree diferent perspectives are used in exploring large data sets.4 One is the international comparison that examines the positioning of Iceland within the large list of participating countries (Table 6.4: Figures 1, 2, and 3). Te second perspective is the Nordic comparison (Table 6.4: Figures 8 and 18), and the third comprises the national data (Table 6.4: Figures 5, 7, 9, and 10), where both longitudinal and areabased comparisons are made.

Te policy discourse in the WP2014 is based on a technical report (UNESCO & IIEP, 2012) published by the United Nations Educational, Scientifc and Cultural Organization (UNESCO) and International Institute for Educational Planning (IIEP). Overall, instead of referring to a list of GPs, the list of references is based on the following: (a) international think tanks, such as McKinsey; (b) information about the education system based on comparative data collected by European institutions like Eurydice and the European Commission; (c) conceptualization, recommendations, and survey results from intergovernmental organizations like OECD; and (d) policy borrowing from other states/nations like Ontario in Canada.

Terefore, it comes as no surprise that 71% of the references in WP2014 are international, and 62% are reports. Te reference list contains 36 citations when including in-text citations, of which 12 originated from OECD, which is the most cited publisher. Tis is not the case for the other two documents, as in the Icelandic database, OECD is the fourth most cited. Further analysis of OECD's impact can be found in Ydesen et al. (Chap. 11 in this volume).

It is very clear from the format of WP2014 that the underlying comparative evidence is quantitative data presented in the fgures presented in the paper, which are described in Table 6.4. Tere are 12 fgures in WP2014 and 11 of them are of international sources mainly from OECD with PISA in the forefront. In 7 fgures the focus is on literacy. All of them originate from the PISA database except one that is based on data from Eurydice


PIAAC = *Programme for the International Assessment of Adult Competencies*; Talis = The OECD Teaching and Learning

International Survey

**Table 6.4**Overview of fgures from WP2014 indicating sources and actual references used

With reference to the quantitative evidence, no fgures or tables are presented in WP2017 nor in the background report prepared by the Icelandic working group.

Te words "teachers," "students," "reading," and "reform" are frequently mentioned in WP2014, while "inclusive," "stakeholders," "system," and "support" are among the most common in WP2017. Te approaches used in these two policy endeavors are strikingly diferent. Tis is, however, in line with the Icelandic tradition of having a wide scope of possibilities to form a policy document, which is further discussed in the next section.

#### **Emphasis on "Evidence-Based" Practice: Focus on Teachers**

Te WPs are quite diferent in terms of aims and openings. In WP2014 the implicit assumption is that international data, like PISA results, address what is most important in education, and by implication, the top priority is to address this issue; in WP2017 it is in the hands of the national Ministry of Education, Science and Culture to defne the main problems and themes to focus on to get Icelandic schools to become more inclusive. In WP2014 the paper sets out two measurable aims—the one related to literacy in the compulsory schools is designed to increase minimum reading PISA standards. Tree main categories of action are included. One is to allot more time to *Icelandic* as a subject, another is to develop standards by measuring reading profciency at various levels as each student progresses toward the end of compulsory school, and the third is to form the work of the teachers in more detail.

An extensive analysis carried out by McKinsey (2007) concluded that the two most important factors infuencing the performance of an education system were the education and the work of teachers. Te same conclusion has been drawn in recent academic writings (Hargreaves and Fullan, 2012). … Recently, an expert panel on the continuing education and professional development of teachers was set up. Te panel's tasks will be to propose ideas and priorities regarding the training and support that teachers should receive in their work, and develop new teaching methods. Te Ministry of Education, Science, and Culture has also started consultations with teacher education institutions on the content of teacher education and increased cooperation in this feld. (WP2014, pp. 38–39)

Tis focus is well known in international research: controlling by numbers through yearly evaluations where teachers are made responsible for achieving better results by adopting the "right" teaching practices (Brian, 2009; Robertson, 2016). On the other hand, WP2017, which also emphasized teaching practices, took a diferent approach intended more so to empower teachers themselves to professionalize in an inclusive way, as illustrated by the following excerpts:

Te need to train teachers on using diverse teaching methods—especially within upper-secondary education—is highlighted. (p. 110)

Tere must be fexible professional development opportunities for teacher educators to support their attitudes, knowledge, skills and ability to model inclusive teaching practice in their work. (p. 130)

Teams of teachers and support professionals should work together to develop fexible frameworks for curriculum and assessment, together with teaching approaches that engage all learners and support their active engagement and participation in learning. (p. 135)

Te discourse in WP2017 is not based on "best practices" of teachers but to enhance practices to be in line with the inclusive ideology.

In WP2014, the negative consequences of the constructivist ideals among Icelandic teachers were one of the assumptions made for the declining reading comprehension. A graph from the Teachers and Learning International Survey (TALIS) survey showed how teachers in Iceland believed more heavily in the constructivist model than in direct transmission, compared to teachers in some other nations (WP2014, p. 40). Te claim was also made by referring (only through in-text citation) to one small-scale research study by Savola (2010) comparing Finnish and Icelandic teaching practices in mathematics, emphasizing the good performance of the Finnish school system that relies more on direct transmission. When systematically observing teaching practices in large-scale local research on compulsory schools (Óskarsdóttir, 2014), the results indicate that teachers in Iceland do still heavily rely on direct transmission.

To promote WP2014, the Ministry of Education, Science and Culture held a conference on literacy to which all teachers in the country were invited.5 Te publication of WP2014 and this conference were the frst steps in what was called a "National Literacy Concord" (i. Þjóðarsáttmáli um læsi) with its own logo and a song. A few weeks later, a critique on Beginning Literacy (i. Byrjendalæsi)6 was issued in the form of a memo from the Directorate of Education.7 Beginning Literacy highlights an interactive approach, collaborative work, and the active participation of pupils as well as integrated language (Eggertsdóttir, 2007). Te argument was that the schools that had adopted the Beginning Literacy approach were, on average, scoring lower than other schools on standardized tests (the 4th grade test) and that it was not an evidence-based practice. Te Minister made a clear statement about this: "From now on, if there is an idea to utilize a particular practice or program it must be based on available evidence that documents its efectiveness" (Skaptadóttir, 2015, p. 10)

Tere was a harsh debate about this in the media, and the diferences in the test scores were contested by scholars; for example, a sociology professor at the University of Akureyri showed how the diference in scores had been exaggerated visually (Þóroddur Bjarnason, 2015). In the same week the founder of Beginning Literacy stated during an interview that it should not be the role of the Minister of Education, Science and Culture to choose the teaching methods used in the classroom (Arnarsson, 2015). Tus, the PISA "shock" was promulgated in the media, and the new institution ("Act of Law on Directorate of Education," 91/2015) took the frst action to deconstruct Beginning Literacy as an accountable reading method in terms of evidence and achievement. Te authors of this chapter are not taking a position on this debate. Rather, this discussion is intended to serve as a follow-up on how the main objective in the WP—to increase achievement in reading—was strongly related to a particular best practice used by teachers and how the mediatization of some sort of a PISA shock was an essential factor in promoting it.

#### **Scarcity of Academic References**

Te data analysis indicated that the use of academic references in policy making is rather low (both in terms of books and academic journals); indeed, few local (Icelandic) or regional (Nordic) academic references were found. In WP2014 two in-text citations were based on a publication in academic journals. Tese two are Fullan (2013) that is a commentary paper, and the other is a small-scale dissertation study comparing Finnish and Icelandic teaching practices in mathematics (Savola, 2010). An examination of the academic references in the database, published either as books or as articles in academic journals, revealed that none of those references includes the keywords "literacy/reading" or "inclusion," which are the main educational themes in the two WPs. Instead, the use of reports to defne and discuss the main concepts—literacy and inclusion—is dominant in both WPs. For the reading profciency discussion, PISA is the main source. Te use of academic papers was also scarce when arguing for changing direction in teaching practices.

Te reason for the limited number of academic references, both from international and national perspectives, was discussed during the interviews. It seems that the research culture in Icelandic academia does not enter into the governmental framework due to the lack of large-scale, quantitative, and comparable research conducted in the academic context of Iceland.

I think we need to use more systematically collected data. … I think if the intention is really to infuence policy, the systematic analysis based on large scale data sets is needed, but also from large scale [national] studies. But it requires an efort to do this. For example, the research from Gerður (referring to Óskarsdóttir, 2014) has a lot of information. Still, it is, however, what you can fnd in it and pull out of it rather than systematically concluded results with statistical data that you can straightforwardly use in public policy making. (Interviewee B, December 2019)

Te Ministry ofcial is asking for a diferent approach that is more convenient for policy makers, that is, research that is more comparable and quantifable (statistical) and easy-to-use results, reducing what then counts as evidence. Te authors of the audit report (WP2017) did not speak or read Icelandic and relied, instead, on international reports and their own data collection and analysis.

It has been suggested that the policy feld is gradually gaining more infuence on the academic world, controlling what can count as valuable knowledge. In recent years the neoliberal discourse on market solutions, evidence-based research, and performativity have become more infuential (Ball & Olmedo, 2012; Dovemark et al., 2018). Tis orientation has challenged what had been the dominant model. In Iceland, stakeholders have most often been included in policy formation, such as with laws and curriculum, which is the case in two of the three documents under review in this paper: WP2017 and GP2014. Icelandic academia, however, has not been one of the stakeholders to have a right to be heard in the democratic process; instead, research is produced by scholars independent of the feld of policy, and therefore, it has been easy to bypass and overlook.

### **The Icelandic Procedure of Education Policy Formulation: Ministerial Governance**

From the spectrum of available documentation, from the content of the documents analyzed, and from the responses of the interviewees, we can conclude that no established procedure exists to direct the construction of educational policy documents. According to Interviewee B, "Tere is no organization or a planned process that covers all policy making. … Tere is no systematic analysis in the background." Interviewee D explained, "To write a green paper with all the documentation and references, we sometimes skip this and enter the proposals for action stage."

As an example, the European Agency authorized the audit report on inclusive education (WP2017) because a long-standing Ministry ofcial had for years led the work with European Association for International Education (EAIE) for Iceland. Te ofcial used their connection to contract with the Agency to prepare the report. Te EAIE asked for a background paper, "Te Critical Refection Paper," which became the third background paper issued by the Ministry of Education, Science and Culture in relation to policy formation. WP2017 was the fourth and fnal paper in this process. Te lesson learned from the interviews is that each Minister determines how to proceed, often by designating someone who is trusted to chair the work, either inside or outside the Ministry of Education, Science and Culture, who, in turn, may have considerable independent views on how to proceed. Indeed, no formal procedures apply. Interviewee A described it this way: "Talking about infuence, the minister's views are much more infuential than something that comes from outside." According to Interviewee E, "Substantial consultation may or may not take place; the evidence is usually collected, but not in a very systematic way, and the work may thoroughly integrate the ministry specialists, or not."

Te policy document formation process has not been institutionalized along certain operational lines, even though considerable expertise is often harnessed into the process. Furthermore, there is neither a tradition to formalize the evidence used (e.g., by systematic referencing) nor consultation obtained.

## **"Tere Is Often a Feeling of an Urgent Need for Action": Te Tight Timeframe**

As mentioned previously, the concepts of the WP and GP have not directly been in use in Icelandic policy making until very recently, which allows extraordinary leeway for a short-cut and powerful agendas to push forward in a short amount of time. "Tere is often a feeling of an urgent need for action. For example, if PISA results are bad, there may be little room for delay" (Interviewee D).

Part of the core curriculum for all school levels was published in the beginning of 2013, and the new Minister of Education, Science and Culture, Illugi Gunnarsson, who took the position in the spring of 2013, managed to publish a WP in June 2014. It would have been impossible to issue a WP with the standard protocol in such a short time. Two of the ofcials in the Ministry of Education, Science and Culture mentioned that the main reason for the focus on PISA in WP2014 was a sort of PISA shock: "Te white paper had mainly focused on OECD material. Other data somehow didn't make it into the paper. I think there was some kind of a PISA shock reaction that pushed this to the fore" (Interviewee D).

Tis is exactly what has been defned as re-articulation of social justice to test-driven data and numbers (Lingard et al., 2014), where evidence becomes the vehicle for reform and renewal in the education sector. Te follow-up of WP2014 was massively resourced fnancially (compared to other projects) according to a Ministry ofcial, and the Minister of Education, Science and Culture himself visited every compulsory school in the country to introduce it to the school communities. WP2014 was never passed through any kind of regulation in the parliament. Te push for writing the frst WP at the Ministry of Education, Science and Culture in such a short time frame, just a year after the subject-based curriculum was issued, could be understood as creating leeway for the Minister of Education, Science and Culture to bypass the new curriculum, change direction, and make his own agenda without the need for any infuence from the parliament or other governance bodies.

Te audit report (WP2017) also had a tight time frame. Te two background papers were issued in 2015 (*Mat á framkvæmd stefnu um skóla án aðgreiningar: Skýrsla starfshóps*, 2015; Ólafsdóttir et al., 2015) and the cooperation with the European Agency started formerly in late 2015. Te "Critical Refection Paper" background paper, according to an interviewee, was written before data collection process, which started in spring 2016. An enormous amount of data was collected in four days by the six audit members, as they spent just few days in the country. Tis included 27 focus groups involving 222 participants, 11 school visits, and 9 individual face-to-face interviews with high-level decision-makers. An online survey was available for six weeks from May–June 2016 (WP2017, pp. 25–27), and the report was launched in Reykjavík on March 2, 2017,8 the same day it was updated as a WP.

#### **Te Small Nordic State in a Globalized World**

After analyzing our interview data, two discursive themes emerged from the reasons given for relying so extensively on external authorities. Tey concern the scarcity of ofcials to participate at the policy-making level and to interpret or translate the OECD discourse and data and the importance of externalization to avoid nepotism or as a strategy to change direction from what is suggested nationally.

Icelandic governance sufers from the lack of capacities at the Ministry of Education, Science and Culture, including the lack of domestic experts and human capital who can participate at the policy level and then flter the OECD infuence and control it in line with the normative way of governance (e.g., through GPs).

It is a major task to be on top of everything within the OECD, and the countries have diferent capacities available in terms of money and staf. Some, like Finland and Estonia, as examples, have people stationed in Paris who only attend to these tasks. And [some] countries don't have education attachés who follow what is happening and prepare meetings … it is only a fraction of my job here to go there and attend meetings … but this is supportive of our work, and we do our best. I have just been talking to a Norwegian colleague who deals with OECD, Nordic issues and bilateral issues—but he has a support of twenty staf. Yes, one feels a bit overwhelmed. (Interviewee B)

Te small state lacks human capital to cover all international cooperative endeavors and translate transnational efects and policy borrowing into a more local frame of policy discourse. In Finland a large group is designated to do that and is authorized as such (Volmari et al., Chap. 5 in this volume).

"Te eye of the guest is keenest" (i. glöggt er gests augað) is a maxim in Iceland that captures the belief that we need to get somebody that is not part of "our group" to explore objectively and tell us the "truth" of what is really happening. Te words from the Minister of Education, Science and Culture that are discussed in WP2017 refect on this belief:

At the audit launch event on November 3, 2015, Mr Illugi Gunnarsson, the Minister for Education, Science and Culture at that time, stated that the main motivation for the Icelandic stakeholders in requesting an audit was to gain an external view of the operation of the Icelandic system for inclusive education. He quoted an Icelandic maxim that says "Te eye of the guest is keenest." (WP2017, p. 12)

Tis viewpoint is based on the belief that Icelanders, in most cases, have dependent and biased viewpoints. For policy formation on inclusive education, two background papers (*Mat á framkvæmd stefnu um skóla án aðgreiningar: Skýrsla starfshóps*, 2015; Steingerður Ólafsdóttir et al., 2015) were requested by the Ministry/Katrín Jakobsdóttir.

Tere was ongoing work on analyzing the stage of inclusive education here in Iceland, which was … not getting us anywhere. Ten there popped up a suggestion of requesting help from the European Agency, just to get an external view, get the guest's eye, and that was the end result. … But a request was sent to them, as their approach is more democratic than the OECD's and not with a tight structure and standards of indicators and questions as OECD. (Interviewee B)

Sensibly, it was not a matter of whether to ask for external help—the choice was made between two international knowledge providers, OECD and EAIE, and EAIE was recommended to the Minister of Education, Science and Culture due to its democratic and inclusive governance practice (according to one interviewee). One can also interpret this as a way to get a diferent political frame of reference than the national context could ofer at that time. Externalization can also be a way of bypassing some local objectives, people, and beliefs.

## **Conclusion and Discussion**

Te external steering and input into policy making were substantial in Iceland from 2013–2017. GP2014 was written by Icelandic experts in line with a structure from OECD, and WP2017 was written and published by an external audit (European Agency) but processed by information from and in consultation with Ministry of Education, Science and Culture ofcials, Icelandic experts, and stakeholders. WP2017 is an example of a document where the externalization is achieved by getting external agencies to bring some possible solutions to the table. English is the original language of two of the three documents explored in this study. WP2014 is exemplary for its dominant use of international reports, especially from OECD, which is the most cited publisher, in defning main concepts, grounding arguments for the weak points in the Icelandic system (teacher practice), and providing solutions.

Te use of references varies widely, both in scope and extent, with GP2014 accounting for 70% of the references used in total. In the combined database the primary knowledge providers are Statistics Iceland; Icelandic Ministry of Education, Science and Culture; the Icelandic Parliament; and OECD. According to the bibliographic analysis, academic papers in general, but especially Icelandic papers, are thought to be irrelevant or not providing "accessible" knowledge in the evidence base for the policy. Strong transnational infuences can be observed, and in WP2014 an excessive policy borrowing in terms of references and values, especially from PISA, OECD, Canada, and international think tanks, is evident. According to the interviews, the procedure in Icelandic education policy making is loosely structured compared to governmental practices in countries where the procedure is more formalized (Kristinsson, 2013).

Te main focus was on the only WP that has been issued as such in Iceland, and analysis of the other two had the aim of providing a wider picture of policy formation. Te over-emphasis on international and quantitative evidence in WP2014 was not as prominent in the other two papers. A more ethnographic approach was detected for the knowledge producing in relation to WP2017 with their own school visits and interviews as main sources of data. In GP2014 the main source was Icelandic governmental data. WP2014 is exemplary when it comes to a scarcity of academic references and time frame. WP2017 did have a short time frame but was better articulated in the three background documents that had been written before getting the European Agency to write the paper. On the other hand, PISA-shock was the driving force for WP2014. Te urgent need to react immediately to declining results in PISA and media attention can support an argument for ministers for policy changes (Sellar & Lingard, 2013).

Prior research on the logic of practice in Icelandic governance resonates with our results in terms of loosely defned protocols and ministerial governance (Kristinsson, 2009, 2013). Tis also holds true in the Ministry of Education, Science and Culture. Despite the thorough discussion, reports, and stricter regulations on the procedures in the governance after the fnancial crash in 2008 (Árnason & Henrysson, 2018) refected in the new law on governance (Icelandic parliament, 2011) the diferent formulation with the two WPs reveals how loosely defned and ministerial governed this process still is. Te short time frame for publishing a WP in 2014, focusing solely on reading performance even though the Ministry of Education, Science and Culture had just recently issued a curriculum (2011/2013) that had not been implemented, is noticeable. On the other hand, this focus on literacy has existed for centuries in Iceland (Guttormsson, 2008).

Te scarcity of people working in international relations for the state education government and the distance between the policy and academia in the feld of education in Iceland provide more space for the external voice to design national policy. It is tempting to rely heavily on international sources, probably to a greater degree than others, when the state governance lacks the human resources required to tackle all their transnational duties and to translate the transnational efects to an Icelandic context (Ydesen et al., Chap. 11 in this volume). Tis is also a way to avoid being accused of nepotism in this small country. However, in a ministerial governance ministers can bypass objectives and ideas that already have gained status in the form of regulations and previous and current discourse, for example, in the local academic arena, and use this danger of nepotism to gain leeway from that and fnd an international knowledge provider abroad that is more in line with their own political agenda.

WP2014, the only document that was authorized and published exclusively by the Ministry, is a clear example of substantial externalization. It reveals the power of OECD and marketization of knowledge through rankings to produce, extract, and choose from the available knowledge to form a policy document. Transnational agencies manage to boil complex issues down to relatively simple numbers, to present sociological issues in terms of quantifcation. Tey come across as politically neutral in the service of local policy making. Te data is seen as comparable and essentially problem free. So even if the evidence used sometimes only refers to the Nordic countries or even to one country, Iceland in our case, the reference base is still defned by OECD. Numbers and statistics have for a long time been used in educational research and also in policy making to understand the distribution of capital, poverty, educational outcomes, and so on. Numbers have been part of the national state for centuries to gain overview and control. Te development in Iceland as it is represented in WP2014 shows the dominance of the international perspective and how governing by numbers is a way to ensure social justice and, thus, depoliticize political decisions. It is a technology of governance that aims at easing out politics and values that are based on other views and concurrently advocate for more efciency, accountability, and efectiveness. It is an attempt at steering from above, especially aimed at teachers in WP2014, by collecting data that translate life in schools and communities into a series of graphs, grids, league tables, and indices and by introducing examples of best practices. Tis discourse is based on the view that competitive and test-driven education policy in the core subjects (reading and math) is the route to real social justice (Lingard et al., 2014; Sellar & Lingard, 2013).

## **Notes**


## **References**


*IV, félagsvísindadeild* (pp. 257–272). Félagsvísindastofnun Háskóla Íslands og University of Iceland Press.


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **7**

## **Structuring School Reform Policy with Evidence: The Inter-mediational Role of Knowledge Sources and Arguments**

**Bernadette Hörmann and Kirsten Sivesind**

In Nordic countries, systematic inquiries have played a crucial role in the nation-states' eforts to reform public education. However, in recent decades, various stakeholders have raised serious concerns about the legitimacy of such inquiries. Both in media and in research, critiques have targeted the quality of professional knowledge, suggesting that education policy should draw on scientifc evidence to reform and evaluate the education system. Tis chapter examines the institutional response to this critique by examining how national authorities have made policy into an evidence-based pursuit of ministries and their governmental bodies.

By inquiring about two white and eight green papers published by the national authorities in Norway, we ask the following questions: How do policymakers and experts provide evidence and expertise in issuing school reforms for basic education? How do they identify options for school reform by deploying knowledge through argumentative forms of

University of Oslo, Oslo, Norway

B. Hörmann (\*) • K. Sivesind

e-mail: bernadette.hormann@iped.uio.no; Kirsten.Sivesind@iped.uio.no

<sup>©</sup> Te Author(s) 2022 **183**

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_7

reasoning? How do various types of evidence structure these options by connecting policy realms and systems? We have divided the chapter into three main parts.

First, the chapter begins by establishing the policy context, the theoretical background for our study, and a typology that classifes various knowledge sources we will use for analytical purposes. We present some contextual information about the school reform we examined and give an overview of the bibliographic meta-data that we collected and the research strategies we used to examine the mediation and use of policy-relevant knowledge. We also introduce perspectives from sociological system theory based on Luhmann (2018) and associated scholars (Andersen, 2019). Tis theory helps us examine how references to knowledge sources are semantically translated within policymaking processes and how evidence informs thematic areas in school reform policy, such as curriculum and assessment. In addition, the theory serves as a link between policy processes and policy systems, such as science, politics, and education.

Second, we present the results obtained from our bibliometric network analysis in the form of the frequency and distribution of prominent references in our dataset. By looking into the ways two white papers and eight green papers refer to various knowledge sources (e.g., research reports, reviews, and governmental documents), we demonstrate how policymakers use bibliographic references to strengthen their arguments for reforming basic education in Norway. In our analysis, we identify how some references acquire a prominent role by being referenced by several sources within and across policy realms. We also uncover their prominence by mapping whether these or other knowledge sources are frequently and explicitly in-text referenced and thereby infuential within translations made by the authors.

Tird, we present a semantical analysis of how state authorities translate knowledge sources in-text referenced within the two white papers. Tese documents prepared for political decision-courses within the Norwegian parliament (see Chap. 10 for further details about the process). We analyze how policymakers formulated options for school reform through postulations and aspirations about structures, processes, and outcomes. Tus, we agree with Colebatch and Hoppe's argument that "policy may be seen as both *ex ante* intention, *ex durante* becoming, and *ex post* outcome" (Colebatch & Hoppe, 2018a, p. 6). Moreover, according to Luhmann (1990), a medium, such as a policy document and its references, can provide meaning through forms of reasoning by semantically linking structures, processes, and outcomes via input/output schemes. Trough these schemes, policy formulation and translation processes become programmatic by character (Luhmann, 2018).

Finally, we question whether the most cited and prominent knowledge has infuenced the semantic repertoire of themes and arguments and thereby the confguration of the decision programs built into the national school reform. Based on a comparison of how references are used within and across two main thematic areas covered by the white papers, we conclude that evidence has a structuring impact on the formation of policy realms and, as a result, options for school reforms. Te empirical investigation shows that claims and recommendations within the white papers are shaped by the type of knowledge source policymakers use, which policy realms they reference, and how they translate knowledge through argumentative modes of reasoning. We also demonstrate how policy documents mediate evidence that structurally connects the political system with the education system. Tis demonstration leads to a discussion of whether and how professional knowledge is restrained or constricted by evidence-based reform within the education sector.

## **The Use of Evidence and Expertise Within Policymaking Processes**

During the last decade, interest in evidence-informed policy and practice has increased in Europe and beyond. Such policy often favors scientifc methodologies and empirical research as a frame of reference. Importantly, the term *evidence* does not bear much signifcance in itself since it is associated with various actors, diferent types of knowledge sources, and multiple forms of knowing (Boaz et al., 2019, p. 5). Moreover, any material base that carries knowledge or any source of information may provide evidence depending on context variables (Sedlačko, 2018). Terefore, what counts as evidence and how evidence is used in reform policies are highly contingent questions.

Tis chapter examines how evidence serves as a medium in policymaking among experts who were mandated to develop ofcial policies aimed at reforming a national education system. We investigate in particular how policymakers and experts selected research-based knowledge alongside other types of documentation, and how the policy documents mediated this knowledge as evidence to inform argumentation and decision-making processes. As Colebatch and Hoppe (2018b) have noted, "Both the documentation and the widespread 'consultation' within administrations can be part of the signature of policy-evidence that 'due process' was followed" (p. 15). As such, evidence-based policy is not merely a question of what kind of knowledge and information are produced and selected; rather, it is a question of how policy processes facilitate the usage and translation of evidence that various actors call for.

Because the term *evidence* has become a buzzword with no clear defnition, both policymakers and practitioners are juggling multiple forms of evidence within these processes and involving various groups of actors that provide support for defning the "best evidence." Within this perspective, the process of evidence use is considered a highly pragmatic enterprise. Despite attempts to establish clear hierarchies of evidence, few policy realms refect a master plan for how to make use of expertise in reform-making processes. For example, in the Oslo Institute for Research on the Impact of Science (OSIRIS) project, Tune (2019) examined the use of evidence within public administrations in Norway. Tune found that both policymakers and practitioners deployed a variety of methods to access information and knowledge, such as contacting colleagues and conducting web searches. Moreover, policymakers used diferent types of media to collect knowledge sources, such as newspapers, publications, presentations, and informal dialogues. Against this background, one might wonder if there are any patterns that actually structure evidence use in policymaking processes.

Despite the micro-politics of providing evidence, researchers have argued for the importance of observing patterns or forms of patterning of policy processes. From this point of view, policy researchers consider the character and strength of the ties of actors as one possible structure that characterizes the use and translation of information (Honig & Venkateswaran, 2012). Among several other researchers, Tune (2019) found that provision of evidence is dependent on the capabilities of actors. According to Tune's survey results, persons who are in senior positions, are well educated, and have work experience from other sectors, especially research sectors, are more capable of collecting and using knowledge sources than persons in junior positions. Moreover, the infuence of these groups of actors is dependent on the relational ties that connect these actors.

O'Day's (2002) study of accountability reform in Chicago during the early 2000s serves as another excellent example of how such ties evolve. O'Day investigated how policymakers and professionals interacted with various forms of governance through the reception and translation of information. In an empirical mixed-methods study, she uncovered critical mechanisms that enforced and constricted the fow of information within the policy-praxis nexus. A key assumption underlying this and similar studies is the changing role of bureaucratic governance. When studying the emergent impact of accountability systems through ties between administrative levels, O'Day (2002) identifed impacts of both an outcome-based bureaucratic mode of governance and a professional mode of governance. Diferent from traditional bureaucratic government, these new modes centered on practice-based knowledge, performance-based standards, and bureaucratic accountability.

In our chapter, we look at similar patterns, but primarily by focusing on documents considered powerful media for policymakers to create options for reform and change. We examine how policy documents mediate knowledge and information through their selection of references and deploy argumentative modes of reasoning that connect politics, policies, and practices. Similar to O'Day (2002), we refer in particular to two models for how policies bring about change by referencing and translating knowledge across policy realms and levels: a traditional bureaucratic mode for policymaking that aligns broad outcomes with general mandates and an outcome-based bureaucratic mode that pursues means-end reasoning along with performance standards. Both these modes operate within the same context and involve diferent degrees of normativity and prescriptive routes of actions, and they can also be combined into mixed forms, as demonstrated in recent research on policy borrowing and lending (Sivesind et al., 2016).

In our study, we draw on Luhmann's (2018) distinction between two program forms to map these alternatives. According to this theory, a conditional program and a purposive program difer by representing two sets of conceptual schemes for observing policymaking processes. According to Luhmann (2018, p. 213), a key diference between the two program forms is that they refect various distinctions for observing decisions. While conditional programs divide between conditions and consequences to build up an argument about change, purposive programs distinguish between means and ends to observe problem-solving processes. Tis difference leads in the next step to various modes of reasoning, based on expectations that can be more or less normative and future-oriented and that can be more or less comprehensive or narrow by ways of issuing a reform within the education sector.

Moreover, by looking at policy documents as media for observing decisions about reform, we can also assess how evolving semantical structures are loose or fxed based on how they are formed. By examining how documents are linked to other media (e.g., other documents) or to systems (e.g., science, education, and politics), we can assess to what degree policymaking is structured (Andersen, 2019, p. 81). Tis examination may lead to an interesting discussion about the structuring role of evidence in policymaking processes if, for example, references to scientifc evidence condition a reform by fxating particular modes of reasoning along with a purpose that national and international stakeholders have actually called for (Burns & Schuller, 2007).

We analyze more or less fxed patterns of evidence use by comparing how documents are connected within and across two thematic areas or realms: curriculum reform and assessment practices. We demonstrate how policy documents for each realm and their references to knowledge sources connect reform and change both in bibliometric networks and through semantical patterns. We show how policymakers create decision programs for education that make up policy options that are more or less future-oriented and normative by their use of evidence and language (Luhmann, 2018, p. 2015). By looking into the way mediating links are confgured between the source documents and referenced documents, by comparing the patterns across policy realms, and by assessing how they connect with science, politics, and education as surrounding systems, we are able to analyze how evidence structures policy options for school reform in both a fxed and a loose form. Tus, we identify how the intermediational roles of both knowledge sources and arguments become decisive for the confguration of policy processes in the feld of school reform.

## **Reform, Data, and Methods**

In this study, we have analyzed the distribution and networks of bibliometric reference in two white and eight green policy papers that were prepared for political processes within the Norwegian parliament on the most recent school reform in basic education (Years 1–13). Tey were all written under the auspices of the Norwegian government and the Ministry of Education and Research. Te reference use and the arguments made for reforming and renewing the education system prepared for political decision-making processes within the parliament. Trough the next steps, these documents resulted in the renewal of the Knowledge Promotion Reform (2016/2020) within the Norwegian education system (Baek et al., 2018; Steiner-Khamsi et al., 2020). Te national government launched the reform in 2017, and it was formally put into efect in August 2020. Tis reform set out to renew the curricula for the core subjects in primary and secondary education and provided a new introductory part that aimed at establishing a coherent framework for organizing and assessing teaching and learning in schools (Sivesind & Karseth, 2019).

To begin our analysis, we examined the bibliometric references in the two most prominent governmental reports (white papers, referenced here as WP#1 and WP#2) that were prepared for decision processes within the parliament. Tereafter, we included references in all public inquiry reports that were referenced in these white papers. Altogether, this sample made explicit references to 2312 knowledge sources, which were listed within the documents' bibliographies and footnotes. By conducting a bibliometric network analysis, we identifed the most co-cited knowledge sources within this corpus of documents.

For the bibliographic network analysis, our data consist of 2312 references from the reference lists of two governmental reports (white papers) and eight ofcial reports (green papers, known as *Norges ofentlige utredninger* [NOUs], written by experts who initially evaluated existing education systems and who addressed how to renew education in the future) referenced in the two white papers (Table 7.1).1

Tis sampling refects the logic of the reform-making process and allows us to analyze the "ofcial knowledge" from which bureaucrats and politicians eventually draw. Te references of our ten source documents were analyzed and edited with the software programs UCINET and Netdraw. Tese programs generated descriptive statistics and visualized relationships between the documents. In this way, we identifed the most prominent references of our dataset: those that were most often cited in the reference lists and played crucial roles in the reform discourse. We established a cut-of point at fve, resulting in a list of 12 documents that were cited fve to eight times in the whole dataset. Since we consider these publications essential knowledge in the process of reform formulation, we decided to track them back to the text in which they were actually quoted. Tese documents are the main data for our semantical analysis.

In addition to the bibliometric study, we conducted a semantical analysis of how policymakers translated references through their argumentative modes of reasoning. In this part of the study, we examined how policymakers formulated arguments to make recommendations for how to reform education in and across policy realms, such as curriculum reform and assessment practices. Te school reform was formally built on two white papers (WP#1 and WP#2) written on behalf of the Norwegian


**Table 7.1** Sampling for the bibliographic network analysis2

government. Tese two white papers referred to various expert commissions and other sources that included references to articles, book chapters, and research reports, among others. Report No. 28 to the Parliament, *Subjects, In-Depth Learning—Understanding. A Renewal of the Knowledge Promotion Reform* (WP#1; Ministry of Education and Research of Norway, 2015), was published in 2015–2016 and presented a clear framework for curriculum revision. Te report suggested continuing with and further developing competence descriptions, setting prioritization by defning core elements in the subjects, including basic skills in curricula, and creating more connections between the subjects by defning three prioritized cross-disciplinary topics (i.e., democracy and citizenship, sustainable development, and health and mastery of life). In addition, the report suggested new goals and assignments for the forthcoming renewal process. Report No. 21 to the Parliament, *Eager to Learn—Early Intervention and Quality in Schools* (WP#2; Ministry of Education and Research of Norway, 2016), was published in 2016–2017 and addressed diversity problems regarding students needing extra support. WP#2 also presented a comprehensive framework to improve equity and quality with new models for monitoring education.

For analytical and heuristic purposes, we identifed two main groups of arguments: (a) those that maintained traditional governance modes that guide actions by regulating conditions and (b) those that purposively emphasized the need for reforming education more strategically. For this part of the analysis, we used Luhmann's (2018) program forms (i.e., conditional versus purposive programs). Conditional programs are pastoriented and emphasize formal and substantial rationales for reforming education, while purposive programs are future-oriented and emphasize value-based and standard-based tools for achieving certain outcomes (Sivesind et al., 2016). For identifying how prominent references were used to legitimize arguments for school reforms and investigate the underlying communication patterns, we consider the combination of semantic and bibliometric network analysis to be a fruitful research approach (Froehlich, 2020).

Finally, we conducted a structural comparison of reference use within and across the two white papers (Sivesind, 1999). Tis comparison aimed at developing insights into the third research question, addressing how various types of evidence structure policy options by linking thematic areas and by connecting systems such as policy with education, politics, and science (Andersen, 2019). First, we examined if and how the most cited references shaped argumentative modes of reasoning through their tendency to favor a conditional program, a purposive program, or both. Trough this comparison, we investigated if and how certain types of evidence can actually structure policy options by representing a particular type of knowledge. Second, we synthesized our fndings and interpretations about the argumentative translation of evidence within the white papers and asked how policymakers shape policy options through the expertise they deploy during the writing process. Moreover, we examined whether we found diferent patterns between the two policy realms in terms of how evidence was translated. Finally, we compared the ways in which the most prominent references in the two white papers connected policy realms by referencing the surrounding world of systems, such as education, science, and politics.

## **Typology**

Along with Stephen Toulmin (1958), we consider a persuasive argument as consisting of both claims and assertions backed by evidence. Tese assertions refer to both facts in terms of data and information, and include reasoning that enhances a persuasive use of the argument. In addition, we categorize various types of knowledge sources from our dataset that were used to build an argument. Moreover, to compare the infuence and usage of various types of knowledge as manifested within the policy papers, we have developed a typology that fts with our data that we can use to build arguments ourselves. Tis typology is based on existing classifcations of research and information that we combined for our particular purpose.

Doyle (2003) referred to Noblit and Hare (1988) when presenting three approaches to conducting research reviews: meta-analysis, literature review, and meta-ethnography. Tese approaches difer in their degree of incorporating contextual knowledge. Meta-analysis synthesizes numbers by aggregating fndings to explain a phenomenon, such as what might increase learning achievement. In this approach, researchers gather all available information from studies that have measured the same variables and explain key mechanisms via either the exhaustive collection of research articles or random sampling. On this basis, experts can present generalizations as a universal standard that can be applied (more or less) independent of contextual knowledge.

Te second alternative is the literature review, where the overall purpose is to create a chain of reasoning that helps illuminate the phenomenon under study. In this case, both the theory and the results are considered relevant knowledge. Data collection must be exhaustive, and researchers are challenged to logically bridge summaries of results and interpretations for diferent studies.

Within the third group, where reviews are based on meta-ethnography, the validity issue is particularly challenging. Doyle (2003) argued that the aim of generalizability in meta-ethnography is based on case studies where the local context shapes the ways in which fndings and results are interpreted. Still, she asserted that it is possible to draw conclusions across case studies through synthetization, which generalizes knowledge beyond what is valid for single cases in the study. Such a generalization requires a particular methodology, which includes alternative analytical steps and aims at developing an increased understanding of a particular phenomenon. In many ways, this procedure is consistent with what Gough et al. (2012) labeled confgurational synthesis, or the compilation of descriptions, interpretations, and results that are reorganized based on analytical steps and concepts rather than on pre-determined concepts or aggregated results.

Beyond evidence-based reviews, knowledge sources can be scientifc by being built on empirical data collected by researchers who follow known procedures and general standards to examine a particular research problem. Such studies can be academic without any purpose of intervention, they can be clinical and result in knowledge to be applied for particular purposes, or they can be part of research projects and evaluations that guide actions through recommendations. According to Rasmussen et al. (2007), knowledge sources of this kind can serve diferent functions. Terefore, they suggested dividing educational research developed for scientifc purposes from educational knowledge sources and theories used for programmatic and practical purposes. Rasmussen et al. (2007) classifed evidence-based studies belonging to the frst category as applied research that can be useful for policymakers and practitioners. A core aim in such applied research is to serve clinical purposes and project best practices by the use of assessment standards.

In our study, we include four types of reference sources that provide knowledge and information to support arguments within policy documents: (a) formal documents that are not primarily a result of research but can indirectly mediate research and serve as one type of evidence, (b) meta-analysis, (c) confgurational reviews, and (d) empirical research studies. Tese four types of knowledge sources do not cover all knowledge and information that could be used; for example, we have not included experience, practice-based innovations, or theoretical knowledge in our analysis. However, through our policy documentation, we can examine how these four types of knowledge sources are used and translated as well as how they serve as a structural condition for how policy options for school reform are expedited.

## **Bibliometric Network Analysis**

An earlier examination of the 2020 reform revealed that the most cited knowledge sources within the bibliographies were predominantly of domestic origin and represented an interesting mixture of references, such as formal documents, research reports, and scientifc publications (Baek et al., 2018). Moreover, the most cited references were research and policy reports that had not undergone a scientifc peer-review procedure.

Figure 7.1 shows the network structure of all references in our database. Source documents are visualized as circles and ordinary references as squares. Te sizes of the nodes indicate their in-degree centrality, which is an indicator of the impact of a text assessed by its number of citations in other works. Te shading shows the geographic origins of the references with regional nodes colored gray, Nordic references white, and international references black.

Te bigger nodes located in the middle visualize references cited by more than one report and how they are connected to each other. Tese

**Fig. 7.1** Complete network structure of all references in our database. (*Note*: The network structure reveals that the publications are highly specialized and issuecentered, with little overlap between the various reports)

co-cited references represent the core knowledge shared by one or more reports and are therefore the focus of our attention. It becomes apparent that there are a relatively small number of co-cited references, which means that the cited publications are highly specialized and issuecentered, with little overlap between the various reports (Baek et al., 2018). For greater insight into the network of co-cited references, we made the following list of the 12 most cited references (i.e., cited more than fve times) in the database (see Table 7.2).

Te document at the top of the list, the white paper *Culture for Learning*, is a governmental report to the Norwegian parliament from 2003 to 2004. Tis foundational paper initiated the Knowledge Promotion Reform that was launched in 2006 and created systemic change in Norwegian education policy by introducing a national test system. Since policymakers presented the new renewal of the Knowledge Promotion Reform (2016/2020) as incremental and not foundational,


the Storting). Ministry of Education, Research, and Church Affairs

**196**

**Table 7.2**

Co-cited references in the full database


#### **7 Structuring School Reform Policy with Evidence…**

(*continued*)


#### **198 B. Hörmann and K. Sivesind**

we fnd it reasonable that this document from 2003 is the most referenced paper.

Te second listed publication is Hattie's *Visible Learning* (2009), a meta-analysis of studies on the efect of diferent educational infuences on student performance. Tis book is not only omnipresent in media and research about direct instruction in schools, but it also is on the top of what is considered to provide evidence according to academic classifcations (Doyle, 2003). Number three on the list is a Danish report on teaching competence and student learning in kindergarten and school that synthesizes international research about the relationship between teacher competence and student learning in schools. Nordenbo et al. (2008) authored this report on behalf of the Norwegian Ministry of Education. Because the synthesis builds on existing research on student learning with a focus on "what works," it represents another typical example of evidence-based knowledge.

Te rest of the publications are predominantly governmental or ofcial reports (NOUs), which we have categorized, together with the white paper *Culture for Learning*, as formal governmental reports. Te two empirical research studies from Aasen et al. (2012) and Nordahl and Hausstätter (2009) contain results from evaluation projects on the Knowledge Promotion Reform from 2006. While Aasen et al. (2012) presented a comprehensive evaluation of reform and governance within the Knowledge Promotion Reform, Nordahl and Hausstätter (2009) analyzed a specifc group of students, those in need of special support. Te research article from Durlak et al. (2011) reviewed existing knowledge about the improvement of student performance by fostering social and emotional learning. Te study by Durlak et al. (2011) was published in the journal *Child Development*, which has a comparatively high impact factor of 5.024, according to the publisher's website. Te article referred to student performance and provided evidence-based knowledge about factors that can improve this performance. Along with Hattie's *Visible Learning*, we classifed the Durlak et al. (2011) article as a scientifc publication in a narrow sense.

All in all, the list consists of publications that provide an interesting overview of prominent knowledge referenced by the school reform. Tis list refects a mixture of formats (i.e., governmental and ofcial reports, academic publications, and policy reports) and origins (i.e., national reports, international research, and one regional/Nordic report). Moreover, the group of knowledge sources represented in Figure 7.1 can be analyzed and compared according to their levels of evidence. Drawing on Doyle (2003), Gough et al. (2012), and Rasmussen et al. (2007), we have differentiated between the following types of knowledge sources: meta-analyses, confgurational reviews (i.e., theory-driven reviews of qualitative and quantitative studies), empirical research studies, and governmental papers. Table 7.2 shows the allocation of references to categories.

## **The Argumentative Translation of References**

As a frst step in our analysis of the argumentative translation of references, we identifed knowledge sources that were most often referenced within the two white and eight green papers. In addition, we identifed how often references appeared as in-text citations within the two white papers that are the basis of our semantical analysis. As shown in Table 7.2, the report on the evaluation of the Knowledge Promotion Reform (Aasen et al., 2012) had an overwhelming in-text citation quote of 20, while the white paper *Culture for Learning* from 2003–2004, which was the most co-cited document within the full database, was cited only six times. Moreover, John Hattie's *Visible Learning* was co-cited seven times within the bibliographies of the ten texts and referenced eight times within the texts themselves. Tis pattern indicates that Hattie's meta-analysis has been among the most infuential references to project options for reforming schools in Norway. However, a comparison of the actual citations with those from the bibliometric network analysis makes it clear that the report by Aasen et al. (2012) was the most infuential reference for the translation of evidence within the two white papers, as it was prominent on the overall lists of the ten documents and was the most cited source in the white papers themselves.

To put the most cited references in perspective of our two white papers, we decided to compare the prominent sources within the full database with the total number of quotations of all references listed in the two white papers. Te frequency of Aasen et al.'s (2012) evaluation report in this analysis was topped only by NOU (2015, p. 8) *Te School of the Future. Renewal of Subjects and Competences*, which was cited 35 times altogether. Te rest of the references in the two white papers were cited eight times or fewer, which underlines the importance of the Aasen et al. (2012) report in the two governmental papers (see Table 7.3) alongside the green paper authored by the public inquiry commission about the school of the future (NOU, 2015, p. 8).

In the list of the most referenced in-text citations (Table 7.3), we also found an expert report (Dahl et al., 2016) that was not a commissioned

**Table 7.3** Frequency of references (in-text) in the two white papers, more than eight citations


*Note*: The two references highlighted in gray, Aasen et al. (2012) and Hattie (2009), were also among the most cited documents in the full database

paper, but still of crucial importance in argumentation for a new reform. In addition, we discovered another research evaluation report by Dale et al. (2011) that analyzed the national curriculum and its implementation. Te list also includes WP#1, which was referenced in WP#2.

To trace the translations of these references and how they were shaped by argumentative modes of reasoning, we analyzed the paragraphs where the knowledge sources were referenced according to Luhmann's (2018) program forms, namely, the past-oriented conditional program versus the future-oriented purposive program. Tis analytical distinction helped us to assess if and how diferent types of reviews and reports were used to stabilize conditions that were already in place or to change education in a more strategic way. In this part of the analysis, we looked at the meaning units within the text, which consisted of one to three paragraphs following the reference. Moreover, we evaluated whether the argumentation manifested within the document referred to normative or descriptive statements. In so doing, we identifed whether statements and arguments referred to past conditions and actions (e.g., decisions about what to teach in schools) or future activities and results (e.g., what to accomplish and achieve). Tis way of conducting semantic analysis helped to clarify how the documents symbolized certain program forms and a regulative or a strategic use of references. Based on this analysis, we classifed how the references were translated within a matrix (see Tables 7.4 and 7.5 in the Appendix).

For example, when statements within the texts referred to governmental papers authorized by state authorities in the past, we classifed them in the frst column (regulative; structures). In this case, we considered the reference use to refect a formal regulation for how to provide education as a service to the population. When the documents made use of arguments to project expectations about performance and/or to discuss standards for how to assess the competence of the future learner, we classifed them in the fourth column (cognitive; outcomes). In the second and third columns we divided sources into arguments that appeared to be merely informed by knowledge in the present (the second column: substantive; content) and arguments that refected normative-oriented statements in the present (third column). Regarding the frst column, it is important to add that normative statements are considered regulative, while normative arguments in the third column are regarded as axiological (i.e., value-based) as they refer to the soft governing side of ongoing activities by expecting learning and change. Tese alternatives resulted in a four-feld classifcation (Sivesind et al., 2016).

In addition to this categorization, we added the titles and subtitles of the chapters in the matrix (Tables 7.4 and 7.5), which helped to create an overview of the thematic realms covered by the documents. Te table includes the titles of only those chapters in the two white papers that contained citations of prominent knowledge sources, and the titles are our own translations.

Finally, we conducted a comparative analysis of the translation of references in the two white papers. In the following section, we present the argumentative use of co-cited sources and most cited in-text references that belong to diferent categories or levels of evidence: meta-analysis, confgurational reviews, empirical research studies, as well as ofcial and governmental papers.

## **A Comparative Analysis of Reference Use**

#### **Meta-analyses**

Te following publications were allocated to this category:


In correspondence with their characteristics as meta-analyses, the two publications refer to studies based on student performance as a benchmark for successful learning and teaching. Tey present strategies, techniques, and approaches intended to increase student performance and, in their own terms, quality in education. Accordingly, they are mostly translated in a purposive way by giving normative directions for teaching and learning and projecting future goals.

While the quotations in WP#1 are more general, those in WP#2 are more specifc and contain precise results and knowledge from the meta-analysis. With one exception, the quotations present suggestions or recommendations that are clearly future-oriented as they refer to increasing student performance as a criterion for success. Te following example uses Hattie to legitimize a core feature of the reform (i.e., three specifc, core curricular goals); as with all quoted examples, the translations are our own:

Te department has defned the following three sector goals for basic education, which sum up core elements in both the objects clause within the education act and the national curriculum framework:


Te three goals are connected to each other and sum up the school's task for society. A good and inclusive learning environment is both a goal in itself and a tool for increasing the students' learning outcomes. Te objects clause points out, among other things, that schools and apprenticeship companies should meet students and apprentices with confdence and respect while working against all forms of discrimination. All children and youth should feel comfortable and be included. Schools that focus on a good and inclusive learning environment also reach better learning outcomes. (Hattie, 2009, Bakken & Seippel, 2012) (WP#2, pp. 15)

Te paragraph states that a good and inclusive learning environment leads to better learning outcomes, which indicates that the matter of student well-being is subordinate to learning outcomes. Since the paragraph explicitly refers to learning outcomes, we have categorized this citation as refecting a cognitive and learning-outcome-oriented program.

Te scientifc article by Durlak et al. (2011) is cited once in each of the two white papers, in both cases for the same argument: that soft skills contribute to student performance. Te frst citation presents knowledge on student learning, while the second one argues that soft skills also need to be supported from early on. By referring to the increase of student performance while subordinating soft skills to it, the reference to this publication clearly promotes a purposive program that suggests specifc measures that, in a normative way, puts student performance on top of the agenda, as shown by the following example:

Social and emotional skills like patience, mastering one's own feelings, curiosity, and mastering resistance, play an important role in student learning. (Backer-Grøndahl & Nærde, 2015; Birch & Ladd, 1997; Heckman & Kautz, 2013; Durlak et al., 2011) (WP#2, p. 42)

In both cases, the journal article is referenced together with additional sources. Te purpose of citing this scientifc article is probably to counter an often-raised critique against testing frameworks.

In general, the two white papers draw on meta-analyses in the following subject areas: student learning and the pedagogical work at school (WP#1) and quality in education, collaboration, early intervention, and competence development in the municipalities (WP#2). By underlining knowledge in these thematic areas, the meta-analyses support core points of the two white papers, namely, the need to strengthen student learning, quality development, collaboration, and competence development on the municipality level, even though some of these areas are not covered by the main arguments in the publications by Hattie (2009) and Durlak et al. (2011). Also striking is that Hattie's book is not used to argue for specifc methods in teaching, such as direct instruction or assessment practices, even though a "fundamental change in teaching" is announced in WP#2. It is more or less used as a justifcation for strengthening a purposive orientation to school reform in general.

### **Confgurational Reviews**

Te following reference was allocated to this category:

• Nordenbo et al. (2008)

Nordenbo et al. (2008) are cited surprisingly few times compared to other sources in this study. Te report is referenced seven times in the whole dataset, whereas it is cited only twice in WP#1 and once in WP#2. It serves as a core source for demanding more competence development to increase collaboration among teachers, to improve student performance, and to demand more teachers with subject-specifc training, especially in subjects that do not (yet) require subject-specifc education, such as art and handicraft education. Tese demands represent purposive programs by anticipating improved student performance, which will follow from a set of normative expectations expressed in this way:

It is thoroughly documented in research that the teachers' subject matter knowledge has an impact on the students' learning outcomes. Teachers who feel confdent about their subject matter are less dependent on predefned teaching designs and methods, and they can make variations and develop their own teaching further (Nordenbo et al., 2008). For example, it is documented that it is important that teachers possess the practical skills that their students are expected to develop (Espeland, 2011). For this reason, the Ministry of Education will evaluate whether one should introduce new competence requirements for teaching in several subjects at the primary school level. (WP#1, p. 74)

An interesting aspect in the example above is how the paragraph introduces Nordenbo et al. (2008). Te argument starts with "it is thoroughly documented in research," indicating that Nordenbo et al.'s report is indeed considered a signifcant knowledge source that provides the best research evidence, even though the report draws on several references that are, in fact, not what is typically regarded as scientifc. We categorize all three citations as refecting a purposive policy program, two of them as a normative type of reference and one as merely informative. Tis means that Nordenbo et al. (2008) is used to project best practices by creating cognitive expectations in one case. For the other two cases, the reference is used for a normative purpose, indicating what should be done to develop competence among teachers in schools.

Interestingly, the white papers make in-text references to Nordenbo et al. (2008) under the following headings: professions in the school, competence development and capacity building, and competence requirements for hiring and teaching. In only one case does the reference appear together with another source, which is an expert report about the role of the teacher (Dahl et al., 2016). Tis ofcial report also synthesizes education research and is used for partly purposive reasons (fve times within WP#2 with a purposive aim and two times within the substantive category, refecting a conditional orientation). Although this expert report describes the teacher role and elaborates on the need for professionalization of teachers, it covers several of the themes in WP#2, such as collaboration among teachers, the role of the teacher, and the department's assessment of the quality of the evaluation system. Tus, this report, which includes some confgurational reviews, covers a broader set of themes than Nordenbo et al.'s (2008) analysis. Tis fnding shows that confgurational reviews that focus on certain evidence might serve diferent functions and purposes than expert reports that are more comprehensive in terms of the knowledge sources they draw on. Yet, by comparing the use of various types of reviews, we fnd that confgurational analyses are used less frequently to specify cognitive outcomes than the metaanalyses of Hattie (2009) and Durlak et al. (2011).

### **Empirical Research Studies**

Te following references were allocated to this category:


It is perhaps self-evident that the evaluation of the Knowledge Promotion Reform plays a crucial role in the two white papers. Tese research-based evaluations were funded by the state and can be classifed as policy research (Christensen & Holst, 2017). Aasen et al.'s (2012) report on the evaluation of the Knowledge Promotion Reform accordingly has the highest number of citations in the two white papers but is co-cited only fve times in the whole dataset. Apart from a number of citations that present background knowledge in the reports, Aasen et al. (2012) is mainly used to legitimize the renewal of the Knowledge Promotion Reform (2016/2020) by showing weaknesses and creating options for discussing new issues in the context of the current reform. In so doing, it delivers core arguments for the basic measures of the renewal of the Knowledge Promotion Reform (2016/2020), including that basic skills should be integrated in the subject matter curricula, that goals and content need to be more explicit, and that the connection between subject matter and the new general part of the curriculum should be strengthened. In addition, Aasen et al. (2012) is used in WP#1 to argue for strengthening the on-site work on curricula, activating the political level (school owners), clarifying the responsibilities and the relationship between diferent levels in the education system, improving schools' ability to understand and make use of quality assessment results, and promoting a more decentralized competence development system in which municipalities should gain more scope of action.

Tis long list of demands shows the impact of the publication and reveals that the authors of WP#1 obviously counted on the evidence provided by Aasen et al.'s (2012) report. However, most of the quotations (12 out of 20) appear in WP#2. Tese quotations mainly deal with the question of how diferent levels in the municipalities and the schools share responsibilities through collaboration and process the information of student assessment tests across contexts. Tis information is used as background information and as additional support for the demand for a more enhanced system of competence development at schools, as the following example shows:

Te evaluation of the Knowledge Promotion Reform showed that the reform contributed to increasing the quality of basic education, but that many small municipalities and schools experienced challenges in implementing the reform. From the perspective of the county governors, many municipalities have gained a stronger grip on the role of school principals and the responsibility for schools in their municipality. But the regional governor also perceives diferences among the municipalities. Tese diferences are frst of all connected to the size of the municipalities, but also to their way of organizing. Small municipalities are in many cases vulnerable concerning their economy and competence. Some of them compensate for this with an established network for collaboration on the level of the municipalities and the schools. (Aasen et al., 2012) (WP#2, p. 31)

#### **7 Structuring School Reform Policy with Evidence…**

Te citation is not normative as such but describes in a rather open way how municipalities have implemented the 2006 reform. It does not suggest a specifc solution but mentions that some municipalities draw on networks of collaboration for coping with the challenges they are facing. We have categorized the citation as oriented to substantial issues and thereby belonging to a conditional type of program.

In WP#1, however, quotations are explicitly used to justify both specifc demands and the renewal of the curriculum in general, as below:

Tere is research that indicates that the General Part and the Principles for Education are part of the local work with curricula only to a small extent under the Knowledge Promotion Reform (Aasen et al., 2012). […] In order to create better cohesion in the curriculum framework, the Department wants to renew the current General Part, Principles for Education, and the subject curricula. Tis is supposed to contribute to a more holistic curriculum framework, updated for today's and the future's society. (WP#1, p. 19)

All in all, we categorized citations from Aasen et al. (2012) either as substantially oriented, refecting a conditional approach, or as normative, refecting a purposive approach. WP#1 uses the research study in a normative way, while WP#2 uses it in a more substantive way. Tis fnding is an interesting diference because we could have expected the opposite pattern. One might interpret this pattern as a dynamic use of evidence: in issues that are more conditionally oriented, such as the curriculum, the reference points to the purposive side; in thematic areas that are more purposive, such as quality development, Aasen et al. (2012) is used to strengthen the conditional side. Another explanation could be that the temporal order of the two white papers and the mandate framed how the policymakers utilized evidence in the two white papers.

Te argument that draws on the reference to Nordahl and Hausstätter (2009) is quite inconspicuous. By citing the number of students who are in need of support for special needs in the course of their education, WP#2 ofers insight into conditions for organizing special needs education. We categorized the citation as informative, associated with a conditional type of program, opposite of a normative or cognitive argument that promotes strategic actions. Both the Aasen et al. (2012) and Nordahl and Hausstätter (2009) publications provide national and local knowledge about the former school reform, the Knowledge Promotion Reform of 2006.

Dale et al. (2011) authored another research report, based on the same conditions for conducting research. Tis report is not among the most frequent co-cited publications, but it is one of the most cited in-text references in WP#1. It is cited eight times, four of which are in conjunction with Aasen et al. (2012). Only one of the eight citations uses the source merely to inform about reform conditions in a substantive way. Instead, WP#1 uses the Dale et al. (2011) source to strengthen the purposive orientation of the reform proposal. None of the reports in this category are, however, used to create narratives or specifcations about outcomes or best practices, which is also an interesting observation.

## **Formal Documents: Governmental and Offcial Reports**

Te following publications were allocated to this category:


Formal documents make up more than half of the co-cited references on the list of the most cited references. Tey play a crucial role in the reform because they serve as a link to previous reforms and arguments for reforming education in earlier periods. Mostly they serve as normative points of reference, from which specifc further steps in the renewal of the Knowledge Promotion Reform (2016/2020) are deducted. In this way, former written and published ofcial knowledge is credited with legitimized power that does not need further justifcation.

Te following quotation is the starting point for arguing for more collaboration among teachers and principals to enhance quality development and student learning:

One of the ambitions of the Knowledge Promotion Reform was that schools should develop a culture for learning to a larger extent. Te starting point for this goal was the St. Meld. 30 (2003–2004) *Culture for Learning*, which stated that schools have to be learning organizations. Te goal was that the schools' ability and willingness to learn and develop them further should be improved. (WP#2, p. 27)

Te governmental and ofcial reports establish continuity in the reform process by providing a history that connects all previous reforms with those that will be implemented. Te use of formal documents in the two white papers appears to be evenly distributed, with a slight tendency for WP#2 to lean more heavily on formal documents. With only a few exceptions, we categorized the use of formal documents as regulative, refecting a conditional program for reforming education and schooling; as such, they stabilize the reform rather than renewing and transforming the schools in radical terms.

## **Conclusions**

In this chapter, we analyzed the distribution and argumentative use of co-cited and in-text references within and across policy realms of Norway's most recent school reform. We aimed to examine how policymakers and experts responsible for writing two white papers and eight public inquiry reports responded to critiques and trends that have generated demands for a more evidence-based policy. We asked how state administrative bodies such as ministries and inquiry bodies made policy evidence-based by referencing diferent types of knowledge sources and how their modes of arguing for reform translated knowledge in terms of two decision programs that each covers two types of arguments. As a result, we assessed how references were embedded in regulative, informative, normative, or cognitive arguments.

Both our bibliometric network and semantical analyses revealed that the authors and bodies involved in this work have reacted to the critique that inquiries have been too practical and general in earlier reform periods. Tis reaction is documented in the extensive usage of references within the white and green papers. Troughout this chapter, we have demonstrated that Norwegian reform documents authorized by the Ministry of Education are thereby heavily evidence-based, as shown through their references to various types of knowledge (e.g., research reviews including meta-analyses and confgurational analyses) and their use of phrases like "as thoroughly documented in research." Te core basis of the most cited documents within the full database contains not only typical evidence-based reviews, but also many governmental documents; in fact, more than half of the most co-cited references are governmental reports or documents. In addition, by taking in-text references into consideration, we showed that the report of Aasen et al. (2012) plays a crucial role by setting the scene for reform and renewal across a variety of topics within both policy realms related to the curriculum reform and to educational governance of the basic education system.

Te argumentative use of the references in the two white papers corresponds, more or less, to the type of category they represent. In other words, reviews in the form of meta-analyses and confgurational analyses tend to be used in a purposive manner, while governmental papers are mainly used to ensure that certain conditions are in place and formal procedures are followed (conditional manner). However, some observations show interesting variations and unexpected translations.

First, there is a commonly held view that reform options become standardized through evidence-based policy that makes use of measurements as quantifable expressions of performance. In our study, relatively few citations specify standards for best practices (see "cognitive outcomes" on the right side of Tables 7.4 and 7.5). Few publications that were cited in the white papers provide measures and standards for how to improve student learning and outcomes by purpose. Te absence of knowledge provided by international organizations such as the Organisation for Economic Co-operation and Development (OECD) among the most co-cited references is possibly a reason for this pattern. However, another important fnding is that the scientifc reviews are in no case used to regulate or inform about past policies or practices in a conditional form, but only in terms of arguments that call for purposive, future-oriented changes within the education system. Tus, the use of scientifc reviews directs attention toward performance-oriented outcomes.

Our analysis shows that meta-analyses and confgurational analyses serve as sources for normative recommendations for how to improve student learning instead of explicitly referring to results of performance studies or specifying cognitive outcomes. Tis fnding holds particularly true for WP#1, whereas WP#2 references Hattie (2009) fve times to specify cognitive outcomes. Interestingly, another important fnding is that the scientifc reviews are in no case used in a conditional way; instead, they are used as arguments for purposive, future-oriented changes within the education system.

Second, our analysis shows that there is signifcant use of governmental reports for regulative use, which means that arguments in the two white papers refer to formal procedures or decisions documented in those governmental reports. Te use of formal documents seems to help with identifying the intention of past decision processes. In addition, they stabilize the reform by creating memories of the past. However, in WP#1, governmental reports are used not only in a regulative manner, but also in substantive, normative, and cognitive arguments. Te use of NOU, 2015: 8 is the prime explanation for this pattern, as it is used frequently to legitimize both conditional and purposive program forms. Tis source is the only one among the most prominent references that is used to make all four types of arguments (i.e., regulative, informative, normative, and cognitive). A dominant pattern is that NOU, 2015: 8 serves a purposive role to legitimize the renewal of the national curriculum. Te report was cited in chapters like "competence aims within the subjects," "an increased cross-disciplinary orientation in teaching and learning," "strengthening clearer priorities within the subjects," "emphasizing improved coherence and progression," "learning strategies and refection on one's own learning," and "competence development and capacity building" (see Table 7.4). Te pattern is in no way surprising, as the authors of WP#1 were mandated to legitimate the renewal of the reform by drawing on earlier governmental and ofcial reports, particularly NOU, 2015: 8 *Te School of the Future*. Te document, however, is not included as a reference in WP#2, which addresses issues related to early intervention and quality in schools.

Tird, we found that the program evaluation reports that contribute empirical research on education reforms are used for various purposes. Interestingly, Aasen et al. (2012) is distributed over two categories, refecting both substantive and normative forms of use. While it tends to be translated in a normative-oriented way in WP#1 in tandem with the Dale et al. (2011) report, authors of the WP#2 apply Aasen et al. (2012) in a substantive, retrospective way. In WP#2, the use of Aasen et al. (2012) difers from references to Dahl et al. (2016), which was formally commissioned by the state as an expert report. Dahl et al. (2016) summarized knowledge about the teacher as a key agent within the education system, for which it is used in a normative way for several themes, such as "teachers' collaborations in school." For the sections that cover quality assessment and the evaluation system, neither the reference to Aasen et al. (2012) nor Dahl et al. (2016) sets the direction in a purposive way (see Table 7.5).

Finally, the reference patterns we have uncovered through our analysis help us explore the role of evidence in education policy. Evidence seems to be structured by the knowledge sources that are used while being deeply dependent on the arguments formulated by policymakers. Looking for examples in the content of the references, we see a clear pattern in which formal documents or governmental papers connect reform options with political ambitions and jurisdictions. Tese connections exist because the formal documents ofer information about various conditions for organizing education and because they are mandated to do so. Moreover, empirical research studies and reviews that refer to the inner life of schools can be used to inform policymakers about educational concerns in renewing teaching and learning. Tus, these references are afliated with topics that policymakers potentially address when they explore options for education reform. If they are also written under the auspices of state-funded programs to evaluate the reform, this condition is likewise an explanation for their prominence. We discovered that the two white papers frequently cite evaluation reports about the former curriculum reform, the Knowledge Promotion Reform 2006. Tis fnding underlines how the evaluation and use of research reports enable a tight connection between the previous and the current reform. Apparently, results and interpretations from the evaluation of the Knowledge Promotion Reform inform policymakers; more importantly, they serve as a normative basis for formulating specifc policy options that aim at curriculum renewal and improvement.

No matter their intended use, references serve the function of bringing evidence into policy, in terms of either bibliometric networks or explicit use to formulate arguments within the bodies of the texts. Trough this function, references align systems of reasoning that are highly diferentiated in practice. In this way, policy knowledge plays an inter-mediational role by shaping reform options connecting both policy realms and systems of reasoning.


**Table 7.4** Classifcation of references from WP#1 St.M. 28 curriculum renewal



Government



Norwegian Government.

WP 20 (2012) QaD = St. Meld. 20 (2012–2013). *På rett vei. Kvalitet og mangfold i fellesskolen.* [On the Right Path. Quality and Diversity in the "School for All"] (White Paper No. 20 to the Storting). Ministry of Education and Research.

WP 29 (1994) PaG = St. Meld. 29 (1994–1995). *Om prinsipper og retningslinjer for 10-årig grunnskole—ny læreplan* [Principles and Guidelines for 10-Year Compulsory Schooling—New Curriculum] (White Paper No. 29 to the Storting). Ministry of Education, Research, and Church Afairs.

WP 30 (2003) CfL = St. Meld. 30 (2003–2004). *Kultur for læring* [Culture for learning] (White Paper No. 30 to the Storting). Ministry of Education and Research.

WP 20 (2012) QaD = St. Meld. 20 (2012–2013). *På rett vei. Kvalitet og mangfold i fellesskolen.* [On the Right Path. Quality and Diversity in the "School for All"] (White Paper No. 20 to the Storting). Ministry of Education and Research.

WP 28 (2015) CR = St. Meld. 28 (2015–2016). *Fag– Fordypning— Forståelse. En fornyelse av Kunnskapsløftet* [Subjects—In-Depth Learning— Understanding. A Renewal of the Knowledge Promotion Reform] (White Paper No. 28 to the Storting). Ministry of Education and Research.

WP 29 (1994) PaG = St. Meld. 29 (1994–1995). *Om prinsipper og retningslinjer for 10-årig grunnskole—ny læreplan* [Principles and Guidelines for 10-Year Compulsory Schooling—New Curriculum] (White Paper No. 29 to the Storting). Ministry of Education, Research, and Church Afairs.

WP 30 (2003) CfL = St. Meld. 30 (2003–2004). *Kultur for læring* [Culture for learning] (White Paper No. 30 to the Storting). Ministry of Education and Research.

## **How to Read the Matrices in Table 7.4 and 7.5**

## *Columns:*

Te column on the left lists the titles of chapters and sub-chapters of the white papers in which we found references from our database in the text. Te next four columns refer to our analytical categories based on Luhmann. Conditional and purposive programs are divided into regulative and substantive forms on the one side and normative and cognitive forms on the other side.

*Rows:*

Each of our references stands for a specifc type of knowledge, which we highlighted in diferent shades of gray (see also Table 7.2 in the text).


References written in *italics* are in-text references, while references written in ordinary style originate from the list of the most cited documents.

Each reference was assigned a program form based on the framework of Luhmann (2000) and Sivesind et al. (2016) according to its use in the text of the white paper.

## **Notes**


## **References**

Aasen, P., Møller, J., Rye, E., Ottesen, E., Prøitz, T., & Hertzberg, F. (2012). *Kunnskapsløftet som styringsreform—et løft eller et løfte? Forvaltningsnivåenes og institusjonenes rolle i implementeringen av reformen [Te knowledge promotion curriculum as a governance reform—Promotion or promise? Te roles of the administrative levels and institutions in the implementation of the reform]*. NIFU Rapport 20/2012.


*intentions, preconditions, and operationalizations of the knowledge promotion reform: An analysis of a curriculum reform]*. University of Oslo.


Toulmin, S. (1958). *Te uses of argument*. Cambridge University Press.

White Paper No. 30 to the Storting, C. f. L. (2003–2004). In *Te Royal Norwegian Ministry of Education, and Research Afairs Oslo: Norwegian Government Service Centre*.

**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **The Complexity of Context in Legitimating National School Reforms: The Case of Sweden**

**Andreas Nordin and Ninni Wahlström**

## **A Theoretical Background**

Policy research has convincingly demonstrated the signifcance of the role of policy entrepreneurs, who bring in new ideas for how to defne and solve policy problems, in policymaking processes (e.g., Verger, 2012). One of the resources that policy entrepreneurs draw upon is empirical research. However, entrepreneurs often advocate for certain ideologies and perspectives, using research in a strategic way to promote a particular solution. Te identity and ideology of the knowledge entrepreneur also matter, including whether they are involved in a particular political party, a certain think tank, or a non-governmental organization or whether they work as a researcher at a university. According to McDonnell and Weatherford (2013), research-based evidence is likely to be used in combination with other evidence in line with the elected ofcials' core values that evoke their positive response. For example, Kingdon (1995) noted

Linnaeus University, Växjö, Sweden

e-mail: andreas.nordin@lnu.se; Ninni.wahlstrom@lnu.se

A. Nordin (\*) • N. Wahlström

<sup>©</sup> Te Author(s) 2022 **227**

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_8

that policymaking and agenda setting are not linear processes, starting with the problem formulation and ending with a policy decision for a solution. Instead, policymaking is characterized by parallel streams of politics and policies, with the implication that a solution can very well be preferred and suggested before a more specifed policy problem is formulated.

Tis chapter draws on two bodies of related research from comparative and international education informing the research questions and making up the interpretative framework to examine the complexity of both layers and sequencing. Te two bodies of research are the role of networks in the shift from government to governance and the study of "traveling reform" in difusion versus reception studies.

Te general shift from government to governance in Western countries is commonly seen as the result of new public management policies that most OECD countries introduced in the wake of the neoliberal reforms of the 1980s and 1990s. In this shift, policy networks play a prominent role (Lubienski, 2019). In the education sector, this shift created a new role for the state, new ways of regulating the education system, and new tools for generating, or alleviating, reform pressure. In turn, the outcomes of new public management reform projects triggered a proliferation of standardized student assessments. International comparisons (including international large-scale student assessments such as PISA, Trends in International Mathematics and Science Study [TIMSS], and Te OECD Teaching and Learning Study [TALIS]) have since been referenced as key sources in policymaking for a variety of reasons. Governments have used them as primary monitoring tools to assess the quality of their teachers, schools, districts, and education systems, as well as to make policy decisions based on these assessments (Addey et al., 2017; Lingard & Sellar, 2016). Te shift from government to governance has fueled a "governance by numbers" (Grek, 2008; Lindblad et al., 2018; Mølstad & Pettersson, 2019) approach, while also empowering non-state actors to participate in the new millennium as key policy actors. Tis has been interpreted as a clear sign of the "disarticulation and diversifcation of the state system," the "destatalisation" of the policy process (Ball & Junemann, 2012, p. 24), and a changing role for the state where the state has handed over previous national infuences and responsibilities to partly new arenas such as international organizations and private actors (Ozga & Lingard, 2007; Wahlström, 2014). Strikingly, the interplay between the state and local actors does not necessarily follow formal decision routes where the state delegates professional responsibility to its civil servants. Instead, at times, the state actively intervenes in the business of local practices through the use of evidence, formal standard setting, and sanctions (Wahlström & Sundberg, 2018).

We also draw on research on traveling reforms, which includes two diferent perspectives. Researchers who adopt a bird's-eye view on the dissemination of international standards or global scripts in education take their point of departure from the global or the world perspective. Tis perspective enables an understanding of why and how some policies become scaled up and spread while others do not. Such a perspective allows for an understanding of the active role of international organizations in lending or disseminating specifc policies and programs, coined as "best practices" or "international standards" (Bromley & Meyer, 2015; Krücken & Drori, 2010). Such a "perspective from above," however, provides only one of several possible angles for understanding the spread of global education policy. By contrast, a "perspective from below" illuminates why, how, and when national or local policy actors selectively borrow global education policies (Anderson-Levitt, 2003; Sivesind & Wahlström, 2016; Steiner-Khamsi, 2012; Steiner-Khamsi et al., 2020; Wahlström, 2020). In these studies, the researchers examine which elements of international policy discourses will be of interest to national policy actors to interpret and reconstruct for the legitimization of national/local policy needs. Te researchers examine the way global policies are used for legitimate and/or delegitimate purposes when recontextualized into national policy agendas.

In this study, the perspective is "from below." From a national level, we explore how the Swedish government makes use of various knowledge sources of evidence for legitimating its reform recommendations. We are interested in the various international, regional, and national policy and knowledge actors that the government turns to for providing evidence for its recommendations for decisions.

## **Purpose and Research Questions**

From this point of departure, the aim of this study is to advance the understanding of the mechanisms at stake in legitimating national school reforms. More precisely, we examine the evidence used by the Swedish government in the 2015/2018 Knowledge Achievement Reform as manifested in the reference lists of the green papers (GPs) and white papers (WPs) analyzed. For this purpose, the following research questions are relevant:


## **Contextualizing the Swedish Decision-Making Process at the State Level**

Over a long period of time, Sweden has appointed commissions as a way for governments to produce evidence for political decision-making and for the "anchoring" of policy recommendations in diferent segments of Swedish society. Te Swedish system stands out because of its strong emphasis on the preparatory stages. Commissions of inquiry have two distinguishing features. First, they are appointed by the Cabinet and have a legal status similar to other government bodies. Second, commissions of inquiry are set up with a mandate to investigate a special thematic area or particular subject (Petersson, 2016). Tis institutional arrangement is not inscribed in Swedish law but has evolved as a praxis over a long period of time. Almost all pieces of legislation are prepared in this way, which means that around 200–300 governmental commissions are at work at any given time in the political system. By international standards, Swedish ministries are quite small; therefore, appointing commissions may also be a way to temporarily expand the staf of the ministry outside the ordinary budget. Once constituted, the commission has a considerable degree of autonomy. Although eforts have been made to limit both the size of the commissions and the time period for their work, it is still not unusual for their work to extend over several years (Trägårdh, 2007). Teir work is fnally presented and handed over to the responsible minister in the format of government ofcial reports (green papers, known by the Swedish acronym SOU). Appointing commissions of inquiry has to do with the political relationship between the state and civil society. Tese commissions have served as a "linchpin in a system of democratic governance whose hallmark is deliberative political practices that involve a mix of civil servants, politicians, academics, experts, and representatives of relevant civil society organizations" (Trägårdh, 2007). As Trägårdh (2007) noted, this extended communicative law-making process also contributes to societal legitimacy and acceptance for a new law or policy and adds trust to the system.

Today, the timespan of the policymaking process has become shorter compared to the praxis during the mid-twentieth century. In the 2010s, the commissions of inquiry were more tightly controlled by the Cabinet and government ministries, while parliamentary representation declined. While the level of striving for consensus in the inquiry process has decreased, the level of conficts has increased and the politics in parliament have become more polarized (Petersson, 2016). Moreover, in recent years, Swedes have demonstrated a decline in their trust of the political system as a whole, with a widened gap between the voters and the elected as a result (Amnå, 2006). In addition, Gunnarsson et al. (1998) have shown a shift in the way commissions were composed between 1988 and 1997, with a decrease in the number of larger commissions with a chairman in favor of one-person commissions with additional members and experts linked to the investigation. Tis development has continued. In a study analyzing the period 1990–2016, Dahlström et al. (2019) showed that 90% of the ongoing commissions of inquiry in 2016 were in the form of one-person commissions. At the same time, barely 3% consisted of parliamentary committees of inquiry. Dahlström et al. (2019) concluded that the kind of committees that once constituted the backbone of the committee system (i.e., broad parliamentary committees) have almost disappeared today. We understand this development as an expression of trying to rationalize and speed up the process of national policymaking in the context of increased global competition (see Kamens, 2013; Lewis & Hogan, 2019).

## **A History of Borrowing**

Swedish school reforms have a well-established history of borrowing policy from other educational systems, particularly for policy domains related to curriculum and governing of schools (Hallsén & Nordin, 2018). Te Governmental School Commission of 1946 talked about the Swedish system of whole-class teaching as being infuenced by "German school life" (SOU 1948:27, p. 112). Te same report also revealed that members of the commission had taken study visits to Denmark, Norway, Belgium, France, Switzerland, the United Kingdom, and the United States. During the period leading up to the realization of a nine-year comprehensive school for all children in 1962, Sweden also became an infuential policy lender, with Stockholm as the leading marketizer of the new Swedish school system that attracted visitors from all over the world (Hallsén & Nordin, 2018). Educational reforms were thoroughly planned in terms of time and resources, and school experiments and investigations were often led by researchers and described as an expression of social engineering (Román et al., 2015). Similarly, in the SOU that preceded the curriculum reform of 1980, *Te School's Inner Working* (SOU 1974:53), an entire chapter is devoted to international examples of ways to improve a school's work environment, with lessons drawn from England, Eastern Europe, and Denmark. Te report *School for Bildung* (SOU 1992:94) featured a shift away from the country-specifc references with talk in terms of a wider ongoing "internationalization," of which Sweden is part, and references to specifc countries as part of organizations such as the United Nations (UN) or the European Community. From 1995 on, following Sweden's membership in the European Union (EU), education policy in the country has been elevated to a priority on the reform agenda. Even though Swedish policymakers have refrained from making explicit references to EU policy, developments in other EU countries have had a visible impact on school reform in Sweden, which has led scholars to talk about the Swedish strategy as a "silent policy borrowing" from the wider European education policy space (see Nordin, 2012; Waldow, 2009). In recent years, the OECD has emerged as an increasingly important source of expertise and authorization for the Swedish government when making reforms. Te OECD today functions as an authoritative reference system in justifying borrowing of international standards created in PISA or national OECD expert reports (Grek, 2019; Ringarp & Waldow, 2016; Wahlström, 2018; Waldow, 2012).

#### **The 2015/2018 Knowledge Achievement Reform**

In 2006, the Swedish government decided to appoint a one-person commission led by a civil servant to investigate the implementation of the new goal-oriented curriculum launched in 1994. In the report that was published in 2007 (SOU 2007:28), the investigator concluded that the new public management of the Swedish school system along managerial principles and decentralized power had not worked out as hoped for. Tis fnding led to the launch of a whole range of fundamental reforms in 2011, including a new school law, a new national teacher education, a new curriculum for the comprehensive school, and a new, more diferentiated grading system, all to enhance monitoring and central control in a decentralized school system. In addition, Sweden at the time sufered from growing concern over continuously declining PISA results. When Sweden received its lowest PISA results ever in 2012, the national school crisis was a fact (Wahlström, 2018) and fueled the national crisis discourse and an emerging "scandalization" (Steiner-Khamsi, 2003) of the comprehensive school within politics and the media (Nordin, 2012, 2019). Against this backdrop, the Swedish government decided to take the unconventional measure to turn to the OECD for help (Pettersson et al., 2017). After a period of analyzing and visiting Sweden, the OECD published a fnal report in 2015. Te OECD stated that more eforts were required to fulfll Sweden's commitment to excellence and equity (OECD, 2015) if the country wanted to come to terms with its national problems. In the report, the OECD suggested a focus on the following three areas: (a) promoting quality with equity across Swedish schools, (b) building a high-quality teaching profession, and (c) steering policy and accountability focused on improvement. To address the OECD report and further refne the reforms of 2011, in April 2015, the Swedish government appointed a government commission called the 2015 School Commission, chaired by the Director-General for the Swedish National Agency for Education. During the last six months of the commission's work, a researcher took over the chair position because the former chair was appointed Minister of Education. Te 2015 School Commission was thus close to the government.

Te commission presented its suggestions in SOU 2017:35 entitled *A Gathering for School—A National Strategy for Knowledge and Equivalence* (Source Document 2). In the white paper 2017/18:182 (Source Document 1) that paved the way for the 2015/2018 Knowledge Achievement Reform, the SOU plays an important role and is explicitly mentioned together with SOU 2016:94 (Source Document 7) and SOU 2017:51 (Source Document 8) as central to the law-making recommendations formulated in the white paper. Te OECD thus played an important role as policy adviser (Lingard & Sellar, 2016) in setting the agenda for the Swedish reform, operating as a "boundary organisation" (Grek, 2019) that pointed out what policy areas to prioritize and indirectly what expert knowledge would be of most use.

## **Methodological Considerations**

We have pursued the research questions by analyzing references made in published policy documents. Te bibliometric network analysis is focused on "ofcial knowledge" (see Baek et al., 2017) or texts produced by the Ministry of Education and Research (white papers) and by its appointed commissions (green papers). Since the white paper had no reference list, we began by searching through the entire volume, resulting in a total of 12 cited green papers of which one lacked a reference list. Of the 12 documents, two documents (SOU 2016:38, an interim document, and SOU 2016:38, the same document as the selected green paper [GP] but published two days earlier) were duplicates and therefore removed. Another two documents (SOU 2016:77 and SOU 2017:49) were not directly related to compulsory education. Tus, the delimitation was made due to (a) duplicate documents and (b) documents not related to compulsory education. We then ended up with eight green papers that served as source documents of which one lacked a reference list. In total, we have nine (eight green papers and one white paper) source documents for which we have frst entered and analyzed a total of 1615 references. In the second step, we adjusted the number of references so that references cited by multiple sources were counted only once, resulting in a total of 1421 references. Table 8.1 illustrates the source documents and the distribution of references.

In addition to the quantitative analyses of citation frequency, we have classifed the reference distribution according to location and type of document. Te location category distinguishes between domestic, regional, and international references, where *regional* refers to the Nordic context. Te type of document distinguishes between the following subcategories: reports, books, academic publications, governmental publications, and others.

Each reference has been given a unique identifcation number, creating a matrix for calculating in-degree centrality and co-citations in order to understand the reference network of the 2015/2018 Knowledge Achievement Reform. We used the software programs UCINET 6.627 and NetDraw for analysis and visualization of the data (Borgatti et al., 2002).1

## **The Bibliometric Network Analysis: Findings**

Te bibliometric network analysis yields several interesting patterns related to the three research questions presented in the introduction of this chapter.

#### **236 A. Nordin and N. Wahlström**

**Table 8.1** Distribution of references in the policy documents of the 2015/2018 Knowledge Achievement Reform


*Note*: ID = Identifcation number of included source documents; WP = white paper; GP = green paper

#### **Reference Patterns in the Swedish School Reform**

Bearing in mind the central role of the OECD in the launch of the 2015/2018 Knowledge Achievement Reform, it is interesting to see that as much as 80% of the references are domestic, mostly referring to various types of laws and regulations on how to govern the Swedish school system at large. Of the remaining references, 19% are international, and only 1% are references to other countries in the Nordic region. As shown in Table 8.2, Sweden thus distinguishes itself both in terms of the amount of domestic references and in the lack of references to the other Nordic countries.

Sweden is an active member of the UN, so the UN Convention on the Rights of the Child has played an important role in Swedish policymaking ever since the parliament ratifed it without any reservations in 1990. Tis importance is evident in the results where the UN Convention on the Rights of the Child is the most cited international document containing regulations. Recently, the parliament has decided to make the UN Convention on the Rights of the Child Swedish law from 2020, increasing its signifcance even more. Looking at the international references in total, the OECD stands out as the most frequently cited. Te OECD is the ffth most cited publisher after the Swedish government, the National


**Table 8.2** Reference distribution

*Note*: References that are cited by multiple sources are counted only once. ID = identifcation number; Int'l = international; Gov't = governmental

Agency of Education, the parliament, and the school inspectorate, in that order.

Another distinct feature of the reform is the limited use of academic publications; in fact, the white paper and three of the green papers do not contain a single academic reference. References to scientifc literature are made in the format of synthesized knowledge. Hattie's publication *Visible Learning* stands out as the most cited document of all, together with Swedish school law. Another research review that has had an impact is the book *Excellent Teaching* by Håkansson and Sundberg. Table 8.3 shows the distribution of the most cited documents in the Swedish reform.

Looking at the level of authors, it is interesting to see that the most cited researcher is Professor Jan-Eric Gustafsson, who became the chair of the 2015 Swedish School Commission after the Director-General for the Swedish National Agency for Education (*Skolverket*) Anna Ekström, who was the original chair, was appointed Minister of Education and Research. Professor Jan-Eric Gustafsson has a total of 11 citations, followed by Linda Darling-Hammond with seven citations and John Hattie with six citations. Te commission led by Professor Gustafsson wrote the green paper SOU 2017:35 *Gathering for School—National Strategy for Knowledge* 


**Table 8.3** Most cited documents

*Note*: Cutoff point: minimum four times. Location: 1 = domestic; 3 = international. Type: 2 = book; 4 = report; 5 = other publications

*and Equality: Final Report from the 2015 School Commission* (Source Document 2), which was central to the 2015/2018 Knowledge Achievement Reform. Even though an entire commission is responsible for writing a green paper, this result shows the importance of the chair of a commission (together with its secretariat) in setting the direction and, in this case, fltering the kinds of references to be included (see Christensen & Holst, 2017). In the Swedish reform, where only 9% of the references were academic, even a small number of references makes a signifcant difference in the percentage of the total number of references in that category.

## **The Structure of the Evidence Base**

Te network design of the study also enables an examination of the relations between the expert commissions included in this study in terms of shared knowledge, refected as explicit references in the source documents. As Fig. 8.1 shows, the overall picture is that the various commissions have produced highly specialized knowledge in their reports.

**Fig. 8.1** Complete network structure. (*Note*: Regional = gray; domestic = white; international = black; source = circle; node size = in-degree centrality)

Te references in Fig. 8.1 are colored based on their domain, where domestic references are white, regional references are gray, and international references are black. Te size of the nodes indicates the document's in-degree centrality, that is, a document's centrality in the network based on how many texts refer to that specifc document.

Te colors illustrate the dominance of domestic references and the lack of Nordic ones in the Swedish reform. Te white paper (WP) (Source Document 1) is marked as number 1 in the middle of the network structure. Te largest feld, located above the central point constituted by the WP, is GP 2, *Gathering for School* (Source Document 2), which represents the commission of inquiry most directly responsible for preparing a succeeding recommendation for a reform and also bears the same name in Swedish as the WP. Among the shared references, most are domestic governmental references, in line with the overall picture. However, two international references stand out as cited by many. Te most cited one is Hattie's book *Visible Learning*, which is cited six times and marked as a black square with the number 1287. Its size also signals importance in terms of shared knowledge, as many cite this reference. Te other international reference that stands out is the UN Convention on the Rights of the Child, marked as a black square with the number 1275 placed on the left side of Hattie in the network. Te references to the UN Convention on the Rights of the Child are part of a larger discourse in Swedish school governing documents, where the UN Convention on the Rights of the Child has played an important role for a long time. Since January 2020, it has also become part of national law.

Also characteristic of the Swedish reform is the limited use of academic references. Out of eight source documents, four do not have a single reference to academic research. As previously mentioned, when referring to research, it is predominantly in the format of synthesized knowledge. In the books authored by Hattie and Håkansson and Sundberg, research knowledge is collected, interpreted, and presented in a supposedly more manageable and accessible format, making them "intermediaries" (Lubienski, 2019) between science and politics in providing policymakers with seemingly "useful" evidence.

**Fig. 8.2** Complete network structure without the white paper

Removing the white paper from the network structure shows the important role of GP 2 *Gathering for School* (Source Document 2). Te feld representing this GP is located in the middle of Fig. 8.2.

GP 2 has mutual references primarily with Source Documents 6, 7, and 8. Characteristic for these three GPs is the fact that they were all produced after the 2015 OECD report with themes in line with the OECD's recommendations for improving quality and knowledge results in Swedish schools. However, the distribution of references does not refect this thematic relationship. While GP 6 (Source Document 6) on the action guarantee for reading, writing, and mathematics has 32% international references, GP 7 (Source Document 7) on students' absences has 81% domestic references and 18% international references. Likewise, GP 8 (Source Document 8) on the adaptation of rules for teacher certifcation has 92% domestic references, 8% international, and not a single reference to other Nordic countries. Tis result adds to the understanding of commissions operating frst and foremost on a "pragmatic mandate" generating "highly specialized knowledge on the specifc topic for which they are tasked" (Steiner-Khamsi et al., 2020, p. 128), which can result in radically diferent reference structures in SOUs, even when related as parts of a wider shared discursive context.

## **Upward Translation in the Swedish School Reform**

Turning to the process of translating expert knowledge upward, the result shows a selective use of expert knowledge at the political level.

Figure 8.3 demonstrates that the white paper (Source Document 1) by the Swedish Ministry of Education and Research shares about 44% of the knowledge produced by the experts in the commissions of inquiry that it has appointed, while about 56% is unique to the white paper. As such, more than half of the number of references is lost in the political process of "upward translation" (Steiner-Khamsi et al., 2020). Additionally, this fnding indicates that the Ministry of Education and Research looked for other sources of expert knowledge apart from the green papers to incorporate when writing the white paper (Source Document 1).

Not all green papers are equally central to a white paper. As signaled in white paper 2017/18:182 (Source Document 1), SOU 2017:35 (Source Document 2), SOU 2016:94 (Source Document 7), and SOU 2017:51 (Source Document 8) are of special importance to the Ministry of Education and Research and therefore hold a specifc mandate visible in

**Fig. 8.3** The political translation of scientifc knowledge in the 2015/2018 Knowledge Achievement Reform. (*Note*: GP = Green paper)

the number of shared references with the white paper. In this case, the process of upward translation thus consisted of two steps of selection, frst by announcing that certain green papers would be given special importance and second by being selective in the use of expert knowledge provided by the green papers used.

Looking into the content of the three green papers mostly drawn upon by the Ministry of Education and Research, SOU 2017:35 (Source Document 2) and SOU 2017:51 (Source Document 8) both provide a wide array of expert knowledge suitable for an incremental reform that builds on and revises previous reforms. It is perhaps more noteworthy that the government also uses quite a lot of expert knowledge from SOU 2016:94 on student absenteeism (Source Document 7). In addition to those three green papers mentioned as especially central to the white paper, the Ministry of Education and Research also drew on expert knowledge from the green paper SOU 2016:59 *On Good Grounds—An Action Guarantee for Reading, Writing and Math* (Source Document 6). Launched as a result of the Swedish decline in PISA, this one-person commission led by an associate professor in education from Stockholm University focuses on developing methods for early interventions to improve children's basic abilities in reading, writing, and math. Te report uses Finland as a reference society and discusses how the early interventions that the commission found to be characteristic of Finland might explain its PISA success. Members of the commission also made study visits to Finland to learn about the Finnish system.

#### **A Final Note on the Commissions in This Study**

A brief look into the commissions that have produced the SOUs included in this study as source documents adds to the conclusion drawn by Dahlström et al. (2019) that there has been a signifcant shift in the Swedish committee system, away from broader parliamentary committees toward one-person commissions with groups of advisory experts. Of the eight committees in this study, six were one-person commissions. Although based on a limited number of cases, the result coincides with the conclusion drawn by Christensen and Holst (2017) that civil servants have been replaced by academics as chairpersons for governmentappointed commissions. Out of the eight commissions in this study, fve of them were led by academics employed at universities or persons holding a doctoral degree when handing over their reports to the Minister of Education and Research. Te most infuential commission for the 2015/2018 Knowledge Achievement Reform was the group behind GP 2 (Source Document 2), frst led by civil servant Anna Ekström as chair before she was replaced by Professor Jan-Eric Gustafsson when she was appointed Minister of Education and Research.

## **Conclusions**

In this conclusion, we will summarize the main fndings of the bibliometric network analyses of the Swedish school reform in relation to the three research questions guiding the analyses. Te frst question concerned whose knowledge was used to legitimate the reform. Here, three characteristics stand out as most signifcant for the Swedish reform. Te frst is the high percentage of domestic, mainly governmental references. Tis fnding shows the possibility for national politics to uphold a high level of self-referentiality even when the national political agenda to a large extent is dictated by international organizations such as the OECD. Te bureaucratic machinery seems to act on its own institutional logic, relatively independent from external infuence in the Swedish reform. Te use of externalization as a way to legitimate the reform difers signifcantly between the diferent green papers and seems to relate to the topic. Some topics seem to be perceived as more national than others, which is illustrated through the structure of the reference list. In addition, the chair of a specifc commission also has an important role in determining what knowledge is used. In the Swedish reform, it is interesting to notice that the chair of the most central green paper (GP 2) also stands out in the material as the most cited academic researcher. As researchers are increasingly replacing civil servants as chairs of government-appointed commissions in both Norway and Sweden (Christensen & Holst, 2017; Dahlström et al., 2019), what Steiner-Khamsi et al. (2020) described as an "expertisation of commissions" seems to create a new quasi-scientifc policy space between science and politics for researchers to marketize their own research. Te results of this study thus harmonize with previous research showing that context matters for what works in the local arena (e.g., Anderson-Levitt, 2003; Steiner-Khamsi, 2013). As shown by the results of this study, context itself has to be understood as a stratifed concept, where aspects afecting what evidence is of the most worth and available for politicians to use in legitimating educational reforms are decided by many diferent interconnected, sometimes contradictory, factors. Tese factors range from how the bureaucratic system functions to which people occupy central positions, such as being chair in a commission.

Te second question addressed the issue of what types of knowledge sources were used in legitimating the Swedish school reform. As hinted above, almost 60% of the references were governmental, which is almost twice as much as the other Nordic countries. Another distinct feature is the low number of academic references, similar to Finland but much less than Norway and Denmark. Of the eight source documents, four do not have any references to academic research, which is remarkably low in a context emphasizing the importance of evidence-based policymaking. When referring to academic research, both the most important references are systematic reviews presenting synthesized knowledge. As the production of scientifc knowledge increases, the role of intermediaries becomes equally important. Tese intermediaries are people, networks, or organizations whose main task is to collect, select, interpret, and present academic knowledge for political and practical use. During the same period as the evolution of the 2015/2018 Knowledge Achievement Reform, there was a broad political consensus in Sweden on the importance of investing in useful knowledge. In 2015, the Swedish government even launched a new governmental institution called the Swedish Institute for Educational Research with the explicit directive to systematically collect and distribute research contributing to more evidence-based and efective teaching in Swedish schools to improve performance on national and international assessments. In a time of an overabundance of academic knowledge, the role of people and organizations operating as intermediaries thus becomes increasingly important as they provide academic research in a simplifed and accessible way (Lubienski, 2019). However, when politicians start to promote such research fnancially, there is a long-term risk that academic research could lose some of its critical stance in order to be perceived as relevant to policy and practice.

Te third and fnal question addressed the issue of what happens when expert knowledge is translated upward into political knowledge. Te results show that, although the white paper drew on expert knowledge produced by the green papers, more than half of the references in the white paper were exclusive to that document. Expert knowledge is thus used very selectively at the political level, and politicians clearly import references of their own as well. Commissions ofer a variety of evidence from which the Ministry of Education and Research can choose to include in the white paper. While these results confrm the idea of an overproduction of evidence (Lubienski, 2019; Steiner-Khamsi et al., 2020) where much expert knowledge is lost in the upward translation, it is reasonable to think that some expert knowledge is imported in a processed form as in-text references and therefore does not turn up in the reference list.

In summary, the results of this study show the importance of understanding context as a complex and dynamic concept where contextual mechanisms at diferent levels afect what knowledge is used in legitimating national educational reforms.

## **Note**

1. For a more elaborated discussion on the method used in this study, see Chap. 3 in this volume by Oren Pizmony-Levy and Chanwoong Baek.

## **References**

Addey, C., Sellar, S., Steiner-Khamsi, G., Lingard, B., & Verger, A. (2017). Te rise of international large-scale assessments and rationales for participation. *Compare: A Journal of Comparative and International Education, 47*(3), 434–452.


*Journal of Education Policy, 35*(1), 119–144. https://doi.org/10.1080/ 02680939.2019.1656289


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **9**

## **Evidence-Based Policymaking in Nordic Countries: Different Settings, Different Practices?**

**Chanwoong Baek, Dijana Tiplic, and Íris Santos**

In recent years, governments have greatly emphasized evidence-based policymaking. By referring to evidence, policymakers attempt to rationalize and scientifcate their political claims when formulating and implementing education policy and practice (Steiner-Khamsi et al., 2020). As a result, references have become an essential part of the institutionalized practice of policymaking. It is now expected that political claims are supported by references to evidence. References indicate that policy statements and claims are not personal or political but scientifc and technical.

Although national policymakers across the world commonly argue that their policy decisions are evidence based, there is little understanding

University of Oslo, Oslo, Norway

e-mail: chanwoong.baek@iped.uio.no; dijana.tiplic@ils.uio.no

Í. Santos

Tampere University, Tampere, Finland

University of Lisbon, Lisbon, Portugal e-mail: iris.santos@tuni.f

C. Baek (\*) • D. Tiplic

<sup>©</sup> Te Author(s) 2022 **253** B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_9

of how they reach such decisions. How does a country practice evidencebased policymaking? Specifcally, what are the mechanisms deployed to legitimize policy proposals and new legislation? What are the commonalities and diferences across countries in terms of what they use as "evidence"? Despite the prevalent use of references in policy documents as evidence, the nature of these references has been underexplored.

In this chapter, we investigate the above questions, focusing on the most recent school reforms in fve Nordic countries: Denmark, Finland, Iceland, Norway, and Sweden. Tere are two reasons to focus on the Nordic countries: frst, because of the geographical proximity and strong historical ties among these fve countries, their education models and policy systems are often commonly perceived to be similar; second, despite the similarities, there are contextual diferences across these fve countries. Tus, we discuss the policy process for an education reform in each Nordic country and examine how the existing mechanisms and systems may lead to diferent practices of evidence-based policymaking.

## **Theoretical Framework**

Te practice of evidence-based policymaking in each country difers by (1) institutionalized forms of policymaking system, (2) degree of selfreferentiality, and (3) type of reform. Each country has various forms of institutionalized policy advisory systems that help the state authorize, validate, or legitimize its policy decisions. Tese arrangements connect the systems of politics and science by bridging policymakers and experts. Interestingly, as these two systems become more coupled, science has faced the crisis of legitimacy. Te public started questioning legitimacy and credibility of science after witnessing how each political assertion could be supported by diferent scientifc expertise and experts (Eyal, 2019; Maasen & Weingart, 2005).

Eyal (2019) introduced four strategies that the state adopts to respond to this legitimacy crisis: objectivity, inclusion, exclusion, and outsourcing. Tese strategies could be categorized by the ways that the state deals with the problem of trust (trust in transparent, objective, public procedures vs. trust in trained judgment of experts) and the problem of extension (technocratic vs. participatory). Inclusion and outsourcing approaches are similar in that they both strive for participatory decision making; however, they difer in that the former places emphasis on open and public procedures, inviting various groups of stakeholders into the process, whereas the latter contracts out evidence production to external groups. By contrast, both exclusion and objectivity strategies take a technocratic approach. Te exclusion strategy establishes various gatekeeping mechanisms to generate an artifcial scarcity of expertise, making it diffcult for one to become an expert or create organizations in charge of setting regulations and managing credentials. Te objectivity strategy excludes any potential involvement of human judgments and highlights quantitative and objective measures.

Tis typology of diferent strategies could be applied to examine the institutionalized process of evidence-based policymaking in the Nordic countries. Here, we hypothesize that there may be diferences across the Nordic countries in terms of how policymakers produce and utilize evidence to claim the legitimacy of their policy proposals and recommendations. For example, policymakers can seek evidence within the bureaucracy, which is closer to the exclusion strategy, or can outsource evidence production outside the bureaucracy, which is more related to the outsourcing strategy. Nevertheless, it is important to note that because the strategies are not mutually exclusive, the institutionalized policy advisory systems may simultaneously adopt multiple strategies.

Te characteristics of the institutionalized process are also closely related to the degree of "self-referentiality" in the system. Te theory of self-referentiality states that sociological systems tend to make internal references and "externalize" when they cannot address the problem through communication within systems (Luhmann, 1990). Tus, we hypothesize that a policy system with a higher degree of self-referentiality may have a more exclusive evidence-based policymaking process than a system with a lesser degree of self-referentiality. Additionally, a selfreferential policy system might make frequent references to government regulations and previous decrees rather than the knowledge produced in external systems.

In policymaking, references play instrumental and legitimizing functions. Policy actors could make a reference to a particular body of knowledge that helps address existing policy problems or signal legitimacy. A reform entailing more controversial ideas often needs references to frst justify the problematization and then authorize and validate the solution (Baek et al., 2018; Howlett & Ramesh, 2003). Tus, fundamental reforms that call for substantial changes are more likely to use references for their legitimizing functions. In addition, there is a greater need for international references in fundamental or controversial reforms because "externalization" helps generate crisis talk and build coalitions among political entities (Steiner-Khamsi, 2004). Terefore, we hypothesize that fundamental or controversial reforms would have a greater number of references in the policy documents and make more frequent references to international sources.

## **The Cases, Data, and Methods**

Te comparability of the policy process has guided our case selection, data, and analytic methods. As illustrated in previous chapters in greater detail, we focus on the most recent school reforms in each Nordic country:


Tese fve reforms were initiated around the same time: between 2013 and 2016. Some reforms had the characteristics of an incremental reform aiming to improve previous reforms implemented relatively recently. Te Reform of 2014/2018 in Iceland builds on the comprehensive Education Act for all school levels in 2008. It laid out two main goals: (1) increasing compulsory school pupils' attainment in reading standards and (2) improving upper secondary students' on-time graduation rates. Te Reform of 2016/2020 in Norway is the renewal of the Knowledge Promotion Reform implemented in 2006. It was designed to refne the Knowledge Promotion Reform of 2006, particularly focusing on the domains of curriculum and quality monitoring. Furthermore, the Reform of 2015/2018 in Sweden is the renewal of a series of fundamental reforms in 2011, including a new education law and new national curriculum for compulsory schooling. Interestingly, this renewal was heavily infuenced by the recommendations made in the OECD study "Improving Schools in Sweden" (OECD, 2015). Chapter 10 explains the incremental aspects of the reforms of Norway and Sweden in detail. Finally, the new National Core Curriculum for Basic Education of 2014 in Finland is the most recent renewal of the national core curriculum. Te new core curriculum did not call for major changes but instead focused on providing more guidance on pedagogy, emphasizing the proactive role of schools in building a future-oriented school culture (Vitikka et al., 2015, p. 84).

By contrast, the 2013 public school reform in Denmark required more fundamental changes in conceptualization and structure. Te 2013 public school reform was one of the most recent major reforms developed in response to Danish students' mediocre performance in international large-scale assessments (ILSAs), introducing many controversial new changes. In particular, a proposal to extend school hours led to much heated debate among stakeholders regarding its scientifc basis.

In an era of evidence-based policymaking when these fve reforms are placed in their policy contexts in terms of national policy process and types of reforms (incremental vs. fundamental), questions arise regarding the similarities and diferences across the countries in relation to what knowledge each government drew on to inform their policy decisions. In this chapter, we examine a set of key policy documents prepared for the selected reforms in each country, along with their references. Regardless of whether the author(s) made a reference to support or reject an assertion, the act of referencing is intended to provide legitimacy and credibility to the information, claim, or evidence presented in the document. Tus, by examining the references used in policy documents, we not only explore what kinds of knowledge are used to inform or justify education reforms, but we also speculate why particular references are utilized more frequently than others.

Te key policy documents were identifed by the research team to best refect the policy mechanisms and institutions in each country. Tese documents were put forward by a group of experts appointed or funded by the government or the government ministry responsible for education policy and practice in each country, respectively (Ministry of Children and Education in Denmark; Ministry of Education and Culture in Finland; Ministry of Education, Science and Culture in Iceland; Ministry of Education and Research in Norway and Sweden).1 In total, there were fve source documents in Denmark, ten in Finland, four in Iceland, ten in Norway, and nine in Sweden (please see Chap. 3 and previous national chapters for the full list of these documents). Te fnal analytic database includes 5443 separate data entries of references extracted from the source documents. We examine these references by paying special attention to the type of references and location of publication across the fve Nordic countries. We used STATA 14.2 for the descriptive and inferential statistical analyses.

## **Education Policymaking Process in Nordic Countries**

Often, the Nordic countries have been externally and internally seen as a coherent group, with similar public policies representing the Nordic model of welfare states (Esping-Andersen, 1996; Hallsén & Nordin, 2020). In relation to education policy, the "Nordic way" of making policy in these fve countries is frequently used as a benchmark that goes well beyond its geographic limits (Dovemark et al., 2018; Ringarp, 2016; Ringarp & Rothland, 2010). However, one may ask the following: are Nordic countries as similar as they are commonly perceived?

Research has shown that despite the existence of some similarities, the diferences among the Nordic countries are evident in terms of governance, policy mechanisms, and institutions (see e.g., Arnesen & Lundahl, 2006; Dovemark et al., 2018). For instance, although all Nordic countries are highly decentralized in their comprehensive education services, with regional and local authorities being the main providers and owning a great level of autonomy in the management of structures, teaching, and even curriculum, the role of private schools and the levels of school choice vary signifcantly among the Nordic countries. Indeed, while in Sweden, Denmark, and Iceland there is a greater number of schools to choose from, in Finland and Norway, school choice is rather limited (Dovemark et al., 2018). Moreover, although the basic governance model for nonpublic actors within primary and lower secondary education in Denmark, Norway, and Sweden appears to be collaborative, diferent policies and legal frameworks are applied to the mix of public, nonproft, and forproft services within education in each country (Segaard & Saglie, 2017). For example, the extent of private nonproft providers is relatively large in Denmark, while for-proft service providers play a more signifcant role in Sweden. In Finland, although there are a few nonproft private providers, for-proft service providers are practically nonexistent (Lundahl, 2017; Dovemark et al., 2018). In Norway, the roles of both of these types of private actors are comparatively modest (Segaard & Saglie, 2017). When it comes to the local governing of Nordic schools, two trends prevail: frst, the degree of municipal control is much lower when it comes to private schools than public schools; and second, there is a higher degree of collaboration between municipalities and public schools than nonproft schools (Tøgersen, 2017). Compared with Denmark and Norway, Sweden is the most marketized of the three Nordic countries and has the largest degree of control and regulation over private schools (Tøgersen, 2017).

Specifcally regarding the institutionalized policy development process, policymaking in Nordic countries has often been perceived as technocratic in that policy actors seek to identify the most efective solutions to address policy problems through scientifc or technical knowledge (Arter, 2008; Christensen & Hesstvedt, 2019; Heclo, 1974). In particular, Nordic countries have a long tradition of developing policies based on the ideas and recommendations presented in Green Papers, which are written by policy advisory commissions. Furthermore, Nordic countries have been commonly characterized as societal corporatist systems in which various interest groups work closely with bureaucracies. Across the fve countries, the involvement of diferent stakeholders in the policy process is noticeable. In addition to interest group participation in advisory commissions, public hearings and stakeholder consultations before formulating legislation to be passed in parliament are institutionalized processes of evidence-based policymaking found in most Nordic countries.

Nevertheless, there are also national variations in their institutionalized policy process. In Norway and Sweden, for example, Green Papers (NOUs/SOUs) are prepared at the beginning stage of policy development by an ad hoc commission appointed by the government when the government is seeking to collect policy ideas and recommendations. Furthermore, Green Papers stimulate debate on a particular policy issue among various stakeholders, organizations, and the public. Based on the collected knowledge and suggestions, the Ministry of Education and Research issues White Papers that outline the policy proposal. Although Green Papers may not necessarily lead to policy formulation or change, White Papers signify the government's intention to pass the policy.

In Denmark, on occasion, Green Papers are released to explore policy issues and initiate the reform process. In the case of the 2013 public school reform, there were no Green Papers specifcally prepared for the policy formulation. Instead, the reform proposal, *Gør en god skole bedre et fagligt løft af folkeskolen* [Make a good school better—improving the academic level of the public school], made references to four documents that were authored by the government or institutions funded by the government. Two of the references were authored by the Agency for the School Council, an independent body that consists of representatives from various interest groups. Te other two were authored by the institute sector, one by *Danmarks Evalueringsinstitut* [the Danish Evaluation Institute (EVA)] and the other by *Socialforskningsinstituttet* [the Danish National Center for Social Research (SFI)].

Although Finland also has a similar policy development process when it comes to Green Papers preceding White Papers for major legislative changes, neither White Papers nor Green Papers are necessarily required for curriculum reforms. For the reform that was carried out in 2014, however, there was the White Paper, "Future Basic Education" and several government-published reports were closely linked to the policy change. Chapter 5 explains in detail how key documents of Finland,equivalent to the White Papers and Green Papers of other countries, were identifed. Indeed, although the curriculum reform was meant to be developed in parliament and to have the participation of diferent stakeholders, it tends to be much more centralized than it appears (see Chap. 5 for details). Te Green and White Papers constitute proposals by the Ministry of Education, which are then sent to obtain the government's approval. Finally, the Finnish National Agency decides on the content of the curriculum using the framework set by the White Paper as a basis.

Te Icelandic policy process is more decentralized. Te concepts of White Papers and Green Papers have not been institutionalized in Icelandic policymaking, except for the recent White Paper published in 2014, *Hvítbók um umbætur í menntun* [White Paper on Education Reform]. Besides this White Paper, there was an audit report prepared by the European Agency for Special Needs and Inclusive Education on behalf of the Icelandic Ministry of Education, Science, and Culture. Policy recommendations from this report were then adopted as education policy and practice, thereby functioning as a draft of the new legislation like a White Paper. A national policy report that provided background information for the Reform of 2014/2018—as a Green Paper would do in other countries—was the "Review of Policies to Improve the Efectiveness of Resource Use in Schools: Country Background Report Iceland." Tis report was prepared by the Iceland Ministry of Education, Science, and Culture as a part of an OECD report. Although the fnal OECD report was never produced, not only was this report cited by the European Agency Report, but also the comprehensive information included in this report often served as a reference point for policymakers in Iceland.

An examination of the institutionalized policymaking process for the most recent education reform in each country shows that although preparation of Green Papers and White Papers for education reforms was not as formally institutionalized across the fve countries, every country had its own ways of seeking expertise and information both from inside and outside the government. Indeed, previous literature has discussed that although policy advisory commissions have played a signifcant role in the Nordic corporatist policymaking context, their numbers, forms, and membership compositions have changed over time. Denmark, Finland, Norway, and Sweden had experienced a decrease in the number of advisory commissions appointed for policy preparation over the past few decades (Ekholm & Moos, 2012; Rommetvedt, 2017). In Sweden and Finland, starting in the 1990s, these commissions were increasingly replaced by one-person commissions led by a special investigator (see Chap. 10; Rommetvedt, 2017). Finland even abolished the commission system in 2003 (Erkkilä, 2012), and the role that the commissions used to play is now played by broad-based working groups (Holli & Turkka, 2019). Te abolition process of commissions in Finland was complex, and there were various reasons behind the decision to remove the advisory commission. Since the 1990s, there had already been a growing criticism that the advisory commission system was inefective (Holli & Turkka, 2019). Furthermore, the government intended to obtain control of the policy process by abolishing the advisory commission institution, which had a fair share of autonomy (Holli & Turkka, 2019). Interestingly, it is also suggested that the government emphasized receiving knowledge beyond the selected group of individuals and actively advocated for public hearings to stress openness and transparency in governance (Erkkilä, 2012).

Under the advisory commission system, commission reports were a formalized part of the policy mechanism that provided information on debates and legislative history regarding policy issues. However, the new system, which includes ad hoc networks and working groups, tends to produce less documentary evidence regarding its deliberations (Erkkilä, 2012). Although much information is now publicly available online, the available information is mostly performance and administrative management (Erkkilä, 2012). Tis also refects the shift in modes of governance: the government now governs by monitoring and controlling the outputs (managerial accountability) instead of managing the inputs.

Furthermore, previous literature has found changes in the composition of advisory commissions in the fve countries. Denmark, Sweden, and Norway have involved more academic researchers in advisory commissions over time (Christensen & Hesstvedt, 2019; Christensen & Holst, 2017; Ekholm & Moos, 2012). In Finland, by contrast, the share of researchers in broad-based working groups as well as their participation in the commissions as chairs, secretaries, and permanent experts have decreased (Holli & Turkka, 2019). Concurrently, interest group representation has dropped in Denmark, Finland, Norway, and Sweden in recent years (Binderkrantz & Christiansen, 2015; Christensen & Hesstvedt, 2019; Rommetvedt, 2017).

Te changes described above signal the overall decline of corporatism in the Nordic countries. Te sources of knowledge have been diversifed, and it has become difcult for governments to develop policies in collaboration with only a few interest groups. As the number of interest groups has increased, the representativeness of the selected groups has become increasingly questionable. Furthermore, because today's policy interests are complex, interest groups cannot promise the support of their members for political exchange, which discourages governments from cooperating and negotiating with interest groups in the policy process (Rommetvedt, 2017).

Strikingly, what is observed in Iceland difers from the rest of the Nordic countries. Óskarsdóttir (2018) examined the number and composition of public commissions in Iceland between 1970 and 2017, fnding that although the number of advisory commissions has declined over time in other Nordic countries, it has signifcantly increased in Iceland. Te results from the study showed that Iceland now has the highest number of preparatory corporatist commissions among the Nordic countries. Interestingly, when looking into the number by policy areas, the Ministry of Education and Culture was the agency with the highest number of commissions, by a wide margin. Óskarsdóttir (2018) explained that perhaps the reason behind why Iceland has demonstrated robust corporatism in recent years compared with other Nordic countries is because the state has undivided control over legislation because of its parliamentary majority and because a number of cohesive interest groups hold representational monopoly. Tis meets the properties required for a corporatist exchange between the state and interest groups (Öberg et al., 2011). Te small size of bureaucracy and limited administrative capacities are other potential contributing factors to Iceland's relatively strong corporatism (Óskarsdóttir, 2018).

## **Reference Patterns in Nordic Policy Documents**

Tis study found varying reference patterns in policy documents across the countries in terms of (1) style and number of references, (2) type of references, and (3) location of publication. Each pattern refects countryspecifc policy processes and reform contexts discussed in the previous sections.

## **Institutionalized Practice of Evidence-Based Policymaking**

Te frequency and style of reference could serve as an indication of the degree of institutionalized evidence-based policymaking practice. Table 9.1 shows that overall, the governments of the fve countries made frequent references to support their policy proposals, ranging from 50 to 264 references per policy document. Tis high number of references may not be surprising given the recent shift toward evidence-based policymaking. Governments are now expected to be transparent about on what they are basing their policy decisions. Indeed, the number of references in national policy documents has increased over time. For example, policy documents prepared for the 1997 reform in Norway seldom made references in their reports, and most of the references were either embedded in the text or listed as footnotes. However, later reforms, such as the Knowledge Promotion Reform of 2006 and the renewal of the Knowledge


**Table 9.1** Number of references in the policy documents

aCV (coeffcient of variation) measures how the standard deviation is related to the mean. In this case, the higher the CV, the greater the dispersion of the number of references across policy documents

Promotion Reform (2016/2020) had separate reference lists and made thousands of references. Even between the Knowledge Promotion Reform and the renewal of the Knowledge Promotion Reform (2016/2020), there was a signifcant increase in the number of references (Baek et al., 2018; Steiner-Khamsi et al., 2020).

Nevertheless, each country appears to engage in diferent levels of the institutionalization of reference practice. Table 9.1 shows that Norway made the largest number of references in a policy document on average (265 references per source document), followed by Sweden (179 references per source). By contrast, Denmark only had about 50 references per policy document on average. Of course, the varying length of policy documents may infuence their average number of references because a longer policy document has a higher chance of having more references. Regardless, the low frequency of references in Danish policy documents is worth highlighting. For example, the reform proposal *Gør en god skole bedre—et fagligt løft af folkeskolen* did not have a separate reference section and had only ten references in the footnotes.

Furthermore, Iceland has the highest variability in the number of references among source documents, followed by Finland. Perhaps this suggests that the reference practice in Iceland and Finland has not become as standardized as in other countries. Both Norway and Sweden did not have much variation across policy documents regarding their average number of references, signifying a greater level of institutionalization of frequent reference utilization. Te patterns in the frequency of references in the fve Nordic countries are also consistent with the institutionalized policy process described in the previous section in this chapter. Although Norway and Sweden had institutionalized the "standard model of bureaucracy" through Green Papers and White Papers prior to issuing a reform (see Steiner-Khamsi et al., 2020; Tullock, 2005), Denmark, Finland, and Iceland did not mandate such a process.

Despite the varying degrees and formats, each country has its own stakeholder review process where interest groups and the public can contribute their expertise to policy formulation. Furthermore, there are government-funded research institutes or groups of academics asked to conduct research for education reforms. In fact, the curriculum reform process in Finland seeks evidence by working closely with a variety of stakeholders throughout the process. Instead of the pyramid structure of bureaucracy, where the government is placed at the top, experts appointed by the government or other stakeholders work with the government in the middle, and the public is at the bottom, all these actors are perceived to coproduce an education reform. Scholars have observed that the democratization of political systems has led to the participation of numerous nongovernment policy actors and organizations in policymaking (Maasen & Weingart, 2005). In addition, the public now monitors and participates in policy knowledge production, democratizing the system of expertise (Weingart, 2003). In other words, the democratization of political systems and systems of expertise has prompted governments to seek legitimacy and credibility beyond the traditional policy process.

## **Types of References**

What constitutes evidence varies over time and is highly context-related (Baek et al., 2018), which translates into diverse patterns of references. Among the Nordic countries, not only do they vary in their styles and numbers of references in policy documents, but there are also variations in what is used as evidence. Table 9.2 shows the distribution of references by type of document. Overall, government-published documents are the most commonly cited type of reference (37.55%). However, there was a statistically signifcant diference in the types of references utilized in each country (χ<sup>2</sup> = 823.93, *p* < 0.001). Although government-published documents are the most cited type of reference in Denmark, Iceland, and Sweden, books and reports are the most cited type in Finland and Norway,


**Table 9.2** Distribution of references by type

*Note*: χ2 = 823.93, *p* < 0.001

respectively. Furthermore, it is noteworthy that journal articles are not frequently cited in the Nordic countries.

#### **Government-Published Documents**

Our analysis reveals that the vast majority of the references in Nordic policy documents were produced by the government. Te high presence of government-published knowledge in Swedish policy documents is particularly striking considering the substantial infuence of international organizations on Sweden's education policy agenda. In Chap. 8, Nordin and Wahlström interpret this as the government's tendency to uphold a great level of self-referentiality (Luhmann, 1990). Te Swedish government favors making references to its institutionalized norms, traditions, and own logic. Following Sweden, policy documents in Iceland made many references to government-published documents. More than half (55.21%) of the references were prepared by the government. When looking into these references more closely, however, it turns out that most of the government-published references were the data and statistics on the education system produced by *Hagstofa Íslands* [Statistics Iceland]. Tis is an interesting contrast to the government-published documents in other countries, which are mostly reports or proposals published by the respective Ministry of Education or the executive agency of the Ministry (e.g., *Utdanningsdirektoratet* [Norwegian Directorate for Education and Training] and *Skolverket* [Swedish National Agency for Education]).

Furthermore, in Finland, only 27.92% of the references used are government productions; however, a deeper analysis into the country context reveals that the infuence of the government might be greater than that (see Chap. 5). When looking at the publishers and the authors, a larger amount of referenced documents are directly related to the government search for knowledge and evidence. Te most cited publishers in Finland are the National Agency of Education (cited 170 times), followed by the Ministry of Education and Culture (57), the University of Jyväskylä (55), and, in the fourth position, the University of Helsinki (32). Of the top four publishers, the frst two are government agencies, and the other two are universities that hold two important and publicly funded research institutes within them, *Koulutuksen tutkimuslaitoksessa* [the Finnish Institute of Educational Research (FIER)] and *Helsingin yliopiston Koulutuksen arviointikeskus HEA* [the Center of Educational Assessment (CEA) in the University of Helsinki], which perform research on the assessment and evaluation of the Finnish education system and are responsible for the implementation of the Programme for International Student Assessment (PISA) surveys, for example. In addition, when we analyzed the authors of the most referenced books, reports, and academic research, many of the authors work for the above institutions, and often, their publications are the result of government-funded projects. Tis phenomenon leads to the understanding that academic and government productions are now often tangled, suggesting the tighter coupling between the systems of politics and science.

## **Reports**

Although Norway had the lowest percentage of government-published documents as references among the fve Nordic countries, it had the greatest percentage of reports. Indeed, reports were the second most cited type of reference in the Nordic countries after government-published documents (see Table 9.2). However, there were signifcant diferences among the Nordic countries regarding the use of reports as the reference type. As previously stated, in Norway, reports represented the most cited reference type (31.01%). In Denmark, the share of reports as the reference type was very close to that of government-published documents (29.87% and 33.77%, respectively). In Iceland, reports were the second most used reference type (25%) after government-published documents (55.21%). By contrast, reports were not as often used in Sweden (11.19%) and Finland (11.08%).

A possible explanation for why reports are the most represented reference type in Norway may be that the sector research in Norway has increased with the expansion of funding and the evaluation of educational research programs (Zapp et al., 2018). Consistent with this speculation, Denmark, another country with a growing infuence of state-funded independent research institutions such as EVA and SFI (which is now VIVE after its merger with the Danish Institute for Local and Regional Government Research [KORA] in 2017), drew many of its references from reports (29.87%). Finally, many references to reports may also refect the infrastructural and epistemological infuence of international organizations on education policymaking (see Addey, 2017; Sellar & Lingard, 2013). For example, in Denmark, about 20% of the references to reports were produced by international organizations. In the following section, we discuss the role of international references in greater depth.

### **Location of Publication**

With the increasing number of international organizations focusing on education, assessment, and improvement, education policymaking is no longer limited by the borders of each nation-state (e.g., Rizvi & Lingard, 2010). For example, the educational reforms in many countries consider twenty-frst-century skills for pupils' learning and well-being as critically important at both individual and societal levels. In addition, in most Nordic countries, the national political, legal, and institutional framework within which education and other local welfare services are provided has been regulated by an EU Directive (Segaard & Saglie, 2017). Te relationships between the global and local identities working on education reform are increasingly tight and diverse, and they are not neccessarily one-directional: although international organizations do have an infuence on local policymaking, national policymakers utilize international organizations and their instruments in the national policymaking process for their own agenda. One such utilization is the reference to the international instruments as authoritative tools to legitimize certain reforms (Steiner-Khamsi, 2004). Our bibliometric analysis reveals active utilization of international references in the Nordic policy documents, with Denmark having the most international references (36.36%) and Sweden presenting the least (18.93%). By contrast, national policy documents made fewer references to other Nordic countries, which remain less than 8% of the references across the fve countries (Table 9.3).


**Table 9.3** Distribution of references by location

*Note*: χ2 = 149.36, *p* < 0.001

Denmark's frequent utilization of international references could also be broken down into the fve types of documents applied in the previous section: report, book, journal article, government-published document, and other. Te analysis informed us that most of the international references in the Danish policy documents were academic literature (42.8%) on the topic of educational leadership (see Chap. 4). Another interesting fnding is that many international references (16.7%) were reports produced by the OECD. In the case of the 2013 public school reform, the OECD reports were referenced to identify the existing problem in the system and to present outsider perspectives. However, the infuence of the OECD was not limited to providing bodies of knowledge and insights; it also contributed to the formulation of coalitions by ideology regarding educational debates, making the policy environment divided and antagonistic (see Chap. 11).

Indeed, the signifcant infuence of the OECD on educational agenda and policy has been discussed by many scholars (e.g., Addey, 2017; Grek, 2009; Hansen & Rieper, 2010; Martens, 2007; Sellar & Lingard, 2013; Takayama, 2013), and this is not only the case in Denmark. Not only had each country used its PISA ranking to diagnose its education system or create reform pressure, but the reforms in Finland, Iceland, Norway, and Sweden examined in this study highlighted global education policies promoted by the OECD, such as competency-based education, twentyfrst-century skills, and accountability reforms. Despite its strong subjectbased tradition, the new core curriculum in Finland introduced and defned seven competence areas related to twenty-frst-century skills (Vitikka et al., 2015). Te Reform of 2014/2018 in Iceland sought to strengthen the competence required in society and economy of the twenty-frst century, which had been emphasized in the 2008 education policy and the National Curriculum Guides. Similarly, the Reform of 2016/2020 in Norway built on the competency-based education introduced in the Knowledge Promotion Reform of 2006. It was also infuenced by the OECD report, titled *OECD Reviews of Evaluation and Assessment in Education: Norway 2011*, which recommended assessments take place through learning goals and quality criteria (Baek et al., 2018; OECD, 2011).

It is important to highlight that the infuence of international organizations such as the OECD is not only direct but also discreet or even silent (see Kallo, 2009; Kauko & Varjo, 2008; Moisio, 2014; Waldow, 2009). For example, a deeper content analysis of the Finnish White Papers and Green Papers found more embedded references to PISA and the OECD and the performative and competitive ideas expressed in their reports than our bibliometric analysis initially identifed. Furthermore, although Sweden demonstrates a lower percentage of international references in its policy documents than that of other Nordic countries, this number does not exactly refect the international efects on the development of the Swedish 2015/2018 Reform. Our additional analysis reveals that the Swedish Green Paper, "Gathering for school—national strategy for knowledge and equality,"2 which had signifcantly shaped the 2015/2018 reform, was specifcally commissioned to review the OECD recommendations in "Improving Schools in Sweden" (OECD, 2015). Steiner-Khamsi et al. call this national adaptation of the OECD "OECDreviews-in-national-disguise" (see Chap. 10). In his study of Swedish educational policymaking, Waldow (2009) documented a trend of "silent borrowing," by which international imports often remain invisible. Waldow (2009) explained that Sweden's silence of borrowing may be because of its leading position in education and welfare systems after World War II. As a pioneer in the feld, referencing external education systems, policies, and bodies of knowledge did not have any rationale for legitimization. Sweden instead relied heavily on obtaining authorization from scientifc rationality (Ringarp & Waldow, 2016; Waldow, 2009). Ringarp and Waldow (2016) found that this culture shifted in the early 2000s as Sweden lost self-confdence in educational performance because of its declining results in PISA; hence, it started to make more international references around 2007. Despite this new orientation, the culture of silent borrowing may still be ingrained in the Swedish policy system compared with other Nordic countries, which could explain Sweden having the lowest number of international references.

When it comes to referencing one's neighboring countries, Norway made the most regional references (see Table 9.3), suggesting its policy borrowing within and across the Nordic region (Steiner-Khamsi, 2004). Indeed, in her study of Norwegian governmental papers, Sivesind (2019) showed that Norwegian policymakers perceive Finland as a country of emulation, particularly in the areas of curriculum, quality and development, and student achievement (Sivesind, 2019). However, considering the common (mis)conception of the Nordic system as one, it is striking that there were not many references to regional documents. Te diferent paths taken by the fve Nordic countries in terms of the organization of the school system and its structures and management (Dovemark et al., 2018) might explain the small usage of regional references. Alternatively, it is possible that the Nordic countries borrow each other's ideas, policies, and practices without referencing them both internationally and unintentionally (e.g., Waldow, 2009).

## **Discussion and Conclusion**

In this chapter, we have examined the policy mechanisms for evidencebased policymaking in the fve Nordic countries and the references that national policymakers have utilized in their policy documents to evidentiate policy ideas and recommendations. In particular, we were interested in whether there are similarities or diferences across the fve countries regarding what knowledge the government used to inform their policy decisions. Te results illustrate that all fve Nordic countries included in this study actively utilize knowledge to support and legitimate their policy proposals; however, they do so in diferent ways and in diferent settings. Te fndings support most of our hypotheses that the practice of evidence-based policymaking varies by (1) institutionalized forms of policymaking system, (2) degree of self-referentiality, and (3) type of reform.

#### **9 Evidence-Based Policymaking in Nordic Countries: Different…**

First, by comparing the policy development process of the most recent school reforms, we found that some countries sought evidence for policy proposals mainly through the policy advisory system within the bureaucracy (e.g., Green Papers in Norway and Sweden), while others outsourced the production of policy advice (e.g., EVA in Denmark and FIER in Finland). Our results also have shown that countries where evidence production is generated within the state bureaucracy (i.e., exclusion strategy) had more references than countries that produced policy knowledge outside the bureaucracy by outsourcing to think-tanks or sponsoring policy research (i.e., outsourcing strategy). Tis may suggest that in countries with an internal reference system, the bureaucracy is more proactive in producing and utilizing evidence or that it at least tries to demonstrate that its policy is evidence-based.

National policy contexts regarding how policymakers seek policy knowledge (e.g., internal or external commissions, public hearings, and stakeholder reviews) signify each nation's political orientation and perception toward democratic and technocratic policymaking. Furthermore, the change in political models, such as the prosperity of corporatism in Iceland and the decline of corporatism in other Nordic countries, shapes who participates in the policy process and what their roles are. Tis information contributes to a more complete understanding of the boundaries of rationality and knowledge, which consequently infuences the reference utilization in each country.

Second, our analysis of reference utilization in the fve Nordic countries showed that reference utilization depends on the extent the policy system is self-referential or receptive to externalization. Te frequent utilization of non-government-published documents such as reports, books, and journal articles in Norway indicates that policymakers were open to external sources of knowledge beyond the system of politics. Indeed, the Norwegian government extensively made use of the knowledge produced by institute-sector organizations such as the Nordic Institute for Studies in Innovation, Research and Education (NIFU) and Norwegian Social Research (NOVA), both of which bridge the systems of politics, science, and practice. Norway also demonstrated a relatively high percentage of international references that were produced across geographical boundaries. Tis is interesting considering that Norway demonstrates a higher level of externalization despite having an exclusive policy advisory system. On the contrary, Sweden, another country where the evidenceseeking process is centered in the state bureaucracy, demonstrated a greater tendency toward self-referentiality or was at least less explicit about its externalization. Sweden had the highest percentage of government-published documents and the lowest percentage of international references compared with the other Nordic countries. Despite this concentration on internally produced knowledge, many of these internal references turned out to be local translations of externally produced knowledge (see Chap. 10; Waldow, 2009). In other words, it is possible that externalization was disguised in the form of internal references. When discussing externalization and translation to understand reference patterns, an additional factor to consider is that there could be certain bodies of knowledge that had become common knowledge within the system and did not require any formal reference. Tus, the absence of references to particular knowledge does not necessarily mean the disregard of the knowledge.

Tird, although our hypothesis that a fundamental or controversial reform would have more references in an attempt to obtain legitimacy and scientifc base did not hold, the results confrmed another hypothesis: a fundamental or controversial reform would utilize more international references than an incremental or noncontroversial reform. An example from our analysis could be the Danish case—a highly controversial reform—which has the most active international reference utilization. In-depth analyses of international references in Denmark showed that policy actors used international references, particularly the ones produced by the OECD, to legitimize the need for the 2013 public school reform by problematizing and diagnosing the existing system (see Chaps. 4 and 11). By contrast, for the 2016/2020 Reform in Norway, which was an incremental reform of the 2006 Knowledge Promotion Reform, policy experts who served on advisory commissions shared that they often referred to international references to collect knowledge about a policy topic that had not been explored in-depth domestically (Baek, 2020).

In conclusion, our fndings show that there are similar and diferent patterns in institutionalized policy processes and reference utilization across the Nordic countries. Tese diferences could be understood by a combination of three factors: institutionalized policymaking system, selfreferentiality, and type of reform. For future studies, we suggest a more in-depth analysis of the *interplay* between the three factors to better understand particularity of the national approaches to evidence-based policymaking.

## **Notes**


## **References**


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **10**

## **How Much Is Policy Advice Changed and Lost in Political Translation?**

**Gita Steiner-Khamsi, Chanwoong Baek, Berit Karseth, and Andreas Nordin**

Tis chapter deals with government-appointed advisory commissions in Norway and Sweden and examines the extent to which their respective governments use evidence in educational reform and policy decisionmaking. Tese commission reports are called NOUs in Norway [*Norges ofentlige utredninger*; English: Norwegian Public Studies] and SOUs in Sweden [*Statens ofentliga utredningar*; English: State Public Studies], and both their composition and purpose have changed signifcantly over the past twenty years.

C. Baek • B. Karseth University of Oslo, Oslo, Norway e-mail: chanwoong.baek@iped.uio.no; Berit.karseth@iped.uio.no

A. Nordin Linnaeus University, Växjö, Sweden e-mail: andreas.nordin@lnu.se

G. Steiner-Khamsi (\*) Teachers College, Columbia University, New York, NY, USA e-mail: gs174@columbia.edu

According to several scholars in political science, governmentappointed advisory commissions are typically established to fulfll three broader purposes: expertise, accountability, and representation (e.g., Boswell, 2017, 2018). Governments rely on experts with insider knowledge who are sufciently familiar with the bureaucracy to provide useful and realistic advice regarding complex matters. Ideally, independent experts—preferably academics working outside the bureaucracy—are needed to provide credible expertise. Tese independent experts are authorized to observe and evaluate past reforms and, by implication, to hold the administration accountable for its technocratic performance. Finally, governments must satisfy the demands of their political environment for participation and representation in government decisions.

In reality, however, the NOUs and SOUs have been directly afected by the "erosion of the corporatist model" (Lundberg, 2015) or "corporatism in decline" (Rommetvedt et al., 2012), a clear trend that political scientists have explored in great detail. In the research literature, the twentieth-century governance models of Denmark, Norway, and Finland have frequently been showcased as "societal corporatist" systems, in which interest groups exerted active infuence on the government by direct participation in commissions and direct interactions with government civil servants. In efect, the rise of this powerful state apparatus sparked a gradual process of depoliticization, whereby power was given to government bureaucrats rather than elected members of parliament. Tis weakening of parliamentary power as a result of societal corporatism has been a recurring theme in political science and policy studies (Rokkan, 1966).

Te perceived attack on the strong state has triggered a series of changes at the heart of government. In the face of "the revival of parliaments" in Norway and Sweden, interest groups such as unions, business organizations, and professional associations have sought out new channels for infuencing political decisions. Rather than following the traditional state corporatist model of providing input to government-appointed advisory commissions, these interest groups chose to change their political arena: withdrawing from the NOUs and SOUs and focusing instead on lobbying politicians and elected members in the parliament directly (Lindvall & Rothstein, 2006; Østerud & Selle, 2006). Te devolution of power from government to parliament has thus impacted not only the interest groups' agenda setting and communication strategies but also the composition of NOUs and SOUs, their political reach, and the wider purpose of the "policy advisory system" (Halligan, 1995; see also Craft & Howlett, 2013).

From a broader policy perspective, this shift has also impacted how education is governed in an era of international comparison and evidencebased policy planning. Over the past decade, public policy scholarship has documented a movement toward network governance (Ball & Junemann, 2012) and multi-centric policymaking (Cairney et al., 2019). Across all public sectors, including education, agenda setting and policymaking is now carried out in multiple sites and involves multiple actors. To complicate the policymaking process further, new actors, notably international actors and non-state actors, have become increasingly infuential in terms of national agenda setting and policymaking: previous studies have highlighted the proliferation of informal channels of consensus- and coalition-building in government (e.g., Rommetvedt et al., 2012). Governments, including those of corporatist states, are evidently under constant pressure to interact and negotiate with, and to mediate between, a plethora of policy actors, ranging from traditional politicians to this new wave of interest groups.

In our comparative bibliometric network analysis of NOUs and SOUs, we analyze reports from these commissions (Green Papers) in terms of the publications referenced to support their assertions (see Chap. 2 in this book). Equally important is the analysis of the ofcial White Papers themselves, which considers the received knowledge on which the respective Ministries of Education and Research draw when explaining and justifying reform. Finally, we investigate the relationship between the two types of documents (advisory versus decision-making): which references, and how many, that are listed in the commission reports also surface in the actual policy documents? We have termed this latter process—the knowledge transfer from Green to White Paper—political translation.

## **Green and White Papers: Important Milestones in the Policymaking Process**

Against the backdrop of the devolution of power from government to parliament in Norway or from executive to legislative in Sweden, the frst research question may be formulated as follows: How have the respective Ministries of Education and Research repurposed their advisory commissions? Te second research question focuses on the ofcial use of committee guidance on policy by the respective Ministries. Specifcally, we examined the extent to which the commission reports (NOUs and SOUs) draw on a similar body of knowledge as the White Papers.1

It is frst necessary to explain the role of Green and White Papers in the larger policymaking process. Figure 10.1 depicts the general process of policy formulation in Norway and Sweden. As shown below, Green Papers constitute the frst stage in preparing a new policy. Green Papers are then shared with stakeholders (interest groups, professional associations, etc.) for review and feedback, known as "hearings." Te Ministry of Education and Research then prepares the White Paper based on the commissioned Green Papers along with the feedback received.

In both countries, White Papers are produced by the executive body in the education sector, the Ministry of Education and Research. In Sweden, White Papers are then signed by both the Minister of Education and Research and the Prime Minister. In Norway, the Council of State (the King and the government) formally approves the White Paper. Tereafter, the Standing Committee on Education and Research submits its recommendations to the legislative body (parliament), which is responsible for

**Fig. 10.1** The policymaking process in Norway and Sweden

making the fnal decision. Our focus here is on the transfer of knowledge from the advisory body to the decision-making authority, a process we call political translation. In addition, comparative analysis of the two policy-advisory systems enables us to distill context-specifc features of the policymaking process in Norway and Sweden.

Ultimately, this study aims to address the common suspicion held by the public when it comes to the seemingly innumerous governmentappointed advisory commissions: Do governments merely use them for window dressing, or do they listen to their advice? If they do, which advice is taken, which is left out, and which is politically reframed, and why?

## **Methodology: The Bibliometric Database, Case Studies, and Comparative Methods of Inquiry**

Certain methodological explanations are in order here. In particular, a more detailed description of the bibliometric database and the comparative research design may help the reader to put our fndings in perspective.

## **Sampling of Source Documents and Bibliometric Database**

As explained in greater detail in previous chapters, our bibliometric database was drawn from the references of White Papers and commission reports. We used a set of White Papers prepared for the renewal of the Knowledge Promotion Reform (2016/2020) (Norway) and the Swedish School Reform of 2015/2018 as source documents. Tose commission reports that were explicitly mentioned in the identifed White Papers were also added. We extracted all references from the source documents and entered them into the database. It is important to note that NOU 2015:2 was cited in both White Papers in Norway; NOU 2014:7 and NOU 2015:8 were produced by the same commission (the Ludvigsen Commission); and SOU 2016:66 did not have any formal references. In total, the White Papers and commission reports from Norway and Sweden cited 2312 documents and 1421 documents, respectively. Table 10.1 presents the composition of the bibliometric database.

In this chapter, we turned our attention to the reference attributes, including types of document and location of publication. We employed the categorization used by Christensen and Holst (2017) for coding. Te eight values for types of documents were as follows: (1) national policy documents; (2) national policy research; (3) national academic research; (4) international policy documents; (5) international policy research; (6) international academic research; (7) interest groups, think tanks, and so on; and (8) others. We then clustered these values into three location categories: (1) national, (2) international, and (3) others. To ensure intercoder reliability, we frst discussed the coding scheme extensively, carried


**Table 10.1** Bibliometric databases (Norway and Sweden)

aTranslation of the full title: *Subjects, in-depth learning, understanding: A renewal of the Knowledge Promotion Reform*; Norwegian: *Fag—fordypning forståelse—En fornyelse av Kunnskapsløftet*

bTranslation of the full title: *Eager to learn: Early intervention and quality in schools*; Norwegian: *Lærelyst—tidlig innsats og kvalitet i skolen* c In Swedish: *Samling för skolan*

dNumber of documents cited directly in both White Papers (WP) and Commission Reports. Source documents are also counted as long as they are cited

out independent reviews, and (in case of divergence) reiterated the review and coding. We used UCINET 6.681 and NetDraw 2.168 for network data analysis and visualization and STATA 14.2 for statistical analysis.

## **Units of Analysis: School Reform in Norway and Sweden**

A brief outline of the substance of school reforms in both countries and of the specifc mandates of the government-appointed commissions allowed us to place the two types of source documents—White Papers and Green Papers—into their larger policy context.

### **Norwegian School Reform 2016/2020**

In Chap. 7 of this book, we examined the renewal of the Knowledge Promotion Reform (2016/2020) in Norway. Tis reform was initially tabled by two White Papers produced by the Ministry of Education and Research and enacted by parliament in 2016 and 2017. Te reform was planned to go into efect in 2020. It is considered an incremental reform, which reconfrms the Knowledge Promotion Reform launched a decade earlier. It is therefore necessary to discuss the current reform in its historical context.

Te Norwegian school reform known as the Knowledge Promotion Reform came into efect in 2006, replacing two previous reforms of primary and lower secondary curricula as well as upper secondary education. It can be considered a fundamental reform (Steiner-Khamsi et al., 2020) as it represented a shift from input-oriented to output-oriented policy instruments, such as measurable objectives, standardized tests, and data-based planning (Møller & Skedsmo, 2013). As pointed out by other researchers, the reform was partly motivated and legitimized by the results of the PISA test released in December 2001 (see Skedsmo, 2018). Alarmed by the PISA fndings, the government concluded that the school system had serious weaknesses in need of immediate repair. To meet these challenges, the Ministry of Education and Research suggested the introduction of a national testing system and improvements to the competencies of teachers, school leaders, and administrators through the establishment of a "culture of learning" (Karseth & Sivesind, 2010). Te reform brought about increased decentralization, on the one hand, and increased accountability, on the other.

Te revised national curriculum targeted specifc competencies for student learning outcomes and emphasized basic or foundational skills that were supposed to be integrated in all subjects and across all grades (Imsen & Volckmar, 2014). Furthermore, a national quality assessment system was introduced alongside the curriculum reform (Møller & Skedsmo, 2013), and national testing was frst implemented in 2004. For the frst time, the test results were published and made publicly available, allowing schools to be benchmarked, ranked, and compared. Undoubtedly, this shift toward outcomes-based monitoring represented a radical break with and departure from the traditionally input-based regulation of Norwegian education (Helgøy & Homme, 2016).

Seven years later, the Ministry concluded in hindsight that the 2006 reform had been an overall success, as exemplifed by rising scores in international large-scale student assessments, among other indicators (Ministry of Education and Research, 2013, p. 12). However, it also found that certain shortcomings in the reform had become a cause for public concern. First, the curriculum was seen as overloaded, leading the Ministry of Education and Research to suggest that priorities—in terms of both content knowledge and subjects—needed to be set based on evidence and formative evaluation. Second, as a result of prioritization and the emphasis on deep learning, the Ministry of Education and Research mandated that the key elements for each school subject be defned in greater detail. Tird, acknowledging the importance of social development, three interdisciplinary topics were given high priority: democracy and citizenship, sustainable development, and public health and wellbeing. Fourth, the reform introduced remedial measures for students with low achievements in reading, writing, and numeracy in grades one to four. Among the many improvements that the 2020 reform intended to achieve, one more is worth mentioning: the reform reafrmed the principle of "test-based accountability" (Verger & Parcerisa, 2018), whereby local authorities are held accountable for student learning outcomes in the schools under their jurisdiction.

Consequently, the Ministry put in motion a two-pronged incremental reform process to renew the existing curriculum and to develop the national quality system further. We have labeled the resulting 2020 school reform the Curriculum Renewal/Quality Monitoring Reform and, for easier reference in this book, merged the two as the renewal of the Knowledge Promotion Reform (2016/2020). Tis is the reform investigated in this chapter. In particular, we examine two White Papers that the Ministry of Education issued in 2016 and 2017, respectively, when it announced the 2020 reform: the Renewal of the Norwegian Knowledge Promotion Reform (Ministry of Education and Research, 2016) 2 and the Early Intervention and Quality Monitoring Reform (Ministry of Education and Research, 2017).3

#### **Swedish School Reform 2015/2018**

In the 1990s, Sweden introduced a series of neoliberal school reforms that featured large-scale privatization and a decentralization of decisionmaking authority from the central to the local level. Within a short period of time, the 290 municipalities were put in charge of overseeing compulsory schooling. Te 2015 Final Report of the School Commission, *Gathering for School* (SOU 2017:35), set the reform in motion, which came into efect in 2018 (White Paper 2017/18:182). In this book, we have therefore labeled the Swedish school reform the 2015/2018 Knowledge Achievement Reform.

Naturally, the deterioration of the strong Swedish welfare state and free public services has not gone unnoticed in the research literature (Englund, 1996; Lundahl, 2007). Weaponized by weak performances in the PISA tests from 2003 to 2012, when Sweden reached its lowest scores, vociferous public debates about the education crisis spread across the country (Nordin, 2019). In 2006, the Swedish government agreed to instate a commission to investigate the challenges in the education sector. Based on that decision, the Ministry of Education and Research appointed a commission with the mandate to explore the reasons for the rapid fall in PISA scores. In practice, the commission was a one-person commission without any members, chaired by the Assistant Under-Secretary of the Ministry of Education and Research, Leif Davidsson. Te commission report, *Clear Goals and Knowledge Demands in Elementary School*<sup>4</sup> (SOU 2007:28), asserted that the compulsory school curriculum was too vague and ideological and criticized the decentralization reform for its detrimental efects on equal opportunities and equality. According to the report, the neoliberal reforms of the 1990s and the turn of the century had beneftted only a precious few municipalities, leaving the overwhelming majority struggling to ofer quality education.

Following the recommendations of the commission report, the Swedish government launched a second series of fundamental reforms in 2011, the most prominent features of which were a new education law and a new national curriculum for compulsory schooling that paid great attention to learning outcomes and the reform of teacher education. A more refned grading system was introduced, and test-based accountability was actively pursued: students had to take a higher number of national, standardized tests, which allowed the Ministry of Education and Research to resume oversight over the quality of education. Tis outcomes-based reorientation was not out of the ordinary per se, but in Sweden it enabled the Ministry of Education and Research to reclaim central control over a school system that had been decentralized only a decade earlier.

However, the introduction of clearly defned standards, unambiguous accountability measures, and massive investments into the Swedish education sector did not yield the expected results. In 2012, only a year after the launch of the extensive reform package, Sweden reached its lowest PISA scores ever. Humiliated, the government turned to the OECD, seeking help with analyzing the problems of the Swedish school system as well as recommendations on how to fx them (see Grek, 2019; Pettersson et al., 2017). Te resulting OECD fndings (2015) were presented in *Improving Schools in Sweden*, a study that proposed comprehensive reforms in three priority areas: (a) to establish conditions that promote quality with equality across Swedish schools; (b) to build capacity for teaching and learning through a long-term HR strategy; and (c) to strengthen steering of policy and accountability with a focus on improvement. As a follow-up, the Swedish government appointed a commission with the explicit mandate to review the OECD proposals for educational reform. Te commission's review of the OECD analyses and recommendations was published in the SOU report *Gathering for School: A National Strategy for Quality and Equivalence. Final Report of the 2015 School Commission* (2017:35).5 It would appear that the Ministry of Education and Research took this particular SOU report to heart, as it became the foundation for the 2015/2018 school reform in Sweden. Indeed, even the name of the subsequent White Paper, *Gathering for School*, was taken from its Green Paper precedent.

## **Comparative Design**

Norway and Sweden are commensurable in terms of their institutionalization of the policymaking process. Te practice of instating advisory councils, for example, dates back to the sixteenth century (Lundberg, 2015). Te system has naturally changed over time and, following the independence of Norway and Sweden from Denmark, bifurcated in two diferent directions. Today, however, the two systems have remained suffciently similar to be considered comparable when it comes to the policymaking process.

As Fig. 10.2 illustrates, the research design enabled us to carry out three types of comparison: (1) of policy-advisory systems; (2) of policydecision systems; and (3) of knowledge transferred from the former to

**Fig. 10.2** Overview of the comparative research design

the latter. We have coined the term political translation to denote this particular transfer process; this third type of comparison entails identifcation of shared knowledge between the commission reports and White Papers in Norway (3a) and Sweden (3b), as well as a comparison of these two patterns of political translation (3c).

Specifcally, the three types of comparison draw on the following data: Type 1—Comparison of the two policy-advisory systems. Te database consists of eight commission reports from Norway (NOUs) and eight commission reports from Sweden (SOUs). How do these two sets of reports difer in terms of their references? In particular, are there diferences in terms of their national/international orientation and the type of documents that they reference?

Type 2—Comparison of the two policy-decision systems. Te Ministry of Education and Research of Norway outlined the renewal of the Knowledge Promotion Reform (2016/2020) in two White Papers, which together contain a total of 294 references. For the 2015/2018 Knowledge Achievement Reform in Sweden, the Ministry of Education and Research issued one White Paper that includes 86 references. For a comparative bibliometric analysis, the key questions are as follows: On what knowledge sources do the two ministries draw? Is there a preference for certain types of texts? Do the two ministries difer in their selection of referenced texts?

Type 3—Comparison of commission reports and White Papers (political translation). A comparison of the two types of documents with different functions (advisory versus decision-making) yields interesting insights in terms of political translation. Methodologically, we compared the ratio of shared knowledge/references as a percentage of all references listed in the commission reports of each respective country. In other words, what percentage of references listed in the commission reports also appear in the reference section of the White Papers? Te ratio of shared references is interpreted as an indication of the political translation process. After completing the analysis for each of the political systems (3a and 3b), we then compared the two political translation processes in Norway and Sweden (3c).

## **Research Findings**

#### **Comparison of the Policy-Advisory Systems**

A juxtaposition of the references listed in the respective commission reports highlights two key similarities and two key diferences between the policy-advisory systems of the two countries that are worth exploring in greater detail.

In terms of shared characteristics, the commissions in both countries substantiate their reviews and their recommendations with a large number of references. For Norway, the number of references listed in NOUs ranges from 146 to 703 references, with an average of 292 references per report. Tis practice of excessive referencing may also be found in the Swedish policy-advisory system: the SOUs show a maximum number of references of 337 and an overall average of 191.6 Te pressure to make knowledge sources transparent and to provide "evidence" for the commission's assertions is clearly discernible. A second commonality is the national orientation of the commissions in terms of their reference literature: approximately two-thirds of the references listed in the commission reports for both countries were published domestically.

Both of these phenomena deserve theorizing. Te frst confrms the belief in knowledge-based or evidence-based policy advice. It is expected of modern-day commissions that they read and reference relevant texts excessively and ostentatiously. Second, the commissions in both countries tend to cite national authors, that is, either themselves or authors known to them. Te large proportion of national references may also refect the practice of commissions to refect and report on contemporary topical debates and controversies (Sweden: low learning outcomes and student absenteeism; Norway: drop-outs and concerns about students' psychosocial environment).

Tere is, however, a statistically signifcant diference (*p* < 0.05) between the types of references listed in the commission reports of the two countries. In Norway, 41.2% of all references in the eight examined commission reports (867 out of 2106 references) are academic research. In stark contrast, academic studies make up only 19.7% of the references


**Table 10.2** Types of references in commission reports and White Papers

in the eight examined SOUs from Sweden (270 out of 1373 references). Instead, the Swedish policy-advisory system relies heavily on national policy documents (34.81%) and national policy research (29.86%) to support commission reviews and recommendations (see Table 10.2).

Te academization of Norway's policy-advisory system is a well-studied phenomenon (Christensen & Holst, 2017). In our bibliometric network analysis of the 2006 Knowledge Promotion Reform in Norway (Steiner-Khamsi et al., 2020), we cursorily examined the composition of government-appointed commissions over time and found that the number of academics appointed had increased dramatically over the previous twenty years. Our observations concur with those of a larger study on expert commissions (Christensen & Holst, 2017; Christensen & Hesstvedt, 2019), which considered the composition of advisory commissions across various ministries over a period of close to ffty years. What they found is striking: the representation of interest groups in education sector commissions has dropped sharply and is now half of what it was four decades ago. During the same time span, the proportion of academics on education-related NOUs has seen a nearly fourfold increase: from 9% in the 1970s to 35% in the 2000s. Te dramatic upsurge of researchers came at the expense not only of the interest groups but also of civil/public servants: whereas forty years ago, more than half of NOU representatives were government ofcials, they now constitute only onethird of commission members.

Today, the two largest groups represented in the NOUs are researchers and government ofcials, which together account for more than 90% of members. In contrast, interest group representation stands at less than 10. Te explosive growth of researcher representation in NOUs has led Christensen and Hesstvedt (2019) to suggest that further investigations are necessary to understand the "expertization" of Norwegian advisory commissions in greater detail. Te breakdown of commissions by chairpersons and secretariats portrays a similar picture. As Christensen and Holst (2017) have asserted, academics have replaced civil servants as chairpersons in the majority of commissions since the year 2000. In other words, academics have become the public face of the commissions.

By contrast, the academization of the Swedish advisory commissions is conspicuous by its absence. Our bibliometric study demonstrates that only 19.7% of the references consist of academic research. Tis is surprising given the heavy representation of academics as chairs of governmentappointed commissions: fve out of the eight commissions included in this study were led by academics who are either employed at universities or have completed their doctoral degree. Two were led by politicians and one by a civil servant. On the one hand, then, academics are visible and infuential as chairs of Swedish advisory commissions, as seen in Norway above. A case in point is the change in chairpersonship for the commission that produced the infuential *Gathering for School* Green Paper in 2017. When the commission was established in April 2015, Anna Ekström, an infuential civil servant and Director-General of the Swedish National Agency for Education (Skolverket), was appointed as the chair, but when she left the commission in September 2016 to assume the position of Minister of Education and Research, she was replaced by Jan-Eric Gustafsson, a professor of education at the University of Gothenburg.

On the other hand, a distinct feature of the Swedish policy-advisory system is the move from larger advisory to one-person commissions known as Special Investigator Commissions. Whereas the former advisory commissions served as a representative and participatory body of various stakeholders (mirroring the composition of parliament), each Special Investigator Commission is led by an individual considered appropriate by the Ministry of Education and Research. Te appointed individual is often a politician, civil servant, or academic with a group of associated experts at his or her disposal for consultation. Tis gradual shift from advisory commissions to Special Investigator Commissions that began more than two decades ago, therefore, has meant a reduction in terms of representativeness and of parliamentary and societal legitimacy as a result.

In the Green Paper SOU 1999:121, Hermansson and colleagues examined 509 commissions and found that the share of Special Investigator Commissions had doubled from 30% in 1960 to 60% in 1995. Tis development has been confrmed in a more recent analysis by Dahlström et al. (2019), who demonstrated that in 2016, as many as 90% of all government-appointed commissions in Sweden were led by Special Investigators. Te removal of politicians and interest group representatives from these commissions tipped the balance between representativeness and efciency: drastically shortening the investigation period from a few years, in extreme cases, to only a few months or even weeks (Dahlström et al., 2020), while downsizing, expertizing, and depoliticizing the commissions themselves (see Petersson, 2016). Indeed, the diminished political infuence of government-appointed commissions is well captured by Lundberg's verdict: "injured but not yet dead" (2015). Te pattern is also repeated in the commissions that advised the Ministry of Education and Research on the most recent reform: of the eight SOUs identifed as relevant to the 2015/2018 Knowledge Achievement Reform, six were produced by a Special Investigator Commission. Against this general pattern, it is worth noting that the *Gathering for School* Green Paper (SOU 2017:35) chaired by Professor Jan-Eric Gustafsson, and central to the 2015/2018 Knowledge Achievement Reform, is one of the two SOUs produced by larger advisory commissions.

Another historical characteristic of Swedish advisory commissions has been a considerable degree of autonomy in relation to central government (Trägårdh, 2007). However, as underlined by Petersson (2016), the shift away from broadly representative and politically powerful commissions to smaller Special Investigation Commissions has led to a tightening of control by the ministries and a subsequent reduction of autonomy. Signifcantly, the twin processes of depoliticization and bureaucratization of the commissions have served to increase governmental power over the policy-advisory process. Tis is also evidenced by the rise, from 1990 to 2014, in the number of commission directives (kommitédirektiv), the written instruction to the appointed commission stating its missions, mandate and timeframe, and the most important tool with which the government can control commissions (Dahlström et al., 2020). Crucially, the government can also adjust or replace these directives over time as a way to exert control within the process.

Clearly, then, both countries have chosen diferent pathways to cope with the corporatist dilemma of the 1980s and 1990s. In Norway, interest groups reduced their presence in the commissions and instead sought more efective means of exerting political infuence. Together with the devolution of power from government to parliament, this led to a depoliticization of the commissions. Longitudinal studies (Christensen & Holst, 2017; Christensen & Hesstvedt, 2019) have suggested that the empty seats left behind by the political interest groups were flled by academics and that political representation was jettisoned in favor of scientifc expertise.

#### **Comparison of the Policy-Decision Systems**

Te heavier reliance on research (both independent/academic and commissioned/applied policy research) in the Norwegian policymaking process also resonates at the ministerial level. As Table 10.2 shows, of the 294 references listed in the two White Papers on Norwegian school reform (WP 2015/16 and WP 2016/17), an overwhelming majority (79.25%) fall under either national policy research (38.10%), national academic research (8.16%), international policy research (14.97%), or international academic research (18.03%). According to the categorization system used by Christensen and Holst (2017), we found that four in fve of the references (233 out of 294) represent analytical publications, that is, national or international policy research or academic research.

Conversely, in Sweden, analytical publications make up less than a quarter (24.42%) of the references listed in the ministerial *Gathering for School* White Paper. It is equally important to note that the Ministry of Education and Research has not cited a single academic publication, whether national or international. Te marked absence of academic references in the Swedish White Paper calls into question whether the Ministry has a troubled relationship with its own academic community and with foundational research in general.

A second fnding has been somewhat surprising, too: in Norway, the Ministry of Education and Research cites signifcantly more researchrelated publications in its White Papers than its appointed expert commissions do in their reports. Specifcally, the bibliographies of the White Papers comprise 79.25% analytical publications, compared with 68.8% for the NOUs (see Table 10.2).

Both of these unexpected results require further unpacking: the frst in light of the diferentiation between "mode 1" and "mode 2" knowledge (Nowotny et al., 2003) and the second in light of the importance of the chairpersonship to the commissions' work.

First, the diferentiation between foundational, academic knowledge (mode 1 knowledge) and applied, policy research (mode 2 knowledge) helps us to understand why, at frst glance, the Ministry of Education and Research of Norway seems to be more committed to research than its own expert commissions. As previously discussed by Steiner-Khamsi et al. (2020), mode 1 knowledge represents foundational research that is primarily concerned with advancing scientifc discovery and disciplines. Mode 2 knowledge, meanwhile, refers to application-oriented, transdisciplinary, local, and involved expertise found outside purely academic settings (Gibbons et al., 1994; Nowotny et al., 2003). Without doubt, sector trends like open access to knowledge products, demand for evidence-based policy planning, and the pluralization of expertise (see Cairney et al., 2019; Eyal, 2019; Maasen & Weingart, 2005) have boosted the importance of mode 2 knowledge in this context. In Norway, this type of research is actively promoted: the so-called institute sector including, for example, the Nordic Institute for Studies in Innovation, Research and Education (NIFU, formerly NIFU STEP)—has become a signifcant knowledge producer in the education sector and is likely to expand in the near future.

Closer analysis of the research publications listed in the White Papers here reveals the ministerial propensity to rely on mode 2 research. Comparing the bibliographies of the White Papers with those of the commission reports, the proportion of both national policy research and international policy research is noticeably higher in the White Papers (38.10% vs. 22.84% and 14.97% vs. 4.8%, respectively), while references to international academic research are roughly equivalent (around 20% for each). In turn, the extent to which these references represent work published by members of the advisory commissions themselves needs to be investigated empirically.

Second, the chairperson of a given commission has a signifcant impact on its research orientation and knowledge production. In the Norwegian context, we considered the high-profle example of Sten Ludvigsen, who chaired the commission that produced NOU 2014:7 and NOU 2015:8. Ludvigsen is a professor in learning and technology at the University of Oslo. He played a pivotal role in the research-based evaluation of the 2006 reform, organized by the Directorate for Education and Training and implemented between 2006 and 2012. Ludvigsen was also the leader of the program board of evaluation from 2008 to 2012. Trough his leadership role in the evaluation, Ludvigsen gained a strong reputation as an expert and policy advisor within the sector before his appointment as commission chair.

A Swedish analogy to Ludvigsen can be found in the fgure of Jan-Eric Gustafsson, who chaired the commission that produced the *Gathering for School* Green Paper (SOU 2017:35),7 which led to the White Paper of the same name as part of the 2015/2018 reform. Gustafsson is a professor of education at the University of Gothenburg and is considered an expert in Swedish school reform, especially in the area of large-scale student assessments. Gustafsson is the most cited academic author, not only in the commission that he chaired, but also in the other Green Papers that the Ministry of Education and Research references in its White Paper.

Figure 10.3 illustrates the author-reference network, where each author is connected to the documents that s/he authored and to their coauthors.

#### **300 G. Steiner-Khamsi et al.**

**Fig. 10.3** Author-reference network. (*Note*: Circle: References. Square: Authors)

A node size indicates how many references s/he served as an author. Te map shows the top 15 authors who were most frequently cited. Among them, Gustafsson was the only individual researcher cited in the relevant SOUs and the Swedish White Paper. All other infuential texts cited in the source documents were authored by institutions, including the Swedish National Agency for Education (Skolverket) and the Swedish Schools Inspectorate (Skolinspektionen), or by legal entities such as the Government (Regeringen) or Parliament (Riksdagen), as visualized in Fig. 10.3.

In contrast to the frequent citation of Gustafsson's work in the SOUs, Ludvigsen's work was not cited once in the NOUs. Tis may seem surprising given his role as chair and his exemplary reputation as a policy advisor and scholar in educational studies (see Baek, 2020). Indeed, according to interviews with several members of the NOUs, Ludvigsen showed his commitment to evidence-based policymaking by distributing reading lists to commission members to ensure informed policy advice.

Te Ludvigsen commission appointed in 2013 consisted of eleven members.8 Te commission was tasked to submit an interim report including historical and comparative analyses of school subjects in primary and secondary education, a feasibility study of international competence-based curriculum frameworks, and recommendations of national stakeholders in terms of students' competency requirements. Te interim report was delivered in 2014 (NOU 2014:7) and examined the knowledge on competencies from "various international organizations, education authorities in a number of countries and comprehensive research and report projects" (NOU 2015:8, p. 16). In addition, the commission appointed a research team that was tasked with providing an overview and making an assessment of diferent competency concepts and frameworks (see Erstad et al., 2014). Te fnal report (NOU 2015:8) drew on both the interim report (2014:8) and the work of the research team and proposed a broad competency concept with four suggested areas as the foundation for the curriculum.

In their role as chairs of their respective advisory commissions, Ludvigsen and Gustafsson publicly provided a scientifc stamp of approval for OECD-informed national education policies. In Norway, however, the OECD link is a more complicated one: several international organizations and commercial actors had a bearing on the recommendations presented by the commission. Tese included the Partnership for 21st Century Skills, the International Society for Technology in Education, the EU and Key Competence Network on School Education, and Cisco, Intel, and Microsoft, who initiated the Meeting Assessment and Teaching of 21st Century Skills (see NOU 2014:7, Chapter 8).

Te two decision-making authorities of Norway and Sweden display marked diferences in how they explain and justify the necessity for reform. In Norway, the Ministry of Education and Research draws on research (with a preference for mode 2 knowledge in the form of applied policy research), whereas the ministry in Sweden legitimizes its policy decisions with greater reference to national policy documents. Tis orientation toward research in Norway versus compliance in Sweden has been discernible at various levels of analysis.

## **Comparison of Political Translation Processes**

Te previous two sections examined the act of knowledge production carried out by the commissions and the ministries, respectively. In this section, we consider the application of this knowledge, guided by two key questions: To what extent do the respective Ministries use the knowledge provided by their expert commissions? And what kind of knowledge (or, more specifcally, which references) from the NOUs and SOUs have been formally adopted in the White Papers?

Te answer to the frst question is that signifcantly more knowledge from commission reports is used in Sweden than in Norway, but overall knowledge transfer is very scarce. As presented in Fig. 10.4, only 30% of the references used in the Norwegian White Papers are identical to those forwarded in the corresponding NOUs. Te remaining 70% are novel references; the Ministry of Education and Research of Norway is very much an evidence-producer in its own right. We chose to describe this esoteric ministerial knowledge, entirely separate from the knowledge ofered in the NOUs, as "political knowledge."

In Sweden, the Ministry of Education and Research seems more inclined to adopt the knowledge sources used by the SOUs to support their assertions. Forty-four percent of the references in the White Papers are identical to those produced in the Green Papers. Te two diagrams in

**Fig. 10.4** Reference distribution in White Papers in Norway and Sweden

Fig. 10.4 tell the ministerial, top-down side of the story; this perspective is explored further in the case study chapter on Sweden in Chap. 8 of this book. Yet we can equally invert the perspective and consider political translation from the bottom-up (i.e., from the viewpoint of the advisory commissions): Were the thorough reviews of policy documents, studies, and other relevant publications worth the time, efort, and resources? Was there any uptake at the higher political level? Te question of whose knowledge (and which kind) is adopted at the political level is important since governments appoint numerous commissions whose output can then go unnoticed in terms of political uptake.

Such uptake is barely visible in the Norwegian case: of the 2106 publications amassed by the commissions to produce evidence for their reviews and recommendations, only 4% were then mentioned in the bibliographies of the two White Papers. As illustrated in Fig. 10.5, this means that, on average, 96% of the expert knowledge references—gathered with great diligence and reviewed in great detail in the NOUs—were not explicitly mentioned in the two White Papers of the 2016/2020 reform.

Te same pattern reemerges in the Swedish case, with only a slightly worse political uptake of commission report references. Te bottom-up perspective suggests a very high cost of political transaction: only 38 documents9 out of a possible 1373 cited in the SOUs (3%) reached the political level. Again, this suggests that a signifcant amount (97%) of expert knowledge has been lost in political translation. Given the highly selective reception of commission reports at the political level (3% in Sweden; 4% in Norway), the question becomes: which references did "make it"?

Te charts in Figs. 10.5 and 10.6 point to the fact that certain commission reports proved more attractive to their respective Ministry than others. In Norway, the Green Papers NOU 2014:7 (*Pupils' Learning in the School of the Future: A Knowledge Base*) 10 and NOU 2015:8 (*Te School of the Future: Renewal of Subjects and Competences*) 11 constitute the two reports with the greatest political infuence. As noted above, both were the products of the Ludvigsen commission. More than any other, this commission was specifcally tasked with identifying the areas where adaptation of the existing curriculum was deemed necessary.

Regardless of report length or number of references, NOU 2014:7 and NOU 2015:8 have clearly caught the attention of the Ministry of

Shared by WPs

Education and Research. In the 2020 reform, for example, NOU 2015:2 includes a thorough literature review (703 publications) and is cited by both White Papers, yet the Ministry of Education and Research only considered 25 of them relevant at the decision-making level.

Similarly, the Swedish Ministry of Education and Research appointed several commissions in preparation for the 2015/2018 reform. However, as shown in Fig. 10.6, not all of them had equal infuence at the political level. As noted above, the Green Papers SOU 2017:35 and SOU 2016:94 were especially integral to the *Samling för skolan*12 White Paper, not just to its title but also to the 2015/2018 reform as a whole. Together with SOU 2013:56, SOU 2017:35 is the only Swedish source document to be produced by a larger advisory commission; all the others came from oneperson commissions, including SOU 2016:94 (*Saknad!*),13 chaired by the psychologist Malin Gren Landell. Such selective use of expert knowledge would appear to support the argument that appointed advisory commissions are becoming weaker (Lundberg, 2015), while the government at the same time grows in strength and independence in relation to its appointed advisory commissions (Petersson, 2016).

We would be remiss if our descriptive bibliometric analysis excluded a qualitative review of the most politically infuential commission reports in both countries. In Norway, the Green Papers NOU 2014:7 and NOU 2015:8 have had the greatest impact in terms of political translation or knowledge transfer from the expert to the political level. Te mandate of the Ludvigsen commission was to assess the degree to which the curriculum covers the competencies that students would need for the future, both professionally and socially. Unlike the Swedish case, the commission was not directly tasked with drawing on expert knowledge from the OECD. In fact, there was no need to do so, because the OECD competency-based Defnition and Selection of Key Competencies curriculum framework (DeSeCo) had already been implemented in the reform of 2006 (see Steiner-Khamsi et al., 2020). Tis time, the question was whether, ten years later, adaptations and modifcations needed to be made.

Te Ludvigsen commission supported the fundamental curriculum reform that was launched in 2006 and recommended that adaptations were made to refect more recent educational frameworks and debates. It considered its work an "advancement of the competence-oriented subject curricula today" (2015:8, p. 15). After reviewing several competency frameworks, notably, OECD's DeSeCo framework, Assessment and Teaching of 21st Century Skills (ATC21S), Partnership for 21st Century Skills (P21), Key Competences for Lifelong Learning (KeyCoNet), and National Educational Technology Standards (NETS), the Ludvigsen Commission recommended in its main report a broad concept of competence comprising cognitive and practical skills as well as social and emotional learning and development (NOU 2015:8, p. 9). It proposed the following four areas of competence as the basis for setting priorities for school activities: subject-specifc competence; competence in learning; competence in communicating, interacting, and participating; and competence in exploring and creating. Tese recommendations aligned with those of the international policy agenda and the OECD. Additionally, they built on learning sciences, and the commission emphasizes the importance of combining learning sciences and subject didactics research (Greeno, 2006, p. 46). Another key concept of the commission was crosscurricular competence; this particular recommendation, however, was not followed up in the White Paper. Instead, the Ministry explicitly stated that the renewal of the curriculum should focus on school subjects and not on cross-curricular competence (Ministry of Education and Research of Norway, 2016, p. 42).

In Sweden, the two reports with the greatest political uptake of shared knowledge or references could be considered OECD reviews in national disguise. Tis applies to SOU 2017:35 in particular, where the commission was explicitly instructed to make proposals based on the OECD *Improving Schools in Sweden* report given to the Swedish government in 2015, a study itself triggered by Sweden's poor performance in the PISA tests. Te OECD contextualization, therefore, lies at the core of the entire 2015/2018 school reform process in Sweden. As for SOU 2016:94, the OECD infuence is somewhat more indirect, given that the commission was not explicitly asked to draw on OECD expert knowledge. However, the OECD infuence is present nonetheless: as early as the introduction, the report refers to student absenteeism as an important factor in declining Swedish PISA performance. More specifcally, the report points to a signifcant correlation between tardiness and students' results in science education (SOU 2016:94, p. 111). Hence, unlike the straightforward national adaptation of the 2015 OECD study in SOU 2017:35, SOU 2016:94 implicitly draws on an OECD governance tool (the PISA test) to generate reform pressure. Te measures to increase student attendance are instead couched in the strong national belief in equality, according to which the school system is charged with generating equal opportunities and life chances for all. Tus, rather than referring to OECD recommendations directly, in the case of SOU 2016:94, the Ministry of Education and Research leverages its policy-advisory system and cites a nationally adopted or "indigenized" version of an OECD recommendation.

## **Summary and Conclusions**

In this bibliometric study, we compared policy documents relevant to the most recent school reforms in Norway and Sweden. Te comparison focused on knowledge produced by the advisory commissions and the Ministries of Education and Research of the two countries. In particular, we were eager to understand the recourse to knowledge—in terms of frequency and type of knowledge used in references—in the reports of the advisory commissions (Green Papers) and in the ministerial decrees (White Papers). As well as knowledge production, we also investigated how much and which knowledge presented in the commission reports was actually taken up by the respective Ministry of Education and Research at the political level. In line with an earlier study (Steiner-Khamsi et al., 2020), we applied the concept of political translation to encapsulate the process of knowledge transfer from science to politics, from Green to White Papers, or from the advisory commissions to the Ministries of Education and Research. Naturally, this will always be a process in which some knowledge gets lost, rebalanced, and reinterpreted.

In this study, we have considered at least three broader interpretations that help to theorize the policymaking process. Both deal to a certain degree with the observed repurposing of advisory commissions: (a) from a tripartite function (accountability, expertise, representation) to a singular function (expertise); (b) as the frst stage in a long process of evidence-based policymaking; and (c) as domestic reviewers and translators of global education policies. In this concluding section, we attempt to "de-Scandinavize" our fndings and refect on the larger phenomenon of network governance (Ball & Junemann, 2012) or polycentric governance (Cairney et al., 2019).

First, the traditional, tripartite purpose of advisory commissions accountability, representation, and expertise—has dissipated, replaced with a solitary focus on expertise. Commissions are no longer appointed to keep the government accountable for its political decisions, but function rather as prolonged arms of the Ministry of Education and Research. In Sweden, for example, the number of instructions that the advisory commissions receive from the Ministry of Education and Research has increased signifcantly over the past twenty-fve years (Dahlström et al., 2020). Some analysts see the close collaboration between commissions and the bureaucracy as an attempt to accelerate the pace of commission work and improve efciency. Others regard the collaboration as too close for comfort, at risk of seriously undermining the independence of the commissions.

Unsurprisingly, representation of diverse political perspectives has ceased to be one of the key requirements of commissions. As shown by Christensen and Holst (2017) in Norway and by Dahlström et al. (2020) in Sweden, political interest group representation in advisory commissions has decreased rapidly over the past twenty years. In Norway in particular, academics and other researchers have flled the spaces left behind by the interest groups. Yet perhaps the most visible signpost of how advisory commissions have been repurposed is the advent of Special Investigator commissions in Sweden, also known as one-person commissions.

Te decline of social corporatism in Scandinavia is a well-documented and well-studied phenomenon in political science. Te traces are equally discernible in the advisory commissions in the education sector: our research has shown that we need only focus on the composition of their members. Compared to earlier compositions that refected a greater, broader representation from various interested parties, the advisory commissions have indeed become depoliticized and are now narrowly charged with producing bodies of "evidence."

#### **310 G. Steiner-Khamsi et al.**

Second, the expertization of advisory commissions does not imply a depoliticization of the policymaking process as a whole. On the contrary, as mentioned in the introductory chapter, interest groups now exert their political infuence elsewhere, most notably during the stakeholder review process (stage 2) and in standing committees or parliamentary committees (stage 4), as shown in Fig. 10.1. In an efort to refne the defnition of the expertization of advisory commissions, we suggest the adoption of a multi-level perspective that brings the entire policymaking process into focus. In particular, it may be useful here to rephrase the fve stages of the policymaking process in terms of evidence-based policymaking. Te same fgure presented in the introductory section (see Fig. 10.1) may be depicted in terms of the binary between science and politics, with the collection of scientifc evidence assigned to the advisory commission (left side) and the issuing of a decree or legally binding act by the legislative body (right side). Figure 10.7 depicts this binary in greater detail.

Nevertheless, the sequence of evidence-based policy planning depicted above—starting with the gathering of evidence in commission and ending with evidence-based bills or parliamentary acts—is not as linear in actual practice as the arrow would have us believe. In reality, until a new bill or act is actually passed, a multitude of policy actors are involved. Tese include experts in advisory commissions; stakeholders, interest groups, and civil society during the referral process; the executive body of government, that is, the Ministry of Education and Research; political parties and interest groups that exert infuence on the parliamentary/ standing committees; and the legislative body in the form of parliament. Tus, the same reform idea (competency-based curriculum reform in Norway, strengthening student performance in Sweden) "morphs as it

**Fig. 10.7** The premise of evidence-based policymaking

moves" (Cowen, 2009) at each subsequent stage of the policymaking process. With each transfer from one level to the next, both science and politics come into play, and there is no smooth continuum from one to the other. Rather, the left-to-right sequence is "interrupted" by several rounds of additional information-gathering (scientifc) and consultation (political). As a result, evidence-based policymaking occurs in an iterative and ubiquitous manner that cannot be neatly confned to a designated stage in the policymaking process.

Two examples may sufce here. On the one hand, the production of evidence, a role designated to the advisory commissions, also occurs in White Papers. In fact, as shown in the political translation section of this chapter, the two Ministries incorporate remarkably few references from their advisory commissions and instead draw on their own bases of knowledge. A meager 3% and 4% of the knowledge sources or references amassed by the advisory commissions in Sweden and Norway, respectively, are actually used by the Ministry of Education. As illustrated in Table 10.2, most of the knowledge referenced in the White Papers is home-spun, taken from domestic policy documents. Tis knowledge qualifes as "regulatory science" (see Eyal, 2019) or mode 2 knowledge; in other words, advisory commissions are not the only entity in the policymaking process that produces knowledge. Diferent kinds of knowledge and "evidence" are produced by diferent actors at the various stages of the policymaking process, and this "evidence" is produced, translated, and changed at each subsequent level of the policymaking process.

On the other hand, political coalition- and consensus-building is not restricted to the last two stages (parliamentary/standing committees and parliament) of the policymaking process, but also occurs at the earlier stages as well. In particular, the referral process or the stakeholder review (known as the "hearing" in Norwegian and Swedish contexts) is meant to ensure democratic participation in the policymaking process. For example, the hearing for the school subject Norwegian Language—conducted in March–June 2019 as part of the renewal of the Knowledge Promotion Reform (2016/2020)—yielded a total of 1074 replies (UDIR, 2019). Similarly, political viewpoints determine to a certain extent which contributions from which members of a standing committee are considered as valid "evidence." In sum, diferent kinds of "evidence" are produced at each and every stage of the policymaking process, and actions taken are political, sometimes more overtly than at other stages. It is therefore important to juxtapose the premise of evidence-based policymaking (Fig. 10.7) with its actual practice.

As shown in Fig. 10.8, information-gathering and consensus-building occurs in practice at each and every step of the policymaking process, blurring the line between science and politics.

A good case in point are advisory commissions that, despite their mandate to fnd facts and gather information, build consensus with stakeholders outside the commission as well as among its constituent members. In Norway, the informal collaboration with the teachers' union illustrates this consensus-building process with external stakeholders: even though the teacher unions were not formally represented in the Ludvigsen commissions, the Union of Education Norway explicitly expressed its support for the commission's work (Utdanningsforbundet, 2015). Likewise, the commission made a point of highlighting its regular meetings with the teacher unions as well as with other interest groups and stakeholders (NOU 2015:8, p. 17). In an efort to reach out to stakeholders and the general public, the commission even set up its own blog. Several experts have interpreted this novel outreach approach as a sign of network governance, in which the perspectives of various stakeholders are informally incorporated at an early stage to secure broad political support later in the process, when the report and the recommendations are released. Tese informal collaboration networks also demonstrate the crucial role of the chairs of the commissions, who may decide whether extra efort is made to include additional experts and stakeholders not otherwise represented in the commission.

**Fig. 10.8** The practice of evidence-based policymaking

In addition to securing political support from diverse stakeholders outside the commission, a Green Paper is in and of itself a compromise and an outcome of negotiations among the members of the advisory commission. In his empirical study of expertise-seeking arrangements in policymaking, Baek (2020) interviewed twelve policy experts who had served on Norwegian expert commissions. Te participants shared that viewpoints that were considered too radical to be accepted at the ministerial and parliamentary levels were sometimes left out from the "consented" reports. Furthermore, the consensus-building process was seen to be infuenced by the existing hierarchy among the committee members.

A closer examination of evidence-based policymaking demonstrates that, rather than conceiving of science and politics as binary opposites, it is more accurate to consider them as structurally coupled. Te overlap between science and politics not only makes the policymaking process non-linear and messier, but also creates room for non-state actors to participate in the policymaking process, including experts, users, professionals, and (in Sweden more so than in Norway) businesses.

Tird, we have shown that a transnational lens is indispensable to understanding why and how advisory commissions have become repurposed in an era of global education policies. Our fndings clearly indicate that the most infuential Green Papers in both countries were those that reviewed OECD recommendations and, in the case of Norway, compared OECD recommendations (the DeSeCo framework) with other international competence-based curriculum frameworks. Tus, the advisory commissions acted as important bridges between global and national reform debates, helping to translate and adapt global education policies, or more narrowly OECD policies, into a national setting. Drawing on the multitude of available international curriculum frameworks, or global education policies more broadly, as sources of authority with which to back up national agendas is not out of the ordinary. What is striking, however, is the preference of national policymakers for a very particular kind of international knowledge: one that provides metrics, relies on international comparisons, and is published by a cluster of afuent countries. In Europe, government ofcials are especially receptive to OECD data, studies, and recommendations (Grek, 2017; Niemann & Martens, 2018; Ydesen, 2019). In this study, we have seen how OECD recommendations were elevated to a gold standard explicitly in Sweden as part of the 2015/2018 reform and implicitly in Norway as part of the 2006 curriculum reform.

We conclude this study with three observations. First, the advisory commissions in Norway and Sweden have been repurposed in ways that place greater emphasis on expertise at the expense of accountability and representation. Second, multi-level analysis reinforces the notion that these commissions represent only one stage in a long sequence of evidence-based policymaking. Finally, the transnational perspective helps us to see how advisory commissions have more recently been used as bridges between global and national policy arenas. In particular, they can review, translate, and recontextualize OECD recommendations—or, in the case of the recent reform in Norway, OECD and other international frameworks—to ft into the specifc national context.

## **Notes**


Ishaq, social commentator; Kjersti Kleven, Chairperson of the Board of the Federation of Norwegian Industries; Tormod Korpås, Head of an upper secondary school; Jens Rasmussen, Professor, Copenhagen, Denmark; Mari Rege, Professor; Sr Sunniva Rose, PhD candidate; Daniel Sundberg, Professor, Växjö, Sweden; Helge Øye, project manager.


## **References**


group lobbyism in Denmark and Norway, 1980–2005. *Comparative Political Studies, 46*(4), 457–485.


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **11**

## **The OECD and the Field of Knowledge Brokers in Danish, Finnish, and Icelandic Education Policy**

**Christian Ydesen, Jaakko Kauko, and Berglind Rós Magnúsdóttir**

Refecting on his long-time service in the Organization for Economic Cooperation and Development (OECD), the Norwegian social economist Kjell Eide (1925–2011)1 described the historical relations between the Nordic countries and the OECD in education. Writing about the 1960s, Eide (1990) contended that "the Nordic countries were in a period of strong expansion and reform, and there too, it was at times valuable to have the OECD's blessing for the political directions underlying the reforms" (p. 20). One of Eide's main points in his 1990 essay was that the

C. Ydesen (\*)

Aalborg University, Aalborg, Denmark e-mail: cy@learning.aau.dk

J. Kauko Tampere University, Tampere, Finland e-mail: Jaakko.kauko@tuni.f

B. R. Magnúsdóttir University of Iceland, Reykjavík, Iceland e-mail: brm@hi.is

Nordic countries and the OECD have had a close relationship in education for a long time. He even hinted that the OECD has played the role of a knowledge broker in the Nordic region.

Recent research has painted the same picture. In their comparative analysis of education policies in the Nordic region, Dovemark et al. (2018) emphasized how OECD country reviews and other expert reports—often commissioned by the national governments—are used to "legitimize economic and strict educational policy decisions" (p. 125). In her recent analysis of the OECD's role in the governing of education in Sweden, Grek (2020) argued that the OECD takes up a position as a boundary organization "constructing a very carefully maintained equilibrium amongst the diferent powers and interests of the actors in the feld" based on "a hybrid of both knowledge and policy closely intertwined." More specifcally, the hybrid consists of "hard numbers, administrative advice, managerial know-how and best practice recommendations in a big, versatile, complex and ever-changing mixture of facts and values" (p. 17). Important nuances to these pictures emerge from the previous chapters in this volume.

In this chapter, we follow this trail of research into the OECD–Nordic region relations in education policy by applying a specifc focus on the relations between the OECD and national knowledge brokers. In other words, the chapter investigates the extent to which the OECD via its relations with national institutions has infused policy change in the Nordic region. As such, we do not focus on the OECD as an actor that impacts national school reform; rather, we examine the relation between the OECD and national policy actors that, at critical stages, draw on the authority of the OECD to develop and substantiate their own national reform strategies. Te chapter ofers an in-depth analysis of Denmark, Finland, and Iceland as empirical cases to understand the nexus or assemblage of the OECD with national institutions serving as knowledge brokers in the Nordic region.

Te relevance of this perspective is supported by the fact that the OECD does not have the mandate or the ability to dictate policies in member countries. As many researchers have noted, the OECD operates with a distinct soft power mode of governance (Bieber & Martens, 2011; Mundy et al., 2016; Steiner-Khamsi, 2019). One example of this soft governance is the peer pressure associated with multilateral surveillance among member countries in the OECD. In this respect, Morgan and Volante (2016) pointed out that "the OECD has pursued a strategy of 'soft' persuasion that naturalizes the idea that performance in a series of measurement exercises represents educational quality" (p. 778). Tis strategy is underpinned by what John Krejsler (2019) has called a "fear of falling behind" among PISA-participating nations.

In terms of education policy reforms, the previous chapters in this book have amply demonstrated that a number of national institutions serve as key providers of knowledge. Tey serve as arbiters, brokers, producers, and mediators of knowledge and policy fows between transnational, national, and local spaces. From an OECD perspective, they might even be described as bridgeheads or intermediaries for the dissemination and impact of OECD policy recommendations and policy instruments.

In a theoretical sense, this observation might be expressed using the concept of "instrument constituencies" (Béland & Howlett, 2016). According to Simons and Voß (2018),

Policy instruments […] are not only "active" or "alive" because they contain scripts for reordering society […] but also because they gather a constituency comprised of practices and actors oriented towards developing, maintaining and expanding a specifc instrumental model of governing. (p. 31)

In their analysis of the OECD, Verger et al. (2019) drew on the same concept using Kingdon's (2003) terms. Verger et al. (2019) pointed out how the potential for the OECD governance mechanisms to advance agendas is rooted in a "[…] capacity to open a policy window through which the problem, policy, and politics streams are afected in a relatively coordinated and coherent way" (p. 236). In this sense, the chapter contributes to our understanding of the OECD as a policy actor—even though that is not our specifc focus—because we unpack the role and relations between the OECD and national institutions in the three case countries.

## **Arguing the Relevance and Context of the Three Case Countries**

Te following sections outline a brief frame of interactions between the OECD and the three case countries. Tis outline serves as the argument for selecting the three case countries.

Te OECD has a history of infuencing and making recommendations for the Danish feld of education (Ydesen, 2021). For instance, in April 1963, the Danish Ministry of Education established an economic and statistical section in response to an OECD request in the program for Educational Investment and Planning (Ydesen & Grek, 2019). In 2004, an OECD report, produced at the request of the Danish government, found that Danish education research was too unfocused and called for the establishment of a clearinghouse for educational research in Denmark (Krejsler, 2017; OECD/CERI, 2004). Te Danish Clearinghouse for Educational Research was established in 2006. Te report also emphasized the importance of establishing an evaluation culture, which led to the implementation of national testing in compulsory education in subsequent years (Shewbridge et al., 2011). Another key initiative following from the 2004 OECD report was the formation of the School Agency, which had an explicit focus on evaluation culture and improving quality in the public school system. In 2012, the OECD identifed Denmark as one of only three countries where the PISA results have had an "extremely" big impact on educational policies and practices.

Te OECD has been a frequent collaborator in Finnish education policy as well, and Finland has received additional international attention due to its high-scoring performance in PISA. As described in Chap. 5, researchers investigating Finnish education disagree on how direct the infuence of the OECD on national education policy is. It is clear that the main higher education reforms in Finland have been preceded by OECD reviews (Kallo, 2009) and that there is an element of using the OECD as a clearinghouse for higher education reforms (Kauko, 2011); however, the efect of the OECD on primary and secondary education is more debatable. In understanding this efect, PISA has been the focus of research. Sahlberg (2011) argued that PISA success has resulted in a lack of innovation in primary and secondary education. However, Seppänen et al. (2019) found that Finnish governments have been noticeably active in comprehensive school policies during the new millennium. Kauko et al. (2021) argued that PISA has been compartmentalized from national reviews and thus has limited efect. Rautalin (2013) pointed out how results were used to strengthen the interest groups' and government ofcials' views with little media criticism.

Te OECD has a long history of infuencing education in Iceland, starting explicitly in the 1960s, when the minister of education introduced a human capital approach through an extensive examination of the education system (Guttormsson, 2008, pp. 88–89). Tis examination was one of the building blocks of the comprehensive schooling act in 1974. Te OECD began undertaking examinations of compulsory schooling in 1986 (Guttormsson, 2008, p. 264f). Te OECD's most apparent infuence on Icelandic education is through the PISA measurements that have substantially infuenced Icelandic educational discourse for the last 18 years. After the fnancial crash in 2008, the economic and governmental system was highly criticized (Oddsdóttir, 2014). In recent years, Iceland has strived to rebuild its education system. Te OECD has played a role in shaping the discourse in recent educational policy papers that have strived for a more professional and transparent system.

#### **Research Questions**

Building upon these exemplary connections and policy fows, we hypothesize that there are very strong interactions between Denmark, Finland, and Iceland as cases and the OECD in education. Building on the fndings of the national chapters in this volume, the objective of the chapter is to analyze policy fows between the OECD and the three case countries and analyze the political capital created by the OECD and its use in the national contexts. More specifcally, we aim to investigate the gearing, entry points, and interactions in the links between the OECD and national institutions in infusing policy change.

In terms of policy reforms, we focus on the same reforms as have been analyzed in the respective national chapters in this volume. In this sense, the chapter ofers supplementary insights into the other chapters of this volume through analyses of the transnational policy fows and the knowledge brokers behind the education reforms in each case country. In pursuing this aim, we follow the guidance of three research questions:


## **Methodology and Chapter Structure**

Our methodological approach takes a starting point in the Foucauldian idea about bringing knowledge and power into one analytical feld, assuming that these two are connected and in interrelation molding each other (Popkewitz & Brennan, 1998). We treat institutions and experts as agents who are positioned in a privileged way that allows them to be the providers of seemingly objective knowledge underpinning education reforms while at the same time exerting and institutionalizing power relations and power discourses in the political feld of education reforms. Understanding the workings of this mechanism is vital for understanding the nexus between the OECD and the Nordic region.

Te methodological recipe employed in the chapter consists of three analytical steps aligned with the three research questions. Te frst step pinpoints the central policy instruments connecting the OECD with each national context and identifes the central institutions associated with these policy instruments. According to Lascoumes and Le Galès (2007), a policy instrument may be defned as:

a device that is both technical and social, that organizes specifc social relations between the state and those it is addressed to, according to the representations and meanings it carries. It is a particular type of institution, a technical device with the generic purpose of carrying a concrete concept of the politics/society relationship and sustained by a concept of regulation. (p. 5)

In the case of the OECD, the main policy instruments are policy reviews (e.g., country or thematic reviews), global progress reports (e.g., *Education at a Glance*), and international large-scale assessments (e.g., PISA and TALIS). Following the defnition above, such policy instruments resonate in the national institutions, allowing knowledge brokers to operate between the OECD and the Danish, Finnish, and Icelandic contexts (e.g., universities, sector research institutions, and consortia).

Te second step relies on descriptive statistics of bibliometric analysis and content analysis of national and Nordic policy documents. Tis step serves the purpose of identifying the signifcance and centrality of the national institutions associated with the OECD. We use documents from three diferent reforms in Denmark, Finland, and Iceland. Te bibliometric data is from white and green papers in each context. We draw on the policy instruments identifed in the frst step and analyze how the national institutions are engaged in the translation of these instruments into the national contexts.

Te third step employs a contextual analysis of the institutions and agents to locate them in the respective national feld of education. We use open sources to conduct the analysis of key institutions and agents in the respective national felds (Menashy & Verger, 2019). Finally, we look at the signifcance and position of these national institutions and the OECD in the national educational feld.

In the concluding discussion, we comparatively look across the three cases and ofer insights into the research questions. In this sense, the chapter illuminates the confguration and workings of the OECDcentered epistemic community forming the modes of knowledge and governance woven into the Nordic education fabric.

## **Denmark: A Contested Field of Education Evidence and Research**

Looking at the most central OECD policy instruments playing into the Danish reform process of 2013, it is clear that global progress reports and international large-scale assessments (ILSAs) are most prevalent in terms of citations. Te OECD was the most frequently cited international publisher, with the most important document being the 2009 PISA results (OECD, 2010a). Tus, OECD policy instruments served as important points of orientation among key agents in the Danish education policy feld.

Despite the large number of studies in Table 11.1, the OECD was only the ffth most cited publisher in the Danish policy documents.


**Table 11.1** Cited documents published by the OECD in the Danish database

Clearly, a number of other knowledge providers were at play in the 2013 Danish education reform. Tree of the knowledge providers that were cited more often than the OECD were government organizations, namely the Ministry of Education itself and government-funded sector research institutions (see Chap. 4 in this volume). Tey were responsible for no less than two-thirds of all citations in the policy documents (Fig. 11.1).

Te role and signifcance of the OECD as a knowledge provider cannot be determined by the number of citations alone. Te OECD also infuenced the mindset and the constituency surrounding the reform process. A key point of orientation here appears in the knowledge brokers between the OECD and the Danish education feld, which can be found in the shifting consortia tasked with conducting the PISA surveys. At the time of the reform, the Danish PISA consortium consisted of the Danish School of Education (Aarhus University), Statistics Denmark, and the Danish Institute of Government Research (AKF). Te consortium included a university, a sector research institution, and the national statistical service, all based in the greater Copenhagen area.

**Fig. 11.1** Most cited publishers in the Danish policy documents

Te chairman of the PISA consortium since 2000 was Professor Niels Egelund from the Danish School of Education. Egelund was the leading fgure behind the report *Danske unge i en international sammenligning* [Danish Youth in International Comparison], which reported on the results of PISA 2009, and he was a member of the School Agency chairmanship. Another leading fgure was Professor Lars Qvortrup, who at the time was dean of the Danish School of Education and who worked closely with Egelund. It is striking that Egelund, Qvortrup, and their colleagues, Professor Jens Rasmussen and Andreas Rasch-Christensen, head of research at VIA University College, served on a number of ministerial committees and institutions surrounding the reform. Generally, these four prominent agents, who command considerable capital in the Danish feld of education, have been very vocal and visible in the whole reform process, beginning with the preparatory work and continuing to the evaluation of the reform.

However, looking at the confgurations of the feld of education research in Denmark paints a picture of a rather acrimonious research environment. Te 2013 school reform has been a particular bone of contention. Te researchers mentioned above, who were associated with OECD policy instruments and the development of education policy, constitute one camp in the feld, whereas a host of critical researchers make up another camp. Most notably, in his PhD dissertation, Keld Skovmand (2017) claimed that the 2013 education reform was not grounded in evidence or knowledge. Tese debates are still ongoing some seven years after the reform was implemented. Tus, it is fair to say that Danish education research often fnds itself in a very toxic environment with signifcant antagonism between at least two main clusters, one being the evidence-based what-works type of research and the other being research adhering to pedagogical ideals about *Bildung* and emancipation as well as a notion of pedagogy being a unique feld with its own values and contributions (Rømer, 2017).

In this environment, the Ministry of Education has followed its own agendas and priorities without engaging or siding explicitly with one camp or the other. Being preoccupied with these agendas and priorities, the Ministry has fallen short of making explicit connections between OECD policy instruments and Danish education reform. Tis somewhat retired role has provided ample space for professional and academic debates to unfold—and perhaps for the trenches to be dug deeper.

A combination of these insights with the fndings in Chap. 4 of this volume indicates that the OECD lent authority to knowledge by providing political capital through evidence. Tis capital was picked up—or extended—to power national agents who were able to shape the Danish education agenda in accordance with the ambitions and instruments featured in the 2013 reform.

## **Finland: State-Centered Production of Data**

Te OECD policy instruments under scrutiny in this chapter most relevant for Finland's education system are OECD policy reviews and international large-scale assessments. Research has documented more national policy changes in relation to the former, while researchers have seen the latter as serving more a legitimation purpose (e.g., Rautalin, 2013; Rinne et al., 2004; Sahlberg, 2011; Seppänen et al., 2019). In the OECD electronic archive,2 which starts from 2005, there are three policy reviews that discuss primary and secondary education in Finland: a thematic review on equity in education (OECD, 2005), a country case study on digital learning resources (OECD, 2008a) as part of a Nordic report (OECD, 2009), and in a school leadership report where Finland was one case country (Pont et al., 2008). Tere is also one infuential (Kallo, 2009; Kauko & Diogo, 2011) review from 2006 (OECD, 2006) addressing tertiary education. Finland features in the global progress reports entitled *Education at a Glance*, and the country has participated in all OECD ILSAs apart from TALIS in 2008 (Sivesind, 2019). Finland was frst in PISA in reading (2001), mathematics (2003), and science (2006) before dipping slightly in reading to second (2009), in science to ffth (2015), and more dramatically in mathematics to 12th (2012; OECD, 2002, 2004, 2006, 2010b, 2014, 2016). While the results might have caused debates resulting in dramatic changes in other countries, the provision of education and the basic principles of the comprehensive school have remained the same in Finland. One of the biggest reforms has been the Core Curriculum Reform of 2014, as described in Chap. 5. In this subsection we analyze the curriculum reform, drawing on the bibliometric database before we scrutinize the OECD reports and analyze the networks working with ILSAs.

Te fndings from our bibliometric analysis reveal that the most important sources of knowledge for the curriculum reform in 2014 can be divided into fve main groups. Table 11.2 displays all publishers with more than ten references, accounting for 51% of all Finnish references in the database. Te most important group contains government organizations, specifcally the Finnish National Agency for Education and the Ministry of Education and Culture. Tese organizations have published around one-third of all references in the Finnish documents. Te next group includes two universities, the University of Jyväskylä and the University of Helsinki, with around 12% of all references. Te third group comprises national and international publishers, focusing on both popular and science publications (PS Publishing, Werner Söderström Limited Company WSOY) or only on science (Taylor & Francis). Tis group is responsible for publishing 6% of the referenced material (35


**Table 11.2** Most cited publishers in the 2014 Finnish curriculum reform green and white papers

*Note*: If a document has been cited many times, all citations are counted

references). Te fourth group consists of OECD Publishing, with 14 references (2%). In summary, over 60% of all references in the Finnish database were published either by state or university actors, and the focus of this chapter, the OECD, played a minor role in the number of direct references.

Table 11.3 lists the 14 OECD-published documents that are referenced in the 2014 Finnish curriculum reform. Half are directly linked to numeric education indicators (i.e., global progress reports and ILSAs);


**Table 11.3** Cited documents published by the OECD in the Finnish database

specifcally, fve are PISA-related, and two are linked to *Education at a Glance*. Apart from one source (*Understanding the Brain*), all policy reviews were closely linked to the content of the curriculum reform: skills, competences, future projections, and career guidance.

Outside the reference database, when looking at the background of all reports on Finland published by the OECD, the social networks reveal a more nuanced picture. Te background information of OECD policy reviews makes it clear that the main informants for the OECD teams come from ministries, universities, and interest groups. In the visiting program of the two review teams (OECD, 2005, 2008a) and the working group writing the background memo (Ministry of Education, 2007) for the leadership report (Pont et al., 2008), 86 names3 of informants are mentioned. Te main groups are 24 ofcials from ministries or the National Agency for Education, 16 university researchers (mainly from the University of Helsinki or University of Jyväskylä), 12 labor market organization representatives, 10 education interest group representatives, 8 schoolteachers or principals, and 6 representatives from cities or municipalities. More women (*n* = 49) than men (*n* = 37) were interviewed by the review teams. Te number of people interviewed does not necessarily communicate the impact of single institutions on the report, as many of the interviews happened in groups.

When considering OECD ILSAs, two institutions and communities of experts are important. Te Ministry of Education and Culture contracted the implementation of PISA to either one or both of two organizations: the Finnish Institute of Educational Research at the University of Jyväskylä and the Centre for Educational Assessment at the University of Helsinki. Te former has also been responsible for TALIS. During and after the curriculum reform, the contractor has been a consortium of these two (Opetus- ja kulttuuriministeriö, 2013, 2015, 2018). Te universities of Jyväskylä and Helsinki and their university research centers seem to be main hubs for OECD data expertise in Finland. Tey were most relevant in the OECD review visits, and they are also responsible for implementing ILSAs in Finland. Te connection seems to be institutional: the names of the experts interviewed for the reviews and those conducting ILSA research do not overlap except for Professor Jouni Välijärvi, who was the head of the Finnish Institute of Educational Research until 2017. Nevertheless, these two universities are also the main publishers featured in the evidence production for the curriculum reform analyzed in Chap. 5.

When these arrangements are considered together, the picture of the main institutions for providing knowledge for decision-making in education starts to unfold. Te production of knowledge in the case of the curriculum reform was much aligned with the picture of the country review visits. As noted in Chap. 5 and as we have seen in the Danish case, there is a distinct state-centeredness in data production through the involvement of the Finnish National Agency for Education and the Ministry of Education and Culture. In addition to these, the University of Helsinki and the University of Jyväskylä stand out from the academic side.

## **Iceland: OECD as a Leading External Source**

Iceland features a well-established and systematic state-centered production of OECD data for all school levels. Te Ministry of Education, Science, and Culture is a key institution that has issued many study reports that played an essential role in the policy papers from 2013 to 2017 along with the OECD ILSA documents. Te previous National Centre for Educational Evaluation was the main knowledge broker between the OECD and Iceland concerning ILSA documentation on PISA from 2000 and TALIS from 2008. In 2015, the institutional structure in the feld of education was reformed when the National Centre for Educational Evaluation and the National Centre for Educational Materials merged into one institution, the Directorate of Education (act of law nr. 91, 2015). Currently, the production of OECD ILSA documents is there.

In Iceland, PISA results have always received much attention in the media. Te reporting of the PISA results in Iceland had considerable stability as the same person, Almar M. Halldórsson, was until recently the project manager of PISA and the main mediator of what was highlighted in the PISA results for Iceland. He authored or co-authored all the Icelandic state reports from the beginning of PISA in 2000 and until 2013 (Björnsson et al., 2004; Halldórsson, 2006; Halldórsson et al., 2007, 2013, 2007, 2013). He has also written some academic journal articles on the Icelandic gender gap in PISA (Halldórsson & Ólafsson, 2009; Ólafsson et al., 2006). When the Directorate of Education took over the PISA project, the institution started to authorize the OECD and the Directorate as the authors (Menntamálastofnun, 2017; Menntamálastofnun & OECD, 2019). Te ILSA documents of TALIS kept its personal authorization by Ragnar F. Ólafsson. Tis diferent governmental process of authoring reports from the same institution tells a story about Icelandic governance and its inconsistency. Tis process also mirrors the emphasis on the PISA results at the Directorate ofce led by the former head of the Education Department at the Ministry of Education Dr. Arnór Guðmundsson, that was appointed by Illugi Gunnarsson, Minister of Education 2013–2017, as the frst director of the new institution after having led the editorial work of the White Paper 2014 (WP2014) (Ministry of Education, Science and Culture, 2014a).

Most of the time, there have been weak ties between the higher education feld in Iceland and OECD data production for the education government body, as revealed by the low percentage of academic references in the bibliographies of the policy papers (Magnúsdóttir & Jónasson, Chap. 6 in this volume). Tis situation changed substantially with a new policy adopted by the Directorate of Education. For PISA 2015 and 2018, there was systematic cooperation among professors from the School of Education at the University of Iceland and specialists from the Directorate in analyzing the Icelandic results of PISA 2015 and introducing in a public forum. In 2016 peer-reviewed special issue on PISA literacy was published by the School of Education, University of Iceland.4 Tus, in recent years, the academization of PISA in Iceland has become markedly more prevalent.

Te institutional arrangement of OECD data production has now been introduced and the next step is to analyze the types and numbers of OECD references in the bibliometric database. Second, we scrutinize the authorship and the use of references in two OECD country case reports (shaded in Table 11.4) focused on the compulsory education in Iceland that were referred to in White Paper 2014 (WP2014). Te analysis of the two case study reports gives some further insight to how the national



(*continued*)



*Note*: The frst two reports (shaded area) are the ones that are further analyzed in this section

institutions are working with the OECD when producing some national evaluation and policy data.

In general, Iceland has few publications or documents that can count as green or white papers, and few of the documents that exist have citations and reference lists. Tere is no tradition for reference lists when framing education acts or curriculum guides. However, the protocol is changing, and the most recent documents published by the state do have citations and reference lists. Only three documents published in the 2013–2018 reform period fulflled all requirements for this study. Tese documents were the only white paper (Ministry of Education, Science and Culture, 2014a), published specifcally as such (WP2014) and two green papers (European Agency for Inclusive Education, 2017; Ministry of Education Science and Culture, 2014b) that had a proper reference list for bibliometric analysis. Te latter green paper (European Agency for Inclusive Education, 2017) was updated to count as a white paper (WP2017) soon after it was published (Magnúsdóttir & Jónasson, Chap. 6 in this volume). Tese three documents contain a total of 203 references, more than half of which appear in the only green paper (GP2014) that is heavily referenced by domestic publishers (Statistics Iceland and the state) (Fig. 11.2).

Combining the bibliographic information for these three documents reveals that the OECD is the third most cited publisher and the only one that is international. Te Icelandic references are overwhelmingly governmental and statistical. As discussed in the national chapter, neither local nor global academia plays a big part in providing knowledge in these documents, according to the bibliography. Of external knowledge providers the OECD is the most cited. Te main reason for the frequency of

**Fig. 11.2** Most cited publishers

OECD references is the many citations in WP2014. Table 11.4 lists all the OECD publications included as references in these three documents.

Te green paper from 2014 was a country background report written as an input to the *OECD Review of Policies to Improve the Efectiveness of Resource Use in Schools*. Te document was prepared in response to guidelines the OECD provided to all countries. Te white paper from 2017 was written by the European Agency of Special Needs, so WP2014 is the only document that was written originally in Icelandic by ofcials at the Ministry of Education without any kind of "external help or guidance" (Magnusdottir & Jonasson, Chap. 6 in this volume). Of these OECD publications in the Icelandic database, 78% are cited in the white paper from 2014. Analyzing WP2014 qualitatively through Atlas.ti software reveals that the OECD is much more prevalent than its reference list accounts for. Te reference list has 36 citations when including in-text citations, 12 of which originated from the OECD. Trough word counting, we determined that the OECD is mentioned 66 times in the document (English version) on 21 of the 45 pages of texts (excluding the reference list). OECD citations in WP2014 are a mix of values, conceptual framework, numerical data (mainly PISA), and advice. Still, numerical data are the dominant form of knowledge that leads the advice given for Icelandic policy in the WP2014.

By exploring the two country case reports (the ones shaded in Table 11.4) one can better understand the knowledge production and procedure. Te working procedure is to hire Icelandic scholars or specialists by the Ministry to write a background report that typically counts for a majority of the fnal country case report. Tese background reports are not published. Tat partly explains the scarce of published green papers in Icelandic governance. *Te OECD Study on Digital Learning Resources in Iceland* (2008b) was based on six case studies in the Icelandic education system. Te appendix provides a list of 47 people who participated, 27 of whom were women. None is authorized but only mentioned as participants in the knowledge process. A group of local experts participates in informative meetings. Skúlína Kjartansdóttir, a former school principal and currently an adjunct at the University of Iceland, wrote the other case study report under review; *OECD Reviews of Vocational Education and Training* (2013). She was hired to the Ministry to work on this report and attended meetings with the OECD authors. Her background report is not mentioned in the reference lists, though her name is mentioned in the acknowledgments along with some ofcials in the Ministry. Tere is no list of participants available in the report. References are 35, thereof 17 of which were published by the OECD. It is very similar to WP2014in terms of heavy use of references from OECD.Conversely, the *OECD Study on Digital Learning Resources in Iceland* (2008b) has 13 references, 5 of which are academic journal articles with only 1 OECD reference which is more in line with GP2014, referencing mainly national knowledge providers rather than OECD documents.

**341**

## **Concluding Discussion**

All case country analyses reveal multiple layers in their OECD-related references. On the one hand, all policy documents—both green papers and white papers—tend to follow the demands and credos of evidencebased policy. Perhaps more importantly, the documents associated with OECD policy instruments carry more weight than what can be seen in the mere list of references. First, this fnding reinforces the fact discussed previously in this volume that a bibliometric reference is more than a reference and signals commitment in addition to relaying information. Second, following the theoretical framework of this chapter, this fnding leads us to conclude that the references are shaped by power relations and in the sense of political capital.

Drawing on all three case analyses, we fnd support for a hypothesis that the power networks that have been formed transnationally are manifest in the use of references in the documents analyzed. Te same Finnish network of knowledge brokers functioned in PISA data collection, national education data collection, and ministry-commissioned national data gathering. In Iceland, the weak ties between national and international organizations formed a base of knowledge selection and use. Tis could be seen as a case where social capital is transferred into knowledge. In the Danish case, we also see the clear contours of a powerful national network of protagonists associated with evidence-based policy advice in general and the results from OECD reports and ILSA data in particular that have been able to exert considerable infuence. Te basis of this infuence is found both in vocal media appearances and in participation in the relevant government bodies and consortia.

A mere quantitative analysis of references would seem to suggest that the OECD plays only a minor role in national education policies due to the fact that its references are in the minority despite being an important international reference in the analyzed country cases. However, it is our understanding that the importance of the OECD was not best visible in these references. When deeply analyzing each of the cases, we identifed an appreciation of knowledge and an increased importance of national institutions through links to the OECD. Tis observation raises serious questions for bibliometric analysis and complex questions for further analysis and conclusions.

Without the pre-existing knowledge from other studies on the OECD, the quantitative analysis would probably have erred toward assigning the OECD a less important role than it has. Tis fnding remains a warning for further analysis. From another perspective, one could argue that this analysis now gives the OECD a more important role than it deserves, as we have dug out diferent networks and connections with the OECD. Some perspective as to whether this argument stands could be drawn from an attempt to understand what would have been the alternative results. For instance, is it coincidental that the Finnish universities with more connections to the OECD are more referenced than the ones without? At the very least, the power of knowledge seems to channel through the same hubs. Te OECD is indeed powerful in forming an epistemic community and a constituency underpinning its policy instruments that is more powerful than other international organizations in the Nordic countries. Tese fndings correspond well with Grek's aforementioned argument about the OECD taking up a position as a boundary organization in the Nordic region. Further studies should now move to unravel the confgurations and workings of the nexus or assemblage that creates the basis for this power.

## **Notes**


## **References**


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **12**

## **Regional Policy Spaces, Knowledge Networks, and the "Nordic Other"**

**Saija Volmari, Kirsten Sivesind, and Jón Torf Jónasson**

Two core beliefs that have helped promote evidence-based policymaking in education are that school knowledge is abstract and universal and that "empirical evidence is an efcient indicator of knowledge and learning" (Wiseman, 2010, p. 1). In fact, these beliefs have popularized the *whatwent-right* or *best practices* approach in education policy planning. Te what-went-right approach implies that it is possible to empirically measure students' learning in ways that produce generalized knowledge and that these transferrable features enable policymakers and others to apply fndings across a variety of contexts. Tis view on the transfer of

University of Helsinki, Helsinki, Finland e-mail: saija.volmari@helsinki.f

K. Sivesind Department of Education, University of Oslo, Oslo, Norway e-mail: Kirsten.Sivesind@iped.uio.no

J. T. Jónasson University of Iceland, Reykjavík, Iceland e-mail: jtj@hi.is

S. Volmari (\*)

knowledge partly explains why, for example, the Programme for International Student Assessment (PISA) has developed into a global project, supported by a large number of stakeholders (Addey et al., 2017).

However, previous research has questioned the smooth transition of empirical evidence across countries and continents as well as the universal character of the fndings and recommendations produced by international organizations (Bieber, 2016). First, the receptiveness to international comparisons and the uptake of global scripts and schemes in policymaking depends on national and local traditions and cultures. Steiner-Khamsi et al. (2020) demonstrated that policymakers under the jurisdiction of the state are selective in their application of evidence from international large-scale studies. Second, for the type of transfer associated with PISA and similar studies, it is a common belief that policy transfer occurs and works best when educational systems are similar in terms of the challenges they are facing (Steiner-Khamsi, 2013, pp. 20–21). Moreover, it is assumed to be more likely that neighboring countries will be more receptive to the same solutions on common problems, than countries in other regions or continents. Based on these arguments, neighboring countries select the same policy solutions since they are committed to the same culture or have established similar systems and models to warrant legal rights on behalf of the citizens. For this reason, neighboring states may refect similar temporal and geographic patterns of policy spread (Dolowitz, 2018).

However, to date, researchers have reached no consensus about the role that regional contexts play in transnational policy transfer and the way this context shapes the receptiveness to evidence of what counts as relevant knowledge in school reform policy. In fact, Dolowitz (2018) reported on the criticism that difusion studies do not capture the cultural and social complexity of policy transfers within the global–local policy nexus. Also Steiner-Khamsi (2013) refers to policy borrowing and lending that takes place despite cultural diferences, simply because policy transfer does not necessarily adhere to a rational logic or pattern. Terefore it is important to apply various models and analytical lenses to study policy transfer in a regional context. Tus, this chapter employs various data sources and methods to examine how policymakers and experts in three Nordic countries locate themselves in a larger political reference space when they develop ideas and collect facts and evidence to justify school reforms in their respective countries. Te following questions drive this study:


By examining the latest (curriculum) reforms in three Nordic countries (Finland, Iceland, and Norway) in more detail, this chapter attempts to identify the actual role and infuence of regional cooperation in relation to education reform and the status and use of regional references. Te chapter also aims to contribute to the discussion on the role of regional context in transferring and translating transnational policy knowledge within the national context of policymaking processes.

## **Policy Borrowing Within and Beyond Regional Contexts**

Nordic countries are regularly seen, from the outside but also within these countries, as one cultural and political territory. Tey are believed to share the same value base of the Nordic welfare state related to democracy, equality, and social justice along with a commitment to welfare policies that strive for societal well-being for the whole population (Arnesen et al., 2014; Delhey & Newton, 2005; Jacobsson et al., 2004). Tese countries also share a common history, as they have at times had the same head of state and even formed one union or state with varied compositions (Berntzen, 2017). Tese political traditions are also evident today. Te Nordic countries have all developed political systems in which the nation-state provides welfare services and largely regulates the public sector. Due to the central role of state authority in public governing, they have developed systems to take public interests into account. Hearings and public inquiries symbolize this kind of system.

In the feld of education, the public school systems likewise share considerable similarities, often referred to as "the Nordic model in education" (Telhaug et al., 2006). Te comprehensive basic education in all fve countries consists of 9–10 years of compulsory education. Tere is an uncontested unanimity in the region that comprehensive basic education should be free of charge. Education policies across the Nordic countries have all been strongly infuenced by social-inclusive aspects. Despite recent tendencies to introduce more market-driven policy solutions, education in the Nordic region is still seen as a crucial instrument for increasing social justice in all Nordic countries. (Arnesen & Lundahl, 2006; Lundahl et al., 2018; Lundahl, 2016)

Considering these presumable common denominators of the Nordic countries, one may expect that there exists a Nordic space for policy cooperation and thereby a considerable policy transfer of knowledge within the region. However, three particular aspects must be taken into consideration in attempts to identify this space. First, as the political scientist Paul Cairney (2016) has suggested, to understand the use of evidence in policymaking processes, one needs to understand both how the policymaking process works (p. 10) and how policymakers ft into it (p. 6). Despite their common history and similar political traditions, the decision-making procedures and the role of diferent policymakers in them may well vary. Hence, it is crucial to take both similarities and differences into account and not to assume that these countries are necessarily uniform in terms of political decision-making processes. Second, since the 1990s, the traditional Nordic values of universal well-being and education have been increasingly infuenced by political interests in neoliberal values, individualization, marketization, and technologies of new public management. However, diferences exist in the extent to which this interest has turned into a dominating feature and infuenced education policy and reforms in various Nordic countries (Arnesen et al., 2014, p. 1). Tird, national reforms, policies, and policy discourses have become globally framed and are partly infuenced by international and supranational organizations, such as the Organisation for Economic Co-operation and Development (OECD) and the World Bank (Rizvi & Lingard, 2010). Tese organizations have become active in producing large-scale comparative data and thus involved in advocating education reform agendas in the Nordic region. Tere are good reasons to consider the infuence of these organizations and their programs in national reform making and national policies as contingent (Sivesind & Wahlström, 2016).

For instance, references to the OECD are partly related to how the data match central topics and issues in national reform processes. Moreover, although the OECD is a politically infuential organization, the use of its data is mostly detached from its ideological views. In particular, numeric and comparative data from such organizations are used as evidence. Numbers are convenient tools in the current culture of evidence-based policymaking, as they appear to be neutral, apolitical, and objective (Stone, 2016). Tey represent what Gil Eyal (2019) called "mechanical objectivity" (p. 115) in policymaking, that is, a legitimation strategy pursued by referencing, rankings, and quantifed comparisons (see discussion in Chap. 2).

Previous research has discussed the regional education space particularly from the point of view of the European Union infuence and established the term "European education space" (see, for instance, Lawn et al., 2011; Grek & Rinne, 2011). Researchers have demonstrated that this policy space has been extensively shaped and sustained by "governing by numbers" (Ozga, 2009; Rose, 1991) as a technology of governing. Grek and Rinne (2011, pp. 29–30) and Martin Lawn et al. (2011) claimed that this policy space has been constructed through harmonization attempts, soft governance, benchmarking, and comparison, which evolved after governments agreed upon the Lisbon treaty (Grek & Rinne, 2011). In addition to law and regulations, transnational expert networks play a signifcant role in how this process is evolving (Lawn et al., 2011). In a recent article, Grek et al. (2020) characterized this form of regional space as resulting from how experts "make meaning and attempt coherence in networked forms" (p. 4). One concern in these contexts is to achieve consensus about what to measure in terms of large-scale assessment.

However, comparisons and numeric data produced by the OECD and similar organizations are not by themselves a sufcient explanation for how regional spaces evolve. As Espeland (2015) has underscored, the objectiveness of numbers and their generic character are appealing since nearly any political narrative can easily be attached to them. In other words, the generic character of numbers supports the creation of political narratives. Tus, the narratives of the regional practices and mindsets can serve as a lens through which to understand cooperation and shared interests that shape Nordic education policy space in particular ways.

Our hypothesis is that this regional Nordic policy space, although partly gaining its signifcance from international networks, operates differently than for instance the European policy space. One reason is that the Nordic region by itself does not produce numbers for assessing education, like the OECD does. Tus, the Nordic community of policymakers and experts relies on something other than agreements upon benchmarks and standards, and we set out to explore this Nordic "other." Tis chapter will examine how this "other" is constructed by relating to a shared policy space that is developed and sustained in various ways.

To analyze Nordic cooperation as a regional space, we will draw on Massey's (1994, p. 2) conceptualization of spatial as "the social stretched out" and her idea of space as not fxed and static but rather "a product of interrelations and constituted through interactions" (p. 7). Following this idea, we look at the Nordic policy space not as a separate layer between the national and transnational, but as a space where national actors interact and form connections to make sense of and give meanings to the international. Tereby, they construct a regional mindset, which we refer to as the *Nordic "other."* As such, the Nordic policy space is in a state of constant becoming and fuidity, created and re-created through the interactions of the involved policymakers, experts, and stakeholders and their narratives of relating to each other.

## **Country Cases**

As we have seen in the previous country chapters, the number of references to other Nordic countries (i.e., regional references) identifed through bibliometric network analysis was surprisingly low. We also found that the country distribution of regional references varied to some degree. Norway used 66 regional references, signifcantly more than Iceland, where only 4 references were regional. Among those regional references used in Iceland source documents were a World Health Organization (WHO) document published in Denmark and documents produced by the European Agency of Special Education, placed in Denmark. In Finland, we identifed 11 references as regional. Te rare use of regional references and the frequent use of national references prompted us to wonder why this pattern had become so biased.

In order to illustrate the regional network pattern of references, we conducted a bibliometric network analysis, which we present in more detail later in this chapter. We also decided to combine this network analysis with complementary data sources to examine the three country cases—Finland, Iceland, and Norway—in more detail. Tese three countries are similar in terms of their legacies as young nation-states compared to Sweden and Denmark (Elgenius, 2011), but they difer in their decision-making procedures and in the role that policy documents play in these processes. Tus, we decided to conduct interviews with experts with knowledge and information about these features of the system. All experts interviewed were involved in the national reform processes that are examined in this book.

We should also mention that the reforms we examined were partly different in terms of their timing, focus, and scope. In the case of Finland, we examined the latest reform of the National Core Curriculum (2014). As explained in Chap. 5 in this volume (Volmari, Kauko, Anturaniemi, & Santos), in the case of Finnish curriculum reform, the research team made the decision to focus on the reform of the objectives of the National Core Curriculum and the distribution of lesson hours from 2012 (Government Decree 422/2012). Te government decree is based on a white paper and it delegates the power to decide on the content of the core curriculum to the National Agency for Education, within the framework the government decree sets for this work. Te most prominent policy documents in this process were the white paper *Future Basic Education* from 2012 (Opetus- ja kulttuuriministeriö, 2012) and the green paper *Basic Education 2020: Common National Aims and Division of Teaching Hours* from 2010 (Opetus- ja kulttuuriministeriö, 2010). Tis green paper was originally designed as a white paper, but it was discarded at the last minute due to political disagreements and a change in political power. Nevertheless, the 2012 white paper was mainly based on the green paper preceding it, as demonstrated in Chap. 5. Both working groups utilized expert hearings and statements and extensively involved diferent stakeholder groups, particularly in the process of the green paper. For example, the working group of the green paper 2010 received commentary from over 200 municipalities in Finland and utilized a survey of over 60,000 students on their experiences in school and their vision of an ideal school.

In the case of Iceland, we base our analysis in this chapter on three documents used to govern Icelandic compulsory education, together with the law, regulations, and formal curriculum (see Chap. 6). Two of these are classifed as white papers and one as a green paper. As discussed previously in Chap. 2, white papers are political by default and therefore infuenced by the government and the minister in charge of the policy processes. However, the political infuence by the administration of the same government varies depending on the continuation of policies over time across various governments. If the background work initiated by one minister leads to the continuation of policy by the previous minister, then the background work sets the agenda and may not even reach the status of being enquired by a green paper. Tis is often the case in Iceland, where recent decades have ofered many examples of extensive background work being conducted and used to underpin reform bills and resolutions in the form of extensive discussion and consultation rather than in the form of public enquiring and formal written documents, which we refer to as green papers. Tus, although the working mode of Icelandic policymaking relies, sometimes extensively, on internal and external documentation, this evidence does not necessarily reach the stage of reference lists or other formal acknowledgment.

Tis situation varies starkly from the Norwegian case. As already presented (see Chaps. 7, 9, and 10), Norway has expended a huge amount of resources on public commissions with a mandate to evaluate and make recommendations to decision-makers regarding how to reform and renew the public sector. Te government appoints members to sit in these commissions. In this chapter, we refer to eight such reports and two white papers. We include references in these documents in our bibliometric network analysis, while two particular inquiries (Bostad and Ludvigsen commissions) and one white paper (Subjects—In-depth Learning— Understanding, see: Appendix 1) serve as the main data source for our qualitative study. While the Bostad commission made inquiries and advised the ministry and parliament on how to revise the education clause formulated within the School Act, the main work task of the Ludvigsen commission was to recommend overall goals, aims, and structures for renewing the national curriculum. A group of ofcials with educational-scientifc backgrounds and experience from the Directorate of Education assisted the experts during the writing process, and civil servants within the Ministry of Education were the main actors in the process of formulating the white paper. Tey worked closely with the political leadership to fnalize the paper. Troughout the process, various groups of stakeholders were invited to provide comments.

Altogether, the three cases selected for this study vary in terms of organizational arrangements for preparing political decision-making processes. For instance, in contrast to Norway, academic participation in Finnish preparatory working groups has declined during the last decade. In the national core curriculum reform of 2010–2016, the working groups we examine in this chapter contained no scientists, although scientifc experts ofered their insights. Te members of the working group of the white paper from 2012 (Source Document 66 in the Finnish sample, see Chap. 5) consisted only of the civil servants of the Ministry of Education and Culture. In the working group of the green paper, members came from the main political parties and stakeholder organizations in addition to civil servants. Tere were no representatives from other countries. Norway chose a rather opposite approach. A scientist led Ludvigsen's committee, which included other scientist members as well. Two of the members represented other Nordic countries, namely Denmark and Sweden. Conversely, Iceland has adopted no formal procedures to produce white papers based on inquiry bodies, although the government plans to move in this direction. It is up to each minister to determine how the work is approached, the amount of party political involvement, and the infuence of stakeholders or various professional groups. Parliament is not involved until a very late stage in the formulation of laws within education. To conclude, the three case studies varied in their timing, focus, and scope, as well as in terms of organizational arrangements. But they also varied in what was valued as expertise and how and in which stage of the reform process expertise was utilized.

## **Data and Methods**

Our research design originally stemmed from the fve national country cases. Across the fve Nordic countries, there was one fnding that particularly puzzled the research teams. In spite of what might be expected and what previous research on education transfer has predicted, we found no solid evidence of regional policy transfer or learning. In fact, the number of regional references in each country was incredibly low, and a shared knowledge base was almost absent. Tis sparked our research question and design, and we decided to combine methods to see if we could fnd any evidence of Nordic policy learning and sharing that were not necessarily authorized within bibliographies and footnotes.

To begin, we conducted a bibliometric network analysis of the regional/ Nordic references to identify any evidence of a common Nordic knowledge base. In the bibliometric network analysis, we used the software program UCINET to generate statistics on Nordic references and draw illustrations of the Nordic knowledge network. Te method of bibliometric analysis and how it was applied in this research project is explained in more detail in Chaps. 1 and 2. However, this method did not appear to ofer a complete picture of Nordic policy cooperation and of the possible knowledge exchange within the Nordic region.

Inspired by Törnberg and Törnberg (2019), we perceive networks as cultural products in which "discursive and cultural elements play out and shape the networks in which they exist" (pp. 61–62) and adopt their suggestion to complement network analysis with other methods, namely with expert interviews. We conducted interviews with members of the key working groups and committees in Finland and Norway and some key ministerial administrators in Iceland. We used the interviews both to obtain new insights into regional knowledge sources and to acquire knowledge about Nordic cooperation and knowledge-sharing, which cannot be disentangled by bibliometric network or document analysis. We used thematic expert interviews with the same list of topics in all three countries. We conducted 18 semi-structured interviews in total, 8 in Norway, 5 in Finland, and 5 in Iceland. We recorded some interviews using Zoom and others with single recorders in physical settings, and all interviews were transcribed to ensure the validity of our approach.

We analyzed the interview data while keeping the following points in mind. First, informants were experts in their feld and had been members of the preparatory committees or working groups in the reforms we examined. Hence, the information they provided can be seen as reliable frsthand information. However, we acknowledge that the informants shared with us their accounts and recollections, sometimes of the work they were involved in almost ten years ago. Attentive to these two vantage points, we decided to categorize our interview fndings into clusters of main narratives.

## **Results**

In our study, we set out to identify the actual role and infuence of the regional, Nordic cooperation in relation to our three country examples (Finland, Iceland, and Norway) of an education reform, and the status and use of regional, Nordic references in these reforms. Our main methods were bibliometric network analysis and thematic expert interviews. We present our fndings below, divided into two subsections according to our two main methods.

## **Bibliometric, Regional References, and the Absent Nordic Other**

Te bibliometric analysis reveals, in part, where diferent countries have looked for inspiration and infuence. Te whole database, including the references of all fve countries, had a total of 225 regional references, meaning references where a country referred to a publication published in another Nordic country. Tere are numerous reasons to expect that one could detect notable policy referencing of information sources between the Nordic countries in our data. However, this does not seem to be the case. Te number of Nordic references used in these reforms was surprisingly low compared to the use of domestic and international references (see Baek, Tiplic, and Santos, Chap. 9, Table 9.3). References from other Nordic countries amounted to approximately 2–7% in all of the countries among the fve cases, which we consider as a key fnding.

In Finland the regional Nordic references amounted only to 1.63% and in Iceland 2.08%. In Norway, 7.09% of all the references were from Nordic sources, but compared to the amount of domestic (66.83%) and international references (26.08%), this percentage was low as well.

In Table 12.1, we can see how references to sources published in another Nordic country were distributed between the fve countries. Altogether 44% of the time, the referenced documents were published in Sweden, while 31% of the time, a country referred to a publication from Denmark. Sources published in Finland or Norway were referenced less than 10% of the time and Icelandic sources only 1% of the time. Based on Table 12.1, one can conclude that the regional evidence base consists predominantly of evidence published in Denmark or Sweden.

Table 12.1 also shows how the regional references were distributed in each country. Documents published in Sweden were the most referenced in Denmark, Finland, and Norway. For example, fve out of eight Nordic references made in the Danish source documents were published in Sweden. In the Finnish source documents, 7 out of 11 cited Nordic references were of Swedish origin. In Norway, the 180 regional references cited in the source documents were distributed slightly more evenly between Denmark (70) and Sweden (77). Iceland served as an exception to this pattern, as in the Icelandic documents only four regional documents were cited, all of which were published in Denmark. However, one was produced by an international agency and three were by a European agency. In the case of the Swedish reform, 44% (8 out of 18) of the regional references cited in the source documents were published in Denmark, with 4 from Finland and 5 from Norway.

Although Danish and Swedish documents were used more than the Finnish ones, the most prominent documents in the Nordic knowledge network would appear to be those used more than once and in more than



one Nordic reform. When we look at co-citations, or those sources referenced by more than one document, the pattern tells a slightly diferent story. Out of the seven most co-cited sources, three were published in Denmark, two in Finland, and two in Sweden. Te Danish and Swedish publications were cited in only the Norwegian policy documents. Both of the Finnish publications were cited by two diferent Nordic countries. Specifcally, the Finnish National Core Curriculum of 2014 was cited two times by diferent Norwegian sources and one time by a Swedish source. Likewise, the Finnish National Core Curriculum of 2004 was cited two times by diferent Norwegian sources and once by a Danish source.

Moreover, the two most cited Finnish documents were actual national core curricula for comprehensive education (2004 and 2014), of which we examine the 2014 curriculum as an example of a Finnish education reform. Tese documents, the 2004 and 2014 National Core Curricula of Finland, were also the only ones among the most-cited regional documents that were actual national core curricula referenced by more than one country. Tey were part of the knowledge base of the reforms in Denmark, Norway, and Sweden, which indicates that other countries have looked upon Finnish curricula as prominent regional references.

Figure 12.1 illustrates the whole network, further confrming the fndings explained above. Norway, in white, clearly co-cited a cluster of regional knowledge most, as indicated by the arrows pointing outward, indicating policy borrowing. Te network consists of predominantly Danish (red quadrants) and Swedish (yellow quadrants) references, which were used in the majority of Norwegian source documents (white dots). Te two Finnish (blue quadrants) documents are medium-sized, as they were referenced less by other Nordic countries than the most co-cited documents of Denmark and Sweden, which appeared in the Norwegian data.

As these results demonstrate, the regional Nordic references were mostly absent in these fve reforms since there were relatively few cocitations. National chapters have explained this as a matter of selfreferencing (Chaps. 5 and 8). However, Norway was a small exception among these countries, with its regional references amounting to almost 7% (see Table 12.1) and with a cluster of references that were co-cited

**Fig. 12.1** Network of the Nordic references. (*Note*: This fgure includes only references for which the publication location differed from the publication location of the source documents. Denmark = red; Finland = blue; Iceland = orange; Norway = white; Sweden = yellow; unclear Nordic = black)

(see Table 12.2). By looking into these co-cited references, we fnd that the reform made use of some reports published in other Nordic countries more actively than the level observed in other Nordic countries' documents. Yet, even in Norway, the number of regional references was lower than originally expected on the basis of previous theories of policy transfer within a region and between similar cultural contexts.

## **Cooperation and Communication Within the Nordic Policy Space**

Te Nordic countries maintain close political and policymaking connections to each other, and their governments have formal connections at multiple levels, for example, through the work of the Nordic Council (founded in 1952) and the Nordic Council of Ministers (founded in 1971). From 1967 national ministries in education initiated Nordic research collaboration to stimulate more advanced use of teaching


#### **Table 12.2** Most co-cited Nordic references

(*continued*)


**Table 12.2** (continued)

*Note*: "Nordic" references refer to those published in a Nordic country that are cited by source documents published in a different Nordic country (e.g., Danish documents cited by Swedish source documents or Finnish documents cited by Norwegian source documents)

technology. From early on, these Nordic institutions initiated evaluations, conferences, and formal inquiries to address issues of signifcance for the governance of Nordic school reforms. One known report published by the Nordic committee for educational research, chaired by Johs Sandven, addressed, for example, how programmed teaching and innovative usage of new technology can improve teaching in schools (Dahllöf & Wallin, 1969). Another report about the administration of the Nordic school systems was published in 1974 after two years of work by a Nordic working group of civil servants who had the mandate to help harmonize the national curricula for the compulsory school systems (Nordiska rådet, 1974). Tey were also asked to provide information about the Nordic school system of relevance for teachers, associations, and administrative bodies. Te Nordic Council (Nordiska Ministerrådet, 1978) made a revision of the report four years later. Here, they presented formal information about the diferent school types within the education system and how they were reformed within the single country.

Later reports ofered evidence of systematic cooperation across the Nordic countries, partly formalized through a governing body of civil servants from the Nordic ministries and a working group that was responsible for meetings and conferences (Nordisk Ministerråd, 1990). One outcome in the early 2000s was the conference report *Vision and Reality* (Sigurðardottir & Harðardóttir, 2000), which addressed evaluation and assessment in Nordic schools. A later report from the Nordic Council of Ministers was *Northern Lights on PISA 2003—A Refection from the Nordic Countries* (Mejding & Roe, 2006).

Tis report, however, did not originate from the work of the Nordic Council; instead, it was written by Nordic researchers who collaborated within the OECD. Te Department of Teacher Education and School Development at the University of Oslo had published an earlier report entitled *Northern Lights on PISA, Unity and Diversity in the Nordic Countries in PISA 2000* (Roe et al., 2003). Tis report stated that PISA represented a new commitment by the governments of OECD countries to monitor the outcomes of education systems with a focus on learning achievement. From then on, it seems that Nordic data and knowledge about the quality of the school systems was hijacked by organizations other than the Nordic Council. Yet, meetings among civil servants and experts within the context of the Nordic Council continued.

To further illustrate the frequency of Nordic cooperation in the feld of education, Table 12.3 presents the number of meetings taking place between top political and administrative levels, as a matter of course, every year within the arena of education and research (and a similar pattern would be obtained for a number of other arenas).

Te informants in all three countries talked about constant and regular formal cooperation, such as annual meetings of the directors of the national education agencies, meetings of Nordic curriculum specialists once or twice a year, and cooperation within the Nordic Council of the Ministers. In addition, ad hoc groups are formed to deal with pressing


**Table 12.3** Number of meetings each year 2013–2019 under the auspices of the Nordic Council and Nordic Council of Ministers

*Source*: https://www.norden.org/en/organisation/nordic-co-operation

*Note*: These meetings related to education involved at different times parliamentary representatives, ministers of education, and civil servants at the ministries of education. Adapted from the webpage Nordic cooperation, which describes the organization of both the Nordic Council and the Nordic Council of Ministers

issues when the need arises. Although the bibliometric network analysis demonstrated a very low percentage of regional references, with Norway being the only Nordic country that seemed to refer substantially more to other Nordic countries, the expert interviews revealed active and vivid cooperation and communication among the Nordic countries. Our informants noted that, in addition to the ofcial meetings, several groups met regularly at the Nordic ministerial level or among the governmental organizations outside the purview of the formal Nordic Council cooperation. Several experts talked about frequent but more informal communication and collaboration, like phone calls and study visits. Coherent documentation of this is not available as formal minutes are not always written or centrally available even though all the meetings are fled within individual ministries. Tere is no question of extensive Nordic contact and discussion at the Nordic policy level, but little evidence is available about the actual impact of those meetings on policy formation.

## **Legitimation, Policy Dynamics, and the Omnipresent Nordic Other**

Since the bibliometric network analysis revealed a very low use of regional Nordic references despite well-established Nordic political and cultural cooperation, we decided to conduct interviews with the key experts involved in the reforms in question in Finland, Iceland, and Norway. In a total of 18 interviews, we asked the informants questions about their role in the reform or working group, their professional background, and the issues and themes they considered relevant during the reform process, in the work of a working group or commission, or in the Nordic cooperation more generally. In addition, we asked about any national, Nordic, or international conferences, workshops, or meetings they attended during the process with a particular focus on any Nordic meetings or other forms of Nordic cooperation.

We also specifcally asked them for possible explanations for the lack of Nordic references in the key policy documents. Te absence of regional bibliometric references in our data came as a surprise to most informants, as they all considered Nordic cooperation important and meaningful. From responses to the question on the possible reasons for the absence of references to Nordic knowledge sources in the actual policy documents, we identifed four main narratives the informants used to explain this absence:


Te frst narrative our informants provided for the absence of Nordic evidence in the actual written documents was that Nordic cooperation is often rather informal in nature and hence not well documented. Earlier in this chapter, we discussed the ofcial forms of cooperation. However, many informants asserted that a signifcant part of the cooperation occurs in more informal contexts and ways. For instance, they described study visits as being very common between the Nordic countries. Tese visits seemed to be particularly common when a country was in the middle of a reform process. Informants in all countries reported this kind of visit as one form of collecting knowledge and ideas. Several Finnish and Norwegian informants mentioned the study visit of the Norwegian Ludvigsen's committee to Finland. Although the Norwegian committee prompted the visit and prepared a set of questions it specifcally wanted to discuss to learn about the recent Finnish core curriculum reform, the Finnish informants described the meeting as one of mutual exchange and learning. In general, the informants described Nordic education cooperation as the "kind of expert kind of cooperation, where we do discuss very profoundly why someone has done something" (working group member, Finland). Since this kind of cooperation is not documented in memos, meeting minutes, or reports, it does not produce data that lends itself to reference in the sense of assertions backed up with information (Cairney, 2016) like the bibliometric references examined in this chapter.

Te second narrative we discovered is that Nordic cooperation is such an implicit part of policy cooperation that it does not need to be mentioned specifcally. Tis second narrative is closely linked to the frst one. Most informants declared that in general other Nordic countries were always looked at and their policy developments carefully followed and that in any reform process, one tended to always start by looking at other Nordic countries. One informant specifcally explained that "we always start by looking at other Nordic countries." More specifcally in the reforms we examined, the informants assured that this was in fact done, even if it may not have been translated into bibliometric references. Tis discrepancy can partly be explained by the frst narrative, in which cooperation and policy infuences are not carefully documented. Another explanation for Nordic evidence being tangential at most in the documents we scrutinized could be that Nordic cooperation is axiomatic and no bibliometric reference is possible or needed. Te following two Icelandic interview excerpts illustrate this point with lucidity. While discussing updating a law on compulsory education, an Icelandic informant noted:

Tus, it was in all this process that the Nordic laws were always on the table, as a sort of a working document, and we used arguments from the Nordic background documents. I am pretty certain. But it was such a matter of course that it didn't need mentioning. (Civil servant, Iceland)

In discussing updating the curriculum, an informant stated:

I am pretty certain, as I remember it, we were all reading the Nordic curricula, and this was taken for granted and didn't need discussing. (Civil servant, Iceland)

For some of these Icelandic informants, the Nordic space was such an implicit part of the process that it did not need to be acknowledged specifcally. Furthermore, we encountered similar narratives in the Finnish and Norwegian interviews. Te narratives we discovered of the Nordic space being so implicit yet simultaneously omnipresent bear similarities to the description of pre-Lisbon European education space, as described by Grek and Rinne (2011). Unlike the European education space, the Nordic is not built on harmonization through benchmarking and numbers, but it still pre-exists in the hearts and minds of the people, like the pre-Lisbon European space did (Grek & Rinne, 2011, pp. 29–30). As such, it is so ubiquitous that no distinct mentioning is required.

According to several informants, the absence of regional Nordic evidence may be related to the dynamics of externalization (the inclination to refer to the education systems of other countries or information produced by global actors in reform proposals, see for instance: Steiner-Khamsi, 2003) and to the tendency to maintain the status quo and avoid including Nordic knowledge that may prove controversial compared to the predominant global policy discourses. In an interview about the Norwegian committee work, one informant described the following:

I think it's about externalization. I think international references are much heavier in those situations. When it comes to writing policy texts, so there were a lot of Nordic references in the discussions but yeah, I can also see that in order to get legitimacy for the policymaking international references are playing a much more, a larger role than the Nordic references so, and I think it's about not to challenge the consensus too much. If you bring in domestic or Nordic references, it could be more controversial because the discussions that usually emerge in reform topics are much more a local thing. (Committee member, Norway)

Tis excerpt illustrates well the third narrative we identifed, which is that the legitimation function and hierarchization of evidence play roles in choosing evidence to reference in written documents. Several informants acknowledged that Nordic cooperation does not produce data that can be used as references in policy documents, but they also stated that, even if it did, international evidence would still enjoy a higher status as a legitimation instrument. Our informants frequently brought up legitimation as an explanation for the absence of Nordic knowledge as evidence. In particular, they mentioned the OECD as one of the main providers of policy evidence used for legitimation of nationally made decisions.

Tese fndings are perfectly in line with the theorizations of Wendy Espeland (2015) and Deborah Stone (2016), who stated that numbers are particularly appealing for policymakers and politicians since their ostensible objectivity makes them ideal in legitimating any political argument. Te OECD produces typically comparative data based on standardized indicators detached from both the multiplicities of local cultures and situations and the organization's own political agenda. Tis perceived objectivity makes this data an excellent tool for mechanical objectivity (Eyal, 2019), as it can be used to match any central topic or issue in the national reform processes.

Te OECD also produces an extensive amount of reports and other written materials where this data is documented and easily usable for bibliometric reference. However, our informants stressed the legitimation status and function of the OECD data over the mere technicalities of it being carefully documented. In fact, one informant even mentioned that country reports were specifcally ordered for national legitimation purposes. Te informant questioned the previous research on transnational governance, claiming that the data production of the OECD often stems from national needs for evidence. Te informant further asserted that the OECD's reports are frequently commissioned by national governments for specifc legitimation needs. He stated:

Well, that transnational governance, there is of course this point of view related to it, that the OECD produces diferent kinds of data. Tat there is real data that can be analyzed nationally and base the decisions on, to use it as basis for decision. But then there are also these country reports that the OECD countries can order, so this kind of commissioned research. And it is of course so, and we have examples of this as well in our state governance, that if not directly that, that they stem from the own interests of the country in question. Tat one decides on a political measure, and then one orders the country report from the OECD, where then international evaluators become involved. And before these reports are published, the countries and the OECD do discuss them. So I think this idea of transnational governance is just one angle, namely this when one uses these country reports. But here one comes to what I said before, the blanks in evaluating research, that one legitimizes one's own views with some material produced by an international actor. (Working group member, Finland)

Tis interview excerpt illustrates the fourth narrative as well. Most informants acknowledged and underlined the fact that in curriculum reforms, the national orientation and political dynamics still played the greatest role. In general, national core curricula, as they have been described by our informants, are securely in the national domain of education policy and politics, as this Norwegian informant emphasized:

Perhaps one of the explanations is that in most countries they probably consider such curriculum work as a very national domain. … Every time in a way curricula are put on the agenda internationally it is like—the frst things everyone says, is that curricula are a national domain. We do not want such an international decision on that. Tat this area is not suitable for that. (Committee member, Norway)

Te fact that curriculum development is still seen as a very national domain of education policy may well explain not only the absence of regional references in our data, but also the self-referential nature of the country cases addressed in more detail in the previous chapters. Perhaps research on another domain of education policy, for instance higher education policy, would have produced data that included more international and regional references. However, the infuence of evidence or the best available knowledge did not seem to be the most important factor in determining the direction for the reforms. Te result of the reforms often arose from negotiations, and issues on the reform agenda sometimes became politicized. In a confict or gridlock situation, compromises were inevitable, even if it meant not utilizing the best knowledge available, as demonstrated in the following interview exchange:


In spite of the narratives explaining the absent Nordic "other," our informants described Nordic policy cooperation as a space of oneness. In the interviews, many described this space as a feeling of belonging. According to our interview fndings, the Nordic education space works as an arena where issues and policy developments are discussed with the like-minded. Still, the actors are simultaneously aware that the education systems, the political situations, and even the degree of marketization tendencies in the fve countries are rather diferent. Tough the cooperation may not produce actual evidence according to the defnition we employ in this chapter, the Nordic education space had concrete benefts as well according to our informants. Tese benefts materialized particularly in the global policy space, for example in the meetings organized by the OECD where the Nordic representatives collaborated, for instance, by voting for same policy solutions and recommendations. An important part of informal cooperation happens in this kind of international setting, as demonstrated in this excerpt from an Icelandic civil servant:


decided to take a stand on some emphasis in new projects, as there are votes on what to research.

Interviewer: And this carries with it some infuence? Nordic infuence? Informant: Absolutely. Te Nordic countries carry much infuence in there. Tey are always, normally, in agreement. One takes the foor, and the others support. It carries some weight. Even China or large countries like the US, which are always on their own, they have their one vote each. Tus, yes, of course, it carries some weight. And people notice. (Civil servant, Iceland)

In this case, the international setting acted as a glue between the Nordic countries, since the Nordic representatives met beforehand during an informal dinner to discuss the formal issues on the OECD meeting agenda. Te Nordic community of sameness was, in this setting, a strategic tool to gain more power in an international setting, as the Nordic countries supported each other in ofcial votes. Tis arrangement indicates that, regardless of the possible diferences in education systems, policymaking procedures, and political situations in these countries, they share some fundamentally similar values and interests that allow them to reach such consensus. Te aforementioned excerpt also validates the previous claim of several scholars that the Nordic countries are seen as one entity from the outside. It demonstrates that other countries and regions acknowledge the weight of Nordic unity.

Moreover, the conduct and practices of the Nordic actors in the international policy space also demonstrate how the national, regional, and international are not layered and separate. Te global dynamics of the education space are not just a matter of international organizations exercising their infuence on the national level, with the regional level acting possibly as a bufer or a mediator in this process. Instead, our interviews painted a picture of an active and rather powerful Nordic education space that stretches out through the connections of the national actors and exercises its infuence as one entity in the international decisions and recommendations of the OECD and other organizations. Even if this infuence cannot be traced back in the policy documents as bibliometric references or as evidence in the way defned by Paul Cairney (2016), the Nordic education space has in fact infuenced what is included in the international policy recommendations (e.g., the OECD reports) now used as evidence in the national documents. Te regional context may shape receptiveness to evidence, but it may also shape the actual sources of evidence when they are created and drafted in the international context of policy cooperation. In short, there certainly is more in the policy process than meets the eye.

## **Conclusions**

Previous research has implied that neighboring countries may be more receptive to the same policy solutions due to their commitment to a common culture and similarities in their legal and political systems. Te Nordic region can be looked upon as a cluster of such countries. Tey are often, both from the inside and the outside, perceived as one cultural and political territory. One could thereby expect considerable transfer of policy knowledge within this region. However, our fndings reveal that the Nordic policy cooperation and exchange of policy knowledge in the feld of education is far more complex and infuenced by multitude of aspects and dynamics, in addition to cultural, legal, and political similarities.

In this chapter, we have investigated the use of regional Nordic policy knowledge in education reforms in three of the Nordic countries: Finland, Iceland, and Norway. Our starting point was the defnition of evidence by Paul Cairney (2016, 3) as "an argument backed by information." By employing both bibliometric network analysis and thematic analysis of expert interviews, we have examined how policymakers and experts in these three countries locate themselves in a larger political reference space when they develop ideas and collect evidence to justify school reforms in their respective countries. We have asked whether government-appointed expert panels and policymakers within the state administration consider knowledge and information from other Nordic countries relevant. Moreover, we examined whether they demonstrate a specifc interest in authorizing such sources by referencing them in public policy, and investigated the factors guiding these decisions.

Our bibliometric network analysis revealed a notable absence of Nordic references in the source documents we included in our study. Regional Nordic references amounted to only 1–7% of all the references, and we found very few references that were cited by more than one country. Tis fnding indicates that publications from other Nordic countries are not considered relevant in terms of "authorized evidence" that is explicitly included in reference lists or footnotes in white and green papers. Nevertheless, our interview data shows that Nordic cooperation in the feld of education policy is vivid, frequent, and ample. Terefore, good reasons remain to argue that the expert panels and policymakers involved in these reforms are very well informed about emerging trends and ongoing reforms in other countries.

In the interviews, we asked our informants for reasons why Nordic knowledge does not appear as the most prominent reference source in policy processes; in other words, we wondered why it is not referenced and used as evidence in the policy documents. We discovered four main explanatory narratives for this phenomenon. First, Nordic cooperation is not thoroughly documented nor does it produce data that lends itself easily to reference. Second, Nordic cooperation is such an implicit part of the policy process that it does not need to be mentioned specifcally. Te third explanatory narrative is related to the legitimation function and hierarchization of policy evidence. Even if Nordic cooperation would produce data that was easy to reference, international evidence still had a higher status as a legitimation device than regional evidence. Finally, the fourth explanatory narrative ofered by our informants was that national orientation and political dynamics still play the greatest role in curriculum reforms. Despite increasing transnational infuences, the curriculum is still characterized as a particular domain in education policymaking and politics, considered predominantly national by character. Although school reforms refect international infuence, the problems addressed turn into national concerns when they are handled. In particular, the policymakers writing the white papers argue that their job is to bridge the gap between political aims and projects on the one hand and broad expertise on the other hand, which implies complex processes that involve a range of stakeholders.

In addition to investigating why Nordic evidence is or is not referenced in the policy documents, our aim in this chapter was to examine how policymakers and experts in Finland, Iceland, and Norway locate themselves in a larger political reference space. We have developed a notion of a Nordic education space in which national experts and policymakers "make meaning and attempt coherence in networked forms" (Grek et al., 2020, p. 4). Te Nordic policy space is characterized by fuidity. We have shown that there are several forms and networks of Nordic cooperation in the feld of education policy, and national experts often participate in multitudes of networks and forms of cooperation. Te reference space of the Nordic experts stretches out from the national to the transnational through their interactions, connections, and networks. Te Nordic education policy space is constructed and maintained in diferent settings of cooperation and communication, with some taking place in the regional and some in the international domain. When the actors meet in international settings, consensus-making is of core importance between representatives from the Nordic countries. Tis difers from regional meetings where they rather explore national solutions to common problems. Tus, we demonstrate that there are several forms and networks of Nordic cooperation that have various functions in terms of evidence-based policymaking depending on the context in which collaboration takes place.

To conclude, the regional Nordic policy space is, in fact, in constant movement and a state of becoming. As a result, both comparative research and the actors involved in this continuously evolving space are challenged to explain its role and function and to pinpoint its changing forms in a comprehensive manner. We have aimed to contribute to this discussion, but we acknowledge that, in line with the nature of the constant becoming of this space, we may have posed more questions than we have answered. Instead of ofering fnite conclusions, we have created new openings for further discussion. Tese openings are much like the policy space we have examined—in a constant state of becoming and hence infnite in nature.

## **Note**

1. NOUs are Norwegian Ofcial Reports.

## **References**


Espeland, W. (2015). Narrating numbers. In R.Rottenburg, S.E. Merry, S.J. Park, & J. Mugler (Eds.), *Te world of indicators: Te making of governmental knowledge through quantifcation* (pp. 56–75). Cambridge University Press.

Eyal, G. (2019). *Crisis of expertise*. Polity Press.


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **13**

## **On Evidence, Impact, and Layers in Education Policy Processes**

**Kerstin Martens**

Te editors invited Antoni Verger and me to refect on this book's contributions by identifying its connections with existing research, pointing out novel ideas, and proposing topics that deserve further scrutiny. I was greatly honored to accept this invitation, as it enabled me to participate in and contribute to this endeavor on evidence and expertise in the Nordic countries. I, therefore, see the purpose of my commentary not as summarizing the fndings or providing feedback on the chapters contained within but, rather, as an exercise in examining cross-cutting themes and future avenues for research resulting from this book. Hence, my comments refect on the empirics of the chapters in conjunction with the guiding theoretical and methodological chapters. A commentary also leaves room for one to argue more freely, to point out the societal contexts of an academic work, and to include "unusual" references, as opposed to being limited to referencing only academic work. Terefore,

© Te Author(s) 2022 **383** B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_13

K. Martens (\*)

University of Bremen, Bremen, Germany e-mail: martensk@uni-bremen.de

some parts of this chapter may seem to stray slightly of the typical academic path but do so in the interest of provoking future discussion.

Overall, from my refections, three substantial themes emerged that I address in this chapter: (1) the conjunction between evidence and politics, (2) the conjunction between referencing and impact, and (3) the conjunction between empirics and theory in global education research. Tese three linkages seem to be part of a broader connection referenced in the book, namely, the connection between science and politics. Science has become a point of reference for politicians due to increased awareness about the challenges our world faces with respect to climate change; moreover, science has become more visible and more clearly defned in political processes in recent years. Tis was particularly apparent during the 2020 Democratic National Convention, at which Joe Biden and Kamala Harris were formally nominated as the Democratic presidential and vice-presidential candidates, respectively. In their remarks at the convention, many speakers (including, among others, Michelle Obama) frequently used the term "science" to make political statements, such as noting that Joe Biden would listen to "science," or would follow "science," or would consider "scientifc reasoning," and so forth. Obviously, these comments were meant to sharply contrast with former president Donald Trump's "alternative facts."

However, we must also remember that science is not static; it is subject to change when new fndings disprove previous knowledge or question conventional knowledge, or when new problems occur that old solutions cannot resolve.1 Te coronavirus pandemic is teaching us to question ourselves on an almost daily basis. Is the Swedish way of believing in herd immunity the better long-term strategy, despite higher casualties in the short run? Was the worldwide lockdown sufcient for keeping the number of infections under control? Are the economic and psychological consequences of that lockdown more disastrous for democracies than we anticipated? Are locally defned solutions the better way to balance restrictions and daily work life (masks for school children, limitations on how many people are allowed to meet, and so on), despite the wide-ranging mobility of people commuting long distances and across borders?

Perhaps we also need to ask ourselves what is special about the social sciences. Without going into the philosophy of science, social science knowledge often seems rather cumulative regarding theoretical approaches, in that theories that have proven (partly) unhelpful in a particular context are not easily discarded. We have not discovered many "black swans," as Popper would claim, but as social scientists we tend to acknowledge that diferent theories may be equally valid, though in different contexts and at diferent times. In addition, the social sciences have a particular "problem": We deal intensively with humans and human behavior. Humans can be highly complex, changing their minds, opinions, attitudes, and actions at any given time. Tey also react to their environment in a variety of ways. To quote a classic political science journal title by Steven Bernstein and colleagues (2000): *God Gave Physics the Easy Problems*.

As much as we believe in science and research, scientifcation and academization have their limits in the social sciences and in education studies, respectively. As my colleague Laura Engel (2016) from George Washington University once argued during a workshop in Hanover, sabermetrics—understood as a method used in baseball for collecting and summarizing all relevant (but also seemingly irrelevant) data, as explained in the book *Moneyball* by Michael Lewis (2003)—cannot easily be applied to education. If it were that simple, all we would have to do is gather sufcient appropriate data and let statisticians run a couple of regressions in order to describe the perfect education system. However, that is not the case, so we need to fnd other and better ways to research education policy. Tis volume can guide future work on global education governance due to its empirical fndings, its theoretical approach, and its methodological fnesse.

## **Conjunction Between Evidence and Politics**

Tis book provides a valuable analytic snapshot of knowledge production processes, of the scientifcation of education (policy), and of the path through which academic knowledge fows into politics. Referencing and analyzing the networks of actors behind these activities provide a deep view into legitimization processes in current politics. Te Nordic countries are characterized as a comparatively homogenous group of states that, despite some diferences, take a similar approach to political decision-making and have a shared history of evidence-based policymaking. Terefore, these countries make ideal examples for this investigation for clarifying and illuminating signifcant aspects of referencing.

From a political science perspective, the dataset of documents on which many of the analyses in this book are based not only produces valuable results, as demonstrated in the diferent empirical chapters, but also provides a fascinating sample that can be used for additional followup analyses. All recent school reform acts that are examined in this volume were introduced in each of the Nordic countries at roughly, and sometimes exactly, the same time: 2014 seems to be the watershed year in education politics in this region (with Norway lagging two years behind). Tus, from a methodological perspective, the Nordic context comes close to what comparativists (in political science) would call an "experimental setting," which includes the factors that best support measuring the impact of the reforms in each country, as some variables are more or less controlled for, including time, institutional background, and regional difusion. Such a setting is not only highly valued methodologically but also rarely possible to establish or to fnd in comparative analyses.

Te existing dataset has provided scholars and researchers with interesting fndings as presented in the diferent chapters of this volume. However, the dataset can also be used to dig deeper into the processes of knowledge production and legitimization in political spheres and identify cross-cutting themes. Te dataset also provides a valuable example of how education research can be systematized more rigorously and under one common research umbrella that guides multiple analyses. As a next step, maintenance of this dataset of documents will preferably be transformed into a standing project involving the continuous collection of documents on school reforms in the respective countries. Such a project would enable future generations of scholars to conduct numerous comparative studies or, at some point, studies about individual countries over a longer period of time.

A comparable dataset with which comparative analyses could be envisioned is, for example, the collection by Marc Helbig and Rita Nikolai (2015a) on the seemingly "incomparable" German Bundesländer (states) with regard to their school politics [*Die Unvergleichbaren—Der Wandel*  *der Schulsysteme in den deutschen Bundesländern seit 1949*]. In that work, the authors rely on their analysis of around 8000 legal texts from the last six decades. Te collection of documents is made publicly available for future work.2 Tus, this project's approach could serve as a blueprint for multiple initiatives: (1) to make the Finnut project (Chap. 1 in this volume) public, (2) to simplify the process of fnding a possible partner for comparative work between the Nordic countries and the German Bundesländer, and (3) to develop similar reform datasets for other countries or regions of the world.

However, considering only the dataset on the Nordic states that already exists, a vast array of possible new research areas comes to mind. For example, the existing databases can be examined for topics and themes that regularly appear in the documents. To identify those topics, the data can be analyzed quantitatively, for example, using topic modeling, a statistical model of machine learning for discovering topics that occur multiple times in a collection of documents. In a recent contribution, Helen Seitzer et al. (2021) applied topic modeling to examine the OECD's PISA project in relation to its other activities and publications on education to discover if and how things change over time. Surprisingly, they fnd that although the OECD is predominantly renowned for the PISA, the themes it deals with are much broader than this, with PISA accounting for only a small part of the OECD's overall output. In fact, the organization addresses higher education policy more intensely than secondary school data, even though the OECD and PISA have become almost synonymous for the international organization's work in education. Tis raises the question as to whether PISA is predominant or perhaps even overrated.

As regards qualitative methodological means, I can also imagine that a rather detailed hermeneutic work on the existing dataset, perhaps even in cooperation with linguists, would be worthwhile. Single passages, where the context in which something is referenced (or not referenced) is evaluated, could be hermeneutically examined. Something else worth considering is the application of a scoring system for statements and references, where each could be assessed in its respective context. Furthermore, digging deeper into the context of these statements could provide insight into connections between themes and produce interesting questions (e.g., does think tank referencing occur more often in the context of equity and human capital in relation to the OECD?). Further network analyses could also be applied to link topics to references.

## **Conjunction Between Referencing and Impact**

With this document dataset, the editors and contributors of this text apply a highly sophisticated analysis to the network of referencing in Nordic education policy processes. Network analysis is especially suited for fgurative visualizations of relations, which are often more impressive and expressive than those produced by many other analyses. Network analysis is also *en vogue* and innovative in education studies from a methodological perspective, despite having been established in the felds of sociology and political science for some time. Particularly when the network of actors in global education governance is to be analyzed, a network approach can make connections or nodes and major actors visible. Te fndings would make interesting contributions to SUNBELT, the signature conference of the International Network for Social Network Analysis (INSNA).

However, although network analysis can do more than visualize relations between references and point to signifcant nodes within the network, centered nodes do not necessarily have the greatest impact. Referencing and visualizing relations between references cannot answer some of the questions that arise from this research endeavor, such as in the context of governmental documents, particularly, the internal importance of green and white books, or how much party ideology comes into play and how this is pictured or becomes pictured in the references. Follow-up questions like those that follow may also emanate from this research: Which document(s) were the most infuential? Which reference(s) have had the most impact? And were these the most cited references?

Possibly even more important for future studies are answers to questions about what we do not see in the documents and, accordingly, what we do not see in the network analyses: What is the intention behind some references being selected and others being (perhaps deliberately) neglected? Considering the somewhat paradigmatic shift brought on by PISA toward perceiving educational outcomes as able to be measured against globally standardized indicators, the references that were omitted and why they were omitted seem to be of even more interest. Where have the boundaries of knowledge been set, and what references have been hidden? With this in mind, a variety of future studies, both across and within countries, is conceivable.

Nevertheless, one must be aware of the limits of our research endeavors. Indeed, measuring "impact" is probably one of the most difcult tasks we encounter in our work. To specifcally pinpoint what factors had an impact as well as when, where, and why those impacts were experienced can usually only be approximated by detailed research. Methodologically, and not without advanced refections on one's own biases, approaches such as process tracing (probably in combination with interviews) may prove efective for investigating the steps through which as well as the substantial or decisive intention with which these government documents were created. Research questions may include inquiries like the following: What were important watershed decisions in the process? How were such decisions made? What were key moments in the process?

Another follow-up step to consider is to examine how these reforms were implemented in practice on the ground, for example, in education districts, in single schools, and so on, or following the thinking of Gita Steiner-Khamsi: How were these reforms translated into the local context? Te written process for implementing the reform does not necessarily describe the same measures as they were implemented in real life. A good example of how diferently an international impetus can be translated into national and local contexts is "inclusive education." While the United Nations (UN) Convention on the Rights of Persons with Disabilities counts as the most quickly ratifed UN convention, the implementation of Article 24 on inclusive education not only varies from country to country but—as evident in the example of Germany—also from state to state and school to school. Hartong and Nikolai (2017) described this as the "local globalness" of policy transfer.

However, to examine international infuences, especially the PISA study, the setting applied in this volume provides a sufcient time frame. Around 15 years passed since the frst PISA study was published and the reform acts of 2014 took place. Tus, considering that political processes take time, especially when international infuences need to be "translated" via national legislation by way of regional administrative bodies before even potentially reaching the classroom, the efects of the international impetus may have reached the local realm. All in all, we can assume that a PISA-afected generation of students has left school with certifcates by now.

## **Conjunction Between Empirics, Theory, and Methods**

Although this book is considered an edited volume, it better resembles a monograph in its set-up. As the result of a longer-lasting research project (the Finnut Project, Chap. 1), one common theoretical frame is applied, including the use of common terminology within the individual chapters and an encompassing research design in which each chapter has its distinct place. Terefore, the book is not only a rich collection of contributions in which each chapter can stand for itself, but the book itself is more than the sum of its parts.

Tis highly sophisticated approach becomes particularly apparent in the systematic and—what I would call—*layered* approach. While Part 1 of the book provides an overview of the study, key concepts, and the methodology applied in the book, Parts 2 and 3 address the empirical fndings for diferent country and cross-country comparisons. In doing so, diferent policy layers are explored—the international or transnational layer (particularly the OECD in this context as the policy source), the national layer (where governments as executors of policies are involved), and the layer of societal actors of diferent kinds (such as trade unions, think tanks, and the like)—all of which are part of the discursive policy practice.

Tis kind of approach within a defned region of the world, such as the Nordic countries, enables us to detect not only patterns and outliers across countries (horizontal approach) but also confgurations and alignments across layers (vertical approach). In my view, this is a highly systematic approach that can serve as a methodological tool or as an element of a research design in many other contexts and for other comparisons. Such an approach could also be extended to analyze the activities of additional global actors, for example, international organizations, such as the World Bank or the UN Educational, Scientifc and Cultural Organization (UNESCO), or more regional organizations active in education, such as the Southeast Asian Ministers of Education Organization (SEAMEO) for the Southeast Asian region or the Arab League Educational, Cultural and Scientifc Organization (ALECSO) for the Arab and Middle East region (see Niemann & Martens, 2021).

Moreover, an additional layer should be considered and can be added to future analyses, namely, individuals. Within the chapters, individuals and their particular impact are sporadically mentioned (particularly in Chap. 10, and even more so in Chap. 11); sometimes a commission is named after them, and sometimes their infuence is hinted at within a particular context. In general, I believe that we underestimate how infuential some individuals may be when we discuss the "Ministry of Education," the "lobby group," or the "education commission." We may only know anecdotally that a particular person shaped the context of a policy document or the direction of a policy. Various studies in the context of the OECD and education policy, however, have shown just how infuential certain individuals have been at key moments (e.g., in Henry et al. 2001; Martens, 2007). As an example, one can imagine how PISA would look or be perceived without its front man, Andreas Schleicher. Tus, more systematic research on the infuential role of individuals in policy processes should be conducted.

In addition to this supplementary layer to be applied to future scholarly work in the feld of education policy, another source of inspiration may be found in the deeper systematic linking of these layers with one another and with theory-driven approaches. What the editors produced with this volume is what comparativists, especially from political science, call a Most Similar Systems Design (MSSD). Comparing a group of similar countries, here the Nordic countries, in a small-N analysis allows for factors that are historically or institutionally similar in the defned sample to be excluded from the analysis in order to concentrate on diferences between cases.

Concentrating on such diferences between otherwise similar systems also allows for contextualizing them in existing approaches that have proven suitable in other studies. Moreover, connecting a MSSD systematically with theoretical approaches that emphasize diferences can allow us to trace mechanisms and detect causations. Tey may also deliver theoretically guided hypotheses, which incorporate the diferent layers and their impact on education policy processes. Depending on the particular research question, a variety of approaches can provide insightful interlinkages and explanations for phenomena of interest, such as the following: approaches to delta-convergence (for linking the international and the national level); welfare state theories and education (for linking the type of system to policy outputs and outcomes); and interest group involvement, such as in the form of corporations or social movements (for linking the national level to internal actors).

## **Conclusion**

Tis volume provides, without question, a particularly rich collection of evidence and expertise on Nordic education policy processes. It includes representation by a pronounced community of scholars from the region who contributed their knowledge to this common endeavor. With this volume, the editors and contributors deliver a systematic and well-written work of research that stands for itself but will also inspire future studies. My commentary focused on three conjunctions that I detected within and across the chapters: the conjunction between evidence and politics, between referencing and impact, and between theory and empirics. By summarizing some of the approaches and fndings of the work in this way, my aim was also to identify and inspire possible projects for the future.

## **Notes**


## **References**


Lewis, M. (2003). *Moneyball: Te art of winning an unfair game*. Norton.


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **14**

## **Evidence-Based Policy Making and Educational Reform in Nordic Europe: Key Contributions of the POLNET Study**

**Antoni Verger**

Evidence-based policy making (EBPM) is currently considered the most appropriate approach to public policy formulation. EBPM implies that both policy decisions and policy changes are increasingly grounded on scientifc reasoning and research fndings. Te most enthusiastic advocates of the evidence shift in policy applaud the fact that policy decisions are increasingly informed by science in contrast to anecdotal information, tacit knowledge, public sentiment, and other forms of support. EBPM is also considered suitable for addressing ideological biases in decision making and for generating broader political consensus on the strategies that should guide policy formulation in diferent domains. In the 1990s, both policy and academic circles in the feld of education began to embrace EBPM to address a legitimacy crisis faced by educational research (Head, 2008). To mitigate this crisis, educational research borrowed rationales and methods from research in medicine and other experimental sciences, with the expectation of promoting a more

A. Verger (\*)

Universitat Autònoma de Barcelona, Cerdanyola, Spain

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_14

cumulative and transferable type of knowledge. Scholars and universities around the world have welcomed the EBPM transition because it makes academic research more socially valuable and visible and, importantly, because it generates new venues for research funding.

To a great extent, evidence-based policy creates opportunities for high quality, knowledgeable policy debates—that is to say, policy debates informed by causal beliefs, sound policy evaluations, and theoretically informed case studies. Politically speaking, EBPM is expected to be embraced in an international environment characterized by the resurgence of populist movements for which mistrust in science pays of politically and in an environment in which policy decisions on sensitive issues, such as global pandemics or climate change, can put population and the future of the planet at risk when those decisions are based on "alternative facts" (see Marten's commentary chapter in this volume).

Nevertheless, despite the many ways that paying more attention to scientifc evidence benefts policy making, EBPM still must be approached with caution, as it has its own limitations and risks for both policy research and practice. Furthermore, some assumptions behind enthusiastic versions of EBPM may not work as expected in real educational policy settings. EBPM discourses assume that, with the emergence of this policy approach, political values, ideologies, and normative beliefs are being sidelined from policy processes in favor of scientifc knowledge and causal beliefs. Tey also assume that EBPM can make policy processes more open to external participation, especially to academic voices and sources. In this respect, EBPM implies that the locus of decision-making shifts from political authorities to networks of experts. Additionally, EBPM embraces a rationalistic ontology to policy transfer according to which policy learning is the main mechanism behind the traveling and selection of "best practices" in diferent policy contexts. Tis implies, to a degree, acknowledging that policy learning has gained centrality as a mechanism of policy transfer against other well-known transfer mechanisms (such as competition, coercion, or emulation).

Te book you have in your hands problematizes these and other assumptions about EBPM through comparative research conducted in the Nordic education policy space. Te book is part of a fve-year research program called POLNET (*Policy Knowledge and Lesson Drawing in Nordic*  *School Reform in an Era of International Comparison*) investigating the implementation of the EBPM approach in contemporary educational reforms in Nordic European countries. Te POLNET study has brought together an international team of outstanding researchers from Nordic European universities and the Teachers College (University of Columbia), under the leadership of Kirsten Sivesind, Berit Karseth, and Gita Steiner-Khamsi. One of the main originalities of this trans-Atlantic research efort is the application of a bibliometric methodology to conduct comparative analyses of educational country reforms. Tat the chapters are grounded on this same methodology contributes to making the volume a cohesive piece of research and strengthens the comparability of the country cases. In the following sections, I refect on the most important contributions of the study and their implications for future research.

## **Science and Politics in Educational Reform: Does Science Rule?**

One of the most persistent critiques of EBPM is that it runs the risk of expertizing policy processes and of generating new forms of social closure within policy networks. According to Biesta (2007, p. 1), through the technifcation of certain policy debates, EBPM "restricts the opportunities for participation in educational decision making." Te POLNET study shows that this would not be the case in most Nordic countries, where EBPM challenges a corporatist form of governance and promotes the transition toward forms of network governance in which new actors such as experts, interest groups, and civil society organizations—are considered in policy formulation processes. In fact, since Nordic governments have embraced EBPM in educational reform, parliamentary debates on education policy have intensifed. Tus, the policy process is becoming more transparent, open, and political, while being confgured through the involvement of a wider nebulous of actors, spaces, and devices for participation (including conferences, commissions, social media interactions, position papers, public blogs, and so on).

Pro-EBPM discourses celebrate the increasing infuence of science over politics in policy making and the scientization of policy debates that were considered too political. However, the POLNET study demonstrates that, in policy processes, the relationship between science and politics is not necessarily a zero-sum game. Te educational reforms analyzed in the book, in fact, show that the bigger presence of science has, indeed, evolved in parallel to more political participation and to the politicization of some stages of the reform processes.

Certainly, in real-life situations the separation between politics and science is precarious and far from linear. Chapter 10 in this text develops this idea eloquently; it graphically depicts the recursive and constant iteration between the political and scientifc domains in education policy making (see Fig. 10.8). In the words of its authors, "informationgathering and consensus-building occur in practice at each and every step of the policy making process, blurring the line between science and politics."

Tese fndings confrm that scientifc evidence is not ontologically distinguishable from political ideologies, normative beliefs, and bureaucratic control in policy-making processes. As illustrated by the POLNET study, the institutionalized forms of EBPM in diferent Nordic countries make clear that politics are at play in decisions that alter the infuence of science over policy at many levels. To start with, by designing, regulating, and funding the EBPM architecture, political actors can dynamically condition the policy outcomes of evidence-based deliberation processes. Decisions regarding the confguration and funding of the agencies in charge of drafting green papers, literature reviews, and position papers are key to understanding the dialectal relationship between science and politics in policy processes, as are decisions about who can integrate the agencies or advisory commissions (including the identifcation of criteria regarding disciplinary background or research experience). As principalagent models would predict, these decisions have important implications for the outcomes of EBPM schemes, as well as for the actual role of independent research in educational reform.

However, politics also condition and, to an extent, restrict scientifc infuence over policy in more indirect ways. EBPM assumes that policy makers, especially in periods of uncertainty and crisis, are receptive to scientifc evidence on best practices. Tus, policy makers increasingly welcome the role of knowledge brokers and experts when selecting new policy instruments or calibrating those that already exist. Yet, they are also aware of the type of knowledge that is more useful to them in both political and policy terms and have at least notions of who can provide them with this kind of knowledge more efectively. Specifcally, policy makers tend to resort to research sources where they can obtain straightforward answers to frequently complex policy problems—sources that ft within what Roger Dale (1994) defned as a "problem-solving" type of research. Policy makers usually rely on technical knowledge to address "what works" questions and are more inclined to base their assessments on quantitative sources rather than on qualitative research and data, with the latter often seen as more interpretative and biased. In doing so, they reproduce ideational frameworks of what type of knowledge counts as policy-actionable evidence. Discursive selectivity, hence, becomes a subtle mechanism for understanding how politics shapes EBPM processes and privileges certain research approaches over others.

Not surprisingly, many of the references cited in the green papers published in the Nordic countries analyzed ft within a mainstream "school efectiveness research" approach—an approach that pays more attention to measure the school efects on student learning than to multiple causal and structural explanations of educational outcomes (Parra, 2018). International scholars with a pragmatic approach to educational research who prescribe clear policy guidelines on how to organize efective schools and classrooms, such as John Hatti and Michael Fullan, are more often cited in the policy papers reviewed in the POLNET study than scholars with a more critical or theoretical understanding of education policy matters.

Nonetheless, a fnding as unexpected as it is important from the POLNET study is that references to international and national educational researchers are relatively scarce in the policy documents used to back educational reform. Even though the green and white papers published in the Nordic countries analyzed cite numerous pieces of evidence, academic evidence in the form of publications in peer-reviewed journals is rather marginal. In contrast, other types of domestic publications (gray literature, ad hoc literature reviews, blog posts, and so on) are much more present. Te response to the question "whose knowledge is used in educational reform in Nordic countries?" is not straightforward; indeed, it seems easier to respond to the question of whose knowledge is *not used*. Educational research produced in national universities is not strongly represented in most of the country cases.

## **Externalization and the Role of International Organizations**

In contrast to the marginal presence of academic publications produced by national universities, reports and papers published by the Organization for Economic Co-operation and Development (OECD) are widely cited in the policy documents produced by state agencies and research institutes in all Nordic countries. Te OECD is not only the most cited international source in the corpus of policy reports gathered: the bibliometric analysis conducted by the POLNET team shows that OECD reports enjoy a high level of in-degree centrality in the networks of publications produced in relation to the diferent country contexts. Tis confrms the OECD's authority in the education policy realm, and in Nordic European education in particular (Grek, 2017; Ydesen, 2019). Still, as acknowledged in Chap. 11, the bibliometric analysis helps to test the reputation of the OECD in the Nordic region but does not capture, on its own, the level of penetration of the OECD in the Nordic education policy space neither the nature of this international organization's infuence in national policies.

OECD reports can be cited in green or white papers for multiple reasons, not all of which are related to the ideational infuence of this international organization—or to how much national policy actors have learned from or within OECD initiatives. In the context of educational reforms, governments may cite OECD sources to legitimize their policy options, and impregnate these options with international status. Moreover, as the authors of Chap. 6 wittily observe, referring to external sources such as the OECD in a country like Iceland, where policy networks are rather compact, is simply a way to "avoid being accused of nepotism." Countries can also cite OECD papers to technify political debates considered too controversial in the domestic policy arena (Rizvi & Lingard, 2009; Browes & Verger, 2020). In fact, as the authors of Chap. 12 observe, the number of references to the OECD increases in those countries in which education reforms have been more contentious. To a degree, externalization intensifes when more sensitive or controversial reforms are being debated in national policy spaces. All of this corroborates what Martin Marcussen observed some time ago: the fact that countries increasingly resort to and cite OECD knowledge products says more about the increasing international legitimacy of this international organization than about its policy infuence (Marcussen, 2004).

Te OECD has a longstanding legitimacy in Nordic countries. Tis international organization has been present in education policy deliberations in the region since the 1990s; it has conducted education policy reviews in several countries repeatedly; and has involved them in diferent training and research initiatives. However, knowledge dissemination and policy evaluations are not the only, or the most important, ways in which the OECD has afected national policy debate in Nordic Europe. As several chapters in this volume refect, the OECD has infuenced substantive policy change through the "scandalization" that came with the Programme for International Student Assessment (PISA) (cf. Steiner-Khamsi & Waldow, 2018). Specifcally, because of the PISA efect, many Nordic countries have adopted performance-based accountability, standardized testing instruments, and curricular standards that were not in place before (see, for instance, Camphuijsen et al., 2020). International performance data has had a signifcant afective type of impact as a catalyst of educational reform (see Sellar & Lingard, 2018). In other words, the adoption of the reforms as described resulted more from governments' political and, to some extent, "emotional" reactions to poor (or, rather, lower than expected) international performance on PISA than to scientifc-based deliberations about policy.

Tese policy dynamics do not neglect that EBPM may play a role in decisions on how to calibrate "governance by numbers" instruments. What they highlight is that the substantive decision on whether to adopt the "governance by numbers" approach and related policy instruments in Nordic countries is more related to the between-countries competition and to the anxieties over educational performance that international large-scale assessments such as PISA have triggered.

## **Is There a Nordic Model of EBPM?**

Te POLNET study also examines education policy transfer within the Nordic region. Te study fndings do not support that policy transfer dynamics within the region have shaped country reforms. Specifcally, the number of references to other countries' sources in educational reform processes is quite low in the documents analyzed: between 1%–7% of all references (see Chap. 12 in this volume). Even references to Finland, a reference educational system at the global level, are very low in the Nordic region. Nevertheless, again, these results need to be considered with caution because policy transfer mechanisms may operate via more subtle forms and informal relations than through explicit mention in policy documents.

At the same time, this book has much to say about the Nordic model of education as such, as well as about the predominant model of EBPM that is being enacted within the region. For external observers, the image of educational systems in Nordic countries includes policy principles such as equity, comprehensiveness, decentralization, publicness, and teachers' professional autonomy. However, as several chapters in this book highlight, the Nordic education model is no longer as cohesive. Growing marketization, school choice demands, and performance-based accountability are transforming the Nordic educational model (with Sweden as the regional outlier in advancing these trends). Te reforms analyzed in this book provide good examples of the tensions that co-exist within the Nordic model. Tese reforms tense educational systems by including policy measures that, on one hand, strengthen the knowledge base of teachers but that, on the other hand, challenge teachers as the main source of educational expertise. Tey also include measures that promote equity and inclusion and, at the same time, strengthen competitive and performative attitudes within schools. Overall, contemporary reforms in Nordic countries have stressed educational systems by reinforcing a post-bureaucratic governance approach that is re-scaling power in education upwards (from teachers to local and national authorities).

Nordic countries are also known internationally for their wellfunctioning democracies and respect for political rights. Despite the limitations of EBPM highlighted in the book, the education policy-making processes described refect and attempt to advance more transparent educational reforms that are open to external voices. As an educational policy researcher from a diferent world region with a more immature democracy, I fnd remarkable how transparent and, to some extent, Cartesian policy formulation processes in the Nordic region seem to be, and the accessibility of the sources of knowledge on which reforms are to be grounded. Citations may be selective and used for legitimation purposes. State agencies and research institutes, in their role as knowledge brokers, can act as gatekeepers and exclude certain voices. But explicit efort to back policy change with evidence is evident. Tus, the opportunity exists to tell policy makers whether they have misinterpreted research fndings or that they have obviated important publications on specifc topics. In contrast, in contexts where policy making seems to more closely follow the garbage-can model than the EBPM model, the opportunities to mobilize knowledge for policy change purposes are much more restricted.

Te POLNET study fnds some regional citation patterns in the reports produced by state agencies and research institutes in preparation for educational reform. Te frst is the above-mentioned marginal presence of academic research produced by national universities; the second is the predominant presence of national sources (in contrast to regional sources); and the third is the important presence of OECD reports and other knowledge products from this international organization. Te contributors to the book also identifed diferences regarding the EBPM approach in the Nordic countries. Bibliometric analysis is a useful tool to make sense of the particularities of the EBPM process that each country has followed and identify variation regarding the number of written sources used in policy papers, the concrete percentage of academic and international sources cited, and the level of externalization on international organizations.

Te case of Denmark is interesting in this regard. Te POLNET study shows that education reform in this country has been backed by a limited number of written sources but has been heavily infuenced by the OECD (see Chap. 4 in this volume). However, what makes this case somewhat exceptional in the Nordic context is that not only has EBPM contributed to the internationalization of the policy process, but the EBPM infrastructure itself has become highly infuenced by international sources, specifcally, through the direct involvement of UK players, such as the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI) of the University College London-Institute of Education.

## **To Conclude: Future Research Directions**

Advocates of EBPM assume that this policy approach favors more open and participatory policy processes in which deliberation based on academic sources intensifes. Tey also assume policy learning is the main mechanism of policy transfer. In this chapter, I explained how the POLNET study challenges these and other assumptions through original comparative research conducted in the Nordic education policy space. Te study shows that the enactment of EBPM has not fattened the terrain of policy making, nor is it a synonym for the increasing infuence of science over politics. In the Nordic region, EBPM translates into a complex policy process in which science and politics interact at diferent levels but in such a way that the political domain retains bureaucratic control over the policy process and its main outcomes. Te study also demonstrates that, more than policy learning, dynamics of legitimation and competition are key to understanding some of the most substantive policy changes Nordic countries have undergone in the educational realm in the last two decades.

Te POLNET study addresses numerous research questions, but as happens with good research, it also generates new questions and ideas about future lines of inquiry. In numerous chapters of the book, the authors show that scientifc knowledge (in particular, the education research produced in national universities) is not a main source of political authority in national education reforms. Overall, the research products cited in policy documents seem to play a bigger role in legitimizing policy decisions than in shaping them. However, under what conditions would policy makers be more genuinely receptive to scientifc evidence? Can changes in the public regulation of state agencies and research institutes contribute to promote more pluralistic approaches to evidence use? Nonetheless, to be fair, the challenges of EBPM implementation do not only originate in the feld of politics. Indeed, the very knowledge base of educational policy research challenges the comprehensive use of research for policy making. Te many themes on educational policy in which research evidence is inconclusive (such as the costs and benefts of pedagogic innovation, school choice, performance-based accountability, public-private partnerships, etc.) is conducive to both knowledge selection biases and the instrumentalization of research for political purposes. To test this statement, future research could analyze whether EBPM is more genuine and rigorous in relation to those reform domains with more cumulative and conclusive research results.

Another question to unpack EBPM processes is: at what stage of the policy process can research evidence become more infuential? As we have seen, in mainstream EBPM frameworks, the soundest and most welcome scientifc evidence comes from "problem-solving" research that mainly informs policy formulation at the policy design stage; in particular, this is research that focuses on school-level efects over learning and packages and sells policy solutions in a rather prescriptive way. However, other types of research can also play a role in policy processes at diferent stages, although this role tends to be less acknowledged. Numerous research initiatives in education are better equipped to problematize existing situations in the educational realm than to prescribe straight-forward solutions. Tis, for instance, is the case of research that focuses on identifying the problems that policy makers need to address, or on constructing policy priorities and preferences that are not central in public agendas yet. Future studies on knowledge uses in policy processes could, thus, pay more attention to the role of research evidence, not only at the policy formulation stage, but also in terms of agenda-setting and problematization.

Te fact that the boundaries between the sites of knowledge production and policy making are being blurred, as the POLNET study reveals, means that the corporatist state may not have been totally left behind in Nordic countries. To some extent, more than entirely over, the corporative tradition has mutated, and the interest groups to which the state resorts to in policy processes have been transformed or, in some cases, replaced. Something I fnd intriguing in this regard is the role of teachers' unions and the political representation of teachers more broadly speaking. Have teachers' unions and other teachers' representatives lost political centrality in the current scenario? Have they been sidelined by the growing centrality of research institutes, state agencies, or new interest groups? More research on the changing role of teachers' unions within EBPM frameworks, and on how and whether EBPM has transformed the functions, power, and nature of collective action within unions, would be welcome as well.

Overall, publications, citations, and references constitute a good entry point for the study of the knowledge base of educational reform. Nonetheless, as the POLNET study makes clear, publications and references are only the tip of the iceberg—the most visible and empirically tangible resource—of more profound and determining political and knowledge mobilization dynamics over policy processes. Te interest in bibliometric analysis is obvious; however, combining it with other methods can boost the potential of this methodology. Te most informative chapters in the book are, in fact, those that combine bibliometric data with interview data or that situate bibliometric fndings within broader research frameworks. Future research can combine diferent forms of social network analysis (including those drawing on bibliometric analysis) with a more qualitative understanding of the EBPM phenomenon. In-depth interviews or observational methods along the lines of political ethnography approaches (see Papanastasiou, 2020) can provide insights into how policy makers make sense of scientifc evidence, the type of evidence they fnd more approachable, and the forms of knowledge to which they resort to construct notions of best practice in public policy.

**Acknowledgments** Tank you to Clara Fontdevila for her comments on this chapter.

## **References**


Steiner-Khamsi, G., & Waldow, F. (2018). PISA for scandalisation, PISA for projection: Te use of international large-scale assessments in education policy making–an introduction. *Globalisation, Societies and Education, 16*(5), 557–565.

Ydesen, C. (2019). *Te OECD's historical rise in education*. Palgrave Macmillan.

**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **15**

## **Conclusion: Toward a Renewed Understanding of Evidence-Based Policy in Education**

**Berit Karseth and Kirsten Sivesind**

## **Evidence-Based Policy Advice and Decisions in the Nordic Region**

In reviewing the research literature on evidence-based policy, very little was found about reference use in policy documentation (see Chaps. 1 and 2). Terefore, it is interesting to conduct research on how various knowledge sources are mobilized in reference networks through policymaking processes. Such sources, as mentioned in the introductory chapter, can be widely defned, including information, ideas, and arguments; well-tested beliefs; and lay, professional, and academic knowledge (Radaelli, 1995).

Te POLNET study focuses on the use of knowledge sources referenced within white and green papers. In our study, we found that, in all fve countries, documents made explicit references to a variety of knowledge sources. Te results of our investigation show that the policy papers

B. Karseth (\*) • K. Sivesind

Department of Education, University of Oslo, Oslo, Norway e-mail: berit.karseth@iped.uio.no; kirsten.sivesind@iped.uio.no

B. Karseth et al. (eds.), *Evidence and Expertise in Nordic Education Policy*, https://doi.org/10.1007/978-3-030-91959-7\_15

reference both governmental documents and draw on other types of documentation published by non-academic authors alongside research-based evidence produced by various research institutions. However, a striking similarity between the fve cases is the near absence of academic references from the educational sciences.

As formulated in the Icelandic chapter, academic papers "are thought to be irrelevant or not providing 'accessible' knowledge in the evidence base for the policy" (Chap. 6, p. 174). Likewise, as spelled out in the Danish chapter, references to Danish academic research are almost nonexistent. Due to international policies that recommend using big data and empirical evidence and the pattern of the relatively few references to academic works in the educational sciences, there seems to be a mismatch between what policy makers and experts consider relevant to cite and what, for example, students at universities and colleges read and discuss to achieve their degrees in the educational sciences. Tere are also relatively few references to articles printed by renowned, national, and international publishers.

Educational science in Europe and the Nordic countries in particular has, for institutional reasons, been characterized by professional-practical scholarship relevant in, for example, teacher education (Heggen et al., 2010). Educational scholarship has also evolved as an academic research feld at universities since the early 1900. Yet, traditions for large-scale empirical research vary between areas of expertise in education as well as between countries. Tere is, for example, a long-standing tradition for comparative research studies in Sweden and Finland, while Denmark, Iceland, and Norway are latecomers in this respect (Sivesind, 2019).

A surprising observation based on our dataset is the outstanding distinction between the numbers of references cited in each of the cases. Norway is clearly on top (2312 references), followed by Sweden (1421), and Finland (677). Te Danish and Icelandic cases have 231 and 203 references, respectively. One obvious explanation for this diference is the selection procedures for collecting documents. In Norway, the team decided to start with two white papers and the green papers on basic education referenced in the white papers and thereafter count all references within this corpus of documents. Te same selection procedure was applied in all countries, but the reforms under study did not produce the same number of source documents (i.e., white papers).

In the particular reform process in Norway, the parliament asked frst for one white paper on reforming the national curricula and second for a white paper that included recommendations for renewing the assessment system. Te minister in education decided to meet this last request by including reform-related themes in a white paper in progress. Tis national case resulted in a larger number of references compared to the other four cases. Due to these concrete circumstances, we cannot conclude that the large variations in the number of references represent an institutionalized pattern. Tat is, we have studied single reforms in fve Nordic countries, and there are good reasons to think of future reforms that will result in other patterns and numbers. Nonetheless, there are obviously various national traditions and institutional practices of policy making within the fve countries that result in a limited number of references, such as in Iceland, and many references, documented in the reference lists and footnotes, such as in Norway. As Christensen and Holst (2017) and Christensen and Hesstvedt (2019) have concluded based on their longitudinal survey of public enquiry reports, the expertization of public enquiry bodies represents an emerging trend in Norway that results in an increasing number of references.

Moreover, as clearly expressed in the Danish chapter, the low number of references in this case does not necessarily refect that the reform did not rely on any evidence. Based on additional qualitative data, the authors argue that stakeholder evidence and practice-based evidence have been of core signifcance in the preparation of the Danish reform. However, this evidence was not substantiated in formal documentation and thereby not available to map in quantitative terms with a bibliometric study. So far, our observation of how Danish governments authorize their national school reforms, seems to be a consequence of a lack of national—institutionalized procedures to write public enquiry reports, like those in Norway or Sweden.

Nonetheless, the relative number of references to domestic governmental documents compared to other types of references is signifcant in all fve countries. However, there are also vast distinctions for this dimension (from around 25% to almost 6%). Sweden and Iceland are on top, Denmark somewhere in the middle, and Norway and Finland at the bottom. As indicated in more detail in each of the chapters, these disparities must be understood and interpreted in the context of national reforms that in all countries are launched under the auspices of national governments. Nevertheless, independent of the institutional arrangement of the bodies that provide the reports, both policy makers and experts are formally expected to reference government-published documents in all Nordic countries.

Our bibliometric analysis reveals the active utilization of international references (cf. Chap. 9). Also for this dimension, our analysis unravels interesting diferences. While Denmark is the country case with the most international references (36.36%), Sweden represents the case with the least amount (18.93%). As Nordin and Wahlström conclude in their chapter, "[T]his fnding shows the possibility for national politics to uphold a high level of self-referentiality even when the national political agenda to a large extent is dictated by international organizations such as the OECD" (p. 244). In the Finnish chapter, the authors write that they expected to see the signifcant use of international sources for policy evidence and, in particular, the use of the OECD; however, data from Finland indicates a strong state involvement and concentration of expertise in state-funded bodies rather than a trust in international expertise. Interestingly, in Finland, there is a longstanding tradition of empirical research in the feld of education and therefore, perhaps, not the same need to draw on OECD studies such as in Iceland, Denmark, and Norway. Another fnding that created a puzzle for the research team was the (mostly) absent use of regional Nordic references in the source documents (cf. Chap. 12). Although Norway stands out as an exception among these countries, with its regional references amounting to almost 7%, this is lower than the authors expected based on previous research on the Nordic education model, which describes the region as a common unit.

Taken together, the bibliometric analyses provide important insights into how policy makers use various types of evidence to inform, back up, and legitimate school reforms. While some fndings are in line with what other researchers have pointed to, that there is a commonness in terms of how the Nordic countries organize their education system (Telhaug et al., 2006), there are also results in our study that nuance and even contradict the idea of a Nordic education model and, thereby, ideas of unity and commonness. Although Nordic policy makers and experts meet in an international context to reach a consensus as stakeholders for advocating their views, they also act highly self-referentially in regional and national settings to deliberate on various possible solutions to their own national problems (see Volmari et al., Chap. 12). Terefore, we consider the national and comparative chapters in this book to provide complementary insights. Reforms are legitimized by national knowledge, which in the Nordic countries are authorized by the state, while global and international knowledge providers place fngerprints on the reforms through their soft governance systems.

## **The Constellation of Knowledge Providers Within a Nordic Policymaking Context**

As Steiner-Khamsi indicates in Chap. 2, references in terms of citations carry epistemological connotations that represent various forms of knowledge as well as sites for knowledge production that connect national, regional, and global policies. Te constellation of knowledge providers in policymaking processes is thereby changing because of new ideas, institutions, and networks that characterize both national and transnational policy (Legrand, 2021). Due to new partnerships and networks that change customary procedures for policy development, the relevance of the traditional distinction between applied research, produced by multiple research institute types, and basic research, produced by universities, is challenged (Gibbons et al., 1994; Nowotny et al., 2003; Stokes, 1997).

In some Nordic countries, governments are funding research programs that involve both universities and research institutes in the same machinery of producing research, expected to make an impact on policymaking processes. A core aspiration of these programs, auspices by, for example, research councils, is to provide policy-relevant evidence. Tese programs may well transcend traditional boundaries between basic research and applied research, since evaluations assess the researchers and their research impact according to the same standards (Smith et al., 2020).

In tandem with new types of research and evaluation programs, the confguration of actors participating in a country's policy advisory system is also in transition (Christensen and Holst 2017; Steiner-Khamsi et al. 2020). As demonstrated in both the national chapters and comparative chapters, the national governments and their state agencies are themselves signifcant knowledge providers. According to Baek et al. (see Chap. 9), government papers that summarize information as well as research comprise more than one quarter of the references in the Nordic sample. In the Swedish case, Nordin and Wahlström show that the state has produced more than half of the references, representing a system of self-referentiality (see Chap. 8). Te role of references produced by governments and their policy departments and national agencies underscores the important role of the state in orchestrating and defning the policy issues at stake in the fve Nordic countries under study. Moreover, as analyzed in the Norwegian chapter by Hörmann and Sivesind (Chap. 7), references published by the government are important for linking to and legitimizing policies from previous reforms and arguments, retrospectively. In Iceland and in Denmark, there are also highly politicized processes, where either the minister has a clear opinion, as in Iceland (see Magningttir & Johansson, Chap. 6), or where political parties are involved in negotiations before a paper is written (see Juul Reder & Ydesen, Chap. 4).

Te number of references produced by research institutes that have had a traditional role of being knowledge suppliers to governments differs among the Nordic countries. Internationally, the label *research institute* encompasses many types of organizations that vary in the degree of how public they are (Late, 2019). Tey can according to Gulbrandsen (2011) be described as boundary organizations, as they often operate as agencies crossing the boundary of science and non-science (Late, 2019, p. 52). Not surprisingly, the use of references produced by such research institutes is central to both Norway's and Denmark's policymaking processes. In Norway, research institutes were initially established to perform R&D areas of interest for sector authorities. As Baek et al. (Chap. 9) and Steiner-Khamsi et al. (Chap. 10) demonstrate, the institute sector is a highly important knowledge provider that conducts educational policy research and research-based evaluations in Norway. Likewise, as Juul Reder and Ydesen's analysis in Chap. 4 depicts, the Danish Evaluation Institute (EVA) and the Danish National Research Centre for Social Research (SFI) represent two large agencies that produce policy-relevant, specialized knowledge under the auspices of two public ministries.

Tere is a striking diference between Norway and Denmark on the one side and Sweden and Finland on the other side. While the research institutes dominate in the two frst, public agencies are more powerful as knowledge providers in the two last. In Sweden, the public agency *Skoleverket* serves the role of being the most important knowledge provider. Tis agency generates ofcial knowledge and research statistics about the school system and childcare in the country. Te agency produces policy-relevant data as well as research reports that are frequently referenced in the white papers in the Swedish data. A similar reference pattern seems to be the case in Finland. Volmari et al. (Chap. 5) underscore the strong expert position and power of the Finnish National Agency of Education.1 Tis organization collects its own data and provides analyses and evaluations. Tis governmental institution is in itself an expert body similar to the Swedish agency albeit shorter history as an independent agency. Nonetheless, Volmari et al. explain that two universities in Finland have two publicly funded research centers that play a vital role in producing sector-based knowledge, not the least, OECDfunded studies, with a special responsibility for PISA. One of these institutes has been part of the EIPPEE network. We fnd similar centers in all Nordic countries that link the universities to globalizing policy spaces with a certain impact on policymaking processes in the Nordic countries.

As mentioned, the scarcity of peer-reviewed academic references, including academic books and journals, is a striking feature of the reference pattern in our documentation. In Chap. 9, Baek et al. demonstrate that 30% of the references in the Nordic sample are journal articles and books. In Chap. 10, Steiner-Khamsi et al. present an analysis of the types of references that receive focus in the white papers and green papers in the Swedish and the Norwegian cases, using a detailed classifcation system that distinguishes between both national and international academic research references. Tey report that, while national and international academic research comprises about 40% and 25% of the references in the Norwegian green papers and white papers, respectively, in the Swedish context, such research constitutes only 20% of the references in green papers and none in white papers. To understand these variations, more in-depth analysis is needed. However, the numbers alone may point to the status of academic research in Sweden, where only 7% of the total number of references represented national academic research. Furthermore, as Wahlström and Nordin conclude in Chap. 8, we need to consider how intermediary organizations mediate academic research, summarizing and translating results and interpretations in a simplifed and accessible way.

Our analysis thus far does not provide much empirical knowledge for the fve countries on the use of references from organizations that present themselves as think tanks. Christensen and Holst (2020) refer to *think tanks* as organizations that aim at infuencing political debates and decisions by referencing knowledge and information. Te think tanks can be considered as Rich (2004, p. 150) defnes the term: "independent, non-interest-based, non-proft organizations that produce and principally rely on expertise and ideas to obtain support and to infuence the policy making process." Medvetz (2012, p. 213) extends this defnition by defning *think tanks* as a "hybrid institutional area situated at the nexus of the political academic, economic and media felds." Notwithstanding, in the Nordic countries, both as organizations and as discursive spaces, think tanks are relatively seen, new inventions, since the organizational fguration of actors as described in the literature on think tanks difers in comparison with the more established research institutes that provide *sector research*.

One key disparity is that research institutes in the Nordic countries receive funding from the Research Council of Norway and conduct commissioned research that are regulated by the contracts with public and private partners. While research institutes must follow formal contracts that to some extent prohibit them from taking an active part in political negotiations, think tanks pursue ideological agendas as discursive tools. Terefore, research institutes do not act and react to political issues in the ways that think tanks can do. Terefore, think tanks claim to generate complementary knowledge in comparison with research carried out by universities and research institutes, both because of their institutional afliation and due to their ways of deploying knowledge and expertise. Findings based on the POLNET study indicate that our documents cite think tanks only to a modest degree, if at all, in the reference lists. Yet, they deserve a closer look in future research, as they may become infuential through advocacy coalitions (Sabatier & Weible, 2007; Steiner-Khamsi, 2021) and policy assemblages where policy is produced through the ancillary of political and environmental contexts (Legrand, 2021).

Taken together, the research and development systems (including innovation) and the policy advisory systems within the fve Nordic countries studied in this book have certain similarities, but the sites at which policy knowledge, such as research and evaluations, are produced do refect signifcant variations. A lesson learned from reading the chapters of this book is that there are no clear boundaries between the sites for knowledge production and the sites for knowledge usage. Tis again leads to unclear separation of roles of policy makers on the one hand and policy advisory commissions, including researchers, on the other hand.

## **International Organizations: The Supremacy of the OECD**

As part of the rise of the evaluative state (Maroy, 2009; Neave, 1988, 2009), research-based evaluations and surveys provide evidence and knowledge that are useful for legitimizing school reforms. Moreover, as Martens et al. (2016, p. 518) declare about the efects of this evolvement: "Tere is little doubt that international assessments established a 'new center of gravity' in the feld of education and that they re-shaped education policy-making and practices in many countries." For this and other reasons; universities, university colleges, and the various forms of research institutes act as competing or collaborating tenderers/bidders in seeking to manage various types of government-funded evaluations in the Nordic countries. Tis is obviously the case in the Norwegian context, where the Ministry, following major educational reforms in 1990, decided to introduce research-based evaluation programs at the turn of the 2000s (see also Zapp et al., 2018).

Besides the formal research and development systems that we consider as domestic knowledge providers, this book unravels the importance of international organizations as core knowledge producers in policymaking processes. As Steiner-Khamsi (2013) suggests, international knowledge and comparative studies in particular can infuence policymaking processes (a) as evidence that informs policy planning within particular contexts, (b) as normative guidelines for how to change educational processes concerning global problems, or (c) as projecting best practices that are evaluated against a set of international performance standards. A key point in Steiner-Khamsi's work is that the production of knowledge, especially the design of comparative research projects, optimizes evidence for the research impact on features that are not necessarily structured in the same ways across the selected sample of cases. Against this background, we expected extensive references to comparative studies for legitimizing reforms of certain features, independent of regional commonalities that are traditionally associated with the Nordic model.

Based on our sample of documents, we fnd that OECD is the most signifcant knowledge producer of the international publications referenced in all the Nordic cases. In the comparative Chap. 11, Ydesen et al. conclude that the fve country-specifc cases reveal multiple layers in OECD-related references. Based on their analysis, the authors emphasize that the OECD policy instrument carries more weight than that found in a quantitative analysis of references only. By forming a powerful epistemic space, the OECD is more powerful than all other international organizations. Nordic countries have a long-lasting tradition of being advised and supported by the OECD on how to govern their educational system (Ydesen, 2019). By the turn of the century, the attention of the OECD became extended by the PISA study, in which all fve countries participated from the year 2000 (Sivesind, 2019).

Verger et al. (2019) recently compiled a literature review on how the OECD infuences education policy in diverse countries by governance mechanisms and demonstrated how the organization has afected member countries diferently by their way of collecting data, evaluating quality, and generating ideas for how to improve education systems. Te POLNET study shows variations in the usage of OECD references. As illustrated by the Swedish case of Nordin et al. (Chap. 8), although the number of references to the OECD is modest compared with the number of references to domestic and government sources, the Ministry turned explicitly to the OECD for analytical help to tackle its national school crisis in 2014. Likewise, in the Finnish case, as shown in Chap. 5, while the bibliometric analysis reveals evidence used in the 2014 curriculum reform was predominantly domestic and self-referential, a content analysis discloses the OECD and, especially, the PISA results were clearly visible in numbers in the policy documents. Te same was demonstrated for the Icelandic case in Chap. 6; OECD evidence was used particularly to legitimize policy recommendations and design together with knowledge sources from Canada and a consultancy company, such as McKinsey.

As Steiner-Khamsi (2013, p. 27) argued, not all forms of comparison will necessarily lead to policy borrowing and lending across countries. Policy transfer depends on various conditions, not least the methodological design used to construct the tests and surveys within OECD studies. Within the Nordic region of Europe, PISA studies, together with similar large-scale assessments, create what Waldow (2019) interestingly conceptualizes as projections of best practices. Tis concept implies that references to best practices are not outcomes of particular conditions that regulate education policy, but rather socially constructed narratives that policy actors make to reduce the complexity they experience (pp. 4–5). Nonetheless, as noted by Steiner-Khamsi in Chap. 2, due to diferences between the successful performance scores of Finnish students in PISA, the Finnish education has been glorifed as one of the most successful systems in the world, and their results are outstanding also in comparison with their Nordic neighbors. Tus we ask: Given the noticeable variations in performance scores between Finland and other Nordic countries, how are policy makers and experts referencing Finland within the policy papers?

Our material does not indicate that national publications about the Finnish education system and Finnish educational practices as such attract extraordinary attention by being highly referenced in our dataset. Interestingly, in the comparative chapter by Volmari et al. (Chap. 12), documents from three Nordic countries refer to two national curricula from Finland, whereas other Nordic curricula are not co-cited. In general, white and green papers from Iceland, Finland, and Norway reference national publications from Sweden and Denmark more often than Sweden and Denmark refer to publications from other Nordic countries. Overall, regional collaboration, as presented by the interviewees, does not seem to lead to policy convergence or difusion albeit Finland is highlighted as an outstanding system. Rather, national authorities decide in their own contexts what to learn from Finland and others, following their own institutional paths for deciding what to do within their own contexts. Tis observation leads us to conclude that option (a) within the typology of Steiner-Khamsi seems to be the most typical feature of policy borrowing and lending among the Nordic countries, while we are also open to the possibility that OECD studies and transnational knowledge about performance scores on PISA infuence national education reforms more silently (Waldow, 2009).

Possible reasons for the self-referentiality of school reform policy are the historical and organizational conditions that policy makers refer to and which difer between countries. Conditions are important levers in reforming schools, which implies the steady need to search for contextdependent knowledge that in the next step shapes the narratives of best practices. Moreover, such disparities may not constitute research objects or topics favored within international assessments because contextsensitive knowledge is neither necessarily capable of being measured by research items used for comparative studies, nor easily transferred between the contexts that we have studied. Terefore, in general, only those dimensions that are constructed as generic, that be, in terms of values and standards, are possible targets for international knowledge transfer.

## **The Knowledge Work of the Government and Its Agencies**

As the chapters in this book uncover, the state plays a vital role in policymaking processes in all the Nordic countries, and besides the ministries, semi-independent central agencies are core bodies that have a say in these processes (Greve et al., 2020). Although the level of independence of these agencies varies, their tasks are to obtain, translate, and implement reform ideas (Røvik et al., 2014). As already mentioned, public agencies are not per se an think tanks organization with political ambitions. Yet, they can organize international cooperation and activities, orchestrated by powerful policy actors, such as the OECD and the European Union (EU). Tereby, they function as agenda setters that mediate knowledge "from the outside" in policymaking processes that are regulated by the state. By coordinating and inviting others to join their research and development fora and to participate in various forms of knowledge work together with for example professional associations (Nerland and Karseth, 2015), the agencies advise ministries on how to deploy both a repertoire of reform ideas and standards of which national policy makers and politicians should be aware.

However, central agencies at the national level can also be viewed as gatekeepers in orchestrating ideas of how to translate, whom to involve, and how to connect topics and realms in policymaking processes. Although ministries mandate and oversee activities within boards and agencies, they can also be considered a hub that regulates its own knowledge fow between policy makers, international organizations, various stakeholders as well as researchers. Tis evolving layer of knowledge can be observed in how knowledge sources produced by the agencies are referenced in governmental papers, and is therefore an interesting question what these agencies prioritize to produce as well as reference in their reports.

Te POLNET study has not identifed reference patterns in reports produced by the state agencies. Yet, as some of the country chapters have demonstrated, national agencies seem to play an important role for providing evaluations and assessments in their education systems, that resonate policy makers and experts' need for evidence. Simultaneously, it is important to recognize that the ministries themselves are both receivers and translators of this evidence (see Chap. 7). Anyhow, the governance arrangements of national agencies seem to be in a mode of constant change through processes of mergers between entities, organizational rearrangements, and new establishments (see Ärlestig & Johansson, 2020). Terefore, it is of interest to study how agencies and ministries interact in their search for policy solutions that potentially results in education reform. We may argue that ongoing transitions refect the search for a workable and legitimate balance of responsibilities and tasks between politicians, bureaucrats, and experts and not at least, between ministries, boards/agencies and coalitions with stakeholders. Tere is a need to study the intersection between these bodies in future research.

Nonetheless, by investigating reference patterns in our database, we observed that the importance of the state and the ethos of a robust public administration system still seem a characteristic of the Nordic mindset (see Volmari et al., Chap. 12). While processes for the deregulation, privatization, and marketization of Nordic education are ongoing, researchers like Dovemark et al. (2018) concluded that the changes are not as dramatic as those occurring in other countries. Nordic countries can be described, according to Maroy et al. (2017), as closer to a Neostatist variant of managerialism than a neoliberal variant. Alternatively, as observed by Greve et al. (2020), a mixture of multiple reform packages is in use in Nordic countries that refects institutionalized forms of coordinating the public sector, although managerial tools are at the forefront (p. 706). Tis leads in the next step to the conception of a "welfare mix" (Sivesind & Saglie, 2017) that characterizes the Nordic education systems. Terefore, unraveling the complexity of governance structures and policy borrowing and lending across nations is imperative. Taken together, to understand the practice of evidence-based policy making, we need to capture how political institutions work: their procedures, routines, regulations, and relations. Moreover, further research on the central educational agencies is needed to capture how science and politics are structurally coupled (see Steiner-Khamsi et al., Chap. 10; Steiner-Khamsi, 2021).

## **Evidence and Expertise in Transitions Between State Government and Network Governance**

Te core aim of this book has been to explore distinctions in reference patterns between reform policies in the fve Nordic countries. We have tried to unravel characteristics of what Eyal (2019, p. 33) labels "distributed cognitions of expertise," that is, how expertise outside individuals is visualized through bibliometric patterns and reference networks. Certainly, it is not easy to uncover why some knowledge sources are selected by actors at the cost of others. Often, policy making is based on tacit, practical knowledge and an outcome of assemblages beyond conscious decision making (Savage, 2020). Evidence-based policies can draw on knowledge from both the outside and inside of public policies. Terefore, the location of knowledge use is equally signifcant as the kind of knowledge produced and used. For that reason, excellent reasons exist for clarifying various reference patterns and for developing what Eyal (2019, p. 33) labeled explicit, abstract knowledge that expands and advances the sociology of expertise. Tis knowledge may well be mediated through books and articles that individual experts and others can read to enlighten conversations on public policies. Moreover, researching the sociology of expertise can provide knowledge that makes sense in contexts where people and bodies develop reforms on behalf of the state. In that case, various types of conditions can stimulate collaborative processes that help policy makers and experts to make recommendations on valid knowledge.

Although not at the forefront of our analyses, the national and the comparative chapters have pointed to strategies that stakeholders deploy in policymaking processes (Chaps. 4 and 10). While public hearings have been important devices for recognizing various opinions and voices in public administrations that have been highly departmentalized (Sivesind & Skedsmo, 2020), today, many more channels are in use for collecting information and infuencing policy. Organizations of stakeholder conferences, blogs, social media, the establishment of diferent types of reference groups, think tanks, and public-private partnerships can evolve in various directions, mobilizing decision processes that call for new constellations of innovative reference patterns that we could not unravel in our analysis of white and green papers and which need further examination. Te rearrangement of public policy implies that the modern state is "beset by a burgeoning array of domestic-global political, social and economic infuence" (Legrand, 2021, p. 37). Ministries and state agencies are core bodies in this landscape of expertise, orchestrating meetings with national and international experts to communicate about kinds of evidence and expertise that are relevant for revising or renewing education policy.

Simultaneously, new technology and co-governing strategies can in many ways give policy makers new opportunities to seek policy-relevant information. Crowd-sourcing procedures are nowadays used to collect information in reform processes from a range of actors; however, the respondents are not necessarily dedicated experts, specialized scientists, representatives of a particular knowledge feld, or powerful stakeholders. Who is invited to participate is technically seen, contingent. Due to the ubiquitous access to digitalized information in society and new constellations of collaborators, there are good reasons to think of policy making as becoming pluralized. Tis pluralization of policy may transcend institutional boundaries that have guaranteed corporate decision making into a discursive policy space that allocates the attention of policy makers to new agenda-setters in education policy.

Nordic countries are known for their corporative traditions where the state and the government have granted access to certain types of interest groups and organizations in arrangements such as public advisory bodies (Åberg et al., 2019). Such arrangements have induced both the stability and legitimacy of policies within civil society that in our case include powerful organizations, such as teacher unions. However, there are good reasons for seeing the policy system as well as traditions for developing professional expertise potentially transformed by evidence-based policy and thereby challenged. Terefore, to create legitimacy for changing policies in the feld of education, the complexity of both conditions and expectations urges academic enquiry.

As Eyal (2019, p. 36) acknowledges, the application of expertise depends on "being connected with a network of expertise composed of other actors, devices and instruments, concepts, and institutional and practical arrangements, distributed in multiple loci, yet assembled into a coherent, collective agency." Tis agency must deal with self-referential problems about the refective use of policy knowledge that, in our case, refers to state-authorized school reforms. However, in our time, it must also look outside its own boundary to seek solutions to global problems in collaboration with others. Terefore, the international orchestration of policy spaces and the interdependent matrix of processes between various knowledge providers, deserve researchers' attention in future research.

## **Note**

1. Tis agency replaced the Finnish national board of education in 2017 as it merged with CEMO (the Centre for International Mobility). Tis shift implies a transition from serving a role as a Directorate for Education to an agency within the national administration of education and training that has a two-tier structure similar to the Swedish case.

## **References**


**Open Access** Tis chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

Te images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.