**Sharon Shoham Yakir Vizel (Eds.)**

# LNCS 13372

# **Computer Aided Verification**

**34th International Conference, CAV 2022 Haifa, Israel, August 7–10, 2022 Proceedings, Part II**

## **Lecture Notes in Computer Science 13372**

#### Founding Editors

Gerhard Goos *Karlsruhe Institute of Technology, Karlsruhe, Germany*

Juris Hartmanis *Cornell University, Ithaca, NY, USA*

#### Editorial Board Members

Elisa Bertino *Purdue University, West Lafayette, IN, USA*

Wen Gao *Peking University, Beijing, China*

Bernhard Steffen *TU Dortmund University, Dortmund, Germany*

Moti Yung *Columbia University, New York, NY, USA* More information about this series at https://link.springer.com/bookseries/558

Sharon Shoham · Yakir Vizel (Eds.)

## Computer Aided Verification

34th International Conference, CAV 2022 Haifa, Israel, August 7–10, 2022 Proceedings, Part II

*Editors* Sharon Shoham Tel Aviv University Tel Aviv, Israel

Yakir Vizel Technion – Israel Institute of Technology Haifa, Israel

ISSN 0302-9743 ISSN 1611-3349 (electronic) Lecture Notes in Computer Science ISBN 978-3-031-13187-5 ISBN 978-3-031-13188-2 (eBook) https://doi.org/10.1007/978-3-031-13188-2

© The Editor(s) (if applicable) and The Author(s) 2022. This book is an open access publication.

**Open Access** This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

## **Preface**

It was our privilege to serve as the program chairs for CAV 2022, the 34th International Conference on Computer-Aided Verification. CAV 2022 was held during August 7–10, 2022. CAV-affiliated workshops were held on July 31 to August 1 and August 11 to August 12. This year, CAV was held as part of the Federated Logic Conference (FLoC) and was collocated with many other conferences in software/hardware verification and logic for computer science. Due to the easing of COVID-19 travel restrictions, CAV 2022 and the rest of the FLoC were in-person events.

CAV is an annual conference dedicated to the advancement of the theory and practice of computer-aided formal analysis methods for hardware and software systems. The primary focus of CAV is to extend the frontiers of verification techniques by expanding to new domains such as security, quantum computing, and machine learning. This puts CAV at the cutting edge of formal methods research, and this year's program is a reflection of this commitment.

CAV 2022 received a high number of submissions (209). We accepted nine tool papers, two case studies, and 40 regular papers, which amounts to an acceptance rate of roughly 24%. The accepted papers cover a wide spectrum of topics, from theoretical results to applications of formal methods. These papers apply or extend formal methods to a wide range of domains such as smart contracts, concurrency, machine learning, probabilistic techniques, and industrially deployed systems. The program featured a keynote talk by Ziyad Hanna (Cadence Design Systems and University of Oxford), a plenary talk by Aarti Gupta (Princeton University), and invited talks by Arie Gurfinkel (University of Waterloo) and Neha Rungta (Amazon Web Services). Furthermore, we continued the tradition of Logic Lounge, a series of discussions on computer science topics targeting a general audience. In addition to all talks at CAV, the attendees got access to talks at other conferences held as part of FLoC.

In addition to the main conference, CAV 2022 hosted the following workshops: Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), On the Not So Unusual Effectiveness of Logic, Formal Methods Education Online, Democratizing Software Verification (DSV), Verification of Probabilistic Programs (VeriProP), Program Equivalence and Relational Reasoning (PERR), Parallel and Distributed Automated Reasoning, Numerical Software Verification (NSV-XV), Formal Reasoning in Distributed Algorithms (FRIDA), Formal Methods for Blockchains (FMBC), Synthesis (Synt), and Workshop on Open Problems in Learning and Verification of Neural Networks (WOLVERINE).

Organizing a flagship conference like CAV requires a great deal of effort from the community. The Program Committee (PC) for CAV 2022 consisted of 86 members – a committee of this size ensures that each member has a reasonable number of papers to review in the allotted time. In all, the committee members wrote over 800 reviews while investing significant effort to maintain and ensure the high quality of the conference program. We are grateful to the CAV 2022 PC for their outstanding efforts in evaluating the submissions and making sure that each paper got a fair chance. Like recent years in CAV, we made the artifact evaluation mandatory for tool paper submissions and optional but encouraged for the rest of the accepted papers. The Artifact Evaluation Committee consisted of 79 reviewers who put in significant effort to evaluate each artifact. The goal of this process was to provide constructive feedback to tool developers and help make the research published in CAV more reproducible. The Artifact Evaluation Committee was generally quite impressed by the quality of the artifacts. Among the accepted regular papers, 77% of the authors submitted an artifact, and 58% of these artifacts passed the evaluation. We are very grateful to the Artifact Evaluation Committee for their hard work and dedication in evaluating the submitted artifacts.

CAV 2022 would not have been possible without the tremendous help we received from several individuals, and we would like to thank everyone who helped make CAV 2022 a success. First, we would like to thank Maria A Schett and Daniel Dietsch for chairing the Artifact Evaluation Committee and Hari Govind V K for putting together the proceedings.We also thank Grigory Fedyukovich for chairing the workshop organization and Shachar Itzhaky for managing publicity. We would like to thank the FLoC organizing committee for organizing the Logic Lounge, Mentoring workshop, and arranging student volunteers. We also thank Hana Chockler for handling sponsorship for all conferences in FLoC. We would also like to thank FLoC chair Alexandra Silva and co-chairs Orna Grumberg and Eran Yahav for the support provided. Last but not least, we would like to thank members of the CAV Steering Committee (Aarti Gupta, Daniel Kroening, Kenneth McMillan, and Orna Grumberg) for helping us with several important aspects of organizing CAV 2022.

We hope that you will find the proceedings of CAV 2022 scientifically interesting and thought-provoking!

June 2022 Sharon Shoham Yakir Vizel

## **Organization**

#### **Steering Committee**


#### **Conference Co-chairs**


#### **Artifact Evaluation Co-chairs**


#### **Publicity Chair**


#### **Workshop Chair**


#### **Proceedings and Talks Chair**


#### **Program Committee**


Andrey Rybalchenko Microsoft Research, UK Anna Lukina TU Delft, The Netherlands Anna Slobodova Intel, USA Caterina Urban INRIA, France Dana Drachsler Cohen Technion, Israel Daniel Kroening Amazon, USA Derek Dreyer MPI-SWS, Germany Guy Avni University of Haifa, Israel Hadar Frenkel CISPA, Germany Hillel Kugler Bar Ilan University, Israel Josh Berdine Meta, UK Krishna Shankaranarayanan IIT Bombay, India

Ana Sokolova University of Salzburg, Austria Anastasia Mavridou KBR Inc., NASA Ames Research Center, USA Andreas Podelski University of Freiburg, Germany Andrei Popescu University of Sheffield, UK Antoine Miné Sorbonne Université, France Armin Biere University of Freiburg, Germany Azalea Raad Imperial College London, UK Bor-Yuh Evan Chang University of Colorado Boulder and Amazon, USA Corina Pasareanu CMU/NASA Ames Research Center, USA David Jansen Institute of Software, Chinese Academy of Sciences, China Deepak D'Souza Indian Institute of Science, India Dejan Jovanovi´c Amazon Web Services, USA Elizabeth Polgreen University of Edinburgh, UK Elvira Albert Complutense University of Madrid, Spain Erika Abraham RWTH Aachen University, Germany Grigory Fedyukovich Florida State University, USA Guy Katz Hebrew University of Jerusalem, Israel Hiroshi Unno University of Tsukuba, Japan Isabel Garcia-Contreras University of Waterloo, Canada Ivana Cerna Masaryk University, Czech Republic Jade Alglave University College London and Arm, UK Jean-Baptiste Jeannin University of Michigan, USA Joost-Pieter Katoen RWTH Aachen University, Germany Joxan Jaffar National University of Singapore, Singapore Kenneth L. McMillan University of Texas at Austin, USA Klaus V. Gleissenthall Vrije Universiteit Amsterdam, The Netherlands Konstantin Korovin University of Manchester, UK Kuldeep Meel National University of Singapore, Singapore

Laura Titolo NIA/NASA LaRC, USA Marijana Lazi´c TU Munich, Germany Marijn J. H. Heule CMU, USA Markus Rabe Google, USA Miriam García Soto IST Austria, Austria Oded Padon VMware Research, USA Parasara Sridhar Duggirala UNC Chapel Hill, USA Rajeev Joshi Amazon, USA Rayna Dimitrova CISPA, Germany Ruben Martins CMU, USA Rupak Majumdar MPI-SWS, Germany Ruzica Piskac Yale University, USA Subodh Sharma IIT Delhi, India Supratik Chakraborty IIT Bombay, India Swarat Chaudhuri UT Austin, USA Swen Jacobs CISPA, Germany Temesghen Kahsai Amazon, USA

Liana Hadarean Amazon Web Services, USA Marieke Huisman University of Twente, The Netherlands Martina Seidl Johannes Kepler University Linz, Austria Mingsheng Ying Chinese Academy of Sciences and Tsinghua University, China Natasha Sharygina University of Lugano, Switzerland Nikolaj Bjørner Microsoft Research, USA Nir Piterman University of Gothenburg, Sweden Parosh Aziz Abdulla Uppsala University, Sweden Pavithra Prabhakar Kansas State University, USA Philippa Gardner Imperial College London, UK Pierre Ganty IMDEA Software Institute, Spain Roderick Bloem Graz University of Technology, Austria Sébastien Bardin CEA List, Université Paris-Saclay, France Shuvendu Lahiri Microsoft Research, USA Sorav Bansal IIT Delhi and CompilerAI Labs, India Taylor T. Johnson Vanderbilt University, USA Wei-Ngan Chin National University of Singapore, Singapore Xavier Rival INRIA Paris/ENS/PSL, France Xujie Si McGill University, Canada Yang Liu Nanyang Technological University, Singapore Yu-Fang Chen Institute of Information Science, Academia Sinica, Taiwan Yuxin Deng East China Normal University, China

#### **Artifact Evaluation Committee**

Guy Amir Hebrew University of Jerusalem, Israel Muqsit Azeem Technical University of Munich, Germany Kshitij Bansal Meta, USA Tianshu Bao Vanderbilt University, USA Fabian Bauer-Marquart University of Konstanz, Germany Anna Becchi Fondazione Bruno Kessler, Italy Ranadeep Biswas Informal Systems, France Christopher Brix RWTH Aachen University, Germany Marek Chalupa IST Austria, Austria Kevin Cheang University of California, Berkeley, USA Lesly-Ann Danie KU Leuven, Belgium Kinnari Dave Certified Kernel Tech, LLC, USA Simon Dierl TU Dortmund, Germany Florian Dorfhuber Technical University of Munich, Germany Benjamin Farinier Technische Universität Wien, Austria Parisa Fathololumi Stevens Institute of Technology, USA Mathias Fleury University of Freiburg, Germany Luke Geeson University College London, UK Pablo Gordillo Complutense University of Madrid, Spain Manish Goyal University of North Carolina at Chapel Hill, USA Akos Hajdu Meta Platforms Inc., UK Alejandro Hernández-Cerezo Complutense University of Madrid, Spain Jana Hofmann CISPA Helmholtz Center for Information Miguel Isabel Universidad Complutense de Madrid, Spain Martin Jonáš Fondazione Bruno Kessler, Italy Samuel Judson Yale University, USA Sudeep Kanav LMU Munich, Germany Daniela Kaufmann Johannes Kepler University Linz, Austria Brian Kempa Iowa State University, USA Ayrat Khalimov Université libre de Bruxelles, Belgium Nishant Kheterpal University of Michigan, USA Edward Kim University of North Carolina at Chapel Hill, USA John Kolesar Yale University, USA Bettina Könighofer Graz University of Technology, Austria Mitja Kulczynski Kiel University, Germany Thomas Lemberger LMU Munich, Germany Julien Lepiller Yale University, USA Sven Linker Lancaster University in Leipzig, Germany Kirby Linvill University of Colorado Boulder, USA Y. Cyrus Liu Stevens Institute of Technology, USA

Security, Germany

Tobias Meggendorfer IST Austria, Austria Felipe R. Monteiro Amazon, USA Daniel Schoepe Amazon, UK Ðor -

Tianhan Lu University of Colorado Boulder, USA Enrico Magnago Fondazione Bruno Kessler and University of Trento, Italy Fabian Meyer RWTH Aachen University, Germany Stefanie Mohr Technical University of Munich, Germany Raphaël Monat LIP6, Sorbonne Université and CNRS, France Marcel Moosbrugger Technische Universität Wien, Austria Marco Muniz Aalborg University, Denmark Neelanjana Pal Vanderbilt University, USA Francesco Parolini Sorbonne University, France Mário Pereira Universidade NOVA de Lisboa, Portugal João Pereira ETH Zurich, Switzerland Sumanth Prabhu Indian Institute of Science and TCS Research, India Cedric Richter University of Oldenburg, Germany Clara Rodríguez Complutense University of Madrid, Spain Bob Rubbens University of Twente, The Netherlands Rubén Rubio Universidad Complutense de Madrid, Spain Stanly Samuel Indian Institute of Science, India Philipp Schröer RWTH Aachen University, Germany Joseph Scott University of Waterloo, Canada Arnab Sharma University of Oldenburg, Germany Salomon Sickert Hebrew University of Jerusalem, Israel Yahui Song National University of Singapore, Singapore Michael Starzinger University of Salzburg, Austria Martin Tappler Graz University of Technology, Austria Michael Tautschnig Queen Mary University of London, UK Mertcan Temel Intel Corporation, USA Saeid Tizpaz-Niari University of Texas at El Paso, USA Deivid Vale Radboud University Nijmegen, The Netherlands Vesal Vojdani University of Tartu, Estonia Masaki Waga Kyoto University, Japan Peixin Wang University of Oxford, UK Tobias Winkler RWTH Aachen University, Germany Stefan Zetzsche University College London, UK Xiao-Yi Zhang National Institute of Informatics, Japan Linpeng Zhang University College London, UK de Žikeli´c IST Austria, Austria

#### **Additional Reviewers**

A. R. Balasubramanian Aaron Gember-Jacobson Abhishek Rose Aditya Akella Alberto Larrauri Alexander Bork Alvin George Ameer Hamza Andres Noetzli Anna Becchi Anna Latour Antti Hyvarinen Benedikt Maderbacher Benno Stein Bettina Koenighofer Bruno Blanchet Chana Weil-Kennedy Christoph Welzel Christophe Chareton Christopher Brix Chun Tian Constantin Enea Daniel Hausmann Daniel Kocher Daniel Schoepe Darius Foo David Delmas David MacIver Dmitriy Traytel Enrique Martin-Martin Filip Cano Francesco Parolini Frederik Schmitt Frédéric Recoules Fu Song Gadi Aleksandrowicz Gerco van Heerdt Grégoire Menguy Gustavo Petri Guy Amir

Haggai Landa Hammad Ahmad Hanna Lachnitt Hongfei Fu Ichiro Hasuo Ilina Stoilkovska Ira Fesefeldt Jens Gutsfeld Ji Guan Jiawei Chen Jip Spel Jochen Hoenicke Joshua Moerman Kedar Namjoshi Kevin Batz Konstantin Britikov Koos van der Linden Lennard Gäher Lesly-Ann Daniel Li Wenhua Li Zhou Luca Laurenti Lukas Armborst Lukáš Holík Malte Schledjewski Martin Blicha Martin Helfrich Martin Lange Masoud Ebrahimi Mathieu Lehaut Michael Sammler Michael Starzinger Miguel Gómez-Zamalloa Miguel Isabel Ming Xu Noemi Passing Niklas Metzger Nishant Kheterpal Noam Zilberstein Norine Coenen

Omer Rappoport Ömer ¸Sakar Peter Lammich Prabhat Kumar Jha Raghu Rajan Rodrigo Otoni Romain Demangeon Ron Rothblum Roope Kaivola Sadegh Soudjani Sepideh Asadi Shachar Itzhaky Shaun Azzopardi Shawn Meier Shelly Garion Shubhani Shyam Lal Karra Simon Spies Soline Ducousso Song Yahui Spandan Das Sumanth Prabhu Teodora Baluta Thomas Noll Tim King Tim Quatmann Timothy Bourke Tobias Winkler Vasileios Klimis Vedad Hadzic Vishnu Bondalakunta Wang Fang Xing Hong Yangjia Li Yash Pote Yehia Abd Alrahman Yuan Feng Zachary Kincaid Ziv Nevo Zurab Khasidashvili

## **Contents – Part II**

#### **Probabilistic Techniques**





## **Contents – Part I**




## **Probabilistic Techniques**

## **PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDP**

Chaitanya Agarwal<sup>1</sup>, Shibashis Guha2(B), Jan Kˇret´ınsk´y<sup>3</sup>, and Pazhamalai Muruganandham<sup>4</sup>

 New York University, New York, USA Tata Institute of Fundamental Research, Mumbai, India Shibashis@tifr.res.in Technical University of Munich, Munich, Germany Chennai Mathematical Institute, Chennai, India

**Abstract.** Markov decision processes (MDP) and continuous-time MDP (CTMDP) are the fundamental models for non-deterministic systems with probabilistic uncertainty. Mean payoff (a.k.a. long-run average reward) is one of the most classic objectives considered in their context. We provide the first algorithm to compute mean payoff probably approximately correctly in unknown MDP; further, we extend it to unknown CTMDP. We do not require any knowledge of the state space, only a lower bound on the minimum transition probability, which has been advocated in literature. In addition to providing probably approximately correct (PAC) bounds for our algorithm, we also demonstrate its practical nature by running experiments on standard benchmarks.

#### **1 Introduction**

Markov decision process (MDP) [7,30,32] is a basic model for systems featuring both probabilistic and non-deterministic behaviour. They come in two flavours: discrete-time MDP (often simply MDP) and continuous-time MDP (CTMDP). While the evolution of MDP happens in discrete steps, their natural real-time extension CTMDP additionally feature random time delays governed by exponential probability distributions. Their application domain ranges across a wide spectrum, e.g. operations research [10,16], power management and scheduling [31], networked and distributed systems [19,22], or communication protocols [28], to name a few. One of the key aspects of such systems is their performance, often formalized as mean payoff (also called long-run average reward), one of the classic and most studied objectives on (CT)MDP [30] with numerous applications [17]. In this context, probabilistic model checking and performance evaluation intersect [5]. While

This work has been partially supported by the DST-SERB project SRG/2021/000466 *Zero-sum and Nonzero-sum Games for Controller Synthesis of Reactive Systems* and by the German Research Foundation (DFG) projects 427755713 (KR 4890/3-1) *Group-By Objectives in Probabilistic Verification (GOPro)* and 383882557 (KR 4890/2-1) *Statistical Unbounded Verification (SUV)*.

c The Author(s) 2022 S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 3–25, 2022. https://doi.org/10.1007/978-3-031-13188-2\_1

the former takes the verification perspective of the worst-case analysis and the latter the perspective of optimization for the best case, they are mathematically dual and thus algorithmically the same.

The range of analysis techniques provided by literature is very rich, encompassing linear programming, policy iteration, or value iteration. However, these are applicable only in the setting where the (CT)MDP is known (whitebox setting). In order to handle the blackbox setting, where the model is unknown or only partially known, statistical model checking (SMC) [37] relaxes the requirement of the hard guarantees on the correctness (claimed precision) of the result. Instead it uses probably approximately correct (PAC) analysis, which provides essentially a confidence interval on the result: with probability (confidence) at least 1 − δ, the result of the analysis is ε-close to the true value. This kind of analysis may be applicable to those systems for which we do not have exclusive access to their internal functionalities, but we can still observe their behaviour.

In this paper, we provide the first algorithm with PAC bounds on the mean payoff in blackbox MDP. We treat both the discrete-time and continuous-time MDP, and the SMC algorithm not only features PAC bounds (returning the result with prescribed precision and confidence), but an anytime algorithm (gradually improving the result and, if terminated prematurely, can return the current approximation with its precision and the required confidence).

The difficulty with blackbox models is that we do not know the exact transition probabilities, not even the number of successors for an action from a state. The algorithm thus must simulate the MDP to obtain any information. The visited states can be augmented to a model of the MDP and statistics used to estimate the transition probabilities. The estimates can be used to compute mean payoff precisely on the model. The results of [12] and [33] then provide a method for estimating the number of times each state-action pair needs to be visited in an MDP to obtain a PAC bound on the expected mean-payoff value of the original MDP. However, notice that this requires that the topology be learnt perfectly, for which we either need some knowledge of the state space or recent development in the spirit of [3]. On the one hand, this simple algorithm thus follows in a straightforward way from the recent results in the literature (although to the best of our knowledge it has not been presented as such yet). On the other hand, the required number of samples using these bounds is prohibitively large, and therefore, giving guarantees with such analysis is not feasible at all in practice. In fact, the numbers are astronomic already for Markov chains with a handful of states [13]. We discuss further drawbacks of such a na¨ıve solution in Sect. 3. Our main contribution in this paper is a practical algorithm. It takes the most promising actions from every state and uses the on-demand value iteration [2], not even requiring an exhaustive exploration of the entire MDP. Using techniques of [3,13], we can show that the partial model captures enough information. Most importantly, instead of using [12,33], the PAC bounds are derived directly from the concrete confidence intervals, reflecting the width of each interval and the topology of the model, in the spirit of the practical SMC for reachability [3].

Our contribution can be summarized as follows:


**Related Work.** SMC of unbounded-horizon properties of MDPs was first considered in [23,29] for reachability. [20] gives a model-free algorithm for ω-regular properties, which is convergent but provides no bounds on the current error. Several approaches provide SMC for MDPs and unbounded-horizon properties with PAC guarantees. Firstly, the algorithm of [18] requires (1) the mixing time T of the MDP (2) the ability to restart simulations also in non-initial states (3) visiting all states sufficiently many times, and thus (4) the knowledge of the size of the state space |S|. Secondly, [9], based on delayed Q-learning [34], lifts the assumptions (2) and (3) and instead of (1) requires only (a bound on) the minimum transition probability pmin. Thirdly, [3] additionally lifts the assumption (4), keeping only pmin, as in this paper. In [13], it is argued that while unboundedhorizon properties cannot be analysed without any information on the system, knowledge of (a lower bound on) the minimum transition probability pmin is a relatively light and realistic assumption in many scenarios, compared to the knowledge of the whole topology. In this paper, we thus adopt this assumption.

In contrast to SMC that uses possibly more (re-started) runs of the system, there are online learning approaches, where the desired behaviour is learnt for the single run. Model-based learning algorithms for mean payoff have been designed both for minimizing regret [4,36] as well as for PAC online learning [25,26].

Due to lack of space, the proofs and some more experimental results and discussions appear in [1].

#### **2 Preliminaries**

A - probability distribution on a finite set X is a mapping ρ : X -→ [0, 1], such that <sup>x</sup>∈<sup>X</sup> <sup>ρ</sup>(x) = 1. We denote by <sup>D</sup>(X) the set of probability distributions on <sup>X</sup>.

**Definition 1. (MDP).** A Markov decision process is a tuple of the form M = (S, s*init*,Act,Av,T, r), where <sup>S</sup> is a finite set of states, <sup>s</sup>*init* <sup>∈</sup> <sup>S</sup> is the initial state, Act is a finite set of actions, Av : <sup>S</sup> <sup>→</sup> <sup>2</sup>Act assigns to every state a set of available actions, <sup>T</sup> : <sup>S</sup> <sup>×</sup> Act → D(S) is a transition function that given a state s and an action a ∈ Av(s) yields a probability distribution over successor states, and <sup>r</sup> : <sup>S</sup> <sup>→</sup> <sup>R</sup>≥<sup>0</sup> is a reward function, assigning rewards to states.

For ease of notation, we write T(s, a, t) instead of T(s, a)(t). We denote by Post(s, a), the set of states that can be reached from s through action a. Formally, Post(s, a) = {<sup>t</sup> <sup>|</sup> <sup>T</sup>(s, a, t) <sup>&</sup>gt; <sup>0</sup>}.

The choices of actions are resolved by strategies, generally taking history into account and possibly randomizing. However, for mean payoff it is sufficient to consider positional strategies of the form π : S → Act. The semantics of an MDP with an initial state sinit is given in terms of each strategy σ inducing a Markov chain <sup>M</sup><sup>σ</sup> <sup>s</sup>init with the respective probability space and unique probability measure PM<sup>σ</sup> <sup>s</sup>init , and the expected value EM<sup>σ</sup> <sup>s</sup>init [F] of a random variable F (see e.g. [6]). We drop <sup>M</sup><sup>σ</sup> <sup>s</sup>init when it is clear from the context.

End Components An end-component (EC) M = (T,A), with ∅ = T ⊆ S and <sup>A</sup> : <sup>T</sup> <sup>→</sup> <sup>2</sup>Act of an MDP <sup>M</sup> is a sub-MDP of <sup>M</sup> such that: for all <sup>s</sup> <sup>∈</sup> <sup>T</sup>, we have that A(s) is a subset of the actions available from s; for all a ∈ A(s), we have Post(s, a) ⊆ T; and, it's underlying graph is strongly connected. A maximal end-component (MEC) is an EC that is not included in any other EC. Given an MDP M, the set of its MECs is denoted by MEC(M). For MEC(M) = {(T1, A1),...,(Tn, An)}, we define MEC<sup>S</sup> <sup>=</sup> <sup>n</sup> <sup>i</sup>=1 T<sup>i</sup> as the set of all states contained in some MEC.

**Definition 2. (continuous-time MDP (CTMDP)).** A continuous-time Markov decision process is a tuple of the form M = (S, s*init*,Act,Av, R, r), where S is a finite set of states, s*init* ∈ S is the initial state, Act is a finite set of actions, Av : <sup>S</sup> <sup>→</sup> <sup>2</sup>Act assigns to every state a set of available actions, <sup>R</sup> : <sup>S</sup> <sup>×</sup> Act <sup>×</sup> <sup>S</sup> <sup>→</sup> <sup>R</sup>≥<sup>0</sup> is a transition rate matrix that given a state <sup>s</sup> and an action a ∈ Av(s) defines the set of successors t of s on action a if R(s, a, t) > 0, and <sup>r</sup> : <sup>S</sup> <sup>→</sup> <sup>R</sup>≥<sup>0</sup> is a reward rate function, assigning a reward function to a state denoting the reward obtained for spending unit time in s.

A strategy in a CTMDP decides immediately after entering a state which action needs to be chosen from the current state. For a given state s ∈ S, and an action a ∈ Av(s), we denote by λ(s, a) = - <sup>t</sup> R(s, a, t) > 0 the exit rate of a in s. The residence time for action a in s is exponentially distributed with mean 1 <sup>λ</sup>(s,a) . An equivalent way of looking at CTMDP is that in state s, we wait for a time which is exponentially distributed with mean λ(s, a), and then with probability Δ(s, a, t) = R(s, a, t)/λ(s, a), we make a transition to state t. The reward accumulated for spending time t in s is r(s) · t.

Uniformization. A uniform CTMDP has a constant exit rate C for all stateaction pairs i.e., λ(s, a) = C for all states s ∈ S and actions a ∈ Av(s). The procedure of converting a non-uniform CTMDP into a uniform one is called uniformization. Consider a non-uniform CTMDP <sup>M</sup>. Let <sup>C</sup> <sup>∈</sup> <sup>R</sup>≥<sup>0</sup> such that C λ(s, a) for all s ∈ S and a ∈ Act. We can obtain a uniform CTMDP M<sup>C</sup> by assigning the new rates.

$$\mathsf{R}'(s,a,t) = \begin{cases} \mathsf{R}(s,a,t) & \text{if } s \neq t \\ \mathsf{R}(s,a,t) + C - \lambda(s,a) & \text{if } s = t \end{cases} \tag{1}$$

For every action a ∈ Av(s) from each state s in the new CTMDP we have a self loop if λ(s, a) < C. Due to a constant transition rate, the mean interval time between two any two actions is constant.

Mean Payoff. In this work, we consider the (maximum) mean payoff (or longrun average reward) of an MDP M, which intuitively describes the (maximum) average reward per step we expect to see when simulating the MDP for time going to infinity. Formally, let Si, Ai, R<sup>i</sup> be random variables giving the state visited, action played, and reward obtained in step i, and for CTMDP, T<sup>i</sup> the time spent in the state appearing in step i. For MDP, R<sup>i</sup> := r(Si), whereas for CTMDP, R<sup>i</sup> := r(Si)·Ti; consequently, for a CTMDP and a strategy π, we have Eπ <sup>s</sup> (Ri) = <sup>r</sup>(Si) <sup>λ</sup>(Si,Ai) .

Thus given a strategy π, the n-step average reward is

$$v\_n^{\pi}(s) := \mathbb{E}\_s^{\pi} \left( \frac{1}{n} \sum\_{i=0}^{n-1} R\_i \right) = \frac{1}{n} \sum\_{i=0}^{n-1} \frac{r(S\_i)}{\lambda(S\_i, A\_i)}.$$

with the latter equality holding for CTMDP. For both MDP and CTMDP, the mean payoff is then

$$v(s) := \max\_{\pi} \liminf\_{n \to \infty} v\_n^{\pi},$$

where the maximum over all strategies can also be without loss of generality restricted to the set of positional strategies ΠMD. A well-known alternative characterization we use in this paper is

$$v(s) = \max\_{\pi \in \Pi^{\mathsf{M}}} \sum\_{M \in \mathsf{M} \mathsf{EC}(\mathcal{M})} \mathbb{P}\_s^{\pi} [\Diamond \Box M] \cdot v\_M,\tag{2}$$

where ♦ and respectively denote the standard LTL operators eventually and always respectively. Further, ♦M denotes the set of paths that eventually remain forever within M and v<sup>M</sup> is the unique value achievable in the (CT)MDP restricted to the MEC M. Note that v<sup>M</sup> does not depend on the initial state chosen for the restriction.

We consider algorithms that have a limited information about the MDP.

**Definition 3. (Blackbox and greybox).** An algorithm inputs an MDP or a CTMDP as blackbox if


imum transition probability.

When input as greybox, it additionally knows the number |Post(s, a)| of successors for each state s and action a. Note that the exact probabilities on the transitions in an MDP or the rates in a CTMDP are unknown for both blackbox and greybox learning settings.

<sup>1</sup> In contrast to practical setups in monitoring, our knowledge of the current state is complete, i.e., the previously visited states can be uniquely identified.

#### **3 Overview of Our Approach**

Since no solutions are available in the literature and our solution consists of multiple ingredients, we present it in multiple steps to ease the understanding. First, we describe a more na¨ıve solution and pinpoint its drawbacks. Second, we give an overview of a more sophisticated solution, eliminating the drawbacks. Third, we fill in its details in the subsequent sections. Besides, each of the three points is first discussed on discrete-time MDPs and then on continuous-time MDPs. The reason for this is twofold: the separation of concerns simplifies the presentation; and the algorithm for discrete-time MDP is equally important and deserves a standalone description.

#### **3.1 Na¨ıve Solution**

We start by suggesting a conceptually simple solution. We can learn mean payoff MP in an MDP M as follows:


Learning the model, i.e. the transition probabilities, can be done by observing the simulation runs and collecting, for each state-action pair (s, a), a statistics of which states occur right after playing a in s. The frequency of each successor t among all successors then estimates the transition probability T(s, a, t). This is the standard task of estimating the generalized Bernoulli variable (a fixed distribution over finitely many options) with confidence intervals. We stop simulating when each transition probability has a precise enough confidence interval (with εT P and δT P yielded by the robustness theorem from the desired overall precision).<sup>2</sup> The drawbacks are (D1: uniform importance) that even transitions with

<sup>2</sup> Several non-trivial questions are dealt with later on: how to resolve the action choices during simulations; when to stop each simulation run and start a new one; additionally, in the black-box setting, when do we know that all successors of each transition have been observed. In particular, the last one is fundamental for the applicability of the robustness theorem. While the literature typically assumes the greybox setting or even richer information, to allow for such an algorithm with PAC bounds, our approach only needs pmin.

little to no impact on the mean payoff have to be estimated precisely (with εT P and δT P ); and (D2: uniform precision required) that, even restricting our attention to "important" transitions, it may take a long time before the last one is estimated precisely (while others are already estimated overly precisely).

Subsequently, using standard algorithms the mean payoff MP can be computed precisely by linear programming [30] or precisely enough by value iteration [2]. The respective MP can then be estimated by the robustness theorem [8], which yields for a given maximum perturbation of transition probabilities (in our case, εT P /2) an upper bound on the respective perturbation of the mean payoff εMP /2. The drawbacks are (D3: uniform precision utilized) that more precise confidence intervals for transitions (obtained due to D2) are not utilized, only the maximum imprecision is taken into account; and (D4: a-priori bounds) that the theorem is extremely conservative. Indeed, it reflects neither the topology of the MDP nor how impactful each transition is and thus provides an a-priori bound, extremely loose compared to the possible values of mean payoff that can be actually obtained for concrete values within the confidence intervals. This is practically unusable beyond a handful of states even for Markov chains [13].

For CTMDP M, we additionally need to estimate the rates (see below how). Subsequently, we can uniformize the learnt CTMDP M . Mean payoff of the uniformized CTMDP is then equal to the mean payoff of its embedded MDP<sup>3</sup>. Hence, we can proceed as before but we also have to compute (i) confidence intervals for the rates from finitely many observations, and (ii) the required precision and confidence of these intervals so that the respective induced error on the mean payoff is not too large. Hence all the drawbacks are inherited and, additionally, also applied to the estimates of the rates. Besides, (D5: rates) while imprecisions of rates do not increase MP-imprecision too much, the bound obtained via uniformization and the robustness theorem is very loose. Indeed, imprecise rates are reflected as imprecise self-loops in the uniformization, which themselves do not have much impact on the mean payoff, but can increase the TP-imprecision and thus hugely the MP-imprecision from the robustness theorem.

Finally, note that for both types of MDP, (D6: not anytime) this na¨ıve algorithm is not an anytime algorithm<sup>4</sup> since it works with pre-computed εT P and δT P . Instead it returns the result with the input precision if given enough time; if not given enough time, it does not return anything (also, if given more time, it does not improve the precision).

#### **3.2 Improved Solution**

Now we modify the solution so that the drawbacks are eliminated. The main ideas are (i) to allow for differences in TP-imprecisions (εT P can vary over

<sup>3</sup> An embedded MDP of a CTMDP is obtained by considering for every state s, actions <sup>a</sup> <sup>∈</sup> Av(s), and transitions <sup>t</sup> <sup>∈</sup> Post(s, a), such that <sup>T</sup>(s, a, t) = <sup>Δ</sup>(s, a, t), and by disregarding the transition rate matrix.

<sup>4</sup> An anytime algorithm can, at every step, return the current estimate with its imprecision, and this bound converges to 0 in the limit.

transitions) and even deliberately ignore less important transitions and instead improve precision for transitions where more information is helpful the most; (ii) rather than using the a-priori robustness theorem, to utilize the precision of each transition to its maximum; and (iii) to give an anytime algorithm that reflects the current confidence intervals and, upon improving them, can efficiently improve the mean-payoff estimate without recomputing it from scratch. There are several ingredients used in our approach.

Firstly, [2] provides an anytime algorithm for approximating mean payoff in a fully known MDP. The algorithm is a version of value iteration, called on-demand, performing improvements (so called Bellman updates) of the meanpayoff estimate in each state. Moreover, the algorithm is simulation-based, performing the updates in the visited states, biasing towards states where a more precise estimate is helpful the most ("on demand"). This matches well our learning setting. However, the approach assumes precise knowledge of the transition probabilities and, even more importantly, heavily relies on the knowledge of MECs. Indeed, it decomposes the mean-payoff computation according to Eq. 2 into computing mean payoff within MECs and optimizing (weighted) reachability of the MECs (with weights being their mean payoffs). When the MECs are unknown, none of these two steps can be executed.

Secondly, [3] provides an efficient way of learning reachability probabilities (in the greybox and blackbox settings). Unfortunately, since it considers TPinconfidence to be the same for all transitions, causing different TP-imprecisions, the use of robustness theorem in [3] makes the algorithm used there practically unusable in many cases. On a positive note, the work identifies the notion of δT P -sure EC, which reflects how confident we are, based on the simulations so far, that a set of states is an EC. This notion will be crucial also in our algorithm.

Both approaches are based on "bounded value iteration", which computes at any moment of time both a lower and an upper bound on the value that we are approximating (mean payoff or reachability, respectively). This yields anytime algorithms with known imprecision, the latter—being a learning algorithm on an incompletely known MDP—only with some confidence. Note that the upper bound converges only because ECs are identified and either collapsed (in the former) or deflated [24] (in the latter), meaning their upper bounds are decreased in a particular way to ensure correctness.

Our algorithm on (discrete-time) MDP M performs, essentially, the following. It simulates M in a similar way as [3]. With each visit of each state, not only it updates the model (includes this transition and improves the estimate of the outgoing transition probabilities), but also updates the estimate of the mean payoff by a Bellman update. Besides, at every moment of time, the current model yields a hypothesis what the actual MECs of M are and the respective confidence. While we perform the Bellman updates on all visited states deemed transient, the states deemed to be in MECs are updated separately, like in [2]. However, in contrast to [2], where every MEC is fully known and can thus be collapsed, and in contrast to the "bounded" quotient of [3] (see Appendix A of [1]), we instead introduce a special action stay in each of its states, which simulates staying in the (not fully known) MEC and obtaining its mean-payoff estimate via reachability:

**Definition 4. (**stay**-augmented MDP).** Let <sup>M</sup> = (S, s*init*,Act,Av,T, r) be an MDP and l, u : MEC(M) → [0, 1] be real functions on MECs. We augment the stay action to M to obtain M = (S , s*init*,Act ,Av ,T , r ), where

$$\begin{array}{l} \mathsf{S} - \mathsf{S}' = \mathsf{S} \cup \{s\_{+}, s\_{-}, s\_{?}\},\\ \mathsf{A} - \mathsf{A} \mathsf{c}\mathsf{t}' = \mathsf{A} \mathsf{c}\mathsf{t} \cup \{\mathsf{stay}\},\\ \mathsf{I} - \mathsf{A} \mathsf{v}'(s) = \begin{cases} \mathsf{A}\mathsf{v}(s) & \text{for } s \in S \cup \bigcup \mathsf{M} \mathsf{EC}(\mathcal{M})\\ \mathsf{A}\mathsf{v}(s) \cup \{\mathsf{stay}\} & \text{for } s \in \bigcup \mathsf{M} \mathsf{EC}(\mathcal{M})\\ \{\mathsf{stay}\} & \text{for } s \in \{s\_{+}, s\_{-}, s\_{?}\} \end{cases} \\ \mathsf{I} - \mathsf{T}' \text{ } \mathsf{c}\mathsf{t}' \text{ ands } \mathsf{T} \text{ by } \mathsf{T}'(s, \mathsf{stay}) = \{s\_{+} \mapsto l(M), s\_{-} \mapsto 1 - u(M), s\_{?} \mapsto u(M) - 1 - r'(s\_{-}) \\ \mathsf{i} - \mathsf{i}'(M) \mathsf{j} \text{ on } s \in M \in \mathsf{MEC}(\mathcal{M}) \text{ and } \mathsf{b} \mathsf{v} \text{ } \mathsf{T}'(s, \mathsf{stay}, s) = 1 \text{ for } s \in \{s\_{+}, s\_{-}, s\_{?}\}.\\ \mathsf{i} - r' \text{ extends } r \text{ by } r'(s\_{+}) = r'(s\_{?}) = r'(s\_{-}) = 0. \mathsf{5} \end{array}$$

**Corollary 1.** If l, u are valid lower and upper bounds on the mean-payoff within MECs of <sup>M</sup> then max<sup>σ</sup> <sup>P</sup>M<sup>σ</sup> [♦{s+}] <sup>v</sup>(s*init*) maxσPM<sup>σ</sup> [♦{s+, s?}] <sup>6</sup> where, max<sup>σ</sup> PM<sup>σ</sup> [♦S] gives the maximum probability of reaching some state in S over all strategies.

This turns the problem into reachability, and thus allows for deflating (defined for reachability in [3]) and an algorithm combining [3] and [2]. The details are explained in the next section. To summarize (D1) and (D2) are eliminated by not requiring uniform TP-imprecisions; (D3) and (D4) are eliminated via updating lower and upper bounds (using deflating) instead of using the robustness theorem.

Concerning CTMDP, in Sect. 5 we develop a confidence interval computation for the rates. Further, we design an algorithm deriving the MP-imprecision resulting from the rate imprecisions, that acts directly on the CTMDP and not on the embedded MDP of the uniformization. This effectively removes (D5).

#### **4 Algorithm for Discrete-Time MDP**

Now that we explained the difficulties of a na¨ıve approach, and the concepts from literature together with novel ideas to overcome them, we describe the actual algorithm for the discrete-time setting. Following a general outline of the algorithm, we give detailed explanations behind the components and provide the statistical guarantees the algorithm gives. Detailed pseudocode of the algorithms for this section is provided in Appendix B of [1].

Overall Algorithm and Details. Our version of an on-demand value iteration for mean payoff in black-box MDP is outlined in Algorithm 1. Initially, the input MDP M is augmented with terminal states ({s+, s−, s?}) to obtain the

<sup>5</sup> A higher transition probability to s<sup>+</sup> indicates that the MEC has high value, a higher transition probability to s? indicates high uncertainty in the value of the MEC, while

a higher transition probability to <sup>s</sup><sup>−</sup> indicates that the MEC has low value. <sup>6</sup> For simplicity of the presentation, we assume the rewards are between 0 and 1, for all states. If they are not, we can always rescale them to [0,1] by dividing them by the maximum reward observed so far and correspondingly adjust <sup>T</sup>(·,stay, ·).

stay-augmented MDP M . We learn a stay-augmented MDP M = (S , sinit, Act ,Av , T , r ) by collecting samples through several simulation runs (Lines 5-8). Over the course of the algorithm, we identify MECs with δT P confidence (Line 13) and gradually increase precision on their respective values (Lines 9-11). As stated earlier, these simulations are biased towards actions that lead to MECs potentially having higher rewards. Values for MECs are encoded using the stay action (Line 12) and propagated throughout the model using bounded value iteration (Lines 14- 19). In Line 14, we reinitialize the values of the states in the partial model since new MECs may be identified and also existing MECs may change. Finally, we claim that the probability estimates T are correct with confidence δMP and if the bounds on the value are precise enough, we terminate the algorithm. Otherwise, we repeat this overall process with improved bounds (Line 20).

Simulation. The SIMULATE function simulates a run over the input blackbox MDP M and returns the visited states in order. The simulation of M is executed by simulating M together with a random choice if action stay is taken. Consequently, a simulation starts from sinit and ends at one of the terminal states ({s+, s−, s?}). During simulation, we enhance our estimate of M by visiting new states, exploring new actions and improving our estimate of T with more samples. When states are visited for the first time, actions are chosen at random, and subsequently, actions with a higher potential reward are chosen. If a simulation is stuck in a loop, we check for the presence of an MEC with δT P confidence. If a δT P -sure MEC is found, we add a stay action with l, u = 0, 1, otherwise we keep simulating until the required confidence is achieved. After that, we take the action with the highest upper bound that is leaving the MEC to continue the simulation. We do several such simulations to build a large enough model before doing value iteration in the next steps.

Estimating Transition Probabilities. [3] gives an analysis to estimate bounds on transition probabilities for reachability objective in MDPs. For completeness, we briefly restate it here. Given an MP-inconfidence δMP , we distribute the inconfidence over all individual transitions as

$$
\delta\_{TP} := \frac{\delta\_{MP} \cdot p\_{\text{min}}}{\left| \{ a \vert \mathbf{s} \in \mathbf{S}' \land a \in \mathsf{Av}'(s) \} \right|},
$$

where <sup>1</sup> <sup>p</sup>min gives an upper bound on the maximum number of possible successors for an available action from a state<sup>7</sup>. The Hoeffding's inequality gives us a bound on the number of times an action a needs to be sampled from state s, denoted #(s, a), to achieve a TP-imprecision εT P - ln δT P <sup>−</sup>2#(s, a) on <sup>T</sup>(s, a, t), such that

$$\widehat{\mathbb{T}}(s, a, t) := \max(0, \frac{\#(s, a, t)}{\#(s, a)} - \varepsilon\_{TP})$$

<sup>7</sup> Knowing additionally max<sup>s</sup>∈S,a∈Av(s) <sup>|</sup>Post(s, a)<sup>|</sup> gives slightly smaller TPimprecision. See Appendix G.4 in [1].


**Input:** MDP M, imprecision εMP > 0, MP-inconfidence δMP > 0, lower bound pmin on transition probabilities in M **Parameters:** revisit threshold k ≥ 2, episode length n ≥ 1 **Output:** upon termination εMP -precise estimate of the maximum mean payoff for M with confidence 1 − δMP , i.e. (εMP , 1 − δMP )-PAC estimate 1: **procedure** ON DEMAND BVI *//Initialization* 2: Set L(s+) = U(s+) = U(s?) = 1, L(s−) = U(s−) = L(s?)=0 Augmentation 3: S = ∅ States of learnt model 4: **repeat** *//Get* n *simulation runs and update MP of MECs where they end up* 5: **for** n times **do** 6: w ← SIMULATE(k) Path taken by the simulation 7: S ← S ∪ w Add states to the model 8: <sup>δ</sup>T P <sup>←</sup> <sup>δ</sup>MP ·pmin |{a| s∈S-∧a∈Av-(s)}| Split inconfidence among all transitions 9: **if** last state of w is s<sup>+</sup> or s? **then** Probably entered a good MEC M 10: M ← MEC from which we entered the last state of w 11: UPDATE MEC VALUE(M) Increase precision using more VI 12: Update T (s,stay) according to Definition 4 for all s ∈ M *//Identify* δT P *-sure MECs and propagate their MP by VI for reachability* 13: *ProbableMECs* ← FIND MECS δT P -sure MECs 14: INITIALIZE VI BOUNDS Reinitialize L, U for all states 15: **repeat** 16: UPDATE(S ) One Bellman update per state 17: **for** T ∈ *ProbableMECs* **do** 18: DEFLATE(T) Ensure safe but converging U 19: **until** L and U close to their respective fixpoints 20: **until** U(sinit) - L(sinit) < <sup>2</sup>εMP <sup>r</sup>max εMP is the absolute error; we use "<sup>&</sup>lt; <sup>2</sup>εMP <sup>r</sup>max " for relative difference between upper and lower values, where <sup>r</sup>max = max <sup>s</sup>∈Sr(s).

where, #(s, a, t) is the number of times t is sampled when action a is chosen in s.

Updating mean-payoff values Using <sup>T</sup>(s, a, t), we compute estimates of the upper and lower bounds of the values corresponding to every action from a state visited in the partial model that is constructed so far. We use the following modified Bellman Eq. [3]:

$$\begin{aligned} \widehat{\mathsf{L}}(s,a) &:= \sum\_{t:\#(s,a,t)>0} \widehat{\mathsf{T}}(s,a,t) \cdot \mathsf{L}(t) \\ \widehat{\mathsf{U}}(s,a) &:= \sum\_{t:\#(s,a,t)>0} \widehat{\mathsf{T}}(s,a,t) \cdot \mathsf{U}(t) + \left(1 - \sum\_{t:\#(s,a,t)>0} \widehat{\mathsf{T}}(s,a,t)\right), \end{aligned}$$

where <sup>L</sup>(t) = max <sup>a</sup>∈Av(t) L (t, a) and <sup>U</sup>(t) = max <sup>a</sup>∈Av(t) <sup>U</sup>(t, a) are bounds on the value of from a state, v(s). When a state is discovered for the first time during the simulation, and is added to the partial model, we initialize L(s), and U(s) to 0, and 1, respectively. Note that - t:#(s,a,t)>0 <sup>T</sup>(s, a, t) <sup>&</sup>lt; 1. We attribute the remaining probability to unseen successors and assume their value to be 0 (1) to safely

under-(over-)approximate the lower (upper) bounds. We call these blackbox Bellman update equations, since it assumes that all the successors of a state-action pair may not have been visited.

Estimating Values of End-Components. End-components are identified with an inconfidence of δT P . As observed in [13], assuming an action has been sampled n times, the probability of missing a transition for that action is at most (1 − pmin)<sup>n</sup>. Thus, for identifying (T,A) as a δT P -sure MEC, every action in A that is available from a state <sup>s</sup> <sup>∈</sup> <sup>T</sup> needs to be sampled at least ln <sup>δ</sup>T P ln(1−pmin) times.

Once a δT P -sure MEC M is identified, we estimate its upper (v<sup>u</sup> <sup>M</sup>) and lower (v<sup>l</sup> <sup>M</sup>) bounds using value iteration.<sup>8</sup> While running value iteration, we assume, with a small inconfidence, that there are no unseen outgoing transitions. So we use the following modified Bellman update equations inside the MEC where we under-(over-)approximate the lower(upper) bound to a much lesser degree.

$$\widehat{\mathsf{L}}(s,a) := \sum\_{t:\#(s,a,t)>0} \widehat{\mathsf{T}}(s,a,t) \cdot \mathsf{L}(t) + \min\_{t:\#(s,a,t)>0} \mathsf{L}(t) \cdot (1 - \sum\_{t:\#(s,a,t)>0} \widehat{\mathsf{T}}(s,a,t))$$

$$\widehat{\mathsf{U}}(s,a) := \sum\_{t:\#\{s,a,t\}>0} \widehat{\mathsf{T}}(s,a,t) \cdot \mathsf{U}(t) + \max\_{t:\#\{s,a,t\}>0} \mathsf{U}(t) \cdot \left(1 - \sum\_{t:\#\{s,a,t\}>0} \widehat{\mathsf{T}}(s,a,t)\right)$$

Following the assumption, we call these greybox (See Definition 3) Bellman update equations. The value iteration algorithm further gives us bounds on v<sup>u</sup> M and v<sup>l</sup> <sup>M</sup>. We say that the upper estimate of v<sup>u</sup> <sup>M</sup> (v<sup>u</sup> <sup>M</sup>) and the lower estimate of v<sup>l</sup> <sup>M</sup> (v<sup>l</sup> <sup>M</sup>) are the overall upper and lower bounds of the mean-payoff value of M, respectively. To converge the overall bounds, we need value iteration to return more precise estimates of v<sup>l</sup> <sup>M</sup> and v<sup>u</sup> <sup>M</sup>, and we need to sample the actions inside M many times to reduce the difference between v<sup>l</sup> <sup>M</sup> and v<sup>u</sup> <sup>M</sup>. We call this procedure, UPDATE MEC VALUE.

Now, some MECs may have very low values or may not be reachable from sinit with high probability. In such cases, no optimal strategy may visit these MECs, and it might not be efficient to obtain very precise mean-payoff values for every MEC that is identified in an MDP. We follow the on-demand heuristic [2] where we progressively increase the precision on mean-payoff values as an MEC seems more likely to be a part of an optimal strategy. The stay action on MECs helps in guiding simulation towards those MECs that have a higher lower bound of the mean-payoff value. In particular, whenever the simulation ends up in s<sup>+</sup> or s?, we run UPDATE MEC VALUE with higher precision on the MEC that led to these states. If the simulation ends up in these states through a particular MEC more often, it indicates that the MEC is likely to be a part of an optimal strategy, and it would be worth increasing the precision on its mean-payoff value.

<sup>8</sup> Note that one requires the ECs to be aperiodic for the VI to converge. [30] suggests a way that deals with this.

Deflate Operation. Unlike in the case of computation of mean payoff for whitebox models [3] where a MEC is collapsed following the computation of its value, for blackbox learning, once a set of states is identified as a δT P -sure MEC, we cannot collapse them. This is because collapsing would prevent a proper future analysis of those states, which is undesirable in a blackbox setting. However, this leads to other problems. To illustrate this, we consider an MDP that only has a single MEC M and one outgoing action from every individual state. Recall from Eq. 2 that we compute the mean-payoff by reducing it to a reachability problem. Once the mean-payoff for the MEC, and the probabilities corresponding to stay action in Line 12 are computed, to compute the reachability probability, the upper and lower bounds of all states in the MECs are initialized to 1 and 0 respectively. Now suppose that the sum of probabilities to s<sup>+</sup> and s? be p denoting the upper bound on the value of the mean-payoff to be p·rmax. Clearly, the upper bound on the reachability value of this MDP is p. Now, when we do BVI to calculate this value, from every state in M, there would be at least two action choices, one that stays inside the MEC, and one that corresponds to the stay action. Initially, all states, except the terminal states, would have upper and lower values set to 0 and 1, respectively. Thus, among the two action choices, one would have upper value p, while the other would have upper value 1, and hence, the Bellman update assigns the upper value of the state to 1. As one can see, this would go on, and convergence wouldn't happen, and hence the true mean-payoff value will not be propagated to the initial state of the MDP. To avoid this, we need the deflate operation which lowers the upper reachability value to the best outgoing action, i.e. in this case, the stay action with value p.

Statistical Guarantees. The following theorem shows that the mean-payoff value learnt by Algorithm 1 is PAC on an input blackbox MDP.

**Theorem 1.** Algorithm 1 has the property that when it stops, it returns an interval for the mean-payoff value of the MDP that is PAC for the given MPinconfidence δMP and the MP-imprecision εMP .

Anytime Algorithm. As a direct consequence, we obtain an anytime algorithm from Algorithm 1 by (1) dropping the termination test on Line 20, i.e. replacing it with **until false**, and (2) upon query (or termination) by the user, we output (U(sinit) + L(sinit))/2 as the estimate and, additionally, we output (U(sinit) - L(sinit))/2 as the current imprecision.

Using Greybox Update Equations During Blackbox Learning. We also consider the variant where we use greybox update equations to estimate the mean-payoff values. However, assuming we keep the TP-imprecision unchanged, the overall TP-inconfidence now has to include the probability of missing some successor of a state s for an action a<sup>9</sup>. Given a number of samples #(s, a), the probability that we miss a particular successor is at most (1 <sup>−</sup> <sup>p</sup>min)#(s,a), and hence the

<sup>9</sup> Assuming #(s, a) to be as small as 200, and pmin = 0.05, the probability of missing a transition is 3.<sup>5</sup> · <sup>10</sup>−<sup>5</sup>.

overall TP-inconfidence corresponding to using greybox equations for blackbox learning increases to <sup>δ</sup>T P + (1 <sup>−</sup> <sup>p</sup>min)#(s,a) .

We also note that the use of greybox update equations on estimating the transition probabilities also gives us a PAC guarantee but with an increased MP-Inconfidence resulting from an increased TP-inconfidence.

#### **5 Algorithm for Continuous-Time MDP**

In this section, we describe an algorithm to learn blackbox CTMDP models for mean-payoff objective while respecting the PAC guarantees. As in the case of MDPs, we reduce the mean-payoff problem to a reachability problem. We follow the same overall framework as in MDPs, where we compute the probability to reach the end-components under an optimal strategy, and we compute their respective mean-payoff values. Computing reachability probabilities in a CTMDP is the same as computing reachability probabilities in the underlying embedded MDP. Similar to estimating T(s, a, t) in Sect. 4 for MDPs, we estimate Δ(s, a, t)<sup>10</sup> for CTMDPs, and follow the simulation-based procedure in Algorithm 1 to compute reachability probabilities. However, unlike MECs in MDPs, where the mean-payoff value depends solely on the transition probabilities, the mean-payoff value in a CTMDP also depends on the rates λ(s, a) for s ∈ T and a ∈ A(s) for an MEC M = (T,A). Thus to compute the value of an MEC, we also estimate the rates of the state-action pairs. Once we get the estimates of the rates, we uniformize the CTMDP to obtain a uniform CTMDP that can be treated as an MDP by disregarding the rates while preserving the mean-payoff value [30]. Detailed pseudocode of the algorithms for this section are provided in Appendix F of [1].

Estimating Rates. Recall that for an action a, the time spent in s is exponentially distributed with a parameter λ(s, a), and <sup>1</sup> <sup>λ</sup>(s,a) is the mean of this distribution. During the simulation of a CTMDP, for every state s reached and action a chosen from s, we construct a sequence τs,a of the time difference between the entry and the corresponding exit from s when action a is chosen. Then, the average over the sequence τs,a gives us an estimate <sup>1</sup> λ-(s,a) of <sup>1</sup> <sup>λ</sup>(s,a) (Abbreviated to <sup>1</sup> <sup>λ</sup> from now on when (s, a) is clear from the context.).

Assuming a multiplicative error α<sup>R</sup> on our estimates of <sup>1</sup> <sup>λ</sup> , the lemma below uses Chernoff bounds<sup>11</sup> to give the number of samples that need to be collected from an exponential distribution so that the estimated mean <sup>1</sup> λ is at most αRfraction away from the actual mean <sup>1</sup> <sup>λ</sup> with probability at least 1 − δR, where αR, δ<sup>R</sup> ∈ (0, 1). Further by Cramer's theorem [15], it follows that this is the tightest possible bound for the number of samples collected.

<sup>10</sup> Recall that an estimate of Δ(s, a, t) is the ratio between #(s, a, t) and #(s, a), and is the probability with which we go to state t from s when action a is chosen from s.

<sup>11</sup> Since λ is not bounded, we cannot use Hoeffding's inequality as in the case of estimating the transition probabilities.

**Lemma 1.** Let X1,...,X<sup>n</sup> be exponentially distributed i.i.d. random variables with mean <sup>1</sup> <sup>λ</sup> . Then we have that

$$\begin{split} \mathbb{P}\left[\left|\frac{1}{\widehat{\lambda}} - \frac{1}{\lambda} > \frac{1}{\lambda} \cdot \alpha\_R\right|\right] &\leqslant \inf\_{-\lambda < t < 0} \left(\frac{\lambda}{\lambda + t}\right)^n \cdot e^{\frac{tn}{\lambda}(1 + \alpha\_R)} + \inf\_{t > 0} \left(\frac{\lambda}{\lambda + t}\right)^n \cdot e^{\frac{tn}{\lambda}(1 - \alpha\_R)}, \\ \text{where } \frac{1}{n} \sum\_{i=1}^n X\_i = \frac{1}{\lambda}. \end{split}$$

Assuming the right-side of the inequality is at most <sup>δ</sup>R, we have that <sup>λ</sup> <sup>∈</sup> [λˆ(1 <sup>−</sup> <sup>α</sup>R), <sup>λ</sup>ˆ(1 + <sup>α</sup>R)], or <sup>λ</sup> <sup>∈</sup> [ <sup>λ</sup> 1+α<sup>R</sup> , <sup>λ</sup> <sup>1</sup>−α<sup>R</sup> ] with probability at least 1 <sup>−</sup> <sup>δ</sup>R. Table <sup>1</sup> shows the number of samples required for various values of α<sup>R</sup> and δ<sup>R</sup> 12.

**Table 1.** Lookup table for number of samples based on α<sup>R</sup> and δ<sup>R</sup>


Given a maximum multiplicative error α<sup>R</sup> on the mean of the exponential distributions of the state-action pairs in a CTMDP, we say that the rate λ is known <sup>α</sup>R-precisely if <sup>λ</sup> <sup>∈</sup> [ <sup>λ</sup> 1+α<sup>R</sup> , <sup>λ</sup> <sup>1</sup>−α<sup>R</sup> ]. We now quantify the bounds on the estimated mean-payoff value. Let M be a CTMDP, v<sup>M</sup> be its actual mean-payoff value, and let <sup>v</sup><sup>M</sup> denote its mean-payoff when the rates of the state-action pairs are known αR-precisely. Then we have the following.

**Lemma 2.** Given a CTMDP M with rates known αR-precisely, with transition probabilities known precisely, and with maximum reward per unit time over all states <sup>r</sup>max, we have <sup>v</sup>M( <sup>1</sup>−α<sup>R</sup> 1+α<sup>R</sup> ) <sup>≤</sup> <sup>v</sup><sup>M</sup> <sup>≤</sup> <sup>v</sup>M( 1+α<sup>R</sup> <sup>1</sup>−α<sup>R</sup> ) and <sup>|</sup>v<sup>M</sup> <sup>−</sup> <sup>v</sup>M| ≤ rmax <sup>2</sup>α<sup>R</sup> <sup>1</sup>−α<sup>R</sup> .

Estimating Mean-Payoff Values of MECs. Using our bounds on the rates of the transitions, we now compute bounds on the mean-payoff values of MECs in CTMDPs. We first show that the mean payoff is maximized or minimized at the boundaries of the estimates of the rates. Intuitively, to maximise the mean-payoff value, for a state s<sup>i</sup> with a high reward, we would like to maximise the time spent in s<sup>i</sup> or equivalently, minimise the rate λ(si, a) for every outgoing action a from si. We do the opposite when we want to find a lower bound on the mean-payoff value in the MEC. Consider an MEC M having states T = {s1, ..., sm}. Assume that λ<sup>i</sup> is the rate of an action a from state si, such that a positional mean-payoff maximizing strategy σ chooses a from si. Then, the expected mean-payoff value of M is given by,

<sup>12</sup> In Appendix E of [1], we show the computation of the number of samples for one of the entries.

$$v\_M = \frac{\sum\_{s\_i \in T} \frac{r(s\_i)\pi\_i}{\lambda\_i}}{\sum\_{s\_i \in T} \frac{\pi\_i}{\lambda\_i}},\tag{3}$$

where π<sup>i</sup> denotes the expected fraction of total time spent in s<sup>i</sup> under σ.

Now, we have estimates <sup>1</sup> λi of <sup>1</sup> <sup>λ</sup> , such that, λ<sup>i</sup> ∈ <sup>λ</sup><sup>i</sup> (1 <sup>−</sup> <sup>α</sup>R), <sup>λ</sup><sup>i</sup> (1 + <sup>α</sup>R) with high probability. Let λ<sup>l</sup> <sup>i</sup> <sup>=</sup> <sup>λ</sup><sup>i</sup> (1 <sup>−</sup> <sup>α</sup>R) and <sup>λ</sup><sup>u</sup> <sup>i</sup> <sup>=</sup> <sup>λ</sup><sup>i</sup> (1 + <sup>α</sup>R).

**Proposition 1.** In Eq. 3, the maximum and the minimum values of v<sup>M</sup> occur at the boundaries of the estimates of λ<sup>i</sup> for each 1 i m.

In particular, v<sup>M</sup> is maximized when,

$$\lambda\_i = \begin{cases} \lambda\_i^l, & \text{if } r(s\_i) \ge v\_M \\ \lambda\_i^u, & \text{otherwise} \end{cases} \tag{4}$$

Once we fix the rates for each of the states in M, we uniformize M to obtain a uniform CTMDP M<sup>C</sup> which is an MEC and can be treated as an MDP for computing its mean-payoff value [30]. Let for a state-action pair, the rate be λ(s, a), and the uniformization constant be C. For a successor t from s under action <sup>a</sup> such that <sup>t</sup> <sup>=</sup> <sup>s</sup>, we have <sup>Δ</sup>(s, a, t) = #(s,a,t) #(s,a) · <sup>λ</sup>(s,a) <sup>C</sup> , and Δ(s, a, s) = 1 − - t=s Δ(s, a, t). Finally, value iteration on M<sup>C</sup> with appropriate confidence width gives us the lower and the upper estimates of the mean-payoff value of the MEC M.

We now describe an iterative procedure to identify those states of the MEC for which the upper bound on the estimates of the rates are assigned, and those states for which the lower bound on the estimates of the rates are assigned in order to maximize or minimize the mean-payoff value of the MEC. Assume w.l.o.g. that the states s1,...,s<sup>m</sup> are sorted in decreasing order of their rewards r(si). In iteration j, we set λ<sup>i</sup> = λ<sup>l</sup> <sup>i</sup> for 1 i j, and we set λ<sup>i</sup> = λ<sup>u</sup> <sup>i</sup> for the remaining states and recompute vM. The maximum value of v<sup>M</sup> across all iterations gives the upper bound on vM. Similarly we can find the lower bound on <sup>v</sup>M. Overall, value iteration is done 2|T<sup>|</sup> times<sup>13</sup>.

Overall Algorithm. As stated in the beginning of this section, an algorithm for computing the mean payoff in blackbox CTMDP models largely follows the same overall framework as stated in Sect. 4. By sampling the actions, we obtain estimates of the rates and the transition probabilities. The reachability probabilities

<sup>13</sup> In our experiments, we use a heuristic to estimate v<sup>M</sup> that provides good approximate bounds and is more efficient. We first compute an initial estimate of <sup>v</sup>-<sup>M</sup> using our current estimates, <sup>λ</sup>-. We then compute the upper bound by assigning the rates as in Eq. <sup>4</sup> where <sup>v</sup><sup>M</sup> is replaced with <sup>v</sup>-<sup>M</sup>. Similarly, the lower bound can also be found. A detailed pseudocode of this algorithm is described in Algorithm 18 of [1].

to the MECs of the CTMDP are estimated using the estimates of the transition probabilities while the mean-payoff values of MECs are estimated using uniformization as decribed above. The confidence widths on the transition probabilities in a uniformized MEC are assigned based on the number of samples #(s, a) for a state-action pair (s, a).

Statistical Guarantees. Let δT P and δ<sup>R</sup> be the TP-inconfidence and the inconfidence on individual transition rates, respectively. Further, let δMP <sup>1</sup> and δMP <sup>2</sup> be the overall inconfidence on the transition probabilities and transition rates, respectively. Then, <sup>δ</sup>T P := <sup>δ</sup>MP <sup>1</sup> · <sup>p</sup>min |{a|<sup>s</sup> <sup>∈</sup> <sup>S</sup> <sup>∧</sup> <sup>a</sup> <sup>∈</sup> Av(s)}|, and <sup>δ</sup><sup>R</sup> :=

δMP <sup>2</sup> |{a|<sup>s</sup> <sup>∈</sup> <sup>S</sup> <sup>∧</sup> <sup>a</sup> <sup>∈</sup> Av(s)}|. Thus, we have that the overall inconfidence on the mean-payoff value, δMP = δMP <sup>1</sup> + δMP <sup>2</sup>. Thus, to achieve a given inconfidence on the mean-payoff value, we fix δT P and δR, and adjust the imprecisions εT P and α<sup>R</sup> accordingly.<sup>14</sup>

As in the case of MDPs, our learning algorithm for blackbox CTMDP models is an anytime algorithm that is PAC for the given MP-inconfidence δMP .

#### **6 Experimental Results**

We implemented our algorithms as an extension of Prism [27] and tested it on 15 MDP benchmarks and 10 CTMDP benchmarks. Several of these benchmarks were selected from the Quantitative Verification Benchmark Set [21] 15. The results for MDP and CTMDP blackbox learning are shown in Table 2 and Table 3 respectively. Here, we scale the upper and lower bounds to 1 and 0, and show the average values taken over 10 experiments. The experiments were run on a desktop machine with an Intel i5 3.2 GHz quad core processor and 16 GB RAM. The MP-imprecision εMP is set to 10−<sup>2</sup>, revisitThreshold k is set to 6, MP-inconfidence δMP is set to 0.1 and n is set to 10000. We further use a timeout of 30 minutes. In the case of a timeout, the reported upper and lower bounds on the mean payoff still correspond to the input MP-inconfidence δMP , although the MP-imprecision may not be the desired one.

Blackbox Learning for MDPs. We see that in Table 2 for blackbox learning, 9 out of 15 benchmarks converge well, such that the precision is within 0.1. In fact, for many of these 9 benchmarks, a precision of 0.1 is achieved much before the timeout (TO). In Fig. 1a and Fig. 1b, we show this for zeroconf and pacman. zeroconf has a large transient part and a lot of easily reachable single state

<sup>14</sup> See Appendix G of [1] for a more detailed calculation of the number of samples required to make transition probabilities and the rates precise.

<sup>15</sup> The CTMDP benchmarks are available as Markov automata models that were converted to CTMDP models using a tool developed in the thesis [11].


**Table 2.** Results on MDP benchmarks.

<sup>a</sup> The number of states and the values are computed using the probabilistic modelchecker Storm [14]

The number of states and the true mean-payoff values are computed by first uniformizing the CTMDP, and then using Storm on the underlying MDP.

MECs. Since it has a true value of 1, the upper and the lower values converge after exploring only a few MECs. Our algorithm only needed to explore a very small percentage of the states to attain the input precision. cs nfail has many significant MECs, and the learning algorithm needs to explore each of these MECs, while in sensor there is a relatively large MEC of around 30 states, and the simulation inside this MEC takes considerable amount of time.

virus consists of a single large MEC of more than 800 states, and its true value is 0. As we simulate the MEC more and more, the TP-imprecision on the transition probabilities decreases and the upper bound on the mean-payoff reduces over time. ij10 contains one MEC with 10 states in it. The value converges faster and reaches a value of 1, during blackbox learning. This model has relatively high number of actions, more than 5, for many of its states outside the MEC. This leads to a higher TP-imprecision. Further, due to the conservative nature of the blackbox update equations, the upper and the lower values converge very slowly.

consensus, ij10, ij3, pacman, wlan were used in [3] for learning policies for reachability objectives. The target states in these benchmarks are sink states with self loops, and we add a reward of 1 on these target states so that the rechability probability becomes the same as the mean payoff. The mean-payoff


**Table 3.** Results on CTMDP benchmarks

results we observe are similar to the bounds reported for reachability probability in [3], and our experiments also take similar time as reported in [3].

The blackjack model [35] is similar to zeroconf model. It has 3829 states and 2116 MECs. It has a large transient part and a lot of single state MECs. However, unlike zeroconf all of the MECs have a value of 0. Thus, simulation takes more time as the TP-imprecision reduces slowly.

Blackbox Learning with Greybox Update Equations. We show the results of these experiments in the right side of Table 2. As observed, convergence is much faster here for all the benchmarks. All our benchmarks converged correctly within a few seconds to a few minutes. Hence for a small degradation in MP-inconfidence use of greybox update equations works well in practice. We show the effect on MP-inconfidence in more detail in Table 8 in Appendix G of [1].

Blackbox Learning for CTMDPs. In Table 3 we show the results for CTMDP benchmarks. The number of states in these benchmarks vary from as low as 12 to more than 7000. All the models used here have a lot of small end-components. We observe that the upper and the lower values take more time to converge as the size of the model grows. Figure 1c and Fig. 1d show the convergence of lower and upper bounds for QueuingSystem and SJS3. As in the case of MDPs, using greybox update equations speeds up the learning process significantly.

Greybox Learning. Recall from Definition 3 that in greybox learning, for every state-action pair, we know the number of successors of the state for the given action. As expected, their convergence is much faster than that for blackbox learning, but the convergence is comparable to the case where we do blackbox learning with greybox update equations. The details of the greybox learning experiments can be found in Appendix G of [1].

**Fig. 1.** Convergence of lower and upper bounds for blackbox update equations and greybox update equations.

#### **7 Conclusion**

We presented the first PAC SMC algorithm for computing mean payoff in unknown MDPs and CTMDPs, where the only information needed is a lower bound on minimum transition probability, as advocated in [13]. In contrast to a naive algorithm, which follows in a quite straightforward way from the literature, our algorithm is practically applicable, overcoming the astronomic number of simulation steps required. To this end, in particular, the inconfidence had to be distributed in non-uniformly over the transitions and then imprecision propagated by value iteration with precision guarantees. In future, we would like to thoroughly analyse how well weakening the PAC bounds can be traded for a yet faster convergence. On the practical side, applying importance sampling and importance splitting could further improve the efficiency.

**Acknowledgements.** The second author would like to thank Subhajit Goswami for insightful discussions on learning transition rate matrix in a CTMDP and for pointing to useful references.

#### **References**

1. Agarwal, C., Guha, S., Pazhamalai, M., Kˇret´ınsk´y, J.: Pac statistical model checking of mean payoff in discrete- and continuous-time mdp (2022). CoRR, abs/2206.01465


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Sampling-Based Verification of CTMCs with Uncertain Rates**

Thom S. Badings1(B) , Nils Jansen<sup>1</sup> , Sebastian Junges<sup>1</sup> , Marielle Stoelinga1,2 , and Matthias Volk<sup>2</sup>

<sup>1</sup> Radboud University, Nijmegen, The Netherlands thom.badings@ru.nl <sup>2</sup> University of Twente, Enschede, The Netherlands

**Abstract.** We employ uncertain parametric CTMCs with parametric transition rates and a prior on the parameter values. The prior encodes uncertainty about the actual transition rates, while the parameters allow dependencies between transition rates. Sampling the parameter values from the prior distribution then yields a standard CTMC, for which we may compute relevant reachability probabilities. We provide a principled solution, based on a technique called scenario-optimization, to the following problem: From a finite set of parameter samples and a user-specified confidence level, compute prediction regions on the reachability probabilities. The prediction regions should (with high probability) contain the reachability probabilities of a CTMC induced by any additional sample. To boost the scalability of the approach, we employ standard abstraction techniques and adapt our methodology to support approximate reachability probabilities. Experiments with various well-known benchmarks show the applicability of the approach.

#### **1 Introduction**

Continuous-time Markov chains (CTMCs) are widely used to model complex probabilistic systems in reliability engineering [51], network processes [36,38], systems biology [11,23] and epidemic modeling [2]. A key verification task is to compute aspects of system behavior from these models, expressed as, e.g., continuous stochastic logic (CSL) formulae [4,7]. Typically, we compute reachability probabilities for a set of horizons, such as: *what is the probability that a target state is reached before time* t1,...,tn*?* Standard algorithms [7] implemented in mature model checking tools such as Storm [37] or Prism [42] provide efficient means to compute these *reachability probabilities*. However, these methods typically require that transition rates and probabilities are precisely known. This assumption is often unrealistic [34] and led to some related work, which we discuss in Sect. 7.

*Illustrative Example.* An epidemic can abstractly be modeled as a finite-state CTMC, e.g., the SIR (susceptible-infected-recovered) model [3], which is shown

c The Author(s) 2022

This work has been partially funded by NWO under the grant PrimaVera, number NWA.1160.18.238, and by EU Horizon 2020 project MISSION, number 101008233.

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 26–47, 2022. https://doi.org/10.1007/978-3-031-13188-2\_2

(a) pCTMC <sup>M</sup> with parameters <sup>λ</sup>i, λr. (b) Distribution P over values for (λi, λr).

**Fig. 1.** An upCTMC (M, <sup>P</sup>) for the SIR (pop=2) model.

**Fig. 2.** The probability of extinction in the SIR (140) model for horizons [100, t]. (Color figure online)

ent rates.

probabilities.

a single CTMC.

transition rates.

in Fig. 1a for a population of two. Such a CTMC assumes a *fixed set of transition rates*, in this case an infection rate λi, and a recovery rate λr. The outcome of analyzing this CTMC for fixed values of λ<sup>i</sup> and λ<sup>r</sup> may yield a *probability curve* like in Fig. 2a<sup>1</sup>, where we plot the probability (y-axis) of reaching a target state that corresponds to the epidemic becoming extinct against varying time horizons (x-axis). In fact, the plot is obtained via a smooth interpolation of the results at finitely many horizons, cf. Fig. 2b. To acknowledge that λi, λ<sup>r</sup> are in fact unknown, we may analyze the model for different values of λi, λr, resulting in a set of curves as in Fig. 2c. These individual curves, however, provide no guarantees about the shape of the curve obtained from another infection and recovery rate. Instead, we assume a *probability distribution* over the transition rates and aim to compute *prediction regions* as those in shown Fig. 2d, in such a way that with a certain (high) probability, any rates λ<sup>i</sup> and λ<sup>r</sup> yield a curve within this region.

*Overall Goal.* From the illustrative example, we state the following goal. Each fixed set of transition rates induces a *probability curve*, i.e., a mapping from horizons to the corresponding reachability probabilities. We aim to construct *prediction regions* around a set of probability curves, such that with high probability and high confidence, sampling a set of transition rates induces a probability curve within this region. Our key contribution is an efficient *probably approximately correct*, or PAC-style method that computes these prediction regions. The remainder of the introduction explores the technical steps toward this goal.

<sup>1</sup> For visual clarity, we plot the reachability probability *between* time 100 and t1,...,tn.

*Uncertain CTMCs.* The setting above is formally captured by parametric CTMCs (pCTMCs). Transition rates of pCTMCs are not given precisely but as (polynomials over) parameters [15,34], such as those shown in Fig. 1a. We assume a *prior* on each parameter valuation, i.e., assignment of values to parameters, similar to settings in [11,44] and in contrast to, e.g., [23,34]. These priors may result from asking different experts which value they would assume for, e.g., the infection rate. The prior may also be the result of Bayesian reasoning [56]. Formally, we capture the uncertainty in the rates by an arbitrary and potentially unknown *probability distribution* over the parameter space, see Fig. 1b. We call this model an *uncertain pCTMC (upCTMC)*. The distribution allows drawing independent and identically distributed (i.i.d.) *samples* that yield (parameter-free) CTMCs.

*Problem Statement.* We consider prediction regions on probability curves in the form of a pair of two curves that 'sandwich' the probability curves, as depicted in Fig. 2d. Intuitively, we then aim to find a prediction region R that is sufficiently large, such that sampling parameter valuations yields a probability curve in R with high probability p. We aim to compute a lower bound on this *containment probability* p. Naturally, we also aim to compute a meaningful, i.e. small (tight), prediction region R. As such, we aim to solve the following problem:

**Problem Statement.** Given a upCTMC with a target state, compute


*The Problem Solved.* In this paper, we present a method that samples probability curves as in Fig. 2c, but now for, say 100 curves. From these curves, we compute prediction regions (e.g., both tubes in Fig. 2d) and compute a lower bound (one for both tubes) on the containment probability that the curve associated with any sampled parameter value will lie in the specific prediction region (tube). Specifically, for a confidence level of 99% and considering 100 curves, we conclude that this lower bound is 79.4% for the red region and 7.5% for the blue region. For a higher confidence level of 99.9%, the lower bounds are slightly more conservative.

*A Change in Perspective.* Toward the algorithm, we make a change in perspective. For two horizons t<sup>1</sup> and t2, reachability probabilities for fixed CTMCs are two-dimensional points in [0, 1]<sup>2</sup> that we call *solution vectors*, as shown in Fig. 3a. Here, these solution vectors represent pairs of the probabilities that the disease becomes extinct before time t<sup>1</sup> and before t2. The prediction regions as in Fig. 2d are shown as the shaded boxes in Fig. 3a.

*Solving the problem algorithmically.* We solve the problem using a samplingbased approach. Starting with a set of solution vectors, we use techniques from *scenario optimization*, a data-driven methodology for solving stochastic optimization problems [18,21]. As such, we construct the prediction region from the solution to an optimization problem. Our method can balance the size of the prediction region with the containment probability, as illustrated by the two boxes in Fig. 3a.

(a) Reachability at time points t<sup>1</sup> and t2. (b) Pareto front for two measures.

**Fig. 3.** Prediction regions on the solutions vectors for two different upCTMCs.

*Extensions.* Our approach offers more than prediction regions on probability curves from precise samples. The change in perspective mentioned above allows for solution vectors that represent *multiple objectives*, such as the reachability with respect to different goal states, expected rewards or even the probability mass of paths satisfying more complex temporal properties. In our experiments, we show that this multi-objective approach —also on probability curves— yields much tighter bounds on the containment probability than an approach that analyzes each objective independently. We can also produce prediction regions as other shapes than boxes, as, for example, shown in Fig. 3b. To accelerate our approach, we significantly extend the methodology for dealing with *imprecise verification results*, given as an interval on each entry of the solution vector.

*Contributions.* Our key contribution is the approach that provides prediction regions and lower bounds on probability curves for upCTMCs. The approach requires only about 100 samples and scales to upCTMCs with tens of parameters. Furthermore: (1) We extend our approach such that we can also handle the case where only imprecise intervals on the verification results are available. (2) We develop a tailored batch verification method in the model checker Storm [37] to accelerate the required batches of verification tasks. We accompany our contributions by a thorough empirical evaluation and remark that our batch verification method can be used beyond scenario optimization. Our scenario optimization results are independent of the model checking and are, thus, applicable to any model where solution vectors are obtained in the same way as for upCTMCs.

*Data Availability.* All source code, benchmarks, and logfiles used to produce the data are archived: https://doi.org/10.5281/zenodo.6523863.

#### **2 Problem Statement**

In this section, we introduce pCTMCs and upCTMCs, and we define the formal problem statement. We use probability distributions over finite and infinite sets; see [9] for details. The set of all distributions over a set X is denoted by *Dist*(*X* ). The set of polynomials over parameters V , with rational coefficients, is denoted by <sup>Q</sup>[<sup>V</sup> ]. An *instantiation* <sup>u</sup>: <sup>V</sup> <sup>→</sup> <sup>Q</sup> maps parameters to concrete values. We often fix a parameter ordering and denote instantiations as vectors, <sup>u</sup> <sup>∈</sup> <sup>Q</sup>|<sup>V</sup> <sup>|</sup> .

**Definition 1 (pCTMC).** *<sup>A</sup>* pCTMC *is a tuple* <sup>M</sup> = (S, s*<sup>I</sup>* ,V, **<sup>R</sup>**)*, where* <sup>S</sup> *is a finite set of states,* <sup>s</sup>*<sup>I</sup>* <sup>∈</sup> *Dist*(*S*) *is the initial distribution,* <sup>V</sup> *are the (ordered) parameters, and* **<sup>R</sup>**: <sup>S</sup> <sup>×</sup> <sup>S</sup> <sup>→</sup> <sup>Q</sup>[<sup>V</sup> ] *is a parametric transition rate function. If* **<sup>R</sup>**(s, s) <sup>∈</sup> <sup>Q</sup>≥<sup>0</sup> *for all* s, s <sup>∈</sup> <sup>S</sup>*, then* <sup>M</sup> *is a (parameter-free)* CTMC*.*

For any pair of states s, s <sup>∈</sup> <sup>S</sup> with a non-zero rate **<sup>R</sup>**(s, s ) > 0, the probability of triggering a transition from <sup>s</sup> to <sup>s</sup> within <sup>t</sup> time units is 1 <sup>−</sup> <sup>e</sup>−**R**(s,s- )·<sup>t</sup> [41].

Applying an *instantiation* <sup>u</sup> to a pCTMC <sup>M</sup> yields an *instantiated* CTMC <sup>M</sup>[u]=(S, s*<sup>I</sup>* ,V, **<sup>R</sup>**[u]) where **<sup>R</sup>**[u](s, s ) = **R**(s, s )[u] for all s, s <sup>∈</sup> <sup>S</sup>. In the remainder, we only consider instantiations <sup>u</sup> for a pCTMC <sup>M</sup> which are *welldefined*. The set of such instantiations is the parameter space VM.

A central *measure* on CTMCs is the *(time-bounded) reachability* Pr(♦≤<sup>τ</sup>E), which describes the probability that one of the error states E<sup>2</sup> is reached within the horizon <sup>τ</sup> <sup>∈</sup> <sup>Q</sup>. Other measures include the expected time to reach a particular state, or the average time spent in particular states. We refer to [41] for details.

Given a concrete (instantiated) CTMC <sup>M</sup>[u], the *solution* for measure <sup>ϕ</sup> is denoted by sol<sup>ϕ</sup> <sup>M</sup>[u] <sup>∈</sup> <sup>R</sup>; the *solution vector* sol<sup>Φ</sup> <sup>M</sup>[u] <sup>∈</sup> <sup>R</sup><sup>m</sup> generalizes this concept to an (ordered) set of m measures Φ = ϕ1,...,ϕm. We abuse notation and introduce the *solution function* to express solution vectors on a pCTMC:

**Definition 2 (Solution function).** *A* solution function sol<sup>Φ</sup> <sup>M</sup> : <sup>V</sup><sup>M</sup> <sup>→</sup> <sup>R</sup>|Φ<sup>|</sup> *is a mapping from a parameter instantiation* <sup>u</sup> ∈ V<sup>M</sup> *to the solution vector* sol<sup>Φ</sup> <sup>M</sup>[u]*.*

We often omit the scripts in sol<sup>Φ</sup> <sup>M</sup>(u) and write sol(u) instead. We also refer to sol(u) as the solution vector of <sup>u</sup>. For <sup>n</sup> parameter samples <sup>U</sup><sup>n</sup> <sup>=</sup> {u1,...,un} with <sup>u</sup><sup>i</sup> ∈ VM, we denote the solution vectors by sol(Un) <sup>∈</sup> <sup>R</sup><sup>m</sup>×<sup>n</sup>.

Using solution vectors, we can define the probability curves shown in Fig. 2c.

**Definition 3 (Probability curve).** *The* probability curve *for reachability probability* <sup>φ</sup><sup>τ</sup> = Pr(♦≤<sup>τ</sup>E) *and CTMC* <sup>M</sup>[u] *is given by* probC : <sup>τ</sup> → sol<sup>ϕ</sup><sup>τ</sup> M[u] *.*

We can approximate the function probC for a concrete CTMC by computing probC(t1),..., probC(tm) for a finite set of time horizons. As such, we compute the solution vector w.r.t. <sup>m</sup> different reachability measures <sup>Φ</sup> <sup>=</sup> {ϕ<sup>t</sup><sup>1</sup> ,...,ϕ<sup>t</sup>m}. By exploiting the monotonicity<sup>3</sup> of the reachability over time, we obtain an upper and lower bound on probC(τ ) as two step functions, see Fig. 2d. We can smoothen the approximation, by taking an upper and lower bound on these step functions.

We study pCTMCs where the parameters follow a probability distribution. This probability distribution can be highly complex or even unknown; we merely assume that we can sample from this distribution.

**Definition 4 (upCTMC).** *<sup>A</sup>* upCTMC *is a tuple* (M, <sup>P</sup>) *with* <sup>M</sup> *a pCTMC and* <sup>P</sup> *a probability distribution over the parameter space* <sup>V</sup><sup>M</sup> *of* <sup>M</sup>*.*

<sup>2</sup> Formally, states are labeled and E describes the label, see [8].

<sup>3</sup> In Definition 3, only the upper limit on the timebound is varied, so measures are monotonic.

A upCTMC defines a probability space (VM, <sup>P</sup>) over the parameter values, whose domain is defined by the parameter space VM. In the remainder, we denote a *sample* from <sup>V</sup><sup>M</sup> drawn according to <sup>P</sup> by <sup>u</sup> ∈ VM.

To quantify the performance of a upCTMC, we may construct a *prediction region* on the solution vector space, such as those shown in Fig. 3a. In this paper, we consider only prediction regions which are compact subsets <sup>R</sup> <sup>⊆</sup> <sup>R</sup>|Φ<sup>|</sup> . We define the so-called *containment probability* of a prediction region, which is the probability that the solution vector sol(u) for a randomly sampled parameter <sup>u</sup> ∈ V<sup>M</sup> is contained in <sup>R</sup>, as follows:

**Definition 5 (Containment probability).** *For a prediction region* R*, the* containment probability contain<sup>V</sup> (R) *is the probability that the solution vector* sol(u) *for any parameter sample* <sup>u</sup> ∈ V<sup>M</sup> *is contained in* <sup>R</sup>*:*

$$\mathsf{contain}\_{\mathscr{V}}(R) = \Pr\{u \in \mathscr{V}\_{\mathscr{M}} \, : \, \mathsf{sol}(u) \in R\}.\tag{1}$$

Recall that we solve the problem in Sect. 1 with a user-specified confidence level, denoted by <sup>β</sup> <sup>∈</sup> (0, 1). Formally, we solve the following problem:

**Formal Problem.** Given a upCTMC (M, <sup>P</sup>), a set <sup>Φ</sup> of measures, and a confidence level <sup>β</sup> <sup>∈</sup> (0, 1), compute a (tight) prediction region <sup>R</sup> and a (tight) lower bound <sup>μ</sup> <sup>∈</sup> (0, 1) on the containment probability, such that contain(R) <sup>≥</sup> <sup>μ</sup> holds with a confidence level of at least <sup>β</sup>.

The problem in Sect. 1 is a special case of the formal problem, with Φ the reachability probability over a set of horizons. In that case, we can overapproximate a prediction region as a rectangle, yielding an interval [¯ c, c¯] for every horizon t that defines where the two step functions (see below Definition 3) change. We smoothen these step functions (similar to probability curves) to obtain the following definition:

**Definition 6 (Prediction region for a probability curve).** *A* prediction region <sup>R</sup> *over a probability curve* probC *is given by two curves* ¯ c, <sup>c</sup>¯ : <sup>Q</sup>≥<sup>0</sup> <sup>→</sup> <sup>R</sup> *as the area in-between:* <sup>R</sup> <sup>=</sup> {(t, y) <sup>∈</sup> <sup>Q</sup> <sup>×</sup> <sup>R</sup> <sup>|</sup> ¯ <sup>c</sup>(t) <sup>≤</sup> <sup>y</sup> <sup>≤</sup> <sup>c</sup>¯(t)}*.*

We solve the problem by sampling a finite set U<sup>n</sup> of parameter values of the upCTMC and computing the corresponding solution vectors sol(Un). In Sect. 3, we solve the problem assuming that we can compute solution vectors exactly. In Sect. 4, we consider a less restricted setting in which every solution is imprecise, i.e. only known to lie in a certain interval.

#### **3 Precise Sampling-Based Prediction Regions**

In this section, we use scenario optimization [16,18] to compute a high-confidence lower bound on the containment probability. First, in Sect. 3.1, we describe how to compute a prediction region using the solution vectors sol(Un) for the parameter samples Un. In Sect. 3.2, we clarify how to compute a lower bound on the containment probability with respect to this prediction region. In Sect. 3.3, we construct an algorithm based on those results that solves the formal problem.

#### **3.1 Constructing Prediction Regions**

We assume that we are given a set of solution vectors sol(Un) obtained from <sup>n</sup> parameter samples. We construct a prediction region R based on these vectors such that we can annotate these regions with a lower bound on the containment probability, as in the problem statement. For conciseness, we restrict ourselves to the setting where <sup>R</sup> is a hyperrectangle in <sup>R</sup>m, with <sup>m</sup> <sup>=</sup> <sup>|</sup>Φ<sup>|</sup> the number of measures, cf. Remark 1 below. In the following, we represent R using two vectors (points) ¯ x, <sup>x</sup>¯ <sup>∈</sup> <sup>R</sup><sup>m</sup> such that, using pointwise inequalities, <sup>R</sup> <sup>=</sup> {<sup>x</sup> <sup>|</sup> ¯ <sup>x</sup> <sup>≤</sup> <sup>x</sup> <sup>≤</sup> <sup>x</sup>¯}. For an example of such a rectangular prediction region, see Fig. 3a.

As also shown in Fig. 3a, we do *not* require R to contain all solutions in sol(Un). Instead, we have two orthogonal goals: we aim to minimize the size of R, while also minimizing the (Manhattan) distance of samples to R, measured in their 1-norm. Solutions contained in R are assumed to have a distance of zero, while solutions not contained in R are called *relaxed*. These goals define a *multiobjective problem*, which we solve by weighting the two objectives using a fixed parameter ρ > 0, called the *cost of relaxation*, that is used to scale the distance to <sup>R</sup>. Then, <sup>ρ</sup> → ∞ enforces sol(Un) <sup>⊆</sup> <sup>R</sup>, as in the outer box in Fig. 3a, while for <sup>ρ</sup> <sup>→</sup> 0, <sup>R</sup> is reduced to a point. Thus, the cost of relaxation <sup>ρ</sup> is a tuning parameter that determines the size of the prediction region R and hence the fraction of the solution vectors that is contained in R (see [19,21] for details).

We capture the problem described above in the following convex optimization problem L<sup>ρ</sup> <sup>U</sup> . We define the decision variables ¯ x, <sup>x</sup>¯ <sup>∈</sup> <sup>R</sup><sup>m</sup> to represent the prediction region. In addition, we define a decision variable <sup>ξ</sup><sup>i</sup> <sup>∈</sup> <sup>R</sup><sup>m</sup> <sup>≥</sup><sup>0</sup> for every sample i = 1,...,n that acts as a slack variable representing the distance to R.

$$\mathfrak{L}\_{\mathcal{U}}^{\rho} \colon \text{minimize} \quad \|\bar{x} - \underline{x}\|\_{1} + \rho \sum\_{i=1}^{n} \|\xi\_{i}\|\_{1} \tag{2a}$$

$$\text{subject to} \quad \underline{x} - \xi\_i \le \mathbf{sol}(u\_i) \le \bar{x} + \xi\_i \quad \forall i = 1, \ldots, n. \tag{2b}$$

The objective function in Eq. (2a) minimizes the size of R —by minimizing the sum of the width of the prediction region in all dimensions— plus ρ times the distances of the samples to R. We denote the optimal solution to problem L<sup>ρ</sup> U for a given ρ by R<sup>∗</sup> ρ, ξ<sup>∗</sup> <sup>ρ</sup>, where R<sup>∗</sup> <sup>ρ</sup> = [¯ x∗ <sup>ρ</sup>, x¯<sup>∗</sup> <sup>ρ</sup>] for the rectangular case.

#### **Assumption 1.** *The optimal solution* R<sup>∗</sup> ρ, ξ<sup>∗</sup> <sup>ρ</sup> *to* L<sup>ρ</sup> <sup>U</sup> *exists and is unique.*

Note that Definition 2 ensures finite-valued solution vectors, thus guaranteeing the existence of a solution to Eq. (2). If the solution is not unique, we apply a suitable tie-break rule that selects one solution of the optimal set (e.g., the solution with a minimum Euclidean norm, see [16]). The following example shows that values of ρ exist for which such a tie-break rule is necessary to obtain a unique solution.

*Example 1.* Figure 4 shows a set of solution vectors in one dimension, labeled A– F. Consider prediction region R<sup>1</sup> = [A, F]. The corresponding objective value Eq. (2a) is <sup>x</sup>¯ <sup>−</sup> ¯ <sup>x</sup> <sup>+</sup> <sup>ρ</sup> · ξ<sup>i</sup> <sup>=</sup> <sup>x</sup>¯ <sup>−</sup> ¯ <sup>x</sup> <sup>=</sup> <sup>δ</sup><sup>1</sup> <sup>+</sup> ··· <sup>+</sup> <sup>δ</sup>5, as all <sup>ξ</sup><sup>i</sup> = 0. For prediction region <sup>R</sup><sup>2</sup> = [B,E], the objective value is <sup>δ</sup>2+δ3+δ4+ρ·δ1+ρ·δ5. Thus, for ρ > 1,

**Fig. 4.** The prediction region changes with the cost of relaxation ρ.

**Fig. 5.** Prediction regions as boxes, for different costs of relaxations ρ.

solving L<sup>ρ</sup> <sup>U</sup> yields <sup>R</sup><sup>1</sup> whereas for ρ < 1, relaxing solutions A and F is cheaper than not doing so, so R<sup>2</sup> is optimal. When ρ = 1, however, relaxing solutions A and F yields the same cost as not relaxing these samples, so a tie-break rule is needed (see above). For ρ < <sup>1</sup> <sup>2</sup> , relaxing samples A, B, E, and F is cost-optimal, resulting in the prediction region containing exactly {C, D}. 

Similarly, we can consider cases with more samples and multiple measures, as shown in Fig. 5 (see [6, Appendix A] for more details). The three prediction regions in Fig. 5 are obtained for different costs of relaxation ρ. For ρ = 2, the region contains all vectors, while for a lower ρ, more vectors are left outside.

*Remark 1.* While problem L<sup>ρ</sup> <sup>U</sup> in Eq. (2) yields a rectangular prediction region, we can also produce other shapes. We may, e.g., construct a Pareto front as in Fig. 3b, by adding additional affine constraints [12]. In fact, our only requirement is that the objective function is convex, and the constraints are convex in the decision variables (the dependence of the constraints on <sup>u</sup> may be arbitrary) [21]. 

#### **3.2 Bounding the Containment Probability**

The previous section shows how we compute a prediction region based on convex optimization. We now characterize a valid high-confidence lower bound on the containment probability w.r.t. the prediction region given by the optimal solution to this optimization problem. Toward that result, we introduce the so-called *complexity* of a solution to problem L<sup>ρ</sup> <sup>U</sup> in Eq. (2), a concept used in [21] that is related to the compressibility of the solution vectors sol(Un):

**Definition 7 (Complexity).** *For* L<sup>ρ</sup> <sup>U</sup> *with optimal solution* <sup>R</sup><sup>∗</sup> ρ, ξ<sup>∗</sup> <sup>ρ</sup>*, consider a set* W⊆U<sup>n</sup> *and the associated problem* <sup>L</sup><sup>ρ</sup> <sup>W</sup> *with optimal solution* <sup>R</sup>˜ρ, ˜ξρ*. The set* W *is* critical*, if*

$$
\tilde{R}\_{\rho} = R\_{\rho}^\* \quad \text{and} \quad \{ u\_i \mid \xi\_{\rho,i}^\* > 0 \} \subseteq \mathcal{W}.
$$

*The* complexity c<sup>∗</sup> <sup>ρ</sup> *of* R<sup>∗</sup> ρ, ξ<sup>∗</sup> <sup>ρ</sup> *is the cardinality of the smallest critical set. We also call* c<sup>∗</sup> <sup>ρ</sup> *the complexity of* L<sup>ρ</sup> U *.*

**Fig. 6.** Lower bounds η on the containment probability as a function of the complexity c, obtained from Theorem 1 for different confidence levels β.

If a sample u<sup>i</sup> has a value ξ<sup>∗</sup> ρ,i <sup>&</sup>gt; 0, its solution vector has a positive distance to the prediction region, R<sup>∗</sup> <sup>ρ</sup>. (i.e., [¯ x∗ <sup>ρ</sup>, x¯<sup>∗</sup> <sup>ρ</sup>] for the rectangular case). Thus, the complexity c<sup>∗</sup> <sup>ρ</sup> is the number of samples for which sol(ui) <sup>∈</sup>/ <sup>R</sup><sup>∗</sup> ρ, plus the minimum number of samples needed on the boundary of the region to keep the solution unchanged. We describe in Sect. 3.3 how we algorithmically determine the complexity.

*Example 2.* In Fig. 5, the prediction region for ρ = 2 contains all solution vectors, so ξ<sup>∗</sup> <sup>2</sup>,i = 0 <sup>∀</sup>i. Moreover, if we remove *all but four* solutions (the ones on the boundary of the region), the optimal solution to problem L<sup>ρ</sup> <sup>U</sup> remains unchanged, so the complexity is c<sup>∗</sup> <sup>1</sup>.<sup>12</sup> = 0 + 4. Similarly, the complexity for ρ = 0.4 is c∗ <sup>0</sup>.<sup>4</sup> = 8 + 2 = 10 (8 solutions outside the region, and 2 on the boundary). 

Recall that Definition 5 defines the containment probability of a generic prediction region R, so contain(R<sup>∗</sup> <sup>ρ</sup>) is the containment probability w.r.t. the optimal solution to L<sup>ρ</sup> <sup>U</sup> . We adapt the following theorem from [21], which gives a lower bound on the containment probability contain(R<sup>∗</sup> <sup>ρ</sup>) of an optimal solution to L<sup>ρ</sup> U for a predefined value of ρ. This lower bound is correct with a user-defined confidence level of <sup>β</sup> <sup>∈</sup> (0, 1), which we typically choose close to one (e.g., <sup>β</sup> = 0.99).

**Theorem 1.** *Let* <sup>U</sup><sup>n</sup> *be a set of* <sup>n</sup> *samples, and let* <sup>c</sup><sup>∗</sup> *be the complexity of problem* L<sup>ρ</sup> <sup>U</sup> *. For any confidence level* <sup>β</sup> <sup>∈</sup> (0, 1) *and any upper bound* <sup>d</sup><sup>∗</sup> <sup>≥</sup> <sup>c</sup>∗*, it holds that*

$$\mathbb{P}^n \left\{ \mathsf{contain} \left( R\_{\rho}^\* \right) \geq \eta (d^\*) \right\} \geq \beta,\tag{3}$$

*where* R<sup>∗</sup> <sup>ρ</sup> *is the prediction region for* L<sup>ρ</sup> <sup>U</sup> *. Moreover,* <sup>η</sup> *is a function defined as* η(n)=0*, and otherwise,* η(c) *is the smallest positive real-valued solution to the following polynomial equality in the* t *variable for a complexity of* c*:*

$$
\binom{n}{c}t^{n-c} - \frac{1-\beta}{2n} \sum\_{i=c}^{n-1} \binom{i}{c} t^{i-c} - \frac{1-\beta}{6n} \sum\_{i=n+1}^{4n} \binom{i}{c} t^{i-c} = 0. \tag{4}
$$

We provide the proof of Theorem 1 in [6, Appendix B.1]. With a probability of at least β, Theorem 1 yields a correct lower bound. That is, if we solve L<sup>ρ</sup> U

**Fig. 7.** Overview of our approach for solving the problem statement.

for many more sets of n parameter samples (note that, as the samples are i.i.d., these sets are drawn according to the product probability P<sup>n</sup>), the inequality in Eq. (3) is incorrect for *at most* a 1 <sup>−</sup> <sup>β</sup> fraction of the cases. We plot the lower bound η(c) as a function of the complexity c = 0,...,n in Fig. 6, for different samples sizes n and confidence levels β. These figures show that an increased complexity leads to a lower η, while increasing the sample size leads to a tighter bound.

*Example 3.* We continue Example 2. Recall that the complexity for the outer region in Fig. 5 is c<sup>∗</sup> <sup>1</sup>.<sup>12</sup> = 4. With Theorem 1, we compute that, for a confidence level of β = 0.9, the containment probability for this prediction region is at least η = 0.615 (cf. Figure 6a). For a stronger confidence level of β = 0.999, we obtain a more conservative lower bound of <sup>η</sup> = 0.455. 

#### **3.3 An Algorithm for Computing Prediction Regions**

We combine the previous results in our algorithm, which is outlined in Fig. 7. The goal is to obtain a set of prediction regions as in Fig. 5 and their associated lower bounds. To strictly solve the problem statement, assume k = 1 in the exposition below. We first outline the complete procedure before detailing Steps 4 and 5.

As preprocessing steps, given a upCTMC (M, <sup>P</sup>), we first (1) sample a set <sup>U</sup><sup>n</sup> of <sup>n</sup> parameter values. Using <sup>M</sup> and <sup>Φ</sup>, a (2) model checking algorithm then computes the solution vector sol<sup>Φ</sup> <sup>M</sup>(u) for each <sup>u</sup> ∈ Un, yielding the set of solutions sol(Un). We then use sol(Un) as basis for (3) the scenario problem <sup>L</sup><sup>ρ</sup> <sup>U</sup> in Eq. (2), which we solve for k predefined values ρ1,...,ρk, yielding k prediction regions R<sup>∗</sup> <sup>ρ</sup><sup>1</sup> ,...R<sup>∗</sup> <sup>ρ</sup><sup>k</sup> . We (4) compute an upper bound <sup>d</sup><sup>∗</sup> <sup>ρ</sup> on the complexity c<sup>∗</sup> <sup>ρ</sup> <sup>∀</sup>ρ. Finally, we (5) use the result in Theorem 1, for a given confidence β, to compute the lower bound on the containment probability η(d<sup>∗</sup> <sup>ρ</sup>) of R<sup>∗</sup> <sup>ρ</sup>. Using Definition 6, we can postprocess this region to a prediction region over the probability curves.

*Step (3): Choosing values for* ρ*.* Example 1 shows that relaxation of additional solution vectors (and thus a change in the prediction region) only occurs at *critical* values of ρ = <sup>1</sup> <sup>n</sup> , for <sup>n</sup> <sup>∈</sup> <sup>N</sup>. In practice, we will use <sup>ρ</sup> <sup>=</sup> <sup>1</sup> <sup>n</sup>+0.<sup>5</sup> for ±10 values of <sup>n</sup> <sup>∈</sup> <sup>N</sup> to obtain gradients of prediction regions as in Sect. 6.

*Step (4): Computing complexity.* Computing the complexity c<sup>∗</sup> <sup>ρ</sup> is a combinatorial problem in general [30], because we must consider the removal of all combinations of the solutions on the boundary of the prediction region R<sup>∗</sup> <sup>ρ</sup>. In practice, we compute an upper bound d<sup>∗</sup> <sup>ρ</sup> <sup>≥</sup> <sup>c</sup><sup>∗</sup> <sup>ρ</sup> on the complexity via a greedy algorithm. Specifically, we iteratively solve L<sup>ρ</sup> <sup>U</sup> in Eq. (2) with *one more sample on the boundary removed*. If the optimal solution is unchanged, we conclude that this sample does not contribute to the complexity. If the optimal solution is changed, we put the sample back and proceed by removing a different sample. This greedy algorithm terminates when we have tried removing all solutions on the boundary.

*Step (5): Computing lower bounds.* Theorem 1 characterizes a computable function B(d∗, n, β) that returns zero for d<sup>∗</sup> = n (i.e., all samples are critical), and otherwise uses the polynomial Eq. (4) to obtain η, which we solve with an approximate root finding method in practice (see [31] for details on how to ensure that we find the smallest root). For every upper bound on the complexity d<sup>∗</sup> and any requested confidence, we obtain the lower bound η = B(d∗, n, β) for the containment probability w.r.t. the prediction region R<sup>∗</sup> ρ.

## **4 Imprecise Sampling-Based Prediction Regions**

Thus far, we have solved our problem statement under the assumption that we compute the solution vectors precisely (up to numerics). For some models, however, computing precise solutions is expensive. In such a case, we may choose to compute an approximation, given as an *interval* on each entry of the solution function. In this section, we deal with such *imprecise solutions*.

*Setting.* Formally, imprecise solutions are described by the bounds sol−(u),sol<sup>+</sup>(u) <sup>∈</sup> <sup>R</sup><sup>m</sup> such that sol−(u) <sup>≤</sup> sol(u) <sup>≤</sup> sol<sup>+</sup>(u) holds with pointwise inequalities. Our goal is to compute a prediction region R and a (high-confidence) lower bound <sup>μ</sup> such that contain(R) <sup>≥</sup> <sup>μ</sup>, i.e., a lower bound on the probability that any *precise solution* sol(u) is contained in R. However, we must now compute R and contain(R) from the imprecise solutions sol−,sol<sup>+</sup>. Thus, we aim to provide a guarantee with respect to the *precise* solution sol(u), based on *imprecise* solutions.

*Challenge.* Intuitively, if we increase the (unknown) prediction region R<sup>∗</sup> from problem L<sup>ρ</sup> <sup>U</sup> (for the unknown precise solutions) while also overapproximating the complexity of L<sup>ρ</sup> <sup>U</sup> , we obtain sound bounds. We formalize this idea as follows.

**Lemma 1.** *Let* R<sup>∗</sup> <sup>ρ</sup> *be the prediction region and* c<sup>∗</sup> <sup>ρ</sup> *the complexity that result from solving* L<sup>ρ</sup> <sup>U</sup> *for the precise (unknown) solutions* sol(Un)*. Given a set* <sup>R</sup> <sup>∈</sup> <sup>R</sup><sup>n</sup> *and* <sup>d</sup> <sup>∈</sup> <sup>N</sup>*, for any confidence level* <sup>β</sup> <sup>∈</sup> (0, 1)*, the following implication holds:*

$$R^\*\_\rho \subseteq R \text{ and } c^\*\_\rho \le d \implies \mathbb{P}^n \left\{ \mathsf{contmin}(R) \ge \eta(d) \right\} \ge \beta,\tag{5}$$

*where* η(n)=0*, and otherwise,* η(d) *is the smallest positive real-valued solution to the polynomial equality in Eq.* (4)*.*

The proof is in [6, Appendix B.2]. In what follows, we clarify how we compute the appropriate R and d in Lemma 1. As we will see, in contrast to Sect. 3, these results do *not* carry over to other definitions L<sup>ρ</sup> <sup>U</sup> (for non-rectangular regions <sup>R</sup>).

**Fig. 8.** Imprecise solutions and the upper bound ¯x- <sup>ρ</sup> of the prediction region.

**Fig. 9.** Complexity of the imprecise solution vs. that of the precise solution.

#### **4.1 Prediction Regions on Imprecise Solutions**

In this section, we show how to compute <sup>R</sup> <sup>⊇</sup> <sup>R</sup><sup>∗</sup> <sup>ρ</sup>, satisfying the first term in the premise of Lemma 1. We construct a *conservative box* around the imprecise solutions as in Fig. 9, containing both sol−(u) and sol<sup>+</sup>(u). We compute this box by solving the following problem G<sup>ρ</sup> <sup>U</sup> as a modified version of <sup>L</sup><sup>ρ</sup> <sup>U</sup> in Eq. (2):

$$\mathfrak{G}\_{\mathcal{U}}^{\rho} \colon \text{minimize } \|\bar{x} - \underline{x}\|\_1 + \rho \sum\_{i=1}^{n} \|\xi\_i\|\_1 \tag{6a}$$

$$\text{subject to} \quad \underline{x} - \xi\_i \le \textsf{sol}^-(u\_i), \ \textsf{sol}^+(u\_i) \le \overline{x} + \xi\_i \quad \forall i = 1, \dots, n. \tag{6b}$$

We denote the optimal solution of G<sup>ρ</sup> <sup>U</sup> by [¯ x <sup>ρ</sup>, x¯ ρ], ξ <sup>ρ</sup> (recall that the optimum to L<sup>ρ</sup> <sup>U</sup> is written as [¯ x∗ <sup>ρ</sup>, x¯<sup>∗</sup> ρ], ξ<sup>∗</sup> <sup>ρ</sup>).<sup>4</sup> If a sample <sup>u</sup><sup>i</sup> ∈ V<sup>M</sup> in problem <sup>G</sup><sup>ρ</sup> U is relaxed (i.e., has a non-zero ξi), part of the interval [sol−(ui),sol<sup>+</sup>(ui)] is not contained in the prediction region. The following result (for which the proof is in [6, Appendix B.3]. relates L<sup>ρ</sup> <sup>U</sup> and <sup>G</sup><sup>ρ</sup> <sup>U</sup> , showing that we can use [¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>] as R in Lemma 1.

**Theorem 2.** *Given* <sup>ρ</sup>*, sample set* <sup>U</sup>n*, and prediction region* [ ¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>] *to problem* Gρ <sup>U</sup> *, it holds that* [ ¯ x∗ <sup>ρ</sup>, x¯<sup>∗</sup> <sup>ρ</sup>] ⊆ [ ¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>]*, with* [ ¯ x∗ <sup>ρ</sup>, x¯<sup>∗</sup> <sup>ρ</sup>] *the optimal solution to* L<sup>ρ</sup> U *.*

We note that this result is not trivial. In particular, the entries ξ<sup>i</sup> from both LPs are incomparable, as are their objective functions. Instead, Theorem 2 relies on two observations. First, due to the use of the 1-norm, the LP G<sup>ρ</sup> <sup>U</sup> can be decomposed into n individual LPs, whose results combine into a solution to the original LP. This allows us to consider individual dimensions. Second, the solution vectors that are relaxed depend on the value of ρ and on their *relative order*, but not on the *precise position* within that order, which is also illustrated by Example 1. In combination with the observation from Example 1 that the *outermost* samples are relaxed at the (relatively) highest ρ, we can provide conservative guarantees on which samples are (or are surely not) relaxed. We formalize these observations and provide a proof of Theorem 2 in [6, Appendix B.3].

<sup>4</sup> We write [¯ x<sup>∗</sup> <sup>ρ</sup>, x¯<sup>∗</sup> <sup>ρ</sup>] and [¯ x- <sup>ρ</sup>, x¯- <sup>ρ</sup>], as results in Sect. 4 apply only to rectangular regions.

#### **4.2 Computing the Complexity**

To satisfy the second term of the premise in Lemma 1, we compute an upper bound on the complexity. We first present a negative result. Let the complexity c <sup>ρ</sup> of problem G<sup>ρ</sup> <sup>U</sup> be defined analogous to Definition 7, but with [¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>] as the region.

**Lemma 2.** *In general,* c<sup>∗</sup> <sup>ρ</sup> <sup>≤</sup> <sup>c</sup> <sup>ρ</sup> *does not hold.*

*Proof.* In Fig. 9, the smallest critical set for the imprecise solutions are those labeled {1, <sup>2</sup>, <sup>7</sup>}, while this set is {1, <sup>3</sup>, <sup>5</sup>, <sup>7</sup>} under precise solutions, so c∗ <sup>ρ</sup> > c <sup>ρ</sup>. 

Thus, we cannot upper bound the complexity directly from the result to G<sup>ρ</sup> <sup>U</sup> . We can, however, determine the samples that are certainly *not* in any critical set (recall Definition 7). Intuitively, a sample is *surely noncritical* if its (imprecise) solution is strictly within the prediction region and does not overlap with any solution on the region's boundary. In Fig. 8, sample u<sup>6</sup> is surely noncritical, but sample u<sup>5</sup> is not (whether u<sup>5</sup> is critical depends on its precise solution). Formally, let δR be the boundary<sup>5</sup> of region [¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>], and let B be the set of samples whose solutions overlap with δR, which is <sup>B</sup> <sup>=</sup> {<sup>u</sup> ∈ U<sup>n</sup> : [sol−(u),sol<sup>+</sup>(u)]∩δR <sup>=</sup> <sup>∅</sup>}.

**Definition 8.** *For a region* [ ¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>]*, let* I ⊂ [ ¯ x <sup>ρ</sup>, x¯ <sup>ρ</sup>] *be the rectangle of largest volume, such that* I ∩ [sol−(u),sol<sup>+</sup>(u)] = <sup>∅</sup> *for any* <sup>u</sup> ∈ B*. A sample* <sup>u</sup><sup>i</sup> ∈ V<sup>M</sup> *is* surely noncritical *if* [sol−(ui),sol<sup>+</sup>(ui)] ⊆ I*. The set of all surely noncritical samples w.r.t. the (unknown) prediction region* [ ¯ x∗ <sup>ρ</sup>, x¯<sup>∗</sup> <sup>ρ</sup>] *is denoted by* X ⊂Un*.*

As a worst case, any sample not surely noncritical can be in the smallest critical set, leading to the following bound on the complexity as required by Lemma 1.

**Theorem 3.** *Let* <sup>X</sup> *be the set of surely noncritical samples. Then* <sup>c</sup><sup>∗</sup> <sup>ρ</sup> ≤ |U<sup>n</sup> \X|*.*

The proof is in [6, Appendix B.4]. For imprecise solutions, the bound in Theorem 3 is conservative but can potentially be improved, as discussed in the following.

#### **4.3 Solution Refinement Scheme**

Often, we can *refine* imprecise solutions arbitrarily (at the cost of an increased computation time). Doing so, we can improve the prediction regions and upper bound on the complexity, which in turn improves the computed bound on the containment probability. Specifically, we propose the following rule for refining solutions. After solving G<sup>ρ</sup> <sup>U</sup> for a given set of imprecise solutions, we refine the solutions on the boundary of the obtained prediction region. We then resolve problem G<sup>ρ</sup> <sup>U</sup> , thus adding a loop back from (4) to (2) in our algorithm shown in Fig. 7. In our experiments, we demonstrate that with this refinement scheme, we iteratively improve our upper bound <sup>d</sup> <sup>≥</sup> <sup>c</sup><sup>∗</sup> <sup>ρ</sup> and the smallest superset <sup>R</sup> <sup>⊇</sup> <sup>R</sup><sup>∗</sup> ρ.

<sup>5</sup> The boundary of a compact set is defined as its closure minus its interior [45].

#### **5 Batch Verification for CTMCs**

One bottleneck in our method is to obtain the necessary number of solution vectors sol(Un) by model checking. The following improvements, while mild, are essential in our implementation and therefore deserve a brief discussion.

In general, computing sol(u) via model checking consists of two parts. First, the high-level representation of the upCTMC —given in Prism [42], JANI [13], or a dynamic fault tree<sup>6</sup>— is translated into a concrete CTMC <sup>M</sup>[u]. Then, from <sup>M</sup>[u] we construct sol(u) using off-the-shelf algorithms [7]. We adapt the pipeline by tailoring the translation and the approximate analysis as outlined below.

Our implementation supports two methods for building the concrete CTMC for a parameter sample: (1) by first instantiating the valuation in the specification and then building the resulting concrete CTMC, or (2) by first building the pCTMC M (only once) and then instantiating it for each parameter sample to obtain the concrete CTMC <sup>M</sup>[u]. Which method is faster depends on the specific model (we only report results for the fastest method in Sect. 6 for brevity).

*Partial models.* To accelerate the time-consuming computation of solution vectors by model-checking on large models, it is natural to abstract the models into smaller models amenable to faster computations. Similar to ideas used for dynamic fault trees [55] and infinite CTMCs [48], we employ an abstraction which only keeps the most relevant parts of a model, i.e., states with a sufficiently large probability to be reached from the initial state(s). Analysis on this partial model then yields bestand worst-case results for each measure by assuming that all removed states are either target states (best case) or are not (worst case), respectively. This method returns imprecise solution vectors as used in Sect. 4, which can be refined up to an arbitrary precision by retaining more states of the original model.

Similar to building the complete models, two approaches are possible to create the partial models: (1) fixing the valuation and directly abstracting the concrete CTMC, or (2) first building the complete pCTMC and then abstracting the concrete CTMC. We reuse partial models for similar valuations to avoid costly computations. We cluster parameter valuations which are close to each other (in Euclidean distance). For parameter valuations within one cluster, we reuse the same partial model (in terms of the states), albeit instantiating it according to the precise valuation.

#### **6 Experiments**

We answer three questions about (a prototype implementation of) our approach:

**Q1.** Can we verify CTMCs taking into account the uncertainty about the rates? **Q2.** How well does our approach scale w.r.t. the number of measures and samples? **Q3.** How does our approach compare to na¨ıve baselines (to be defined below)?

*Setup.* We implement our approach using the explicit engine of Storm [37] and the improvements of Sect. 5 to sample from upCTMCs in Python. Our current

<sup>6</sup> Fault trees are a common formalism in reliability engineering [51].


**Table 1.** Excerpt of the benchmark statistics (sampling time is per 100 CTMCs).

<sup>a</sup> Computed using approximate model checking up to a relative gap between upper bound sol<sup>+</sup>(u) and lower bound sol−(u) below 1% for every sample <sup>u</sup> ∈ VM. <sup>b</sup> Model size is unknown, as the approximation does not build the full state-space.

**Fig. 10.** Prediction regions for the SIR (60) benchmark with n = 400.

implementation is limited to pCTMC instantiations that are *graph-preserving*, i.e. for any pair s, s <sup>∈</sup> <sup>S</sup> either **<sup>R</sup>**(s, s )[u] = 0 or **R**(s, s )[u] > 0 for all u.We solve optimization problems using the ECOS solver [29]. All experiments ran single-threaded on a computer with 32 3.7 GHz cores and 64 GB RAM. We show the effectiveness of our method on a large number of publicly available pCTMC [35] and fault tree benchmarks [50] across domains (details in [6, Appendix C]).

#### **Q1. Applicability**

An excerpt of the benchmark statistics is shown in Table 1 (see [6, Appendix C] for the full table). For all but the smallest benchmarks, sampling and computing the solution vectors by model checking is more expensive than solving the scenario problems. In the following, we illustrate that 100 samples are sufficient to provide qualitatively good prediction regions and associated lower bounds.

**Table 2.** Lower bounds ¯μ and standard deviation (SD), vs. the observed number of 1 000 additional solutions that indeed lie within the obtained regions.

```
(a) Kanban (3).
```

```
(b) Railway crossing (1,1,hc).
```


*Plotting prediction regions.* Figure 10 presents prediction regions on the extinction probability of the disease in the SIR model and is analogous to the tubes in Fig. 2d (see [6, Appendix C.1] for plots for various other benchmarks). These regions are obtained by applying our algorithm with varying values for the cost of relaxation ρ. For a confidence level of β = 99%, the widest (smallest) tube in Fig. 10 corresponds to a lower bound probability of μ = 91.1% (μ = 23.9%). Thus, we conclude that, with a confidence of at least 99%, the curve created by the CTMC for any sampled parameter value will lie within the outermost region in Fig. 10 with a probability of at least 91.1%. We highlight that our approach supports more general prediction regions. We show n = 200 solution vectors for the buffer benchmark with two measures in Fig. 11 and produce regions that approach the Pareto front. For a confidence level of β = 99%, the outer prediction region is associated with a lower bound probability of μ = 91.1%, while the inner region has a lower value of μ = 66.2%. We present more plots in [6, Appendix C.1].

*Tightness of the solution.* In Table 2 we investigate the tightness of our results. For the experiment, we set ρ = 1.1 and solve L<sup>ρ</sup> <sup>U</sup> for different values of <sup>n</sup>, repeating every experiment 10 times, resulting in the average bounds ¯μ. Then, we sample 1 000 solutions and count the *observed* number of solutions contained in every prediction regions, resulting in an empirical approximation of the containment probability. Recall that for ρ > 1, we obtain a prediction region that contains all solutions, so this observed count grows toward n. The lower bounds grow toward the empirical count for an increased n, with the smallest difference (RC, n = 800, β = 0.9) being as small as 0.9%. Similar observations hold for other values of ρ.

*Handling imprecise solutions.* The approximate model checker is significantly faster (see Table 1 for SIR (140) and RC), at the cost of obtaining imprecise solution vectors.<sup>7</sup> For SIR (140), the sampling time is reduced from 49 to 9 min, while the scenario optimization time is slightly higher at 129 s. This difference only grows larger with the size of the CTMC. For the larger instances of RC and HECS, computing exact solutions is infeasible at all (one HECS (2,2) sample alone takes 15 min). While the bounds on the containment probability under imprecise solu-

<sup>7</sup> We terminate at a relative gap between upper/lower bound of the solution below 1%.

(a) SIR (population 20).


(b) Railway crossing (1,1,hc).

**Table 3.** Run times in [s] for solving the scenario problems for SIR and RC with ρ = 0.1 (timeout (TO) of 1 hour) for different sample sizes n and measures m.

tions may initially be poor (see Fig. 12a, which results in μ = 2.1%), we can improve the results significantly using the refinement scheme proposed in Sect. 4.3. For example, Fig. 12c shows the prediction region after refining 31 of the 100 solutions, which yields μ = 74.7%. Thus, *by iteratively refining only the imprecise solutions on the boundary of the resulting prediction regions, we significantly tighten the obtained bounds on the containment probability*.

#### **Q2. Scalability**

In Table 3, we report the run times for steps (3)–(5) of our algorithm shown in Fig. 7 (i.e., for solving the scenario problems, but not for computing the solution vectors in Storm). Here, we solve problem L<sup>ρ</sup> <sup>U</sup> for <sup>ρ</sup> = 0.1, with different numbers of samples and measures. Our approach scales well to realistic numbers of samples (up to 800) and measures (up to 400). The computational complexity of the scenario problems is largely *independent of the size of the CTMC*, and hence, similar run times are observed across the benchmarks (cf. Table 1).

#### **Q3. Comparison to baselines**

We compare against two baselines: (1) Scenario optimization to analyze each measure independently, yielding a separate probabilistic guarantee on each measure. (2) A frequentist (Monte Carlo) baseline, which samples a large number of parameter values and counts the number of associated solutions within a region.

**Fig. 12.** Refining imprecise solution vectors (red boxes) for RC (2,2), n = 100. (Color figure online)

*Analyzing measures independently.* To show that analyzing a full *set of measures* at once, e.g., the complete probability curve, is essential, we compare our method to the baseline that analyzes *each measure independently* and combines the obtained bounds on each measure afterward. We consider the PCS benchmark with precise samples and solve L<sup>ρ</sup> <sup>U</sup> for <sup>ρ</sup> = 2 (see [6, Table 5] for details). For <sup>n</sup> = 100 samples and β = 99%, our approach returns a lower bound probability of μ = 84.8%. By contrast, the na¨ıve baseline yields a lower bound of only 4.5%, and similar results are observed for different values of n (cf. [6, Table 5 in Appendix C]). There are two reasons for this large difference. First, the baseline applies Theorem 3 once for each of the 25 measures, so it must use a more conservative confidence level of <sup>β</sup>˜ = 1 <sup>−</sup> <sup>1</sup>−<sup>β</sup> <sup>25</sup> = 0.9996. Second, the baseline takes the conjunction over the 25 independent lower bounds, which drastically reduces the obtained bound.

*Frequentist baseline.* The comparison to the frequentist baseline on the Kanban and RC benchmarks yields the previously discussed results in Table 2. The results in Tables 1 and 3 show that *the time spent for sampling is (for most benchmarks) significantly higher than for scenario optimization.* Thus, our scenario-based approach has a relatively low cost, while resulting in valuable guarantees which the baseline does not give. To still obtain a high confidence in the result, a much larger sample size is needed for the frequentist baseline than for our approach.

#### **7 Related Work**

Several verification approaches exist to handle uncertain Markov models.

For (discrete-time) *interval* Markov chains (DTMCs) or Markov decision processes (MDPs), a number of approaches verify against all probabilities within the intervals [32,39,46,53,54]. Lumpability of interval CTMCs is considered in [22]. In contrast to upCTMCs, interval Markov chains have no dependencies between transition uncertainties and no distributions are attached to the intervals.

*Parametric* Markov models generally define probabilities or rates via functions over the parameters. The standard parameter synthesis problem for discrete-time models is to find all valuations of parameters that satisfies a specification. Techniques range from computing a solution function over the parameters, to directly solving the underlying optimization problems [24,28,33,40]. Parametric CTMCs are investigated in [23,34], but are generally restricted to a few parameters. The work [15] aims to find a robust parameter valuation in pCTMCs.

For all approaches listed so far, the results may be rather conservative, as no prior information on the uncertainties (the intervals) is used. That is, the uncertainty is not quantified and all probabilities or rates are treated equally as likely. In our approach, we do not compute solution functions, as the underlying methods are computationally expensive and usually restricted to a few parameters.

Quantified uncertainty is studied in [44]. Similarly to our work, the approach draws parameter values from a probability distribution over the model parameters and analyzes the instantiated model via model checking. However, [44] studies DTMCs and performs a frequentist (Monte Carlo) approach, cf. Sect. 6, to compute estimates for a single measure, without prediction regions. Moreover, our approach requires significantly fewer samples, cf. the comparison in Sect. 6.

The work in [10,11] takes a sampling-driven Bayesian approach for pCTMCs. In particular, they take a prior on the solution function over a single measure and update it based on samples (potentially obtained via statistical model checking). We assume no prior on the solution function, and, as mentioned before, do not compute the solution function due to the expensive underlying computations.

*Statistical model checking* (SMC) [1,43] samples path in stochastic models to perform model checking. This technique has been applied to numerous models [25–27,47], including CTMCs [52,57]. SMC analyzes a *concrete* CTMC by sampling from the known transition rates, whereas for upCTMC these rates are parametric.

Finally, scenario optimization [16,21] is widely used in control theory [14] and recently in machine learning [20] and reliability engineering [49]. Within a verification context, closest to our work is [5], which considers the verification of single measures for uncertain MDPs. [5] relies on the so-called sampling-anddiscarding approach [17], while we use the risk-and-complexity perspective [31], yielding better results for problems with many decision variables like we have.

#### **8 Conclusion**

This paper presents a novel approach to the analysis of parametric Markov models with respect to a set of performance characteristics. In particular, we provide a method that yields statistical guarantees on the typical performance characteristics from a finite set of samples of those parameters. Our experiments show that high-confidence results can be given based on a few hundred of samples. Future work includes supporting models with nondeterminism, exploiting aspects of parametric models such as monotonicity, and integrating methods to infer the distributions on the parameter space from observations.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Playing Against Fair Adversaries in Stochastic Games with Total Rewards**

Pablo F. Castro1,3(B) , Pedro R. D'Argenio2,3,4 , Ramiro Demasi2,3 , and Luciano Putruele1,3

<sup>1</sup> Departamento de Computaci´on, FCEFQyN, Universidad Nacional de R´ıo Cuarto, R´ıo Cuarto, Argentina

<sup>2</sup> FAMAF, Universidad Nacional de C´ordoba, C´ordoba, Argentina

<sup>3</sup> Consejo Nacional de Investigaciones Cient´ıficas y T´ecnicas (CONICET),

Buenos Aires, Argentina

pcastro@dc.exa.unrc.edu.ar

<sup>4</sup> Saarland University, Saarland Informatics Campus, Saarbr¨ucken, Germany

**Abstract.** We investigate zero-sum turn-based two-player stochastic games in which the objective of one player is to maximize the amount of rewards obtained during a play, while the other aims at minimizing it. We focus on games in which the minimizer plays in a fair way. We believe that these kinds of games enjoy interesting applications in software verification, where the maximizer plays the role of a system intending to maximize the number of "milestones" achieved, and the minimizer represents the behavior of some uncooperative but yet fair environment. Normally, to study total reward properties, games are requested to be stopping (i.e., they reach a terminal state with probability 1). We relax the property to request that the game is stopping only under a fair minimizing player. We prove that these games are determined, i.e., each state of the game has a value defined. Furthermore, we show that both players have memoryless and deterministic optimal strategies, and the game value can be computed by approximating the greatest-fixed point of a set of functional equations. We implemented our approach in a prototype tool, and evaluated it on an illustrating example and an Unmanned Aerial Vehicle case study.

#### **1 Introduction**

Game theory [25] admits an elegant and profound mathematical theory. In the last decades, it has received widespread attention from computer scientists because it has important applications to software synthesis and verification. The analogy is appealing, the operation of a system under an uncooperative environment (faulty hardware, malicious agents, unreliable communication channels, etc.) can be modeled as a game between two players (the system and the environment), in which the system tries to fulfill certain goals, whereas the environment tries to prevent this from happening. This view is particularly useful for

This work was supported by ANPCyT PICT-2017-3894 (RAFTSys), ANPCyT PICT 2019-03134, SeCyT-UNC 33620180100354CB (ARES), and EU Grant agreement ID: 101008233 (MISSION).

*controller synthesis*, i.e., to automatically generate decision-making policies from high-level specifications. Thus, synthesizing a controller consists of computing optimal strategies for a given game.

In this paper we focus on zero-sum, perfect-information, two-player, turnbased stochastic games with (non-negative) rewards [18]. Intuitively, these games are played in a graph by two players who move a token in turns. Some vertices are probabilistic, in the sense that, if a token is in a probabilistic vertex, then the next vertex is randomly selected. Furthermore, the players select their moves using strategies. Associated with each vertex there is a reward (which, in this paper, is taken to be non-negative). The goal of Player 1 is to maximize the expected amount of collected rewards during the game, whereas Player 2 aims at minimizing this value. This is what [28] calls *total reward objective*. These kinds of games have been shown useful to reason about several classes of systems such as autonomous vehicles, fault-tolerant systems, communication protocols, energy production plants, etc. Particularly, in this paper we consider those games in which one of the players employs fair strategies.

Fairness restrictions, understood as fair resolutions of non-determinism of actions, play an important role in software verification and controller synthesis. Especially, fairness assumptions over environments make possible the verification of liveness properties on open systems. Several authors have indicated the need for fairness assumptions over the environment in the controller synthesis approach, e.g., [2,16]. As a simple example consider an autonomous vehicle that needs to traverse a field where moving objects may interfere in its path. Though the precise behavior of the objects may be unknown, it is reasonable to assume that they will not continuously obstruct the vehicle attempts to avoid them. In this sense, while stochastic behavior may be a consequence of the vehicle faults, we can only assume a fair behavior of the surrounding moving objects. In this work, we consider stochastic games in which one of the players (the one playing the environment) is assumed to play only with strong fair strategies.

In order to guarantee that the expected value of accumulated rewards is well defined in (perhaps infinite) plays, some kind of stopping criteria is needed. A common way to do this is to force the strategies to decide to stop with some positive probability in every decision. This corresponds to the so-called discounted stochastic games [18,27], and has the implications that the collected rewards become less important as the game progresses (the "importance reduction" is given by the discount factor). Alternatively, one may be interested in knowing the expected *total* reward, that is, the expected accumulated reward *without* any loss of it as time progresses. For this value to be well defined, the game itself needs to be stopping. That is, no matter the strategies played by the players, the probability of reaching a terminal state needs to be 1 [13,18]. We focus on this last type of game. However, we study here games that may not be stopping in general (i.e., for every strategy), but instead, require that they become stopping only when the minimizer plays in a fair way. We use a notion of (almost-sure) strong fairness, mostly following the ideas introduced in [7] for Markov decision processes. We show that these kinds of games are determined, i.e., each state of the game has a value defined. Furthermore, we show that memoryless and deterministic optimal strategies exist for both players. Moreover, the value of the game can be calculated via the greatest fixed point of the corresponding functionals. It is important to remark that most of the properties discussed in this paper hold when the fairness assumptions are made over the minimizer. Similar properties may not hold if the role of players is changed. However, these conditions encompass a large class of scenarios, where the system intends to maximize the total collected reward and the environment has the opposite objective.

In summary, the contributions of this paper are the following: (1) we introduce the notion of stopping under fairness stochastic game, a generalization of stopping game that takes into account fair environments; (2) we prove that it can be decided in polynomial time whether a game is stopping under fairness; (3) we show that these kinds of games are determined and both players possess optimal stationary strategies, which can be computed using Bellman equations; and (4) we implemented these ideas in a prototype tool embedded in the PRISM-games toolset [22], which we used to evaluate the viability of our approach through illustrative case studies.

The paper is structured as follows. Section 2 introduces an illustrating example to motivate the use of having fairness restrictions over the minimizer. Section 3 fixes terminology and introduces background concepts. In Sect. 4 we describe a polynomial procedure to check whether a game stops under fairness assumptions, we also prove that determinacy is preserved in these games as well as the existence of (memoryless and deterministic) optimal strategies. Experimental results are described in Sect. 5. Finally, Sects. 6 and 7 discuss related work and draw some conclusions, respectively.

#### **2 Roborta vs. the Fair Light (A Motivating Example)**

Consider the following scenario. Roborta the robot is navigating a grid of 4 × 4 cells. Roborta's moves respond to a traffic light: if the light is yellow, she must move sideways (at a border cell, Roborta is allowed to wrap around to the other side); if the light is green she ought to move forward; if the light is red, she cannot perform any movement; finally, if the light is off, Roborta is free to move either sideways or forward. The light and Roborta change their states in turns. In addition, a (non-negative) reward is associated with each cell of the grid. Also, some cells restrict the sideway movement to only one direction. Moreover, we consider possible failures on the behavior of the robot and the light. If Roborta fails, she loses her turn to move. If the light fails, it turns itself off. The failures occur with a given probability and are not permanent (they only affect the current play). The goal of Roborta is to collect as many rewards as possible. In opposition, the light aims at minimizing this value.

The specification of this game is captured in Fig. 1 (using PRISM-like notation [23]). In this model, WIDTH and LENGTH are constants defining the dimension of the grid. MOVES is a two-dimensional array modeling the possible sideways movements in the grid (0 allows the robot to move only to the left, 1, to either side, and 2, only to the right). The light plays when it is red (light=0) and it

```
module Roborta vs the light
 col : [0..WIDTH] init 0;
 row : [0..LENGTH] init 0;
 light : [0..3] init 0; // current light color
                        // 0: red (light's turn)
                        // 1: yellow (Roborta moves sideways)
                        // 2: green (Roborta moves foreward)
                        // 3: off (light fails, any move)
 // light moves
 [l y] (light=0) -> (1-Q) : (light'=1) + Q : (light'=3);
 [l g] (light=0) -> (1-Q) : (light'=2) + Q : (light'=3);
 // Roborta moves
 [r l] ((light=1) | (light=3)) & (MOVES[col,row] <= 1)
                 -> (1-P) : (light'=0) & (col'=(col-1)%WIDTH) +
                    P : (light'=0) ;
 [r r] ((light=1) | (light=3)) & (MOVES[col,row] >= 1)
                 -> (1-P) : (light'=0) & (col'=(col+1)%WIDTH) +
                    P : (light'= 0);
 [r f] ((light=2) | (light=3)) & (row < LENGTH)
                 -> (1-P) : (light'=0) & (row'=row+1) +
                    P : (light'= 0);
endmodule
```
**Fig. 1.** Model for the Game

can choose whether to turn on the yellow light (transition labelled with l <sup>y</sup>) or green (transition labelled l <sup>g</sup>). Notice that with any choice, the light may fail with probability Q, in which case it turns itself off (light'=3). If the light is not red, then it is Roborta's turn to play. If the light is yellow (light=1) or off (light=3), Roborta can chose whether to

move left (r <sup>l</sup>) or right (r <sup>r</sup>), provided the grid allows the movements. If the light is green (light=2) or off (light=3), she can choose to move forward (notice that if light=2 this is the only possible move). Like the light, each of Roborta's choices has a failure probability of P, in which case, she does not move and only passes the turn to the light (by setting light'=0). For completeness, we mention that the rewards are stored in a secondary matrix which is not shown in Fig. 1.

Figure 2 shows the assignment of rewards to each cell of the 4×4 grid as well as the sideway movement restrictions (shown on the bottom-right of each cell with white arrows). The game starts at the cell (0, 0) and it stops when Roborta escapes through the end of the grid (i.e., row <sup>=</sup> LENGTH).

A possible scenario in this game is as follows. Roborta starts in cell (0, 0) and, in an attempt to minimize the rewards accumulated by the robot, the environment switches the yellow light on. For the sake of simplicity, we assume no failures on the light, i.e., Q = 0. Notice that, if the environment plays always in this way (signaling a yellow light), then Roborta does not collect rewards (since all rewards in the first row are 0) but also she will never reach the goal and

**Fig. 2.** A robot on a 4 × 4 grid

the game never stops. This scenario occurs when the light plays in an unfair way, i.e., an action (the one that turns the green light on) is enabled infinitely often, but it is not executed infinitely often. Assuming fairness over the environment, we can ensure that a green light will be eventually switched on, allowing the robot to move forward.

For the case in which Q = 0, the best strategy for Roborta when the light is yellow is shown in black arrows on the top-right of each cells with no movement restrictions (restricting cells provide only one choice). As a result, when both players play their optimal strategies, the path taken by Roborta to achieve the goal can be observed in the yellow-highlighted portion of the grid in Fig. 2. In Sect. 5, we evaluate this problem experimentally with different configurations of the game.

## **3 Preliminaries**

We introduce some basic definitions and results on stochastic games that will be necessary across the paper.

A (discrete) *probability distribution* μ over a denumerable set S is a function μ : S → [0, 1] such that μ(S) = - <sup>s</sup>∈<sup>S</sup> <sup>μ</sup>(s) = 1. Let <sup>D</sup>(S) denote the set of all probability distributions on S. Δ<sup>s</sup> ∈ D(S) denotes the Dirac distribution for s ∈ S, i.e., Δs(s) = 1 and Δs(s ) = 0 for all s ∈ S such that s = s. The *support* set of μ is defined by *Supp*(μ) = {s| μ(s) > 0}.

Given a set V , V <sup>∗</sup> (resp. V <sup>∞</sup>) denotes the set of all finite sequences (resp. infinite sequences) of elements of V . Concatenation is represented using juxtaposition. We use variables ω, ω , ···∈ V <sup>∞</sup> as ranging over infinite sequences, and variables ˆω, ωˆ , ··· ∈ V <sup>∗</sup> as ranging over finite sequences. The i-th element of a finite (resp. infinite) sequence ˆω (resp. ω) is denoted ˆω<sup>i</sup> (resp. ωi). Furthermore, for any finite sequence ˆω, |ωˆ| denotes its length. For ω ∈ V <sup>∞</sup>, inf(ω) denotes the set of items appearing infinitely often in <sup>ω</sup>. Given <sup>S</sup> <sup>⊆</sup> <sup>V</sup> <sup>∗</sup>, <sup>S</sup><sup>k</sup> is the set obtained by concatenating k times the sequences in S.

A *stochastic game* [11,28] is a tuple G = (V,(V1, V2, VP), δ), where V is a finite set of vertices (or states) with V1, V2, V<sup>P</sup> ⊆ V being a partition of V , and δ : V × V → [0, 1] is a probabilistic transition function, such that for every v ∈ V<sup>1</sup> ∪ V2, δ(v, v ) ∈ {0, 1}, for any v ∈ V ; and δ(v, ·) ∈ D(V ) for v ∈ VP. If V<sup>P</sup> = ∅, then G is called a two-player game graph. Moreover, if V<sup>1</sup> = ∅ or V<sup>2</sup> = ∅, then G is a *Markov decision process* (or MDP). Finally, in case that V<sup>1</sup> = ∅ and V<sup>2</sup> = ∅, G is a *Markov chain* (or MC). For all states v ∈ V we define *post* <sup>δ</sup>(v) = {v <sup>∈</sup> <sup>V</sup> <sup>|</sup> <sup>δ</sup>(v, v ) > 0}, the set of successors of v. Similarly, *pre*<sup>δ</sup>(v ) = {v ∈ V | δ(v, v ) > 0} as the set of predecessors of v , we omit the index δ when it is clear from context. Also, when useful, we fix an initial state for a game, in such a case we use the notation G<sup>v</sup> to indicate that the game starts from v. Furthermore, we assume that *post*(v) = ∅ for every v ∈ V . A vertex v ∈ V is said to be *terminal* if δ(v, v) = 1, and δ(v, v ) = 0 for all v = v . Most results on MDPs rely on the notion of *end component* [5], we straightforwardly extend this notion to two-player games: an end component of G is a pair (V , δ ) such that (a) V ⊆ V ; (b) δ (v) = <sup>δ</sup>(v) for <sup>v</sup> <sup>∈</sup> <sup>V</sup>P; (c) <sup>∅</sup> <sup>=</sup> *post* <sup>δ</sup>- (v) <sup>⊆</sup> *post* <sup>δ</sup>(v) for <sup>v</sup> <sup>∈</sup> <sup>V</sup><sup>1</sup> <sup>∪</sup>V2; (d) *post* <sup>δ</sup>- (v) ⊆ V for all v ∈ V ; (e) the underlying graph of (V , δ ) is strongly connected. Note that an end component can also be considered as being a game. The set of end components of G is denoted *EC* (G).

A *path* in the game G is an infinite sequence of vertices v0v<sup>1</sup> ... such that <sup>δ</sup>(vk, vk+1) <sup>&</sup>gt; 0 for every <sup>k</sup> <sup>∈</sup> <sup>N</sup>. *Paths*<sup>G</sup> denotes the set of all paths, and *FPaths*<sup>G</sup> denotes the set of finite prefixes of paths. Similarly, *Paths*G,v and *FPaths*G,v denote the set of paths and the set of finite paths starting at vertex v.

A *strategy* for Player i (for i ∈ {1, 2}) in a game G is a function π<sup>i</sup> : V <sup>∗</sup>V<sup>i</sup> → D(V ) that assigns a probabilistic distribution to each finite sequence of states such that πi(ˆωv)(v ) > 0 only if v ∈ *post*(v). The set of all the strategies for Player i is named Πi. A strategy π<sup>i</sup> is said to be *pure* or *deterministic* if, for every ˆωv ∈ V <sup>∗</sup>Vi, πi(ˆωv) is a Dirac distribution, and it is called *memoryless* if <sup>π</sup>i(ˆωv) = <sup>π</sup>i(v), for every ˆ<sup>ω</sup> <sup>∈</sup> <sup>V</sup> <sup>∗</sup>. Let <sup>Π</sup>*<sup>M</sup>* <sup>i</sup> and Π*<sup>D</sup>* <sup>i</sup> be respectively the set of all memoryless strategies and the set of all deterministic strategies for Player i. Π*MD* <sup>i</sup> = Π*<sup>M</sup>* <sup>i</sup> <sup>∩</sup> <sup>Π</sup>*<sup>D</sup>* <sup>i</sup> is the set of all its deterministic and memoryless strategies.

Given two strategies π<sup>1</sup> ∈ Π1, π<sup>2</sup> ∈ Π<sup>2</sup> and an initial vertex v, the *result* of the game is a Markov chain [11], denoted <sup>G</sup><sup>π</sup>1,π<sup>2</sup> <sup>v</sup> . An event A is a measurable set in the Borel σ-algebra generated by the cones of *Paths*G. The *cone* or *cylinder* spanned by the finite path ˆω ∈ *FPaths*<sup>G</sup> is the set *cyl*(ˆω) = {ω ∈ *Paths*<sup>G</sup> | ∀0 ≤ i < <sup>|</sup>ωˆ<sup>|</sup> : <sup>ω</sup><sup>i</sup> = ˆωi}. *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v is the associated probability measure obtained when fixing strategies π1, π2, and an initial vertex v [11]. Intuitively, *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (A) is the probability that strategies π<sup>1</sup> and π<sup>2</sup> generates a path belonging to the set A when the game G starts in v. When no confusion is possible, we just write *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (ˆω) instead of *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (*cyl*(ˆω)). Similar notations are used for MDPs and MCs. A stochastic game (defined as above) is said to be *stopping* [14] if for all pair of strategies π1, π<sup>2</sup> the probability of reaching a terminal state is 1. We use LTL notation to represent specific set of paths, e.g., -T = {ω ∈ *Paths*<sup>G</sup> | ∃i ≥ 0 : ω<sup>i</sup> ∈ T} is the set of all the plays in the game that reach vertices in T.

A *quantitative objective* or *payoff function* is a measurable function f : V <sup>∞</sup> → R. Let E<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f] be the expectation of measurable function <sup>f</sup> under probability *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v . The goal of Player 1 is to maximize this value whereas the goal of Player 2 is to minimize it. Sometimes quantitative objective functions can be defined via *rewards*. These are assigned by a *reward function r* : <sup>V</sup> <sup>→</sup> <sup>R</sup><sup>+</sup>. We usually consider stochastic games augmented with a reward function. Moreover, we assume that for every terminal vertex v, *r* (v) = 0.

The value of the game for Player 1 at vertex v under strategy π<sup>1</sup> is defined as the infimum over all the values resulting from Player 2 strategies in that vertex, i.e., inf<sup>π</sup>2∈Π<sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f]. The *value of the game* for Player 1 is defined as the supremum of the values of all Player 1 strategies, i.e., sup<sup>π</sup>1∈Π<sup>1</sup> inf<sup>π</sup>2∈Π<sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f]. Similarly, the value of the game for a Player 2 under strategy π<sup>2</sup> and the value of the game for Player 2 are defined as sup<sup>π</sup>1∈Π<sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f] and inf<sup>π</sup>2∈Π<sup>2</sup> sup<sup>π</sup>1∈Π<sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f], respectively. We say that a game is *determined* if both values are the same, that is, sup<sup>π</sup>1∈Π<sup>1</sup> inf<sup>π</sup>2∈Π<sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f] = inf<sup>π</sup>2∈Π<sup>2</sup> sup<sup>π</sup>1∈Π<sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [f]. Martin [24] proved the determinacy of stochastic games for Borel and bounded objective functions.

In this paper we focus on the *total accumulated reward payoff* function, i.e., *rew*(ω) = -∞ <sup>i</sup>=0 *r* (ωi). Since *rew* is unbounded, the results of Martin [24] do not apply to this function. In this paper we restrict ourselves to non-negative rewards, as shown in the next sections, non-negative rewards are enough to deal with interesting case studies, we briefly discuss in Sect. 7 the possible extension of the results presented here to games having negative rewards.

#### **4 Stopping Games and Fair Strategies**

We begin this section by introducing the notions of *(almost sure) fair strategy* and *stopping games under fairness*. From now on, we assume that Player 2 represents the environment, which tries to minimize the amount of rewards obtained by the system, thus fairness restrictions will be applied to this player.

**Definition 1.** *Given a stochastic game* G = (V,(V1, V2, VP), δ)*. The set of fair plays for Player* 2 *(denoted FP*<sup>2</sup>*) is defined as follows:*

$$FP^2 = \{ \omega \in Paths\_{\mathcal{G}} \mid \forall v' \in V\_2 : v' \in \inf(\omega) \Rightarrow post(v') \subseteq \inf(\omega) \}$$

Alternatively, if we consider each vertex as a proposition, *FP*<sup>2</sup> can be written using LTL notation as: v∈V<sup>2</sup> v-∈*post*(v)(v ⇒ v ). This property is ωregular, thus it is measurable in the σ-algebra generated by the cones of *Paths*<sup>G</sup> (see e.g., [5, p.804]). This is a state-based notion of fairness, but it can be straightforwardly extended to settings where transitions are considered. For the sake of simplicity we do not do so in this paper.

Next, we introduce the notion of (almost-sure) *fair strategies* for Player 2.

**Definition 2.** *Given a stochastic game* G = (V,(V1, V2, VP), δ)*, a strategy* π<sup>2</sup> ∈ Π<sup>2</sup> *is said to be* almost-sure fair *(or simply* fair*) iff it holds that: Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (*FP*<sup>2</sup>)=1*, for every* <sup>π</sup><sup>1</sup> <sup>∈</sup> <sup>Π</sup><sup>1</sup> *and* <sup>v</sup> <sup>∈</sup> <sup>V</sup> *.*

The set of all the fair strategies for Player 2 is denoted by Π<sup>F</sup> <sup>2</sup> . We combine this notation with the notation introduced in Sect. 3, e.g., Π*<sup>M</sup>*<sup>F</sup> <sup>2</sup> refers to the set of all memoryless and fair strategies for Player 2. The previous definition is based on the notion of fair scheduler as introduced for Markov decision processes [5,7].

Note that for stopping games, every strategy is fair, because the probability of visiting a vertex infinitely often is 0. Also notice that there are games which are not stopping, but they become stopping if Player 2 uses only fair strategies. This is the main idea behind the notion of *stopping under fairness* as introduced in the following definition.

**Definition 3.** *A stochastic game* G = (V,(V1, V2, VP), δ) *is said to be* stopping under fairness *iff for all strategies* π<sup>1</sup> ∈ Π1, π<sup>2</sup> ∈ Π<sup>F</sup> <sup>2</sup> *and vertex* v ∈ V *, it holds that Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (-T)=1*, where* T *is the set of terminal vertices of* G*.*

*Checking stopping criteria.* This section is devoted to the effective characterization of games that are stopping under fairness. The following lemma states that, for every game that is not stopping under fairness, there is a *memoryless deterministic* strategy for Player 1 and a fair strategy for Player 2 that witnesses it.

**Lemma 1.** *Let* G = (V,(V1, V2, VP), δ) *be a stochastic game,* v ∈ V *, and* T *the set of terminal states of* <sup>G</sup>*. If Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (-T) < 1 *for some* π<sup>1</sup> ∈ Π<sup>1</sup> *and* π<sup>2</sup> ∈ Π<sup>F</sup> <sup>2</sup> *, then, for some memoryless and deterministic strategy* π <sup>1</sup> <sup>∈</sup> <sup>Π</sup>*MD* <sup>1</sup> *and fair strategy* π <sup>2</sup> ∈ Π<sup>F</sup> <sup>2</sup> *, Prob*<sup>π</sup>- 1,π- 2 <sup>G</sup>,v (-T) < 1*.*

The proof of this lemma follows by noticing that, if *Prob*π1,π<sup>2</sup> <sup>G</sup>,v (-T) < 1, there must be a finite path that leads with some probability to an end component not containing a terminal state and which is a trap for the fair strategy π2. This part of the game enables the construction of a memoryless deterministic strategy for Player 1 by ensuring that it follows the same finite path (but skipping loops) and that it traps Player 2 in the same end component.

The next theorem states that checking stopping under fairness in a stochastic game G can be reduced to check the stopping criteria in a MDP, which is obtained from G by fixing a strategy in Player 2 that selects among the output transitions according to a uniform distribution. Thus, this theorem enables a graph solution to determine stopping under fairness.

**Theorem 1.** *Let* G = (V,(V1, V2, VP), δ) *be a stochastic game and* T *its set of terminal states. Consider the Player 2 (memoryless) strategy* π<sup>u</sup> <sup>2</sup> : V<sup>2</sup> → D(V ) *defined by* π<sup>u</sup> <sup>2</sup> (v)(v ) = <sup>1</sup> #*post*(v) *, for all* v ∈ V<sup>2</sup> *and* v ∈ post(v)*. Then,* G *is stopping under fairness iff Prob*<sup>π</sup>1,π<sup>u</sup> 2 <sup>G</sup>,v (-T)=1 *for every* v ∈ V *and* π<sup>1</sup> ∈ Π1*.*

While the "only if" part of the theorem is direct, the "if" part is proved by contraposition using Lemma 1.

Theorem 1 introduces an algorithm to check if the stochastic game G is stopping under fairness: transform <sup>G</sup> into the MDP <sup>G</sup><sup>π</sup><sup>u</sup> <sup>2</sup> by fixing π<sup>u</sup> <sup>2</sup> in G and check whether *Prob*<sup>π</sup><sup>1</sup> Gπ<sup>u</sup> <sup>2</sup> ,v(-T) = 1 for all v ∈ V . As a consequence, we have the following theorem.

**Theorem 2.** *Checking whether the stochastic game* G *is stopping under fairness or not is in* O(*poly*(*size*(G)))*.*

Alternatively, we can use Theorem 1 to provide a direct algorithm on G and avoiding the construction of the intermediate MDP. The main idea is to use a modification of the standard *pre* operator, as shown in the following definition:

$$\begin{aligned} \exists Pr e\_f(C) = \{v \in V \mid \delta(v, C) > 0\} \\ \forall Pr e\_f(C) = \{v \in V\_2 \cup V\_{\mathbb{P}} \mid \delta(v, C) > 0\} \cup \{v \in V\_1 \mid \forall v' \in V: \delta(v, v') > 0 \Rightarrow v' \in C\} \end{aligned}$$

As usual we consider the transitive closures of these operators denoted ∃*Pre*<sup>∗</sup> f and ∀*Pre*<sup>∗</sup> <sup>f</sup> , respectively.

**Theorem 3.** *Let* G = (V,(V1, V2, VP), δ)*, be a stochastic game and let* T *be the set of its terminal states. Then, (1) Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (-T)=1 *for every* π<sup>1</sup> ∈ Π<sup>1</sup> *and* π<sup>2</sup> ∈ Π<sup>F</sup> <sup>2</sup> *iff* v ∈ V \ ∃*Pre*<sup>∗</sup> <sup>f</sup> (V \ ∀*Pre*<sup>∗</sup> <sup>f</sup> (T))*, and (2)* G *is stopping under fairness iff* ∃*Pre*<sup>∗</sup> <sup>f</sup> (V \ ∀*Pre*<sup>∗</sup> <sup>f</sup> (T)) = ∅*.*

*Determinacy of Stopping Games under Fairness.* The determinacy of stochastic games with Borel and bounded payoff functions follows from Martin's results [24]. The function *rew* is unbounded, so Martin's theorems do not apply to it. In [18], the determinacy of a general class of stopping stochastic games (called *transient*) with total rewards is proven. However, note that we restrict Player 2 to only play with fair strategies and hence, the last result does not apply either. In [26] the authors classify Player 2's strategies into proper (those ensuring termination) and improper (those prolonging the game indefinitely). For proving determinacy, the authors assume that the value of the game for Player 2's improper strategies is ∞. It is worth noting that, for proving the results below, we do not make any assumption about unfair strategies. In the following we prove that the restriction to fair plays does not affect the determinacy of the games.

Figure 3 shows the dependencies of the lemmas that eventually lead to our main results, namely, Theorem 4, which states that the general problem can be limited to only memoryless and deterministic strategies, and Theorem 5, which establishes determinacy and the correctness of the algorithmic solution through the Bellman equations. To prove Theorem 4 we use the intermidiate notion of *semi-Markov* strategies [18] and a first step to this reduction is presented in Lemma 2. Lemmas 3 and 4 ensure the transient carachteristics of stop-

**Fig. 3.** A roadmap to proving Theorems 4 and 5

ping under fairness problems. They are essential to prove that every possible total reward play yields a solution (Lemma 5). Already approaching Theorem 4, Lemma 6 states that there is always a minimizing fair strategy that is memoryless and deterministic, and Lemma 7 helps to reduce the problem from the domain of semi-Markov strategies to the domain of memoryless deterministic strategies. Using Theorem 4 and Proposition 1, which states that the Bellman equations are well behaved in the lattice of solutions, Theorem 5 is finally proved.

Intuitively, a semi-Markov strategy only takes into account the length of a play, the initial state, and the current state to select the next step in the play.

**Definition 4.** *Let* G = (V,(V1, V2, VP), δ) *be a stochastic game. A strategy* π<sup>i</sup> ∈ Π<sup>i</sup> *is called semi-Markov if:* πi(vωvˆ ) = πi(vωˆ v )*, for every* v ∈ V *and* ω, ˆ ωˆ ∈ V <sup>∗</sup> *such that* |ωˆ| = |ωˆ |*.*

Notice that, by fixing an initial state v, a semi-Markov strategy π<sup>i</sup> can be thought of as a sequence of memoryless strategies π<sup>0</sup>,v <sup>i</sup> <sup>π</sup><sup>1</sup>,v <sup>i</sup> <sup>π</sup><sup>2</sup>,v <sup>i</sup> ... where πi(v) = π<sup>0</sup>,v <sup>i</sup> (v) and πi(vωvˆ ) = π<sup>|</sup>ωˆ|+1,v <sup>i</sup> (v ). The set of all semi-Markov (resp. semi-Markov fair) strategies for player i is denoted Π*<sup>S</sup>* <sup>i</sup> (resp. Π*<sup>S</sup>*<sup>F</sup> <sup>i</sup> ).

The importance of semi-Markov strategies lies in the fact that, when Player 2 plays a semi-Markov strategy, any Player 1's strategy can be mimicked by a semi-Markov strategy as stated in the following lemma.

**Lemma 2.** *Let* G *be a stopping under fairness stochastic game, and let* π<sup>2</sup> ∈ Π*<sup>S</sup>*<sup>F</sup> <sup>2</sup> *be a fair and semi-Markov strategy. Then, for any* π<sup>1</sup> ∈ Π1*, there is a semi-Markov strategy* π<sup>∗</sup> <sup>1</sup> <sup>∈</sup> <sup>Π</sup>*<sup>S</sup>* <sup>1</sup> *such that* E<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] = <sup>E</sup><sup>π</sup><sup>∗</sup> <sup>1</sup> ,π<sup>2</sup> <sup>G</sup>,v [*rew*]*.*

*Proof (Sketch).* The proof follows the arguments of Theorem 4.2.7 in [18] adapted to our setting.

Consider the event <sup>k</sup>v <sup>=</sup> {<sup>ω</sup> <sup>∈</sup> *Paths*<sup>G</sup> <sup>|</sup> <sup>ω</sup><sup>k</sup> <sup>=</sup> <sup>v</sup> }, for k ≥ 0. That is, the set of runs in which v is reached after exactly k steps. We define π<sup>∗</sup> <sup>1</sup> as follows. For v with *Prob*π1,π<sup>2</sup> <sup>G</sup>,v (<sup>k</sup>v ) > 0 and |ωvˆ | = k,

$$
\pi\_1^\*(\widehat{\omega}v')(v'') = \operatorname{Prob}\_{\mathcal{G},v}^{\pi\_1,\pi\_2}(\Diamond^{k+1}v'' \mid \Diamond^k v').
$$

For v with *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (<sup>k</sup>v ) = 0 and |ωvˆ | = k we define π<sup>∗</sup> 1(ˆωv ) to be the uniform distribution on *post*(v ). Notice that π<sup>∗</sup> <sup>1</sup> is a semi-Markov strategy. We prove that π<sup>∗</sup> <sup>1</sup> is the strategy that satisfies the conclusion of the lemma. For this, we first show that *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (<sup>k</sup>v ) = *Prob*<sup>π</sup><sup>∗</sup> <sup>1</sup> ,π<sup>2</sup> <sup>G</sup>,v (<sup>k</sup>v ) by induction on k, and use it to conclude the following.

$$\begin{split} \mathbb{E}\_{\mathcal{G},v}^{\pi\_{1},\pi\_{2}}[rew] &= \sum\_{N=0}^{\infty} \sum\_{\hat{\omega} \in V^{N+1}} \operatorname{Prob}\_{\mathcal{G},v}^{\pi\_{1},\pi\_{2}}(\hat{\omega}) r(\hat{\omega}\_{N}) = \sum\_{N=0}^{\infty} \sum\_{v' \in V} \operatorname{Prob}\_{\mathcal{G},v}^{\pi\_{1},\pi\_{2}}(\bigcirc^{N} v') r(v') \\ &= \sum\_{N=0}^{\infty} \sum\_{v' \in V} \operatorname{Prob}\_{\mathcal{G},v}^{\pi\_{1}^{\pi},\pi\_{2}}(\bigcirc^{N} v') r(v') = \mathbb{E}\_{\mathcal{G},v}^{\pi\_{1}^{\pi},\pi\_{2}}[rew] \end{split}$$

In a stopping game, all non-terminal states are transient (a state is transient if the expected time that both players spend in it is finite). In fact, [18] defines a stopping game with terminal states in - T as a *transient game*, i.e., a game in which ∞ N=1 - <sup>ω</sup>ˆ∈(<sup>V</sup> \T)<sup>N</sup> *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (ˆω) <sup>&</sup>lt; <sup>∞</sup> for all strategies <sup>π</sup><sup>1</sup> <sup>∈</sup> <sup>Π</sup><sup>1</sup> and <sup>π</sup><sup>2</sup> <sup>∈</sup> <sup>Π</sup>2. Obviously, this generality does not hold in our case since unfair strategies make the game dwell infinitely on a set of non-terminal states. Therefore, we prove a weaker property in our setting. Roughly speaking, the next lemma states that, in games that stop under fairness, non-terminal states are transient, provided that the two players play memoryless strategies, and in particular, that Player 2 plays only fair.

**Lemma 3.** *Let* G = (V,(V1, V2, VP), δ) *be a stochastic game that is stopping under fairness with* <sup>T</sup> *being the set of terminal states. Let* <sup>π</sup><sup>1</sup> <sup>∈</sup> <sup>Π</sup>*<sup>M</sup>* <sup>1</sup> *be a memoryless strategy for Player 1 and* <sup>π</sup><sup>2</sup> <sup>∈</sup> <sup>Π</sup>*<sup>M</sup>*<sup>F</sup> <sup>2</sup> *a memoryless fair strategy for Player 2. Then* -∞ N=1 - <sup>ω</sup>ˆ∈(<sup>V</sup> \T)<sup>N</sup> *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (ˆω) <sup>&</sup>lt; <sup>∞</sup>*.*

This result can be extended to all the strategies of Player 1. The main idea behind the proof is to fix a stationary fair strategy for Player 2 (e.g., a uniform distributed strategy). This yields an MDP that stops for every strategy of Player 1, and furthermore, it can be seen as a one-player *transient game* (as defined in [18]). Hence, the result follows from Lemma 3 and Theorem 4.2.12 in [18].

**Lemma 4.** *Let* G *be a stochastic game that is stopping under fairness and let* T *be the set of terminal states. In addition, let* π<sup>1</sup> ∈ Π<sup>1</sup> *be a strategy for Player* <sup>1</sup> *and* <sup>π</sup><sup>2</sup> <sup>∈</sup> <sup>Π</sup>*<sup>M</sup>*<sup>F</sup> 2 - *be a fair and memoryless strategy for Player* 2*. Then* ∞ N=0 - <sup>ω</sup>ˆ∈v(<sup>V</sup> \T)<sup>N</sup> *Prob*<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v (ˆω) <sup>&</sup>lt; <sup>∞</sup>*.*

Using the previous lemma, some fairly simple calculations lead to the fact that the value of the total accumulated reward payoff game is well-defined for any strategy of the players. As a consequence, the value of the game is bounded from above for any Player 1's strategy. This is stated in the next lemma.

**Lemma 5.** *Let* G = (V,(V1, V2, VP), δ, *r* ) *be a stochastic game that is stopping under fairness,* π<sup>1</sup> ∈ Π<sup>1</sup> *a strategy for Player 1. Then, for all memoryless fair strategy* <sup>π</sup><sup>2</sup> <sup>∈</sup> <sup>Π</sup>*M*<sup>F</sup> <sup>2</sup> *for Player 2 and all* <sup>v</sup> <sup>∈</sup> <sup>V</sup> *,* <sup>E</sup>π1,π<sup>2</sup> <sup>G</sup>,v [*rew*] <sup>&</sup>lt; <sup>∞</sup>*. Moreover, for every vertex* v ∈ V *,* inf<sup>π</sup>2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup>π1,π<sup>2</sup> <sup>G</sup>,v [*rew*] <sup>&</sup>lt; <sup>∞</sup>*.*

The following lemma is crucial and plays an important role in the rest of the paper. Intuitively, it states that, when Player 1 plays with a memoryless strategy, Player 2 has an optimal deterministic memoryless fair strategy. This lemma is the guarantee of the eventual existence of a minimizing memoryless deterministic fair strategy for Player 2 in general.

**Lemma 6.** *Let* G = (V,(V1, V2, VP), δ, *r* ) *be a stochastic game that is stopping under fairness and let* <sup>π</sup><sup>1</sup> <sup>∈</sup> <sup>Π</sup>*<sup>M</sup>* <sup>1</sup> *be a memoryless strategy for Player 1. There exists a deterministic memoryless fair strategy* π<sup>∗</sup> <sup>2</sup> <sup>∈</sup> <sup>Π</sup>*MD*<sup>F</sup> <sup>2</sup> *such that* inf<sup>π</sup>2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] = <sup>E</sup><sup>π</sup>1,π<sup>∗</sup> 2 <sup>G</sup>,v [*rew*]*, for every* <sup>v</sup> <sup>∈</sup> <sup>V</sup> *.*

*Proof (Sketch).* Though it differs in the details, the proof strategy is inspired by the proof of Lemma 10.102 in [5]. We first construct a reduced MDP <sup>G</sup><sup>π</sup><sup>1</sup> min which preserves exactly the optimizing part of the MDP <sup>G</sup><sup>π</sup><sup>1</sup> . Thus <sup>δ</sup><sup>π</sup><sup>1</sup> min(v, v ) = δ<sup>π</sup><sup>1</sup> (v, v ) if v ∈ V<sup>1</sup> ∪ VP, or v ∈ V<sup>2</sup> and x<sup>v</sup> = *r* (v) + x<sup>v</sup>- , where, for every v ∈ V , x<sup>v</sup> = inf<sup>π</sup>2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] (which exists due to Lemma 5). Otherwise, δ<sup>π</sup><sup>1</sup> min(v, v ) = 0. <sup>G</sup><sup>π</sup><sup>1</sup> min can be proved to be stopping under fairness.

Then, the strategy π<sup>∗</sup> <sup>2</sup> for <sup>G</sup><sup>π</sup><sup>1</sup> min is constructed as follows. For every v ∈ V , let v be the length of the shortest path fragment to some terminal vertex in T in the MDP <sup>G</sup><sup>π</sup><sup>1</sup> min. Define π<sup>∗</sup> 2(v)(v ) = 1 for some v such that δ<sup>π</sup><sup>1</sup> min(v, v ) = 1 and v = v + 1. By definition, π<sup>∗</sup> <sup>2</sup> is memoryless. We prove first that π<sup>∗</sup> <sup>2</sup> yields the optimal solution of <sup>G</sup><sup>π</sup><sup>1</sup> by showing that the vector (xv)<sup>v</sup>∈<sup>V</sup> (i.e., the optimal values of <sup>G</sup><sup>π</sup><sup>1</sup> ) is a solution to the set of equations for expected rewards of the Markov chain <sup>G</sup><sup>π</sup>1,π<sup>∗</sup> <sup>2</sup> . Being the solution unique, we have that x<sup>v</sup> = E Gπ1,π<sup>∗</sup> <sup>2</sup> ,v[*rew*] for all v ∈ V and hence the optimality of π<sup>∗</sup> <sup>2</sup>. To conclude the proof we show by contradiction that π<sup>∗</sup> <sup>2</sup> is fair.

As already noted, semi-Markov strategies can be thought of as sequences of memoryless strategies. The next lemma uses this fact to show that, when Player 2 plays a memoryless and fair strategy, semi-Markov strategies do not improve the value that Player 1 can obtain via memoryless deterministic strategies. The proof of the following lemma adapts the ideas of Theorem 4.2.9 in [18] to our games.

**Lemma 7.** *For any stochastic game* G *that is stopping under fairness, and vertex* v*, it holds that:*

$$\sup\_{\pi\_1 \in \Pi\_1^S, \pi\_2 \in \Pi\_2^{MD\mathcal{F}}} \inf\_{\pi\_2 \in \Pi\_2^{MD\mathcal{F}}} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2}[rew] = \sup\_{\pi\_1 \in \Pi\_1^{MD}, \pi\_2 \in \Pi\_2^{MD\mathcal{F}}} \inf\_{\pi\_2 \in \Pi\_2^{MD\mathcal{F}}} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2}[rew]$$

Using the previous lemma, we can conclude that the problem of finding supπ1∈Π<sup>1</sup> infπ2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup>π1,π<sup>2</sup> [*rew*], for any vertex <sup>v</sup>, can be solve by only focusing on deterministic memoryless strategies as stated and proved in the following theorem.

**Theorem 4.** *For any stochastic game* G *that is stopping under fairness we have:*

$$\sup\_{\pi\_1 \in \Pi\_1} \inf\_{\pi\_2 \in \Pi\_2^{\mathcal{F}}} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2}[rew] = \sup\_{\pi\_1 \in \Pi\_1^{MD}} \inf\_{\pi\_2 \in \Pi\_2^{MD\mathcal{F}}} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2}[rew]$$

*Proof.* First, we prove that the left-hand term is less than or equal to the righthand one:

$$\begin{split} \sup\_{\pi\_{1} \in \Pi\_{1}} \inf\_{\pi\_{2} \in \Pi\_{2}^{\mathcal{F}}} \mathbb{E}\_{\mathcal{G}, \boldsymbol{v}}^{\pi\_{1}, \pi\_{2}}[\boldsymbol{re}\boldsymbol{w}] &\leq \sup\_{\pi\_{1} \in \Pi\_{1}} \inf\_{\pi\_{2} \in \Pi\_{2}^{\text{MDF}}} \mathbb{E}\_{\mathcal{G}, \boldsymbol{v}}^{\pi\_{1}, \pi\_{2}}[\boldsymbol{re}\boldsymbol{w}] \\ &\leq \sup\_{\pi\_{1} \in \Pi\_{1}^{\mathcal{S}}} \inf\_{\pi\_{2} \in \Pi\_{2}^{\text{MDF}}} \mathbb{E}\_{\mathcal{G}, \boldsymbol{v}}^{\pi\_{1}, \pi\_{2}}[\boldsymbol{re}\boldsymbol{w}] \\ &\leq \sup\_{\pi\_{1} \in \Pi\_{1}^{\text{MD}}} \inf\_{\pi\_{2} \in \Pi\_{2}^{\text{MDF}}} \mathbb{E}\_{\mathcal{G}, \boldsymbol{v}}^{\pi\_{1}, \pi\_{2}}[\boldsymbol{re}\boldsymbol{w}]. \end{split}$$

The first inequality follows from Π*MD*<sup>F</sup> <sup>2</sup> ⊆ Π<sup>F</sup> <sup>2</sup> , the second inequality is due to Lemma 2 and the fact that memoryless strategies are semi-Markov, and the last inequality is obtained by applying Lemma 7.

To prove the other inequality, we calculate:

$$\begin{split} \sup\_{\pi\_{1}\in\Pi\_{1}^{MD}} \inf\_{\pi\_{2}\in\Pi\_{2}^{MD}} \mathbb{E}\_{\mathcal{G},v}^{\pi\_{1},\pi\_{2}}[rew] &= \sup\_{\pi\_{1}\in\Pi\_{1}^{MD}} \inf\_{\pi\_{2}\in\Pi\_{2}^{F}} \mathbb{E}\_{\mathcal{G},v}^{\pi\_{1},\pi\_{2}}[rew] \\ &\leq \sup\_{\pi\_{1}\in\Pi\_{1}} \inf\_{\pi\_{2}\in\Pi\_{2}^{F}} \mathbb{E}\_{\mathcal{G},v}^{\pi\_{1},\pi\_{2}}[rew]. \end{split}$$

The first equality is a consequence of Lemma 6 and the second inequality is due to properties of suprema.

The standard technique to prove the determinacy of stopping games is by showing that the Bellman operator

$$\Gamma(f)(v) = \begin{cases} r(v) + \sum\_{v' \in post(v)} \delta(v, v') f(v') & \text{if } v \in V\_{\mathbb{P}} \backslash T \\ \max\{r(v) + f(v') \mid v' \in post(v)\} & \text{if } v \in V\_1 \backslash T, \\ \min\{r(v) + f(v') \mid v' \in post(v)\} & \text{if } v \in V\_2 \backslash T, \\ 0 & \text{if } v \in T. \end{cases}$$

has a unique fixpoint. However, in the case of games stopping under fairness, Γ has several fixpoints as shown by the next example.

*Example 1.* Consider the (one-player) game in Fig. 4, where Player 1's vertices are drawn as boxes, Player 2's vertices are drawn as diamonds, and probabilistic vertices are depicted as circles. Note that, in that game, the greatest fixpoint is (1, 1, 1, 0). Yet, (0.5, 0.5, 1, 0) is also a

**Fig. 4.** A game with infinite fixpoints

fixpoint as Γ(0.5, 0.5, 1, 0) = (0.5, 0.5, 1, 0). In fact, the Bellman operator for this game has infinite fixpoints: any f of the form (x, x, 1, 0) with x ∈ [0, 1].

Thus, the standard approach cannot be used here. Instead, we use the greatest fixpoint for proving determinacy, but this cannot be done directly on Γ. A main difficulty is that the Knaster-Tarski theorem does not apply for <sup>Γ</sup> since (R<sup>V</sup> , <sup>≤</sup>) is not a complete lattice. Using instead the extended reals ((R∪{∞})<sup>V</sup> ) is not a solution, as in some cases the greatest fixpoint will assign ∞ to some vertices (e.g., (∞,∞, 0) would be the greatest fixpoint in the Markov chain of Fig. 5). One possible approach is to approximate the greatest fixpoint from an estimated upper bound via value iteration. Unfortunately, there may not be an order relation between f and Γ(f) and it may turn out that for some vertex v, Γ(f)(v) > f(v) before converging to the fixpoint. This is shown in the next example.

*Example 2.* Consider the game depicted in Fig. 5. The (unique) fixpoint in this case is (100, 90, 0). Observe that, we have that Γ(120, 100, 0) = (110, 108, 0), thus the value at v<sup>1</sup> increases after one iteration. Several iterations are needed then to reach the greatest fixpoint. Thus, in general, starting value iteration from an estimated upper bound does not guarantee a monotone convergence to the greatest fixpoint.

We overcome the aforementioned issues by using a modified version of Γ. Roughly speaking, we modify the Bellman operator in such a way that it operates over a complete lattice.

Notice that, by Lemma 5, the value E<sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] is finite for every stopping game under fairness

**Fig. 5.** A game where value iteration may go up

<sup>G</sup> and strategies <sup>π</sup><sup>1</sup> <sup>∈</sup> <sup>Π</sup>*MD* <sup>1</sup> , <sup>π</sup><sup>2</sup> <sup>∈</sup> <sup>Π</sup>*MD*<sup>F</sup> <sup>2</sup> . Furthermore, because the number of deterministic memoryless strategies is finite, we also have that the number max{inf<sup>π</sup>2∈Π*MD*<sup>F</sup> <sup>2</sup> sup<sup>π</sup>1∈Π*MD* <sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] <sup>|</sup> <sup>v</sup> <sup>∈</sup> <sup>V</sup> } is well defined. From now on, fix a number **U** ≥ max{inf<sup>π</sup>2∈Π*MD*<sup>F</sup> <sup>2</sup> sup<sup>π</sup>1∈Π*MD* <sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] <sup>|</sup> <sup>v</sup> <sup>∈</sup> <sup>V</sup> }. We define a modified Bellman operator Γ<sup>∗</sup> : [0, **U**] <sup>V</sup> <sup>→</sup> [0, **<sup>U</sup>**] <sup>V</sup> as follows.

$$\boldsymbol{I}^{\*}(f)(v) = \begin{cases} \min\left(r(v) + \sum\_{v' \in post(v)} \delta(v, v')f(v'), \mathbf{U} \mid \right) & \text{if } v \in V\_{\mathbb{P}} \mid T\\\min\left(\max\{r(v) + f(v') \mid v' \in post(v)\}, \mathbf{U} \mid \right) & \text{if } v \in V\_{1} \mid T, \\\min\left(\min\{r(v) + f(v') \mid v' \in post(v)\}, \mathbf{U} \mid \right) & \text{if } v \in V\_{2} \nmid T, \\\ 0 & \text{if } v \in T. \end{cases}$$

Note that Γ<sup>∗</sup> is monotone, which can be proven by observing that maxima, minima and convex combinations are all monotone operators. Furthermore, Γ<sup>∗</sup> is also Scott continuous (it preserves suprema of directed sets), this can be proven similarly as in [10]. The following proposition formalizes these properties.

#### **Proposition 1.** Γ<sup>∗</sup> *is monotone and Scott-continuous.*

Note that ([0, **U**] <sup>V</sup> , <sup>≤</sup>) is a complete lattice. Thus by Proposition <sup>1</sup> and the Knaster-Tarski theorem [15], the (non-empty) set of fixed points of Γ<sup>∗</sup> forms a complete lattice, and the greatest fixpoint of the operator can be approximated by successive applications of Γ<sup>∗</sup> to the top element (i.e., **U**) [15]. In the following we denote by νΓ<sup>∗</sup> the greatest fixed point of Γ∗.

The following theorem states that games restricted to fair strategies on Player 2 are determinate. Furthermore, the value of the game is given by the greatest fixpoint of Γ∗.

**Theorem 5.** *Let* G *be a stochastic game that is stopping under fairness. It holds that:*

$$\inf\_{\pi\_2 \in \Pi\_2^{\mathcal{F}}} \sup\_{\pi\_1 \in \Pi\_1} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2}[rew] = \sup\_{\substack{\pi\_1 \in \Pi\_1 \ \pi\_2 \in \Pi\_2^{\mathcal{F}}}} \inf\_{\pi\_2 \in \Pi\_2^{\mathcal{F}}} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2}[rew] = \nu \Gamma^\*(v)$$

*Proof.* First, note that inf<sup>π</sup>2∈Π*MD*<sup>F</sup> <sup>2</sup> sup<sup>π</sup>1∈Π*MD* <sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] is a fixed point of <sup>Γ</sup>∗. Thus we have:

$$\begin{aligned} \sup\_{\pi\_1 \in \Pi\_1} \inf\_{\pi\_2 \in \Pi\_2^F} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2} [new] &\leq \inf\_{\pi\_2 \in \Pi\_2^F} \sup\_{\pi\_1 \in \Pi\_1} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2} [new] \\ &\leq \inf\_{\pi\_2 \in \Pi\_2^{MDF}} \sup\_{\pi\_1 \in \Pi\_1^{MD}} \mathbb{E}\_{\mathcal{G}, v}^{\pi\_1, \pi\_2} [new] \leq \nu \Gamma^\*(v), \end{aligned}$$

for any v. The first inequality is a standard property of suprema and infima [21], the second inequality holds because Π*MD*<sup>F</sup> <sup>2</sup> ⊆ Π<sup>F</sup> <sup>2</sup> and standard properties of MDPs: by fixing a deterministic memoryless fair strategy for Player 2 we obtain a transient MDP, the optimal strategy for Player 1 in this MDP is obtained via a deterministic memoryless strategy [20]. The last inequality holds because inf<sup>π</sup>2∈Π*MD*<sup>F</sup> <sup>2</sup> sup<sup>π</sup>1∈Π*MD* <sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] is a fixpoint of <sup>Γ</sup>∗.

Rest to prove that sup<sup>π</sup>1∈Π<sup>1</sup> inf<sup>π</sup>2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] <sup>≥</sup> νΓ∗(v). Note that, if there is π<sup>1</sup> ∈ Π<sup>1</sup> such that inf<sup>π</sup>2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*] <sup>≥</sup> νΓ∗(v) the property above follows by properties of supremum. Consider the strategy π<sup>∗</sup> <sup>1</sup> defined as follows: π<sup>∗</sup> <sup>1</sup>(v) ∈ argmax{νΓ∗(v ) + *r* (v) | v ∈ *post*(v)}. Note that π<sup>∗</sup> <sup>1</sup> is a memoryless and deterministic strategy. For any memoryless, deterministic and fair strategy <sup>π</sup><sup>2</sup> <sup>∈</sup> <sup>Π</sup>*MD*<sup>F</sup> <sup>2</sup> we have νΓ∗(v) <sup>≤</sup> <sup>E</sup><sup>π</sup><sup>∗</sup> <sup>1</sup> ,π<sup>2</sup> <sup>G</sup>,v [*rew*] (by definition of <sup>Γ</sup>∗). Thus, νΓ∗(v) <sup>≤</sup> inf<sup>π</sup>2∈Π*MD*<sup>F</sup> <sup>2</sup> <sup>E</sup>π<sup>∗</sup> <sup>1</sup> ,π<sup>2</sup> <sup>G</sup>,v [*rew*] and then: νΓ∗(v) <sup>≤</sup> sup<sup>π</sup>1∈Π*MD* <sup>1</sup> inf<sup>π</sup>2∈Π*MD*<sup>F</sup> <sup>1</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*]. Finally, by Theorem 4 we get: νΓ∗(v) ≤ sup<sup>π</sup>1∈Π<sup>1</sup> inf<sup>π</sup>2∈Π<sup>F</sup> <sup>2</sup> <sup>E</sup><sup>π</sup>1,π<sup>2</sup> <sup>G</sup>,v [*rew*].

*Considerations for an algorithmic solution.* Value iteration [9] has been used to compute maximum/minimum expected accumulated reward in MDPs, e.g., in the PRISM model checker. Usually, the value is computed by approximating the least fixpoint from below using the Bellman equations [9]. In [6], the authors propose to approach these values from both a lower and an upper bound (known as interval iteration [19]). To do so, [6] shows a technique for computing upper bounds for the expected total rewards for MDPs. This approach is based on the fact that, given a stopping MDP <sup>G</sup>, <sup>E</sup><sup>π</sup><sup>1</sup> <sup>G</sup>,v[*rew*] = - v-<sup>∈</sup>R(v) <sup>ζ</sup><sup>π</sup><sup>1</sup> <sup>v</sup> (v ) ∗ *r* (v ), where R(v) denotes the set of reachable states from v, and ζ<sup>π</sup><sup>1</sup> <sup>v</sup> (v ) denotes the expected number of times to visit v in the Markov chain induced by π<sup>1</sup> when starting at v. [6] describes how to compute a value ζ<sup>∗</sup> <sup>v</sup> (v ), such that ζ<sup>∗</sup> <sup>v</sup> (v ) <sup>≥</sup> sup<sup>π</sup>1∈Π<sup>1</sup> <sup>ζ</sup><sup>π</sup><sup>1</sup> <sup>v</sup> (v ). Thus, - v-<sup>∈</sup>R(v) <sup>ζ</sup><sup>∗</sup> <sup>v</sup> (v )∗*r* (v ) gives an upper bound for sup<sup>π</sup><sup>1</sup> <sup>E</sup><sup>π</sup><sup>1</sup> G,v[*rew*]. Our algorithm uses these ideas to provide an upper bound for two-player games. Roughly speaking, the above defined functional Γ<sup>∗</sup> presents a form of Bellman equations that enables a value iteration algorithm to solve these games. We need to start with some value vector larger than such a fixpoint. Given a stopping under fairness game, we fix a (memoryless) fair strategy for the environment, thus obtaining an MDP. We then use the techniques described above to find an

upper bound for this MDP, which in turn is an upper bound in the original game. The obvious fair strategy to use is the one based on the uniform distribution (as in Theorem 1). This idea is described in Algorithm 1. It is worth noting that, instead of using a unique upper bound for every vertex (as in the definition of Γ∗), the algorithm may use a different upper bound for each component of the value vec-


tor, this improves the number of iterations performed by the algorithm. We have implemented Algorithm 1 as a prototype embedded in the PRISM-games toolset [22], as described in the next section.

#### **5 Experimental Validation**

In order to evaluate the viability of our approach we have extended the model checker PRISM [22,23] with an operator to compute the expected rewards for stochastic games that stop under fairness. The prototype also allows one to check whether a game is stopping under fairness. The tool takes as input a model describing the game in PRISM notation and returns as output the optimal expected total reward for a given initial state as well as the synthesized optimal controller strategy (under fairness assumptions). The experimental evaluation shows that our approach can cope with non-trivial case studies. For computing these values we set a relative error of at most ε = 10−<sup>6</sup>.

*Roborta vs. the Fair Light.* Table 1 shows the results of the example introduced in Sect. 2 for multiple configurations. We considered three variants of the case study: version A (the light does not fail), version B (the light can only fail when trying to signal a green light), and version C (the light can fail when trying to signal any kind of light). We assumed that, when Roborta fails, she cannot move (this is beneficial to Roborta since she can re-collect the reward); when the light fails, the robot can freely move into any allowed direction. The grid configuration (movement restrictions and rewards) are randomly generated. For each setting, Table 1 describes the results for three different scenarios generated starting at different seeds. For the grid configuration shown in Sect. 2 with parameters P = 0.1 and Q = 0, the tool derived the optimal strategy depicted in Fig. 2 and reports an expected total reward of 5.55.

*Autonomous UAV vs. Human Operator.* We adapted the case study analyzed in [17]. A remotely controlled Unmanned Aerial Vehicle (UAV) is used to perform intelligence, surveillance, and reconnaissance (ISR) missions over a road network. The UAV performs piloting functions autonomously (selecting a path to fly between *waypoints*). The human operator (environment) controls the onboard

sensor to capture imagery at a waypoint as well as the piloting functions on certain waypoints (called checkpoints). Note that an operator can continuously try to get a better image by making the UAV loiter around a certain waypoint, this may lead to an unfair behavior. Each successful capture from an unvisited waypoint grants a reward. Figure 6 shows an example of road network consisting of six surveillance waypoints labeled w0, w2, ..., w5, the edges represent connecting paths, a red-dashed line means that the path is dangerous enough to make the

**Fig. 6.** UAV Network for ISR missions adapted from [17]

UAV stop working with probability 1, while on any other path, this probability is S. Checkpoints are depicted as pink nodes, therein the operator can still delegate the piloting task to the UAV with probability D. Each node is annotated with three possible rewards. For instance, for S = 0.3 and D = 0.5 and the leftmost reward values in each triple, the synthesized strategy for the UAV tries to follow the optimal circuit w0, w1, w2, w3, w4, w5. While for the middle and rightmost reward values, the optimal circuits to follow are w0, w5, w0, w1, w2, w3, w<sup>4</sup> and w0, w5, w4, w1, w2, w3, respectively. Table 2 shows the results obtained for this game for several randomly generated road networks.

Tables 1 and 2 do not report the time taken to compute the results, but in all cases the output was computed in less than 400 s.All the experiments were run on a MacBook Air with Intel Core i5 at 1.3 GHz and 4 Gb of RAM.

#### **6 Related Work**

Stochastic games with payoff functions have been extensively investigated in the literature. In [18], several results are presented about *transient games*, a generalized version of stopping stochastic games with total reward payoff. In transient games, both players possess optimal (memoryless and deterministic) strategies. Most importantly, the games are determined and their value can be computed as the least fixed point of a set of equations. Most of these results are based on the fact that the Γ functional (see Sect. 4) for transient games has a unique fixed point. Notice that in this paper we have dealt with games that are stopping only under fairness assumptions. Thus, the corresponding functional may have several fixed points. Hence, the main results presented in [18] do not apply to our setting.

[12] and [28] present logical frameworks for the verification and synthesis of systems. While [12] provides a solution for a probabilistic branching temporal **Table 1.** Results for Roborta vs. Light Game. First column describes the grid size. Second column indicates the fault probability for the robot (P) and light (Q). The other columns describe the size of the model, the expected total reward for the optimal strategy, and the number of iterations performed, respectively, for three different randomly generated grid configurations.


**Table 2.** Results for the UAV vs. Operator Game. First column describes the number of waypoints used. Second column indicates probability of delegation (D), and the probability that the UAV stops working (S). The other columns show the size of the model, the expected total reward for the optimal strategy, and the number of iterations performed, respectively, for three different randomly generated roadmap configurations.


logic extended with expected total, discounted, and average reward objective functions, [28] does the same in a similar extension of a probabilistic linear temporal logic. Both frameworks were implemented in the tool PRISM [22,23]. Although a vast class of properties can be expressed in these frameworks, none of them are presented under fair environments. In fact, these works are on stochastic multiplayer games in which each player is treated equally.

However, of all the operators in [12,22,28], p1 Rmax=?[F∞T] is the closest to our proposal and it deserves a deeper comparison. p1 Rmax=?[F∞T] returns the expected accumulated reward until reaching T in which infinite plays receive an infinite value [12,22]. PRISM approximates this value by computing a greatest fixpoint. It uses a two-phase algorithm to do so: (i) it first replaces zero rewards with a small positive value and applies value iteration on this modification to get an estimated upper bound, and (ii) this upper bound is used to start another value iteration process aimed to compute the greatest fixpoint.

This heuristic could return erroneous approximations of the greatest fixpoint. We illustrate this with a simple example. Consider the game depicted in Fig. 7, For any p, the value of the greatest fixpoint in vertex v<sup>0</sup> is 2. However, by taking p = 0.99 and tolerance  = 10−<sup>6</sup>, PRISM returns a value close to 39608. This occurs because PRISM changes 0 to

**Fig. 7.** A simple two-player game: only probability less than 1 are shown

the value 0.02, which results in an extremely large upper bound. Obviously, it also returns an incorrect strategy for vertex v0. We have checked this example with our tool, and it returned the correct value for vertex v<sup>0</sup> in 2 iterations, regardless of the value of p. We have chosen a large value for p to make the difference noticeable. Small values also may produce different values in, e.g., v<sup>1</sup> only that it could be blamed on approximation errors. We have also run this operator on our case studies and observed small differences in many of them (particularly on Roborta) that get larger when the fault probabilities get larger as well.

*Stochastic shortest path games* [26] are two-player stochastic games with (negative or positive) rewards in which the minimizer's strategies are classified into *proper* and *improper*, proper strategies are those ensuring termination. As proven in [26], these games are determined, and both players posses memoryless optimal strategies. To prove these results, the authors assume that the expected game value for improper strategies is ∞, this ensures that the corresponding functional is a contraction and thus it has a unique fixpoint. In contrast, we restrict ourselves to non-negative rewards but we do not make any assumptions over unfair strategies, as mentioned above the corresponding functional for our games may have several fixpoints. Furthermore, we proved that the value of the game is given by the greatest fixpoint of Γ. In recent years, several authors have investigated stochastic shortest path problems for MDPs (i.e., one-player games), where the assumption over improper strategies is relaxed (e.g., [3]); to the best of our knowledge, these results have not be extended to two-player games.

In [4] the authors tackle the problem of synthesizing a controller that maximizes the probability of satisfying an LTL property. Fairness strategies are used to reduce this problem to the synthesis of a controller maximizing a PCTL property over a product game. However, this article does not address expected rewards and game determinacy under fairness assumptions.

Interestingly, in [2] the authors consider the problem of winning a (nonstochastic) two-player games with fairness assumptions over the environment. The objective of the system is to guarantee an ω-regular property. The authors show that winning in these games is equivalent to almost-sure winning in a Markov decision process. It must be noted that this work only considers nonstochastic games. Furthermore, payoff functions are not considered therein.

Finally, we remark that in *qualitative* ω-regular stochastic games [1] strong fairness can easily be consider by properly transforming the original ω-regular objective. Notably, in this setting, [8] shows that qualitative Rabin conditions on stochastic games can be solved by translating this problem into a two-player (non-stochastic) game with the same Rabin condition under extreme fairness following a somewhat inverse direction to that we used to prove Theorem 2.

#### **7 Concluding Remarks**

In this paper, we have investigated the properties of stochastic games with total reward payoff under the assumption that the minimizer (i.e., the environment) plays only with fair strategies. We have shown that, in this scenario, determinacy is preserved and both players have optimal memoryless and deterministic strategies; furthermore, the value of the game can be calculated by approximating a greatest fixed point of a Bellman operator. We have only considered non-negative rewards in this paper. A possible way of extending the results presented here to games with negative rewards is to adapt the techniques presented in [3] for MDPs with negative costs, we leave this as a further work.

In order to show the applicability of our technique, we have presented two examples of applications and an experimental validation over diverse instances of these case studies using our prototype tool. We believe that fairness assumptions allow one to consider more realistic behavior of the environment.

We have not investigated other common payoff functions such as discounted payoff or limiting-average payoff. A benefit of these classes of functions is that the value of games are well-defined even when the games are not stopping. At first sight, the notion of fairness is little relevant for games with discounted payoff, since these kinds of payoff functions take most of their value from the initial parts of runs. For limiting-average the situation is different, and fairness assumptions may be relevant as they could change the value of games, we leave this as further work.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Automated Expected Amortised Cost Analysis of Probabilistic Data Structures**

Lorenz Leutgeb1(B) , Georg Moser<sup>2</sup> , and Florian Zuleger<sup>3</sup>

<sup>1</sup> Max Planck Institute for Informatics and Graduate School of Computer Science, Saarland Informatics Campus, Saarbr¨ucken, Germany lorenz@mpi-inf.mpg.de <sup>2</sup> Department of Computer Science, Universit¨at Innsbruck, Innsbruck, Austria <sup>3</sup> Institute of Logic and Computation 192/4, Technische Universit¨at Wien, Vienna, Austria

**Abstract.** In this paper, we present the first fully-automated *expected amortised cost analysis* of self-adjusting data structures, that is, of *randomised splay trees*, *randomised splay heaps* and *randomised meldable heaps*, which so far have only (semi-)manually been analysed in the literature. Our analysis is stated as a type-and-effect system for a first-order functional programming language with support for sampling over discrete distributions, non-deterministic choice and a ticking operator. The latter allows for the specification of fine-grained cost models. We state two soundness theorems based on two different—but strongly related—typing rules of ticking, which account differently for the cost of non-terminating computations. Finally we provide a prototype implementation able to fully automatically analyse the aforementioned case studies.

**Keywords:** amortised cost analysis · functional programming · probabilistic data structures · automation · constraint solving

#### **1 Introduction**

*Probabilistic* variants of well-known computational models such as automata, Turing machines or the λ-calculus have been studied since the early days of computer science (see [16,17,25] for early references). One of the main reasons for considering probabilistic models is that they often allow for the design of more efficient algorithms than their deterministic counterparts (see e.g. [6,23,25]). Another avenue for the design of efficient algorithms has been opened up by Sleator and Tarjan [34,36] with their introduction of the notion of *amortised complexity*. Here, the cost of a single data structure operation is not analysed in isolation but as part of a sequence of data structure operations. This allows for the design of algorithms where the cost of an expensive operation is averaged out over multiple operations and results in a good overall *worst-case cost*. Both methodologies—*probabilistic programming* and *amortised complexity*—can be combined for the design of even more efficient algorithms, as for example in *randomized splay trees* [1], where a rotation in the splaying operation is only performed with some probability (which improves the overall performance by skipping some rotations while still guaranteeing that enough rotations are performed).

In this paper, we present the first fully-automated *expected amortised cost analysis* of probabilistic data structures, that is, of *randomised splay trees*, *randomised splay heaps*, *randomised meldable heaps* and a *randomised analysis* of a *binary search tree*. These data structures have so far only (semi-)manually been analysed in the literature. Our analysis is based on a novel type-and-effect system, which constitutes a generalisation of the type system studied in [14,18] to the nondeterministic and probabilistic setting, as well as an extension of the type system introduced in [37] to sublinear bounds and non-determinism. We provide a prototype implementation that is able to fully automatically analyse the case studies mentioned above. We summarise here the main contributions of our article: (i) We consider a first-order functional programming language with support for *sampling over discrete distributions*, *non-deterministic choice* and a *ticking* operator, which allows for the specification of fine-grained cost models. (ii) We introduce compact *small-step* as well as *big-step* semantics for our programming language. These semantics are equivalent wrt. the obtained normal forms (i.e., the resulting probability distributions) but differ wrt. the cost assigned to non-terminating computations. (iii) Based on [14,18], we develop a novel type-and-effect system that strictly generalises the prior approaches from the literature. (iv) We state two soundness theorems (see Sect. 5.3) based on two different—but strongly related typing rules of ticking. The two soundness theorems are stated wrt. the smallstep resp. big-step semantics because these semantics precisely correspond to the respective ticking rule. The more restrictive ticking rule can be used to establish (positive) almost sure termination (AST), while the more permissive ticking rule supports the analysis of a larger set of programs (which can be very useful in case termination is not required or can be established by other means); in fact, the more permissive ticking rule is essential for the precise cost analysis of randomised splay trees. We note that the two ticking rules and corresponding soundness theorems do not depend on the details of the type-and-effect system, and we believe that they will be of independent interest (e.g., when adapting the framework of this paper to other benchmarks and cost functions). (v) Our prototype implementation ATLAS strictly extends the earlier version reported on in [18], and all our earlier evaluation results can be replicated (and sometimes improved).

With our implementation and the obtained experimental results we make two contributions to the complexity analysis of data structures:

1. *We automatically infer bounds on the expected amortised cost, which could previously only be obtained by sophisticated pen-and-paper proofs. In particular, we verify that the amortised costs of randomised variants of self-adjusting data structures improve upon their non-randomised variants.* In Table 1 we state the expected cost of the randomised data structures considered and their deterministic counterparts; the benchmarks are detailed in Sect. 2.


**Table 1.** Expected Amortised Cost of Randomised Data Structures. We also state the deterministic counterparts considered in [18] for comparison.

2. *We establish a novel approach to the expected cost analysis of data structures.* Our research has been greatly motivated by the detailed study of Albers et al. in [1] of the expected amortised costs of *randomised splaying*. While [1] requires a sophisticated pen-and-paper analysis, our approach allows us to fully-automatically compare the effect of different rotation probabilities on the expected cost (see Table 2 of Sect. 6).

*Related Work.* The generalisation of the model of computation and the study of the expected resource usage of *probabilistic* programs has recently received increased attention (see e.g. [2,4,5,7,10,11,15,21,22,24,27,37,38]). We focus on related work concerned with automations of expected cost analysis of deterministic or non-deterministic, probabilistic programs—imperative or functional. (A probabilistic program is called *non-deterministic*, if it additionally makes use of non-deterministic choice.)

In recent years the *automation* of expected cost analysis of probabilistic data structures or programs has gained momentum, cf. [2–5,22,24,27,37,38]. Notably, the Absynth prototype by [27], implement Kaminski's ert-calculus, cf. [15] for reasoning about expected costs. Avanzini et al. [5] generalise the ert-calculus to an expected cost transformer and introduce the tool eco-imp, which provides a modular and thus a more efficient and scalable alternative for non-deterministic, probabilistic programs. In comparison to these works, we base our analysis on a dedicated type system finetuned to express sublinear bounds; further our prototype implementation ATLAS derives bounds on the expected amortised costs. Neither is supported by Absynth or eco-imp.

Martingale based techniques have been implemented, e.g., by Peixin Wang et al. [38]. Related results have been reported by Moosbrugger et al. [24]. Meyer et al. [22] provide an extension of the KoAT tool, generalising the concept of alternating size and runtime analysis to probabilistic programs. Again, these innovative tools are not suited to the benchmarks considered in our work. With respect to probabilistic *functional* programs, Di Wang et al. [37] provided the only prior expected cost analysis of (deterministic) probabilistic programs; this work is most closely related to our contributions. Indeed, our typing rule (ite : coin) stems from [37] and the soundness proof wrt. the big-step semantics is conceptually similar. Nevertheless, our contributions strictly generalise their results. First, our core language is based on a simpler semantics, giving rise to cleaner formulations of our soundness theorems. Second, our type-and-effect provides two different typing rules for ticking, a fact we can capitalise on in additional strength of our prototype implementation. Finally, our amortised analysis allows for logarithmic potential functions.

A bulk of research concentrates on specific forms of *martingales* or *Lyapunov ranking functions*. All these works, however, are somewhat orthogonal to our contributions, as foremostly *termination* (i.e. AST or PAST) is studied, rather than computational complexity. Still these approaches can be partially suited to a variety of quantitative program properties, see [35] for an overview, but are incomparable in strength to the results established here.

*Structure.* In the next section, we provide a bird's eye view on our approach. Sections 3 and 4 detail the core probabilistic language employed, as well as its small- and big-step semantics. In Sect. 5 we introduce the novel type-and-effect system formally and state soundness of the system wrt. the respective semantics. In Sect. 6 we present evaluation results of our prototype implementation ATLAS. Finally, we conclude in Sect. 7. All proofs, part of the benchmarks and the source codes are given in [19].

#### **2 Overview of Our Approach and Results**

In this section, we first sketch our approach on an introductory example and then detail the benchmarks and results depicted in Table 1 in the Introduction.

#### **2.1 Introductory Example**

Consider the definition of the function descend, depicted in Fig. 1. The *expected* amortised complexity of descend is log2(|t|), where <sup>|</sup>t<sup>|</sup> denotes the size of a tree t (defined as the number of leaves of the tree).<sup>1</sup> Our analysis is set up in terms of template potential functions with unknown coefficients, which will be instantiated by our analysis. Following [14,18], our potential functions are composed of two types of resource functions, which can express *logarithmic* amortised cost: For a sequence of <sup>n</sup> trees <sup>t</sup>1,...,t*<sup>n</sup>* and coefficients <sup>a</sup>*<sup>i</sup>* <sup>∈</sup> <sup>N</sup>, b <sup>∈</sup> <sup>Z</sup>, with *n <sup>i</sup>*=1 <sup>a</sup>*<sup>i</sup>* <sup>+</sup> <sup>b</sup> -

<sup>1</sup> An amortised analysis may always default to a wort-case analysis. In particular the analysis of descend in this section can be considered as a worst-case analysis. However, we use the example to illustrate the general setup of our amortised analysis.

74 L. Leutgeb et al.

**Fig. 1.** descend function

0, the resource function <sup>p</sup>(*a*1*,...,an,b*) (t1,...,t*n*) log<sup>2</sup> (a<sup>1</sup> · |t1<sup>|</sup> <sup>+</sup> ··· <sup>+</sup> <sup>a</sup>*<sup>n</sup>* · |t*n*<sup>|</sup> <sup>+</sup> <sup>b</sup>) denotes the logarithm of a linear combination of the sizes of the trees. The resource function rk(t), which is a variant of Schoenmakers' potential, cf. [28,31,32], is inductively defined as (i) rk(leaf) := 1; (ii) rk(node ldr) := rk(l) + log2(|l|) + log2(|r|) +rk(r), where <sup>l</sup>, <sup>r</sup> are the left resp. right child of the tree node ldr, and <sup>d</sup> is some data element that is ignored by the resource function. (We note that rk(t) is not needed for the analysis of descend but is needed for more involved benchmarks, e.g. randomised splay trees.) With these resource functions at hand, our analysis introduces the coefficients <sup>q</sup>∗, <sup>q</sup>(1*,*0), <sup>q</sup>(0*,*2), <sup>q</sup> <sup>∗</sup>, <sup>q</sup> (1*,*0), <sup>q</sup> (0*,*2) and employs the following *Ansatz* : 2

$$\begin{aligned} q\_\* \cdot \mathsf{rk}(t) + q\_{(1,0)} \cdot p\_{(1,0)}(t) + q\_{(0,2)} \cdot p\_{(0,2)}(t) &\geqslant c\_{\mathsf{descend}}(t) \\ + q'\_\* \, \mathsf{rk}(\mathsf{descend} \, t) + q'\_{(1,0)} \cdot p\_{(1,0)}(\mathsf{descend} \, t) + q'\_{(0,2)} \cdot p\_{(0,2)}(\mathsf{descend} \, t) \, \dots \end{aligned}$$

Here, cdescend(t) denotes the expected cost of executing descend on tree t, where the cost is given by the ticks as indicated in the source code (each tick accounts for a recursive call). The result of our analysis will be an instantiation of the coefficients, returning q(1*,*0) = 1 and zero for all other coefficients, which allows to directly read off the logarithmic bound log2(|t|) of descend.

Our analysis is formulated as a *type-and-effect system*, introducing the above *template potential functions* for every subexpression of the program under analysis. The typing rules of our system give rise to a constraint system over the unknown coefficients that capture the relationship between the potential functions of the subexpressions of the program. Solving the constraint system then gives a valid instantiation of the potential function coefficients. Our type-andeffect system constitutes a generalisation of the type system studied in [14,18] to the non-deterministic and probabilistic setting, as well as an extension of the type system introduced in [37] to sublinear bounds and non-determinism.

In the following, we survey our type-and-effect system by means of example descend. A partial type derivation is given in Fig. 2. For brevity, type judgements and the type rules are presented in a simplified form. In particular, we restrict our attention to tree types, denoted as T. This omission is inessential to the actual complexity analysis. For the full set of rules see [19]. We now discuss this type derivation step by step.

Let e denote the body of the function definition of descend, cf. Fig. 1. Our automated analysis infers an *annotated type* by verifying that the type

<sup>2</sup> For ease of presentation, we elide the underlying semantics for now and simply write "descend t" for the resulting tree *t* - , obtained after evaluating descend t.

$$\begin{array}{lcl} \text{descend} & \mathsf{descend} \colon \mathsf{T}[Q \to \mathsf{T}[Q'] \\ \hline l: \mathsf{T}[Q\_{5} \vdash \mathsf{descend} \text{ 1} : \mathsf{T}[Q\_{6} \vdash \\ \qquad l: \mathsf{T}[Q\_{4} \vdash (\mathsf{descend} \text{ 1})^{\circ} : \mathsf{T}[Q\_{6} \vdash \\ \qquad\qquad l: \mathsf{T}\_{r} \mathrel{\mathsf{T}[Q\_{3} \vdash \mathsf{let} \; xl = (\mathsf{descend} \text{ 1})^{\circ} \text{ in node } x\_{l} \; a \; r: \mathsf{T}[Q']} \; (\mathsf{let} \; \mathsf{t}) \\ \hline & l: \mathsf{T}\_{r} \mathrel{\mathsf{T}\_{r} \mathrel{\mathsf{T}[Q\_{2} \vdash \mathsf{if} \; \mathsf{co} \; \mathsf{in} \; 1/2 \; \mathsf{then} \; e\_{2} \; \mathsf{else} \; e\_{3}: \mathsf{T}[Q']} \; (\mathsf{the}: \mathsf{co} \mathsf{in}) \\ \hline & l: \mathsf{T}\_{r} \mathrel{\mathsf{T}[Q\_{1} \vdash \mathsf{if} \; \mathsf{do} \; 1/2 \; \mathsf{then} \; e\_{2} \; \mathsf{else} \; e\_{3}: \mathsf{T}[Q']} \; (\mathsf{we} \\ \hline t: \mathsf{T}[Q \vdash \mathsf{match} \; t \; \mathsf{with} \; \mathsf{1oaf} \; \mathsf{1} \; \mathsf{not} \; \ \mathsf{let} \; \mathsf{node } \; l \; \mathsf{a} \; r \rightarrow e\_{1}: \mathsf{T}[Q']} \; (\mathsf{match}) \end{array}$$

**Fig. 2.** Partial Type Derivation for Function descend

judgement <sup>t</sup>:T|<sup>Q</sup> <sup>e</sup>:T|Q is derivable. Types are decorated with *annotations* <sup>Q</sup> := [q∗, q(1*,*0), q(0*,*2)] and <sup>Q</sup> := [q ∗, q (1*,*0), q (0*,*2)]—employed to express the potential carried by the arguments to descend and its results. Annotations fix the coefficients of the resource functions in the corresponding potential functions, e.g., (i) <sup>Φ</sup>(t:T|Q) := <sup>q</sup><sup>∗</sup> · rk(t) + <sup>q</sup>(1*,*0) · <sup>p</sup>(1*,*0)(t) + <sup>q</sup>(0*,*2) · <sup>p</sup>(0*,*2)(t) and (ii) <sup>Φ</sup>(e:T|Q ) := q <sup>∗</sup> · rk(e) + <sup>q</sup> (1*,*0) · <sup>p</sup>(1*,*0)(e) + <sup>q</sup> (0*,*2) · <sup>p</sup>(0*,*2)(e).

By our soundness theorems (see Sect. 5.3), such a typing guarantees that the *expected* amortised cost of descend is bounded by the expectation (wrt. the value distribution in the limit) of the difference between <sup>Φ</sup>(t:T|Q) and <sup>Φ</sup>(descend <sup>t</sup>:T|Q ). Because e is a match expression, the following rule is applied (we only state a restricted rule here, the general rule can be found in [19]):

$$\begin{array}{c} \varepsilon|\mathcal{Q}\vdash\mathsf{1aeaf}\colon\mathsf{T}|Q'\quad l\colon\mathsf{T},r\colon\mathsf{T}|Q\_{1}\vdash\ e\_{1}\colon\mathsf{T}|Q'\\\hline t\colon\mathsf{T}|Q\vdash\mathsf{match}\,t\,\mathsf{with}\,\mathsf{1eaaf}\to\mathsf{1eaaf}\,\mathsf{inode}\,\,l\,\mathsf{a}\,\,r\to\,e\_{1}\colon\mathsf{T}|Q' \end{array} (\mathsf{match}) $$

Here e<sup>1</sup> denotes the subexpression of e that corresponds to the node case of match. Apart from the annotations Q, Q<sup>1</sup> and Q , the rule (match) constitutes a standard type rule for pattern matching. With regard to the annotations Q and Q1, (match) ensures the correct distribution of potential by inducing the constraints

$$q\_1^1 = q\_2^1 = q\_\* \qquad q\_{(1,1,0)}^1 = q\_{(1,0)} \qquad q\_{(1,0,0)}^1 = q\_{(0,1,0)}^1 = q\_\* \qquad q\_{(0,0,2)}^1 = q\_{(0,2)} \text{ ,} $$

where the constraints are immediately justified by recalling the definitions of the resource functions <sup>p</sup>(*a*1*,...,an,b*) (t1,...,t*n*) := log<sup>2</sup> (a<sup>1</sup> · |t1<sup>|</sup> <sup>+</sup> ··· <sup>+</sup> <sup>a</sup>*<sup>n</sup>* · |t*n*<sup>|</sup> <sup>+</sup> <sup>b</sup>) and rk(t) = rk(l) + log2(|l|) + log2(|r|) + rk(r).

The next rule is a structural rule, representing a *weakening* step that rewrites the annotations of the variable context. The rule (w) allows a suitable adaptation of the coefficients based on the following inequality, which holds for any substitution <sup>σ</sup> of variables by values, <sup>Φ</sup>(σ; <sup>l</sup>:T, r :T|Q1) -<sup>Φ</sup>(σ; <sup>l</sup>:T, r :T|Q2).

$$\frac{l:\mathsf{T},r:\mathsf{T}|Q\_2 \vdash e\_1:\mathsf{T}|Q'}{l:\mathsf{T},r:\mathsf{T}|Q\_1 \vdash e\_1:\mathsf{T}|Q'} \left(\mathsf{w}\right)$$

In our prototype implementation this comparison is performed *symbolically*. We use a variant of Farkas' Lemma [19,33] in conjunction with simple

**Fig. 3.** Partial meld function of Randomised Meldable Heaps

mathematical facts about the logarithm to linearise this symbolic comparison, namely the monotonicity of the logarithm and the fact that 2+log2(x)+log2(y) 2 log2(x + y) for all x, y - 1. For example, Farkas' Lemma in conjunction with the latter fact gives rise to

$$\begin{aligned} q\_{(0,0,2)}^1 + 2f &\geqslant q\_{(0,0,2)}^2 & q\_{(1,1,0)}^1 - 2f &\geqslant q\_{(1,1,0)}^2\\ q\_{(1,0,0)}^1 + f &\geqslant q\_{(1,0,0)}^2 & q\_{(0,1,0)}^1 + f &\geqslant q\_{(0,1,0)}^2 \end{aligned}$$

for some fresh rational coefficient f - 0 introduced by Farkas' Lemma. After having generated the constraint system for descend, the solver is free to instantiate <sup>f</sup> as needed. In fact in order to discover the bound log2(|t|) for descend, the solver will need to instantiate f = <sup>1</sup>/2, corresponding to the inequality log2(|l<sup>|</sup> <sup>+</sup> <sup>|</sup>r|) -<sup>1</sup>/<sup>2</sup> log2(|l|) + <sup>1</sup>/<sup>2</sup> log2(|r|) + 1.

So far, the rules did not refer to sampling and are unchanged from their (non-probabilistic) counterpart introduced in [14,18]. The next rule, however, formalises a coin toss, biased with probability p. Our general rule (ite : coin) is depicted in Fig. 12 and is inspired by a similar rule for coin tosses that has been recently been proposed in the literature, cf. [37]. This rule specialises as follows to our introductory example:

$$\begin{aligned} l: \mathsf{T}, r: \mathsf{T}|Q\_4 &\vdash e\_3: \mathsf{T}|Q'\\ l: \mathsf{T}, r: \mathsf{T}|Q\_3 &\vdash \mathsf{1et}\ x\_l = (\mathsf{desend }l)^\vee \text{ in node }\ x\_l \text{ a } r: \mathsf{T}|Q'\\ \hline l: \mathsf{T}, r: \mathsf{T}|Q\_2 &\vdash \mathsf{if } \mathsf{coin}\ 1/2 \text{ then } e\_2 \text{ else } e\_3: \mathsf{T}|Q' \end{aligned} \text{ (ite : } \mathsf{coin})$$

Here e<sup>2</sup> and e<sup>3</sup> respectively, denote the subexpressions of the conditional and in addition the crucial condition <sup>Q</sup><sup>2</sup> <sup>=</sup> <sup>1</sup>/<sup>2</sup> · <sup>Q</sup><sup>3</sup> <sup>+</sup> <sup>1</sup>/<sup>2</sup> · <sup>Q</sup><sup>4</sup> holds. This condition, expressing that the corresponding annotations are subject to the probability of the coin toss, gives rise to the following constraints (among others)

$$\begin{aligned} q\_{(0,0,2)}^2 &= 1/2 \cdot q\_{(0,0,2)}^3 + 1/2 \cdot q\_{(0,0,2)}^4 & q\_{(0,1,0)}^2 &= 1/2 \cdot q\_{(0,1,0)}^3 + 1/2 \cdot q\_{(0,1,0)}^4\\ q\_{(1,0,0)}^2 &= 1/2 \cdot q\_{(1,0,0)}^3 + 1/2 \cdot q\_{(1,0,0)}^4 \end{aligned}$$

In the following, we will only consider one alternative of the coin toss and proceed as in the partial type derivation depicted in Fig. 1 (ie. we state the then-branch and omit the symmetric else-branch). Thus next, we apply the rule for the let expression. This rule is the most involved typing rule in the system proposed in [14,18]. However, for our leading example it suffices to consider the following simplified variant:

**Fig. 4.** Partial splay function of Randomised Splay Trees (zigzig-case)

$$\frac{l:\mathsf{T}|Q\_4\vdash(\mathsf{descend }l)^\swarrow\mathsf{T}|Q\_6\quad l:\mathsf{T}|Q\_7\vdash\mathsf{node}\ x\_l\ a\ r:\mathsf{T}|Q'\quad\text{(let}\\\mathsf{t}\text{)}}{l\colon\mathsf{T},r\colon\mathsf{T}|Q\_3\vdash\mathsf{1et}\ x\_l=(\mathsf{descend }l)^\swarrow\text{in node}\ x\_l\ a\ r:\mathsf{T}|Q'}\text{ (let)}$$

Focusing on the annotations, the rule (let : tree) suitably distributes potential assigned to the variable context, embodied in the annotation Q3, to the recursive call within the let expression (via annotation Q4) and the construction of the resulting tree (via annotation Q7). The distribution of potential is facilitated by generating constraints that can roughly be stated as two "equalities", that is, (i) "Q<sup>3</sup> = Q<sup>4</sup> + D", and (ii) "Q<sup>7</sup> = D + Q6". Equality (i) states that the input potential is split into some potential Q<sup>4</sup> used for typing (descend l) and some remainder potential D (which however is not constructed explicitly and only serves as a placeholder for potential that will be passed on). Equality (ii) states that the potential Q<sup>7</sup> used for typing node x*<sup>l</sup>* a r equals the remainder potential D plus the leftover potential Q<sup>6</sup> from the typing of (descend l) -. The (tick : now) rule then ensures that costs are properly accounted for by generating constraints for Q<sup>4</sup> = Q<sup>5</sup> + 1 (see Fig. 2). Finally, the type derivation ends by the application rule, denoted as (app), that verifies that the recursive call is well-typed wrt. the (annotated) signature of the function descend: <sup>T</sup>|<sup>Q</sup> <sup>→</sup> <sup>T</sup>|Q , ie. the rule enforces that Q<sup>5</sup> = Q and Q<sup>6</sup> = Q . We illustrate (a subset of) the constraints induced by (let), (tick : now) and (app):

$$\begin{aligned} q\_{(1,0,0)}^3 &= q\_{(1,0)}^4 & q\_{(0,1,0)}^3 &= q\_{(0,1,0)}^7 & q\_1' &= q\_1^6 & q\_{(0,2)}^4 &= q\_{(0,2)}^5 + 1 \\ q\_{(0,0,2)}^3 &= q\_{(0,2)}^4 & q\_2^3 &= q\_2^7 & q\_{(1,0)}^7 &= q\_{(1,0)}^6 & q\_{(1,0)}^4 &= q\_{(1,0)}^5 \\ q\_1^3 &= q\_1^4 & q\_{(0,2)}^/ &= q\_{(0,2)}^6 & q\_1^6 &= q\_1^7 & q\_{(1,0)}^5 &= q\_{(1,0)} \end{aligned}$$

where (i) the constraints in the first three columns—involving the annotations Q3, Q4, Q<sup>6</sup> and Q7—stem from the constraints of the rule (let : tree); (ii) the constraints in the last column—involving Q4, Q5, Q and Q —stem from the constraints of the rule (tick : now) and (app). For example, q<sup>3</sup> (1*,*0*,*0) <sup>=</sup> <sup>q</sup><sup>4</sup> (1*,*0) and q3 (0*,*1*,*0) <sup>=</sup> <sup>q</sup><sup>7</sup> (0*,*1*,*0) distributes the part of the logarithmic potential represented by Q<sup>3</sup> to Q<sup>4</sup> and Q7; q<sup>6</sup> <sup>1</sup> = q<sup>7</sup> <sup>1</sup> expresses that the rank of the result of evaluating the recursive call can be employed in the construction of the resulting tree node x*<sup>l</sup>* a r; q<sup>4</sup> (1*,*0) <sup>=</sup> <sup>q</sup><sup>5</sup> (1*,*0) and <sup>q</sup><sup>4</sup> (0*,*2) <sup>=</sup> <sup>q</sup><sup>5</sup> (0*,*2) + 1 relate the logarithmic resp. constant potential according to the tick rule, where the addition of one accounts for the cost embodied by the tick rule; q<sup>5</sup> (1*,*0) <sup>=</sup> <sup>q</sup>(1*,*0) stipulates that the potential at the recursive call site must match the function type.

Our prototype implementation ATLAS collects all these constraints and solves them fully automatically. Following [14,18], our implementation in fact searches 78 L. Leutgeb et al.

**Fig. 5.** insert function of a Binary Search Tree with randomized comparison

for a solution that minimises the resulting complexity bound. For the descend function, our implementation finds a solution that sets q(1*,*0) to 1, and all other coefficients to zero. Thus, the logarithmic bound log2(|t|) follows.

#### **2.2 Overview of Benchmarks and Results**

*Randomised Meldable Heaps.* Gambin et al. [13] proposed meldable heaps as a simple priority-queue data structure that is guaranteed to have expected logarithmic cost for all operations. All operations can be implemented in terms of the meld function, which takes two heaps and returns a single heap as a result. The partial source code of meld is given in Fig. 3 (the full source code of all examples can be found in [19]). Our tool ATLAS fully-automatically infers the bound log2(|h1|) + log2(|h2|) on the expected cost of meld.

*Randomised Splay Trees.* Albers et al. in [1] propose these splay trees as a variation of deterministic splay trees [34], which have better expected runtime complexity (the same computational complexity in the O-notation but with smaller constants). Related results have been obtained by F¨urer [12]. The proposal is based on the observation that it is not necessary to rotate the tree in every (recursive) splaying operation but that it suffices to perform rotations with some fixed positive probability in order to reap the asymptotic benefits of self-adjusting search trees. The theoretical analysis of randomised splay trees [1] starts by refining the cost model of [34], which simply counts the number of rotations, into one that accounts for recursive calls with a cost of c and for rotations with a cost of d.

We present a snippet of a functional implementation of randomised splay trees in Fig. 4. We note that in this code snippet we have set c = d = <sup>1</sup>/2; this choice is arbitrary; we have chosen these costs in order to be able to compare the resulting amortised costs to the deterministic setting of [18], where the combined cost of the recursive call and rotation is set to 1; we note that our analysis requires fixed costs c and d but these constants can be chosen by the user; for example one can set c = 1 and d = 2.75 corresponding to the costs observed during the experiments in [1]. Likewise the probability of the coin toss has been arbitrarily set to p = <sup>1</sup>/<sup>2</sup> but could be set differently by the user. (We remark that to the best of our knowledge no theoretical analysis has been conducted on how to chose the best value of p for given costs c and d.) Our prototype implementation is able to fully automatically infer an amortised complexity bound of <sup>9</sup>/<sup>8</sup> log2(|t|) for splay (with c, d and p fixed as above), which improves on the complexity

**Fig. 6.** delete\_max function of a Coin Search Tree with one rotation

bound of <sup>3</sup>/<sup>2</sup> log2(|t|) for the deterministic version of splay as reported in [18], confirming that randomisation indeed improves the expected runtime.

We remark on how the amortised complexity bound of <sup>9</sup>/<sup>8</sup> log2(|t|) for splay is computed by our analysis. Our tool ATLAS computes an annotated type for splay that corresponds to the inequality

$$r^3/4\operatorname{rk}(t) + ^9/s\log\_2(|t|) + ^3/4 \geqslant c\_{\mathtt{sp1ay}}(t) + ^3/4\operatorname{rk}(\mathtt{sp1ay}\ t) + ^3/4\ \mathtt{s}$$

By setting φ(t) := <sup>3</sup>/<sup>4</sup> rk(t) + <sup>3</sup>/<sup>4</sup> as potential function in the sense of Tarjan and Sleator [34,36], the above inequality allows us to directly read out an upper bound on the amortised complexity asplay(t) of splay (we recall that the amortised complexity in the sense of Tarjan and Sleator is defined as the sum of the actual costs plus the output potential minus the input potential):

$$a\_{\mathbf{s}\mathbf{p}\mathbf{1}\mathbf{a}\mathbf{y}}(t) = c\_{\mathbf{s}\mathbf{p}\mathbf{1}\mathbf{a}\mathbf{y}}(t) + \phi(\mathbf{s}\mathbf{p}\mathbf{1}\mathbf{a}\mathbf{y}\ \mathrm{ }t) - \phi(t) \leqslant {}^{9}/\mathbf{s} \cdot \log\_{2}(|t|)\ .$$

*Probabilistic Analysis of Binary Search Trees.* We present a probabilistic analysis of a deterministic binary search tree, which offers the usual contains, insert, and delete operations, where delete uses delete\_max given in Fig. 6, as a subroutine (the source code of the missing operations is given in [19]). We assume that the elements inserted, deleted and searched for are equally distributed; hence, we conduct a probabilistic analysis by replacing every comparison with a coin toss of probability one half. We will refer to the resulting data structure as Coin Search Tree in our benchmarks. The source code of insert is given in Fig. 5.

Our tool ATLAS infers an logarithmic expected amortised cost for all operations, ie., for insert and delete\_max we obtain (i) <sup>1</sup>/<sup>2</sup> rk(t) + <sup>3</sup>/<sup>2</sup> log2(|t|) + <sup>3</sup>/<sup>2</sup> <sup>c</sup>insert(t) + <sup>1</sup>/<sup>2</sup> rk(insert <sup>t</sup>) + 1; and (ii) <sup>1</sup>/<sup>2</sup> rk(t) + <sup>3</sup>/<sup>2</sup> log2(|t|)+1 - cdelete\_max(t) + <sup>1</sup>/<sup>2</sup> rk(delete\_max t) + 1, from which we obtain an expected amortised cost of <sup>3</sup>/<sup>2</sup> log2(|t|) + <sup>1</sup>/<sup>2</sup> and <sup>3</sup>/<sup>2</sup> log2(|t|) respectively.

#### **3 Probabilistic Functional Language**

*Preliminaries.* Let R<sup>+</sup> <sup>0</sup> denote the non-negative reals and R<sup>+</sup><sup>∞</sup> <sup>0</sup> their extension by ∞. We are only concerned with *discrete distributions* and drop "discrete" in the following. Let A be a countable set and let D(A) denote the set of *(sub)distributions* <sup>d</sup> over <sup>A</sup>, whose support supp(μ) := {<sup>a</sup> <sup>∈</sup> <sup>A</sup> <sup>|</sup> <sup>μ</sup>(a) = 0} is countable. Distributions are denoted by Greek letters. For <sup>μ</sup> <sup>∈</sup> <sup>D</sup>(A), we may

**Fig. 7.** A Core Probabilistic (First-Order) Programming Language

write <sup>μ</sup> <sup>=</sup> {a*<sup>p</sup><sup>i</sup> <sup>i</sup>* }*<sup>i</sup>*∈*<sup>I</sup>* , assigning probabilities <sup>p</sup>*<sup>i</sup>* to <sup>a</sup>*<sup>i</sup>* <sup>∈</sup> <sup>A</sup> for every <sup>i</sup> <sup>∈</sup> <sup>I</sup>, where <sup>I</sup> is a suitable chosen index set. We set <sup>|</sup>μ<sup>|</sup> := - *<sup>i</sup>*∈*<sup>I</sup>* <sup>p</sup>*i*. If the support is finite, we simply write <sup>μ</sup> <sup>=</sup> {a*<sup>p</sup>*<sup>1</sup> <sup>1</sup> ,...,a*<sup>p</sup><sup>n</sup> <sup>n</sup>* } The *expected value* of a function <sup>f</sup> : <sup>A</sup> <sup>→</sup> <sup>R</sup><sup>+</sup> <sup>0</sup> on <sup>μ</sup> <sup>∈</sup> <sup>D</sup>(A) is defined as <sup>E</sup>*µ*(f) := - *<sup>a</sup>*∈supp(*µ*) <sup>μ</sup>(a) · <sup>f</sup>(a). Further, we denote by - *<sup>i</sup>*∈*<sup>I</sup>* <sup>p</sup>*<sup>i</sup>* · <sup>μ</sup>*<sup>i</sup>* the *convex combination of distributions* <sup>μ</sup>*i*, where - *<sup>i</sup>*∈*<sup>I</sup>* <sup>p</sup>*<sup>i</sup>* 1. As by assumption - *<sup>i</sup>*∈*<sup>I</sup>* <sup>p</sup>*<sup>i</sup>* 1, - *<sup>i</sup>*∈*<sup>I</sup>* <sup>p</sup>*<sup>i</sup>* · <sup>μ</sup>*<sup>i</sup>* is always a (sub-)distribution.

In the following, we also employ a slight extension of (discrete) distributions, dubbed *multidistributions* [4]. Multidistributions are countable *multisets* {a*<sup>p</sup><sup>i</sup> <sup>i</sup>* }*<sup>i</sup>*∈*<sup>I</sup>* over pairs <sup>p</sup>*<sup>i</sup>* : <sup>a</sup>*<sup>i</sup>* of *probabilities* <sup>0</sup> < p*<sup>i</sup>* 1 and *objects* <sup>a</sup>*<sup>i</sup>* <sup>∈</sup> <sup>A</sup> with - *<sup>i</sup>*∈*<sup>I</sup>* <sup>p</sup>*<sup>i</sup>* 1. (For ease of presentation, we do not distinguish notationally between sets and multisets.) Multidistributions over objects A are denoted by <sup>M</sup>(A). For a multidistribution <sup>μ</sup> <sup>∈</sup> <sup>M</sup>(A) the induced distribution <sup>μ</sup> <sup>∈</sup> <sup>D</sup>(A) is defined in the obvious way by summing up the probabilities of equal objects.

*Syntax.* In Fig. 7, we detail the syntax of our core probabilistic (first-order) programming language. With the exception of ticks, expressions are given in let-normal form to simplify the presentation of the operational semantics and the typing rules. In order to ease the readability, we make use of mild syntactic sugaring in the presentation of actual code (as we already did above).

To make the presentation more succinct, we assume only the following types: a set of *base types* <sup>B</sup> such as Booleans Bool <sup>=</sup> {true, false}, integers Int, or rationals Rat, product types, and binary trees T, whose internal nodes are labelled with elements b :B, where B denotes an arbitrary base type. *Values* are either of base types, trees or pairs of values. We use lower-case Greek letters (from the beginning of the alphabet) for the denotation of types. Elements t:T are defined by the following grammar which fixes notation. <sup>t</sup> ::= leaf <sup>|</sup> node <sup>t</sup><sup>1</sup> b t2. The size of a tree is the number of leaves: <sup>|</sup>leaf<sup>|</sup> := 1, <sup>|</sup>node tau<sup>|</sup> := <sup>|</sup>t<sup>|</sup> <sup>+</sup> <sup>|</sup>u|.

We skip the standard definition of integer constants <sup>n</sup> <sup>∈</sup> <sup>Z</sup> as well as variable declarations, cf. [29]. Furthermore, we omit binary operators with the exception of essential comparisons. As mentioned, to represent sampling we make use of a dedicated if-then-else expression, whose guard evaluates to true depending on a coin toss with fixed probability. Further, non-deterministic choice is similarly rendered via an if-then-else expression. Moreover, we make use of *ticking*, denoted by an operator · *<sup>a</sup>*/*<sup>b</sup>* to annotate costs, where a, b are optional

**Fig. 8.** One-Step Reduction Rules

and default to one. Following Avanzini et al. [2], we represent ticking · as an operation, rather than in let-normal form, as in Wang et al. [37]. (This allows us to suit a big-step semantics that only accumulates the cost of terminating expressions.) The set of all expressions is denoted E.

A *typing context* is a mapping from variables V to types. Type contexts are denoted by upper-case Greek letters, and the empty context is denoted ε. A program P consists of a signature F together with a set of function definitions of the form f x<sup>1</sup> ... x*<sup>n</sup>* = e*<sup>f</sup>* , where the x*<sup>i</sup>* are variables and e*<sup>f</sup>* an expression. When considering some expression e that includes function calls we will always assume that these function calls are defined by some program P. A *substitution* or (*environment*) σ is a mapping from variables to values that respects types. Substitutions are denoted as sets of assignments: <sup>σ</sup> <sup>=</sup> {x<sup>1</sup> → <sup>t</sup>1,...,x*<sup>n</sup>* → <sup>t</sup>*n*}. We write dom(σ) to denote the domain of σ.

#### **4 Operational Semantics**

*Small-Step Semantics.* The small-step semantics is formalised as a (weighted) non-deterministic, probabilistic abstract reduction system [4,9] over M(E). In this way (expected) cost, non-determinism and probabilistic sampling are taken care of. Informally, a probabilistic abstract reduction system is a transition systems where reducts are chosen from a probability distribution. A reduction wrt. such a system is then given by a stochastic process [9], or equivalently, as a reduction relation over *multidistributions* [4], which arise naturally in the context of non-determinism (we refer the reader to [4] for an example that illustrates the advantage of multidistributions in the presence of non-determinism).

Following [5], we equip transitions with (positive) weights, amounting to the cost of the transition. Formally, a *(weighted) Probabilistic Abstract Reduction System* (PARS) on a countable set <sup>A</sup> is a ternary relation · · →·⊆ <sup>A</sup>×R<sup>+</sup> <sup>0</sup> <sup>×</sup>D(A). For <sup>a</sup> <sup>∈</sup> <sup>A</sup>, a rule <sup>a</sup> *<sup>c</sup>* → {b*µ*(*b*)}*<sup>b</sup>*∈*<sup>A</sup>* indicates that <sup>a</sup> reduces to <sup>b</sup> with probability <sup>μ</sup>(b) and cost <sup>c</sup> <sup>∈</sup> <sup>R</sup><sup>+</sup> <sup>0</sup> . Note that any right-hand-side of a PARS is supposed to be a *full* distribution, ie. the probabilities in μ sum up to 1. Given two objects a and b, a *<sup>c</sup>* → {b<sup>1</sup>} will be written <sup>a</sup> *<sup>c</sup>* → <sup>b</sup> for brevity. An object <sup>a</sup> <sup>∈</sup> <sup>A</sup> is called *terminal* if there is no rule a *<sup>c</sup>* → <sup>μ</sup>, denoted <sup>a</sup> →. We suit the one-step reduction relation → given in Fig. 8 as a (non-deterministic) PARS over multidistributions.

$$\begin{array}{cc} e \stackrel{c}{\rightarrow} \{e\_i^{p\_i}\}\_{i \in I} \\ \{\mathbb{C}[e^1]\} \stackrel{c}{\rightarrow} \{\mathbb{C}[e\_i]^{p\_i}\}\_{i \in I} \end{array} \text{(Step)} \qquad \begin{array}{cc} \mu\_i \stackrel{c\_i}{\rightarrow} \nu\_i \quad \sum\_i p\_i \leqslant 1 \\ \{\sharp\_i \, p\_i \cdot \mu\_i \quad \sum\_i \, \sharp\_i \, p\_i \cdot \nu\_i \\ \{\ast\} \, p\_i \cdot \{\ast\} \end{array} \text{(Conv)}$$

**Fig. 9.** Probabilistic Reduction Rules of Distributions of Expressions

As above, we sometimes identify Dirac distributions {e<sup>1</sup>} with <sup>e</sup>. *Evaluation contexts* are formed by let expressions, as in the following grammar: C ::= <sup>|</sup> let <sup>x</sup> <sup>=</sup> <sup>C</sup> in <sup>e</sup>. We denote with <sup>C</sup>[e] the result of substitution the empty context with expression e. Contexts are exploited to lift the one-step reduction to a ternary weighted reduction relation · −→ ⊆ <sup>M</sup>(E) <sup>×</sup> <sup>R</sup>+<sup>∞</sup> <sup>0</sup> × M(E), cf. Fig. 9. (In (Conv), refers to the usual notion of multiset union.)

The relation · −→ constitutes the operational (small-step) semantics of our simple probabilistic function language. Thus <sup>μ</sup> *<sup>c</sup>* −→ <sup>ν</sup> states that the submultidistribution of objects μ evolves to a submultidistribution of reducts ν in one step, with an expected cost of c. Note that since · → is non-deterministic, so is the reduction relation · −→. We now define the evaluation of an expression <sup>e</sup> ∈ E wrt. to the small-step relation · −→: We set <sup>e</sup> *<sup>c</sup>* −→<sup>∞</sup> <sup>μ</sup>, if there is a (possibly infinite) sequence {e<sup>1</sup>} *<sup>c</sup>*<sup>1</sup> −→ <sup>μ</sup><sup>1</sup> *<sup>c</sup>*<sup>2</sup> −→ <sup>μ</sup><sup>2</sup> *<sup>c</sup>*<sup>3</sup> −→ ... with <sup>c</sup> <sup>=</sup> - *<sup>n</sup>* <sup>c</sup>*<sup>n</sup>* and <sup>μ</sup> = lim*<sup>n</sup>*→∞ <sup>μ</sup>*n<sup>V</sup>* , where <sup>μ</sup>*n<sup>V</sup>* denotes the restriction of the distribution <sup>μ</sup>*<sup>n</sup>* (induced by the multidistribution μ*n*) to a (sub-)distribution over values. Note that the μ*n<sup>V</sup>* form a CPO wrt. the pointwise ordering, cf. [39]. Hence, the fixed point <sup>μ</sup> = lim*<sup>n</sup>*→∞ <sup>μ</sup>*n<sup>V</sup>* exists. We also write <sup>e</sup> −→<sup>∞</sup> <sup>μ</sup> in case the cost of the evaluation is not important.

*(Positive) Almost Sure Termination.* A program P is *almost surely terminating* (*AST*) if for any substitution <sup>σ</sup>, and any evaluation eσ −→<sup>∞</sup> <sup>μ</sup>, we have that μ forms a full distribution. For the definition of positive almost sure termination we assume that every statement of P is enclosed in an ticking operation with cost one; we note that such a cost models the length of the computation. We say P is *positively almost surely terminating* (*PAST*), if for any substitution σ, and any evaluation eσ *<sup>c</sup>* −→<sup>∞</sup> <sup>μ</sup>, we have c < <sup>∞</sup>. It is well known that PAST implies AST, cf. [9].

*Big-Step Semantics.* We now define the aforementioned big-step semantics. We first define approximate judgments σ *<sup>n</sup> <sup>c</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup>, see Fig. 10, which say that in derivation trees with depth up to n the expression e evaluates to a subdistribution μ over values with cost c. We now consider the cost c*<sup>n</sup>* and subdistribution μ*<sup>n</sup>* in σ *<sup>n</sup> c<sup>n</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup>*<sup>n</sup>* for <sup>n</sup> → ∞. Note that the subdistributions <sup>μ</sup>*<sup>n</sup>* in <sup>σ</sup> *<sup>n</sup> c<sup>n</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup>*<sup>n</sup>* form a CPO wrt. the pointwise ordering, cf. [39]. Hence, there exists a fixed point <sup>μ</sup> = lim*<sup>n</sup>*→∞ <sup>μ</sup>*n*. Moreover, we set <sup>c</sup> = lim*<sup>n</sup>*→∞ <sup>c</sup>*<sup>n</sup>* (note that either <sup>c</sup>*<sup>n</sup>* converges to some real <sup>c</sup> <sup>∈</sup> <sup>R</sup><sup>+</sup><sup>∞</sup> <sup>0</sup> or we have <sup>c</sup> <sup>=</sup> <sup>∞</sup>). We now define the big-step judgments <sup>σ</sup> *<sup>c</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup> by setting <sup>μ</sup> = lim*<sup>n</sup>*→∞ <sup>μ</sup>*<sup>n</sup>* and <sup>c</sup> = lim*<sup>n</sup>*→∞ <sup>c</sup>*<sup>n</sup>* for <sup>σ</sup> *<sup>n</sup> c<sup>n</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup>*n*.

$$\begin{array}{c|c|c}\hline\hline\pi\_{0}\text{ is a total value} & \sigma\_{0}\left|\frac{\pi}{0}\right| \quad\sigma\_{0}\left|\frac{\pi}{0}\right| \quad\sigma\_{0}\left|\frac{\pi}{0}\right| \quad\sigma\_{0}\left|\frac{\pi}{0}\right| \quad\sigma\_{1}\left|\frac{\pi}{0}\right| \quad\sigma\_{1}\left|\frac{\pi}{0}\right| \quad\sigma\_{2}\left|\frac{\pi}{0}\right| \quad\sigma\_{3}\left|\frac{\pi}{0}\right| \quad\sigma\_{3}\left|\frac{\pi}{0}\right| \quad\sigma\_{4}\left|\frac{\pi}{0}\right| \quad\sigma\_{5}\left|\frac{\pi}{0}\right| \quad\sigma\_{6}\left|\frac{\pi}{0}\right| \quad\sigma\_{7}\left|\frac{\pi}{0}\right| \quad\sigma\_{8}\left|\frac{\pi}{0}\right| \quad\sigma\_{9}\left|\frac{\pi}{0}\right| \quad\sigma\_{10}\left|\frac{\pi}{0}\right| \quad\sigma\_{11}\left|\frac{\pi}{0}\right| \quad\sigma\_{21}\left|\frac{\pi}{0}\right| \quad\sigma\_{31}\left|\frac{\pi}{0}\right| \quad\sigma\_{4}\left|\frac{\pi}{0}\right| \quad\sigma\_{5}\left|\frac{\pi}{0}\right| \quad\sigma\_{5}\left|\frac{\pi}{0}\right| \quad\sigma\_{6}\left|\frac{\pi}{0}\right| \quad\sigma\_{7}\left|\frac{\pi}{0}\right| \quad\sigma\_{8}\left|\frac{\pi}{0}\right| \quad\sigma\_{9}\left|\frac{\pi}{0}\right| \quad\sigma\_{10}\left|\frac{\pi}{0}\right| \quad\sigma\_{11}\left|\frac{\pi}{0}\right| \quad\sigma\_{12}\left|\frac{\pi}{0}\right| \quad\sigma\_{13}\left|\frac{\pi}{0}\right| \quad\sigma$$

**Fig. 10.** Big-Step Semantics.

We want to emphasise that the cost <sup>c</sup> in <sup>σ</sup> *<sup>c</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup> only counts the ticks on terminating computations.

**Theorem 1 (Equivalence).** *Let* P *be a program and* σ *a substitution. Then, (i)* <sup>σ</sup> *<sup>c</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup> *implies that* eσ *<sup>c</sup>*- −→<sup>∞</sup> <sup>μ</sup> *for some* <sup>c</sup> <sup>≥</sup> <sup>c</sup>*, and (ii)* eσ *<sup>c</sup>* −→<sup>∞</sup> <sup>μ</sup> *implies that* σ *<sup>c</sup>*- <sup>e</sup> <sup>⇒</sup> <sup>μ</sup> *for some* <sup>c</sup> <sup>c</sup>*. Moreover, if* eσ *almost-surely terminates, we can choose* c = c *in both cases.*

The provided operational big-step semantics generalises the (big-step) semantics given in [18]. Further, while partly motivated by big-step semantics introduced in [37], our big-step semantics is technically incomparable—due to a different representation of ticking—while providing additional expressivity.

$$\begin{array}{c} \begin{array}{c} \Gamma \mid Q \vdash e : \alpha \mid Q'\\ \hline \Gamma \mid Q + {}^{a}/{}\_{b} \vdash e^{\sqrt{{}^{a}} \wedge {}^{b}} : \alpha \mid Q' \end{array} \text{ (tick}: \text{now} \end{array}$$

$$\begin{array}{c} \begin{array}{c} \Gamma \mid Q \vdash e : \alpha \mid Q'\\ \hline \Gamma \mid Q \vdash e^{\sqrt{{}^{a}} \wedge {}^{b}} : \alpha \mid Q' - {}^{a}/{}\_{b} \end{array} \text{ (tick}: \text{defer}) \end{array}$$

**Fig. 11.** Ticking Operator. Note that *a*, *b* are not variables but literal numbers.

#### **5 Type-and-Effect System for Expected Cost Analysis**

#### **5.1 Resource Functions**

In Sect. 2, we introduced a variant of Schoenmakers' potential function, denoted as rk(t), and the additional potential functions p(*a*1*,...,an,b*) (t1,...,t*n*) = log<sup>2</sup> (a<sup>1</sup> · |t1<sup>|</sup> <sup>+</sup> ··· <sup>+</sup> <sup>a</sup>*<sup>n</sup>* · |t*n*<sup>|</sup> <sup>+</sup> <sup>b</sup>), denoting the log<sup>2</sup> of a linear combination of tree sizes. We demand *n <sup>i</sup>*=1 <sup>a</sup>*<sup>i</sup>* <sup>+</sup> <sup>b</sup> - 0 (a*<sup>i</sup>* <sup>∈</sup> <sup>N</sup>, b <sup>∈</sup> <sup>Z</sup>) for well-definedness of the latter; log<sup>2</sup> denotes the logarithm to the base 2. Throughout the paper we stipulate log2(0) := 0 in order to avoid case distinctions. Note that the constant function 1 is representable: 1 = λt. log2(0 · |t<sup>|</sup> + 2) = <sup>p</sup>(0*,*2). We are now ready to state the resource annotation of a sequence of trees.

**Definition 1.** *A* resource annotation *or simply* annotation *of length* m *is a sequence* <sup>Q</sup> = [q1,...,q*m*] <sup>∪</sup> q(*a*1*,...,am,b*) <sup>a</sup>*i*, b <sup>∈</sup> <sup>N</sup> *, vanishing almost everywhere. The length of* <sup>Q</sup> *is denoted* <sup>|</sup>Q|*. The empty annotation, that is, the annotation where all coefficients are set to zero, is denoted as* ∅*. Let* t1,...,t*<sup>m</sup> be a sequence of trees. Then, the potential of* t*m*,...,t*<sup>n</sup> wrt.* Q *is given by*

$$\Phi\left(t\_1, \ldots, t\_m \mid Q\right) := \sum\_{i=1}^m q\_i \cdot \text{rk}\left(t\_i\right) + \sum\_{a\_1, \ldots, a\_m \in \mathbb{N}, b \in \mathbb{Z}} q\_{(a\_1, \ldots, a\_m, b)} \cdot p\_{(a\_1, \ldots, a\_m, b)}\left(t\_1, \ldots, t\_m\right).$$

In case of an annotation of length 1, we sometimes write <sup>q</sup><sup>∗</sup> instead of <sup>q</sup>1. We may also write <sup>Φ</sup>(<sup>v</sup> : <sup>α</sup>|Q) for the potential of a value of type <sup>α</sup> annotated with Q. Both notations were already used above. Note that only values of tree type are assigned a potential. We use the convention that the sequence elements of resource annotations are denoted by the lower-case letter of the annotation, potentially with corresponding sub- or superscripts.

*Example 1.* Let <sup>t</sup> be a tree. To model its potential as log2(|t|) in according to Definition 1, we simply set <sup>q</sup>(1*,*0) := 1 and thus obtain <sup>Φ</sup>(t|Q) = log2(|t|), which describes the potential associated to the input tree t of our leading example descend above. 

Let <sup>σ</sup> be a substitution, let <sup>Γ</sup> denote a typing context and let <sup>x</sup><sup>1</sup> : ,...,x*<sup>n</sup>* : denote all tree types in <sup>Γ</sup>. A *resource annotation for* <sup>Γ</sup> or simply *annotation* is an annotation for the sequence of trees x1σ, . . . , x*n*σ. We define the *potential* of the annotated context <sup>Γ</sup>|<sup>Q</sup> wrt. a substitution <sup>σ</sup> as <sup>Φ</sup>(σ; <sup>Γ</sup> <sup>|</sup> <sup>Q</sup>) := <sup>Φ</sup>(x1σ, . . . , x*n*<sup>σ</sup> <sup>|</sup> <sup>Q</sup>).

$$\frac{\Gamma|Q\_1 \vdash e\_1 : \alpha|Q' \quad \Gamma|Q\_2 \vdash e\_2 : \alpha|Q' \quad p = ^{a}b \quad Q = p \cdot Q\_1 + (1 - p) \cdot Q\_2}{\Gamma|Q \vdash \mathtt{i}\mathtt{f} \cdot \mathtt{c} \text{in} \ a/b \text{ then } e\_1 \text{ o1sse } e\_2 : \alpha|Q'} \text{ (ite : \text{coin})}$$

**Fig. 12.** Conditional expression that models tossing a coin.

**Definition 2.** *An* annotated signature <sup>F</sup> *maps functions* <sup>f</sup> *to sets of pairs of annotated types for the arguments and the annotated type of the result:*

<sup>F</sup>(f) := {α<sup>1</sup> ×···× <sup>α</sup>*<sup>n</sup>* <sup>|</sup><sup>Q</sup> <sup>→</sup> <sup>β</sup><sup>1</sup> ×···× <sup>β</sup>*k*<sup>|</sup> <sup>Q</sup> <sup>|</sup><sup>m</sup> <sup>=</sup> <sup>|</sup>Q|, 1 = <sup>|</sup>Q |} .

*We suppose* f *takes* n *arguments of which* m *are trees;* m n *by definition. Similarly, the return type may be the product* <sup>β</sup><sup>1</sup> × ··· × <sup>β</sup>*i. In this case, we demand that at most one* β*<sup>i</sup> is a tree type.*<sup>3</sup>

Instead of <sup>α</sup><sup>1</sup> ×···× <sup>α</sup>*<sup>n</sup>* <sup>|</sup><sup>Q</sup> <sup>→</sup> <sup>β</sup><sup>1</sup> ×···× <sup>β</sup>*k*<sup>|</sup> <sup>Q</sup> ∈ F(f), we sometimes succinctly write <sup>f</sup> : <sup>α</sup>|<sup>Q</sup> <sup>→</sup> <sup>β</sup>|Q where <sup>α</sup>, <sup>β</sup> denote the product types <sup>α</sup><sup>1</sup> ×···× <sup>α</sup>*n*, <sup>β</sup>1×···×β*k*, respectively. It is tacitly understood that the above syntactic restrictions on the length of the annotations Q, Q are fulfilled. For every function f, we also consider its *cost-free* variant from which all ticks have been removed. We collect the cost-free signatures of all functions in the set <sup>F</sup>cf.

*Example 2.* Consider the function descend depicted in Fig. 2. Its signature is formally represented as <sup>T</sup>|<sup>Q</sup> <sup>→</sup> <sup>T</sup>|Q , where <sup>Q</sup> := [q∗] <sup>∪</sup> [(q(*a,b*))*a,b*∈<sup>Z</sup>] and <sup>Q</sup> := [q <sup>∗</sup>] <sup>∪</sup> [(q (*a,b*))*a,b*∈<sup>Z</sup>]. We leave it to the reader to specify the coefficients in <sup>Q</sup>, <sup>Q</sup> so that the rule (app) as depicted in Sect. 2 can indeed be employed to type the recursive call of descend.

Let <sup>Q</sup> = [q∗] <sup>∪</sup> [(q(*a,b*))*a,b*∈<sup>N</sup>] be an annotation and let <sup>K</sup> be a rational such that q(0*,*2) + K - 0. Then, <sup>Q</sup> := <sup>Q</sup> <sup>+</sup> <sup>K</sup> is defined as follows: <sup>Q</sup> = [q∗] <sup>∪</sup> [(q (*a,b*))*a,b*∈<sup>N</sup>], where <sup>q</sup> (0*,*2) := <sup>q</sup>(0*,*2) <sup>+</sup> <sup>K</sup> and for all (a, b) = (0, 2) <sup>q</sup> (*a,b*) := <sup>q</sup>(*a,b*). Recall that <sup>q</sup>(0*,*2) is the coefficient of function <sup>p</sup>(0*,*2)(t) = log2(0|t|+ 2) = 1, so the annotation Q+K increments or decrements cost from the potential induced by Q by <sup>|</sup>K|, respectively. Further, we define the multiplication of an annotation <sup>Q</sup> by a constant <sup>K</sup>, denoted as <sup>K</sup> ·<sup>Q</sup> pointwise. Moreover, let <sup>P</sup> = [p∗]∪[(p(*a,b*))*a,b*∈<sup>N</sup>] be another annotation. Then the addition P +Q of annotations P, Q is similarly defined pointwise.

#### **5.2 Typing Rules**

The non-probabilistic part of the type system is given in [19]. In contrast to the type system employed in [14,18], the cost model is not fixed but controlled by the ticking operator. Hence, the corresponding application rule (app) has been adapted. Costing of evaluation is now handled by a dedicated *ticking* operator, cf. Fig. 11. In Fig. 12, we give the rule (ite : coin) responsible for typing probabilistic conditionals.

<sup>3</sup> The restriction to at most one tree type in the resulting type is non-essential and could be lifted. However, as our benchmark functions do not require this extension, we have elided it for ease of presentation.

86 L. Leutgeb et al.

**Fig. 13.** Function foo illustrates the difference between (tick : now) and (tick : defer).

We remark that the core type system, that is, the type system given by Fig. 12 together with the remaining rules [19], ignoring annotations, enjoys subject reduction and progress in the following sense, which is straightforward to verify.

**Lemma 1.** *Let* e *be such that* e: α *holds. Then: (i) If* e *<sup>c</sup>* → {<sup>e</sup> *pi <sup>i</sup>* }*<sup>i</sup>*∈*<sup>I</sup> , then* <sup>e</sup>*<sup>i</sup>* : <sup>α</sup> *holds for all* <sup>i</sup> <sup>∈</sup> <sup>I</sup>*. (ii) The expression* <sup>e</sup> *is in normal form wrt. <sup>c</sup>* → *iff* <sup>e</sup> *is a value.*

#### **5.3 Soundness Theorems**

A program <sup>P</sup> is called *well-typed* if for any definition <sup>f</sup> (x1,...,x*n*) = <sup>e</sup> <sup>∈</sup> P and any annotated signature <sup>f</sup> : <sup>α</sup><sup>1</sup> ×···× <sup>α</sup>*n*|<sup>Q</sup> <sup>→</sup> <sup>β</sup>|Q , we have a corresponding typing <sup>x</sup><sup>1</sup> : <sup>α</sup>1,...,x*<sup>k</sup>* : <sup>α</sup>*k*|<sup>Q</sup> <sup>e</sup>: <sup>β</sup>|Q . A program P is called *cost-free* well-typed, if the cost-free typing relation is used (which employs the cost-free signatures of all functions).

**Theorem 2 (Soundness Theorem for** (tick : now)**).** *Let* P *be well-typed. Suppose* <sup>Γ</sup>|<sup>Q</sup> <sup>e</sup>: <sup>α</sup>|Q *and* eσ *<sup>c</sup>* −→<sup>∞</sup> <sup>μ</sup>*. Then* <sup>Φ</sup>(σ; <sup>Γ</sup>|Q) <sup>c</sup> <sup>+</sup> <sup>E</sup>*µ*(λv.Φ(v|Q ))*. Further, if* <sup>Γ</sup>|<sup>Q</sup> *cf* <sup>e</sup>: <sup>α</sup>|Q *, then* <sup>Φ</sup>(σ; <sup>Γ</sup>|Q) - <sup>E</sup>*µ*(λv.Φ(v|Q ))*.*

**Corollary 1.** *Let* P *be a well-typed program such that ticking accounts for all evaluation steps. Suppose* <sup>Γ</sup>|<sup>Q</sup> <sup>e</sup>: <sup>α</sup>|Q *. Then* e *is positive almost surely terminating (and thus in particular almost surely terminating).*

**Theorem 3 (Soundness Theorem for** (tick : defer)**).** *Let* P *be well-typed. Suppose* <sup>Γ</sup>|<sup>Q</sup> <sup>e</sup>: <sup>α</sup>|Q *and* <sup>σ</sup> *<sup>c</sup>* <sup>e</sup> <sup>⇒</sup> <sup>μ</sup>*. Then, we have* <sup>Φ</sup>(σ; <sup>Γ</sup>|Q) c + <sup>E</sup>*µ*(λv.Φ(v|Q ))*. Further, if* <sup>Γ</sup>|<sup>Q</sup> *cf* <sup>e</sup>: <sup>α</sup>|Q *, then* <sup>Φ</sup>(σ; <sup>Γ</sup>|Q) - <sup>E</sup>*µ*(λv.Φ(v|Q ))*.*

We comment on the trade-offs between Theorems 2 and 3. As stated in Corollary 1 the benefit of Theorem 2 is that when every recursive call is accounted for by a tick, then a type derivation implies the termination of the program under analysis. The same does not hold for Theorem 3. However, Theorem 3 allows to type more programs than Theorem 2, which is due to the fact that (tick : defer) rule is more permissive than (tick : now). This proves very useful, in case termination is not required (or can be established by other means).

We exemplify this difference on the foo function, see Fig. 13. Theorem 3 supports the derivation of the type rk(t) + log2(|t|)+1 rk(foo t) + 1, while Theorem 2 does not. This is due to the fact that potential can be "borrowed" with Theorem 3. To wit, from the potential rk(t) + log2(|t|) + 1 for foo one can derive the potential rk(l ) + rk(r ) for the intermediate context after both letexpression (note there is no +1 in this context, because the +1 has been used to

**Table 2.** Coefficients *q* such *q*·log2(|*t*|) is a bound on the expected amortized complexity of splay depending on the probability *p* of a rotation and the cost *c* of a recursive call, where the cost of a rotation is 1 − *c*. Coefficients are additionally presented in decimal representation to ease comparison.


pay for the ticks around the recursive calls). Afterwards one can restore the +1 by weakening rk(l ) + rk(r ) to rk(foo t) + 1 (using in addition that rk(t) - 1 for all trees t). On the other hand, we cannot "borrow" with Theorem 2 because the rule (tick : now) forces to pay the +1 for the recursive call immediately (but there is not enough potential to pay for this). In the same way, the application of rule (tick : defer) and Theorem 3 is essential to establish the logarithmic amortised costs of randomised splay trees. (We note that the termination of foo as well as of splay is easy to establish by other means: it suffices to observe that recursive calls are on sub-trees of the input tree).

#### **6 Implementation and Evaluation**

*Implementation.* Our prototype ATLAS is an extension of the tool described in [18]. In particular, we rely on the preprocessing steps and the implementation of the weakening rule as reported in [18] (which makes use of Farkas' Lemma in conjunction with selected mathematical facts about the logarithm as mentioned above). We only use the fully-automated mode reported in [18]. We have adapted the generation of the constraint system to the rules presented in this paper. We rely on Z3 [26] for solving the generated constraints. We use the optimisation heuristics of [18] for steering the solver towards solutions that minimize the resulting expected amortised complexity of the function under analysis.

*Evaluation.* We present results for the benchmarks described in Sect. 2 (plus a randomised version of splay heaps, the source code can be found in [19]) in Table 1. Table 3 details the computation time for type checking our results. Note that type inference takes considerably longer (tens of hours). To the best of our knowledge this is the first time that an expected amortised cost could be inferred for these data structures.

By comparing the costs of the operations of randomised splay trees and heaps to the costs of their deterministic versions (see Table 1), one can see the randomised variants have equal or lower complexity in all cases (as noted in Table 2 we have set the costs of the recursive call and the rotation to <sup>1</sup>/2, such that in the deterministic case, which corresponds to a coin toss with p = 1, these



costs will always add up to one). Clearly, setting the costs of the recursion to the same value as the cost of the rotation does not need to reflect the relation of the actual costs. A more accurate estimation of the relation of these two costs will likely require careful experimentation with data structure implementations, which we consider orthogonal to our work. Instead, we report that our analysis is readily adapted to different costs and different coin toss probabilities. We present an evaluation for different values of <sup>p</sup>, recursion cost <sup>c</sup> and rotation cost 1 <sup>−</sup> <sup>c</sup> in Table 2. In preparing Table <sup>2</sup> the template <sup>q</sup><sup>∗</sup> · rk(t) + <sup>q</sup>(1*,*0) · log2(|t|) + <sup>q</sup>(0*,*2) was used for performance reasons. The memory usage according to Z3's "max memory" statistic was 7129MiB per instance. The total runtime was 1H45M, with an average of 11M39S and a median of 2M33S. Two instances took longer time (36M and 49M).

*Deterministic Benchmarks.* For comparison we have also evaluated our tool ATLAS on the benchmarks of [18]. All results could be reproduced by our implementation. In fact, for the function SplayHeap.insert it yields an improvement of <sup>1</sup>/<sup>4</sup> log2(|h|), ie. <sup>1</sup>/<sup>2</sup> log2(|h|) + log2(|h<sup>|</sup> + 1) + <sup>3</sup>/<sup>2</sup> compared to <sup>3</sup>/<sup>4</sup> log2(|h|)+log2(|h|+ 1)+3/2. We note that we are able to report better results because we have generalised the resource functions p(*a*1*......am,b*) (t1,...,t*m*) := log<sup>2</sup> (a<sup>1</sup> · |t1<sup>|</sup> <sup>+</sup> ··· <sup>+</sup> <sup>a</sup>*<sup>m</sup>* · |t*m*<sup>|</sup> <sup>+</sup> <sup>b</sup>) to also allow negative values for <sup>b</sup> (under the condition that - *<sup>i</sup>* <sup>a</sup>*<sup>i</sup>* <sup>+</sup><sup>b</sup> <sup>≥</sup> 1) and our generalised (let : tree) rule can take advantage of these generalized resource functions (see [19] for a statement of the rule and the proof of its soundness as part of the proof of Theorem 3).

## **7 Conclusion**

In this paper, we present the first fully-automated *expected amortised cost analysis* of self-adjusting data structures, that is, of *randomised splay trees*, *randomised splay heaps* and *randomised meldable heaps*, which so far have only (semi-)manually been analysed in the literature.

In future work, we envision to extend our analysis to related probabilistic settings such as skip lists [30], randomised binary search trees [20] and *randomised treaps* [8]. We note that adaptation of the framework developed in this paper to new benchmarks will likely require to identify new potential functions and the extension of the type-effect-system with typing rules for these potential functions. Further, on more theoretical grounds we want to clarify the connection of the here proposed expected amortised cost analysis with Kaminski's ert-calculus, cf. [15], and study whether the expected cost transformer is conceivable as a potential function.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Murxla: A Modular and Highly Extensible API Fuzzer for SMT Solvers**

Aina Niemetz(B) , Mathias Preiner , and Clark Barrett

Stanford University, Stanford, USA *{*niemetz,preiner,barrett*}*@cs.stanford.edu

**Abstract.** SMT solvers are highly complex pieces of software with performance, robustness, and correctness as key requirements. Complementing traditional testing techniques for these solvers with randomized stress testing has been shown to be quite effective. Recent work has showcased the value of input fuzzing for finding issues, but this approach typically does not comprehensively test a solver's API. Previous work on modelbased API fuzzing was tailored to a single solver and a small subset of SMT-LIB. We present Murxla, a comprehensive, modular, and highly extensible model-based API fuzzer for SMT solvers. Murxla randomly generates valid sequences of solver API calls based on a customizable API model, with full support for the semantics and features of SMT-LIB. It is solver-agnostic but extensible to allow for solver-specific testing and supports option fuzzing, cross-checking with other solvers, translation to SMT-LIBv2, and SMT-LIBv2 input fuzzing. Our evaluation confirms its efficacy in finding issues in multiple state-of-the-art SMT solvers.

#### **1 Introduction**

Satisfiability Modulo Theories (SMT) solvers determine the satisfiability of formulas over first-order theories and their combinations. They serve as back-end reasoning engines for a wide range of applications in academia and industry [18, 27], including hardware and software verification [14,29,31,35,38,40], model checking [23,24,46], security [12,33], automated test-case generation [22,50], and synthesis [10,34]. Notable SMT solvers include Bitwuzla [42], Boolector [46], cvc5 [13], MathSAT [26], OpenSMT2 [36], SMTInterpol [25], SMT-RAT [28], STP [32], veriT [20], Yices2 [30], and Z3 [41]. State-of-the-art SMT solvers are complex pieces of software with up to hundreds of thousands lines of code. Because of their frequent use as back-ends in higher-level tool chains, strong requirements include performance, robustness, and a high level of trust. Due to their complex nature, full verification of SMT solvers has so far remained out of reach. Furthermore, most SMT solvers are under active development, meaning that there is a constant risk of introducing new issues. While traditional testing techniques

c The Author(s) 2022

This work was supported in part by DARPA (award no. FA8650-18-2-7861), ONR (award no. N68335-17-C-0558), and by an Amazon Research Award.

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 92–106, 2022. https://doi.org/10.1007/978-3-031-13188-2\_5

such as unit testing and a regression test suite are important, these techniques alone are insufficient for achieving high levels of robustness.

SMT solvers usually provide two user-facing interfaces: (i) a textual interface (expecting input in either SMT-LIBv2 [15] or some solver-specific format); and (ii) the application programming interface (API), which allows users to directly integrate the solver into a tool chain. Randomized stress testing (fuzz testing) can be used as a complement to traditional testing to attack these interfaces and has been shown to be very effective at finding issues and thereby helping to improve the correctness and robustness of SMT solvers. In 2009, Brummayer et al. [21] presented a grammar-based generative black-box input fuzzer for the SMT-LIBv1 language [48] called FuzzSMT, and in 2017, Niemetz et al. [45] presented a model-based API fuzz testing framework called BtorMBT for the SMT solver Boolector. More recently, fuzz testing of SMT solvers via their textual interface has gained even more traction with a series of papers on the subject in top venues [19,39,47,49,51,52]. Note that these approaches (and this paper) assume full knowledge of the input structure, i.e., they only generate valid textual input or sequences of API calls. Fuzz testing approaches that are unaware of the input structure can also be useful for testing whether invalid inputs or API calls are handled correctly. This is, however, not a direction we address in this paper.

As mentioned, recent work has focused on fuzzing the textual interface. This is not surprising, as it typically requires significantly less effort than API fuzzing. Input fuzzers generate a new input file or mutate an existing (so-called) seed input file and pass it to a solver binary. Fuzz testing of the solver API is more involved since it requires interaction with the solver—API fuzzers generate sequences of calls to the solver API and typically link against the solver library.

There are, however unique advantages that API fuzzers have. For example, API call sequences generated by API fuzzers may include features and extensions that are not supported by or cannot be expressed via the textual interface. Moreover, even if restricted to standard features, API fuzzers may be able to generate sequences of calls that are not possible using the textual interface, even if the textual interface is built on top of the user-facing API, and especially if it is not. On the other hand, API fuzzing cannot find bugs in parser code. Thus, both fuzzing strategies have unique benefits.

API fuzzing has been an integral part of the development workflow of the SMT solver Boolector [46] since 2013. Boolector supports quantified bit-vector formulas and quantifier-free formulas in the theories of fixed-size bit-vectors, arrays and uninterpreted functions. It ships with BtorMBT [45], an API fuzzer tailored to Boolector, which covers all features of Boolector except quantifiers. BtorMBT has been regularly and rigorously applied during active development of Boolector (locally, prior to major commits to master, and in a cluster setting on 30 nodes prior to every release), with great success. Notably, recent SMT fuzzing campaigns did not report any issues in code covered by BtorMBT [39]; in particular, the few that have been reported [2] all made use of quantified formulas, which are unsupported by BtorMBT. To the best of our knowledge,<sup>1</sup>

<sup>1</sup> The first two authors of this paper are the main developers of the SMT solvers Bitwuzla [42] and Boolector [46], and all three authors are part of the development team of the SMT solver CVC4 [16] and its successor cvc5 [13].

Boolector is the only SMT solver for which API fuzzing has been integrated as a core component of the development workflow.

One of BtorMBT's major weaknesses, however, is that it cannot (easily be extended to) be used with other SMT solvers—it is monolithic, tailored towards the supported theories, and directly calls Boolector's API. Further, it lacks support for quantified formulas, only supports a subset of the theories standardized in SMT-LIB, and even for those, not the full feature set since Boolector only supports a subset. For recording API call sequences, it relies on the API tracing feature of Boolector, the system under test. And for replaying and minimizing such recorded sequences, it requires additional tools.

*Contributions.* In this paper, we present Murxla, a modular and highly extensible model-based API fuzzer for SMT solvers. Murxla is a comprehensive fuzzing tool that generates valid sequences of solver API calls, records these sequences in a simple text-based trace format, and provides support for minimizing and replaying these traces while preserving the original behavior of the solver. Murxla builds on top of a generic solver interface that can be used with any SMT solver and provides full SMT-LIB support in terms of semantics, features, and standard theories. It further has experimental support for some non-standard theories (sequences, sets, bags) and is fully compatible with and configurable for solverspecific features, extensions, and restrictions. Murxla provides support for option fuzzing (randomly configuring solver options based on the options model of the solver) and can be run in cross-checking mode, where the answers of two different solvers are compared with each other. It additionally implements correctness checks for retrieved model values, unsat assumptions, and unsat cores. Finally, it can optionally translate generated API traces to SMT-LIBv2 (provided that the traces do not contain solver-specific extensions), and can thus be used as a textual interface fuzzer for any solver that supports SMT-LIBv2.

Murxla currently supports the SMT solvers Bitwuzla [42], Boolector [46], cvc5 [13], and Yices2 [30]. Our goal so far has been to fully cover solvers we are actively developing (the first three). We additionally added support for Yices2 as a proof of concept for showing that the tool is sufficiently general and modular to be used with solvers other than our own.

*Related Work.* The first application of model-based API fuzzing in the context of verification back-ends was proposed by Artho et al. [11] for the SAT solver Lingeling [17]. In the context of SMT solvers, the first and only integration of model-based API fuzzing as a core component of the development workflow was for the solver Boolector [45], as described above. In both instances, the authors showed the effectiveness of the approach for testing solvers, in particular in combination with option fuzzing and delta debugging.

The first input fuzzer for the SMT-LIB language was FuzzSMT [21], a generative grammar-based fuzzer supporting most of SMT-LIBv1 [48]. In 2018, Blotsky et al. [19] presented an SMT-LIBv2 input fuzzer specifically for strings, which generates and mutates SMT-LIBv2 input and mainly targets performance issues. In 2020, Winterer et al. [51,52] proposed two mutational approaches, one based on merging two inputs and the other based on mutating operators. The former supports only integers, reals, and strings, whereas the latter supports all benchmarks in SMT-LIB but only mutations for the most basic operators. In the same year, Mansur et al. [39] presented Storm, an SMT-LIBv2 fuzzer based on mutating the Boolean structure of an input. Most recently, Park et al. [47] presented TypeFuzz, a hybrid approach for integers, reals, and strings which mutates SMT-LIBv2 by replacing expressions with newly generated expressions. Finally, Scott et al. [49] recently proposed a mutational fuzzer for all of SMT-LIB which leverages reinforcement learning and targets performance issues.

#### **2 Model-Based API Fuzzing for SMT Solvers**

Generally speaking, model-based API fuzzing can be seen as lifting grammarbased input fuzzing to the API level: it requires a "model" of the solver that defines what sequences of API calls are valid. For convenience, we consider this model to be made up of three distinct parts: (i) the *semantic (or data) model*, which defines constructs (such as theories, sorts, operators, and commands) and their semantics (usually based on the SMT-LIBv2 [15] standard); (ii) the *API model*, which defines the usage of the API itself; and (iii) the *options model*, which defines configuration options and how they may or may not be combined.

The main requirements for SMT solvers, especially when used as back-ends of higher-level tool chains, are correctness, performance, and robustness. Within the SMT community, the notion of "issue" is thus commonly defined as one of the following: (i) *soundness* issues—either refutation unsoundness (the solver answers *unsat* when the input is *sat*) or model unsoundness (the solver answers *sat* when the input is *unsat*); (ii) *incorrect witnesses*—models (values), proofs, unsat cores, or unsat assumptions; (iii) *crashes*—assertion failures, segmentation faults; and (iv) *performance regressions*. The most critical issues are soundness issues. Refutation unsoundness is especially problematic, as most solvers provide limited or no means for checking the correctness of an *unsat* result. Model unsoundness is less problematic, since state-of-the-art SMT solvers usually provide models for satisfiable formulas, which are easier to check for correctness. The easiest way to identify soundness issues is to check one solver against a second solver, unless the satisfiability of the input formula is known or can be determined by construction. Witnesses are very often checked inside the solver when in debug mode, and their correctness can be determined outside the solver with relatively little effort for all but proofs, which require more involved checking.

As SMT solver developers, we are interested in catching issues as close to the source as possible. For that purpose, in the context of model-based API fuzzing, we configure solvers under test in debug mode with assertions enabled.

#### **3 Murxla**

Murxla is a modular model-based API fuzzing tool for SMT solvers which generates valid solver API call sequences and supports the recording, replaying, and minimizing of these sequences for debugging purposes. Murxla is written in C++ and available under the GPLv3 at [43]. Extensive documentation is

**Fig. 1.** Murxla architecture. (Color figure online)

available at [44]. A high-level view of its architecture is given as a call graph in Fig. 1. Murxla can integrate any SMT solver (provided that it exposes an API in a programming language that can be integrated). Murxla provides a solver API abstraction, the *Generic Solver API*, which is then specialized via a solver wrapper for a specific solver. Solver-specific components are indicated in blue in Fig. 1 and consist of the solver wrappers and solver-specific extensions of the general API model and options model implemented by Murxla. The four main components of Murxla (green) are the *API Fuzzer*, the *Tracer*, the *Untracer*, and the *Trace Minimizer*.

The API Fuzzer is responsible for generating random but valid API call sequences to the solver under test. The Tracer records these sequences in an *API trace*, which stores all the information required to replay the trace with the Untracer. Replaying a trace with the Untracer executes the exact same API call sequence that was executed when recording the trace. This is particularly useful for replicating interesting behavior that was uncovered while fuzzing the API of the solver under test. The Trace Minimizer takes an API trace as input and tries to minimize it while preserving its behavior with respect to the solver under test. Murxla's core connects all of these components. It is also responsible for interfacing with the SMT solvers and maintaining all sorts and terms created by a solver. In the following, we will describe these components in more detail.

#### **3.1 The Core**

Murxla's Core manages communication and the sorts and terms created by a solver. It consists of three modules: the *Actions*, the *Solver Manager*, and the *Generic Solver API*.

**Actions.** An action is an abstraction defining a particular interaction with the solver under test. These interactions are represented internally as a set of calls to the Generic Solver API. Actions are responsible for three tasks: (1) randomly *generating* API call arguments; (2) *executing* API calls with a given set of arguments; and (3) *replaying* a traced copy of the action.

Murxla currently implements a base set of 25 actions which wrap the methods of the Generic Solver API and include creating and deleting a solver instance, configuring solver options, creating sorts and terms, asserting formulas, altering the context levels via push and pop, checking satisfiability of asserted formulas (with assumptions), and many more. When executing API calls, actions may perform sanity checks on results retrieved from solver API calls. For this, Murxla provides a macro MURXLA\_TEST which allows C-style assertion checks. These remain in the code even if the tool is compiled without assertions. If a solver supports more functionality than that covered by the Generic Solver API, the solver wrapper can extend the base set with *solver-specific actions* which directly interact with the solver API.

**Solver Manager.** The Solver Manager is the central manager for sorts, operators, and terms created by the solver. It exposes an interface for actions to (i) randomly pick enabled sorts and operators based on certain criteria, and (ii) notify the manager of new terms and sorts. The Solver Manager further maintains *solver-specific configurations* of supported theories, sorts, and operators. It configures solver-specific behavior by querying the solver wrapper to obtain solver-specific configuration information (e.g., solver-specific operators) and restrictions (e.g., unsupported sorts and operators).

**Generic Solver API.** The Generic Solver API provides a common solver interface for interacting with a solver under test. It covers the majority of the features defined in SMT-LIB, and defines abstract base classes for sort, term and solver implementations. It further provides an interface for configuring the Solver Manager as mentioned above. Integrating a new SMT solver into Murxla amounts to implementing these three classes, and optionally, solver-specific configurations, in a solver wrapper.

The Generic Solver API aims at being as general as possible while supporting all semantic features of the SMT-LIB data model. The Generic Solver API further supports "meta" solvers for different purposes. Murxla implements meta solvers for: (i) performing checks of witnesses that require additional solver instances (model value, unsat core, and unsat assumptions checks); (ii) checking the results of one solver instance against another to identify soundness issues; (iii) translating API call sequences to the SMT-LIBv2 format; and (iv) SMT-LIBv2 input fuzzing of SMT solver binaries in interactive SMT-LIB mode.

#### **3.2 API Fuzzer**

The *API Fuzzer* is responsible for generating random but valid API call sequences and is the central component of Murxla. Valid API call sequences are generated based on an API model which is implemented as a weighted finitestate machine (FSM), where states correspond to the current state of the SMT solver, and transitions have a weight, a pre-condition, and an associated action. Each state of the FSM may provide a pre-condition that defines when it is legal to transition into that state. Taking a transition also executes its action. The associated action of a transition may be empty, in which case it leads to the next state without calling the solver. The pre-condition of a transition and the pre-condition of its next state define the conditions under which the transition can be selected, whereas its weight determines the probability of it being taken in cases where multiple transitions are enabled at the same time.

By default, the FSM implements an API model that captures the functionality and constraints defined in the SMT-LIB standard. And as described above, its associated actions call the Generic Solver API. Murxla supports arbitrary solver-specific modifications to this FSM by providing a configuration interface for solver wrappers (which we discuss in Sect. 3.3 below).

Configuration of the API Fuzzer and execution of its FSM to generate API call sequences for a single run is performed using the following steps.


In contrast to some recent mutation-based SMT-LIBv2 input fuzzing approaches [39,49,51,52], the API Fuzzer is *generation-based*: it generates expressions that, importantly, respect the semantic and API models of the solver under test. Non-leaf terms are generated by combining leaf terms (variables or theory-specific constants) and previously generated terms via any of the enabled operators. To bias the generated terms towards more variety and structure, each term maintains a reference count, and terms with lower reference counts are selected with a higher probability when constructing new terms. For indexed operator kinds (e.g., the extract operator in the theory of fixed-size bit-vectors), random integer values up to a configured maximum value (if not otherwise restricted by the semantics of the operator) are selected. Similarly, arguments to sort constructors (e.g., Array) are sampled from previously generated sorts, and sorts with numeric parameters (e.g., bit-vector and floating-point sorts) are constructed from randomly selected integer values up to a configured maximum value.

The API fuzzer utilizes a random number generator (RNG) for random decisions, which is deterministic in the sense that it is guaranteed to produce the same sequence of values when given the same starting seed. The API fuzzer supports two usage modes: (i) single run, starting with a specific seed; and (ii) continuous, consisting of repeated single runs with seeds selected by a dedicated Seed Generator, which uses the current time and process ID to generate seeds. Each mode can be restricted to a given set of theories (with or without quantifiers) via the command line (in this case, step two of the fuzzer configuration detailed above is skipped). When in *single run mode*, Murxla by default sends a trace of the run to stdout (and optionally to a file). In *continuous mode*, each run is first executed without tracing. If a run uncovers an issue, it is replayed with the same seed and recorded to a trace file. In this mode, Murxla maintains a statistics summary with the current number of issues, timeouts, and *sat*, *unsat*, and *unknown* results. When an issue is discovered, it reports the corresponding seed and solver output. On termination, it provides an overview of all issues, *deduplicated* based on fuzzy matching on the solver output.

Murxla only reports false positives in rare cases where false positives may only be avoided with unreasonable effort, e.g., implementing well-formedness checks for algebraic datatypes.

#### **3.3 Solver Wrappers**

As mentioned above, a solver wrapper is used to connect Murxla to a solver. Solver wrappers are typically 2k–4k LOC in size and implement the Generic Solver API. If a solver provides features that are different from those covered by the Generic Solver API, a solver wrapper can accommodate these differences by reconfiguring the FSM of the API Fuzzer to add or remove states, transitions, and actions (added actions can be configured to call the API of the solver under test directly). Solver wrappers are further responsible for configuring the semantic model of the API Fuzzer by (i) adding or removing supported theories and their corresponding sorts and operators; and (ii) extending or restricting the set of operators for supported theories. Solver wrappers may also implement sanity checks of arbitrary complexity by utilizing the MURXLA\_TEST macro.

The option model of a solver is implemented as part of the Generic Solver API. For Bitwuzla, Boolector, and cvc5, this amounts to 15–55 LOC since all three can be queried for available options and valid configuration values via the API. This allows an automated registration of options with the Solver Manager. Yices2 does not provide this feature which requires that options are registered explicitly. Note that its option model is currently not implemented.

Each solver wrapper maintains its own RNG which is used to make choices when there are multiple alternative solver API calls for one specific task. This RNG is independent from the main RNG of the API Fuzzer and is seeded with a value generated by the main RNG for each action execution. These seeds are recorded by the Tracer to ensure that random choices can be deterministically replicated when replaying a traced run of the API Fuzzer.

#### **3.4 Tracer, Untracer, Trace Minimizer**

The **Tracer** records all action executions with their corresponding arguments and return values in a text-based format. Each action line in the trace follows the pattern <seed> <action> [<args...>], optionally followed by a return statement of the form return <values...> for actions that create sorts or terms. The <seed>

```
74761 new
65471 set -logic QF_BV
33949 mk -sort SORT_BOOL
      return s1
64345 mk -sort SORT_BV 8
      return s2
49391 mk -const s2 "a"
      return t1
89712 mk -const s2 "b"
      return t2
 6548 mk -term OP_EQUAL SORT_BOOL 2 t1 t2
      return t3 s1
20351 assert -formula t3
47017 check -sat
74496 delete
```
**Fig. 2.** Murxla trace for checking *a* = *b* for bit-vectors *a* and *b* of size 8.

in an action is the seed of the solver wrapper's RNG when executing the action. It is recorded to ensure that random choices made by the solver wrapper can be deterministically replicated. This is especially important when minimizing a trace, since modifying trace lines may change the way the main RNG behaves when replaying the trace. Sorts are recorded as s<id> and terms as t<id>, and the <args...> of an action line determine all sort, term and numerical arguments required to replay the execution of the given action. Similarly, the <values..> of a return statement record all of its sort and term return values. Figure 2 shows an example of a trace generated by Murxla. It records the action sequence for checking the satisfiability of *a* = *b*, where *a* and *b* are bit-vectors of size 8. Note that when creating terms via mk-term, we trace argument lists while also providing the number of arguments, e.g., 2 t1 t2. The same applies for indices of indexed operators. Further, for any action that creates terms that are added to the term database (e.g., mk-term), we also need to trace the sort of the created term, e.g., return t3 s1. This is due to the fact that some operators create terms of new sorts that may not have been encountered in the trace yet.

The **Untracer** takes a trace as input and replays each recorded action, thereby replicating the behavior of the original execution. This is especially useful for debugging erroneous behavior of the solver under test. Additionally, if a trace does not contain any solver-specific extensions, the Untracer can replay it using a different solver or translate it to the SMT-LIBv2 format. Tracing actions instead of calls to the Generic Solver API has the advantage that both the API Fuzzer and the Untracer can use the same infrastructure for communicating with the solver under test. Furthermore, supporting solver-specific actions does not require changes to any component other than the solver wrapper.

The **Trace Minimizer** is built on top of the Untracer and minimizes a given trace while preserving the behavior of the original execution. It implements simple ddmin-style [53] minimization techniques in three phases: (i) line-based minimization to reduce the number of trace lines; (ii) minimization of action lines to reduce the number of arguments; and (iii) term substitution, where terms are replaced with simpler terms of the same sort. Even though all of these minimization techniques are rather basic, the Trace Minimizer typically reduces the size of a trace to less than 10% of the original trace. If the minimized trace can be translated to SMT-LIB, then it can often be further reduced using a delta-debugging tool such as ddSMT [37]. Even if a minimized trace cannot be expressed in SMT-LIB due to solver-specific extensions, we have found that in practice, the reduction due to the Trace Minimizer is typically good enough to allow efficient debugging.

#### **4 Evaluation**

We evaluate the efficacy of Murxla in three experiments, comparing: (1) Murxla and BtorMBT, testing Boolector; (2) Murxla and the current state-of-the-art input fuzzers STORM [39] and TypeFuzz [47]; and (3) Murxla with and without option fuzzing. For this evaluation, we target soundness issues and crashes, and do not consider performance regressions. In the following, we use *issues* to mean crashes unless explicitly otherwise noted. We use Bitwuzla commit eea0973 [5], Boolector commit b157b10 [6], cvc5 commit 0f5ee6b [7], and Yices2 commit 09f1621 [8]. For each experiment we compare the number of issues uncovered by each tool, and the code coverage of the solver under test. Code coverage was measured with gcov, which is part of the GNU Compiler Collection [9]. We performed all experiments in an Ubuntu 21.04 Docker container on a machine with an AMD Threadripper 3970X CPU and 128GB of memory and used a one hour wall-clock time limit for each experiment and tool.

**Murxla vs. BtorMBT.** We compare the effectiveness of fuzzing Boolector with Murxla against that of its own custom API fuzzer BtorMBT. We ran both tools in continuous mode with a one second time limit per single run. Murxla achieves a line (function) coverage of 81% (88%) and finds 18 issues (including 3 known reported issues). BtorMBT achieves 72% (81%) coverage, but does not find any issues. BtorMBT does not support quantifiers, and three of the issues found by Murxla are located in Boolector's quantifiers module. The other issues, however, occur in code that is covered by BtorMBT.

**Murxla vs. STORM, TypeFuzz.** We test cvc5 with Murxla, STORM, and TypeFuzz on QF SLIA problems. We use all QF SLIA benchmarks in the SMT-LIB benchmark library as seed files for STORM and TypeFuzz. Both Storm and TypeFuzz mainly target soundness issues. TypeFuzz requires using at least two SMT solvers as it relies on comparing their results, whereas Storm creates satisfiable formulas by construction and does not require cross-checking. Hence, we additionally use a cross-checking configuration of Murxla (*Murxla-cc*), which compares Z3 version 4.8.14 and cvc5. Since Murxla does not yet integrate Z3, we use it via Murxla's SMT-LIBv2 interface in interactive SMT-LIB mode (the input fuzzing mode). Note that this requires disabling solver-specific extensions


**Table 1.** Number of issues (I), and line (L) and function (F) coverage for experiments two (top) and three (bottom). Option fuzzing for Yices2 is not yet implemented (-).

of cvc5, since they are unsupported by Z3. The results are shown in Table 1. Murxla and Murxla-cc have consistently higher coverage than the other tools and find 8 issues, whereas the other tools find none. Most notably Murxla-cc was able to find a model unsoundness issue in cvc5, where cvc5 incorrectly reports *satisfiable* due to an incorrect rewrite rule for the re.loop operator [1].

**Option Fuzzing.** We evaluate the effectiveness of Murxla with and without option fuzzing on all supported solvers. We use the default configuration of Murxla, which tests all supported features for each solver. The results are shown in Table 1 and showcase the efficacy of option fuzzing both for improving coverage and for finding issues. In its best configuration, Murxla achieves an API function coverage of 85% for Bitwuzla, 94% for Boolector, 68% for cvc5, and 46% for Yices2. cvc5 provides the richest API, supporting not only all of SMT-LIB but also non-standard theories and non-SMT features like SyGuS and high-order reasoning, which are not yet supported in Murxla. Bitwuzla and Boolector export parsing via the API, which is currently only supported in Murxla for Boolector. Coverage for Yices2 is low in comparison as it was integrated as a proof of concept, and its wrapper does not yet implement all of its features nor its option model.

The artifact containing the experimental data of this evaluation is available at https://zenodo.org/record/6494381.

#### **5 Conclusion**

Our experimental evaluation shows that Murxla quickly and effectively finds issues in multiple state-of-the-art SMT solvers—even for logics like QF SLIA which have been the subject of month-long fuzzing campaigns [39,47,51,52] over the last two years. Furthermore, during the past few months, while finalizing and testing Murxla, we found many more issues in these solvers—more than 100 for cvc5 alone, and some of them critical [3,4]. Based on this success, we believe that Murxla will be a valuable tool for stress-testing SMT solvers and thereby improving their correctness and robustness. We are currently in the process of integrating it into the development workflow of Bitwuzla, Boolector and cvc5.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

**Automata and Logic**

## **FORQ-Based Language Inclusion Formal Testing**

Kyveli Doveri1,2 , Pierre Ganty1(B) , and Nicolas Mazzocchi1,3

<sup>1</sup> IMDEA Software Institute, Madrid, Spain {kyveli.doveri,pierre.ganty, nicolas.mazzocchi}@imdea.org <sup>2</sup> Universidad Polit´ecnica de Madrid, Madrid, Spain

<sup>3</sup> Institute of Science and Technology Austria, Klosterneuburg, Austria

**Abstract.** We propose a novel algorithm to decide the language inclusion between (nondeterministic) B¨uchi automata, a PSpace-complete problem. Our approach, like others before, leverage a notion of quasiorder to prune the search for a counterexample by discarding candidates which are subsumed by others for the quasiorder. Discarded candidates are guaranteed to not compromise the completeness of the algorithm. The novelty of our work lies in the quasiorder used to discard candidates. We introduce FORQs (family of right quasiorders) that we obtain by adapting the notion of family of right congruences put forward by Maler and Staiger in 1993. We define a FORQ-based inclusion algorithm which we prove correct and instantiate it for a specific FORQ, called the structural FORQ, induced by the B¨uchi automaton to the right of the inclusion sign. The resulting implementation, called Forklift, scales up better than the state-of-the-art on a variety of benchmarks including benchmarks from program verification and theorem proving for word combinatorics. **Artifact:** https://doi.org/10.5281/zenodo.6552870

**Keywords:** Language inclusion · B¨uchi automata · Well-quasiorders

#### **1 Introduction**

In verification [19,20] and theorem proving [31], B¨uchi automata have been used as the underlying formal model. In these settings, B¨uchi automata respectively encode 1) the behaviors of a system as well as properties about it; and 2) the set of valuations satisfying a predicate. Questions like asking whether a system complies with a specification naturally reduce to a language inclusion problem and so does proving a theorem of the form ∀x ∃y, P(x) ⇒ Q(y).

This work was partially funded by the ESF Investing in your future, the Madrid regional project S2018/TCS-4339 BLOQUES, the Spanish project PGC2018-102210- B-I00 BOSCO, the Ram´on y Cajal fellowship RYC-2016-20281, and the ERC grant PR1001ERC02.

In this paper we propose a new algorithm for the inclusion problem between ω-regular languages given by B¨uchi automata. The problem is PSpacecomplete [23] and significant effort has been devoted to the discovery of algorithms for inclusion that behave well in practice [8,10,14,18,22,25]. Each proposed algorithm is characterized by a set of techniques (e.g. Ramsey-based, rank-based) and heuristics (e.g. antichains, simulation relations). The algorithm we propose falls into the category of Ramsey-based algorithms and uses the antichain [11] heuristics: the search for counterexamples is pruned using quasiorders. Intuitively when two candidate counterexamples are comparable with respect to some considered quasiorder, the "higher" of the two can be discarded without compromising completeness of the search. In our setting, counterexamples to inclusion are ultimately periodic words, i.e., words of the form uv<sup>ω</sup>, where u and v are called a *stem* and a *period*, respectively. Therefore pruning is done by comparing stems and periods of candidate counterexamples during the search.

In the work of Abdulla et al. [7,8] which was further refined by Clemente et al. [10] they use a single quasiorder to compare both stems and periods. Their effort has been focused on refining that single quasiorder by enhancing it with simulation relations. Others including some authors of this paper, followed an orthogonal line [13,22] that investigates the use of two quasiorders: one for the stems and another one, independent, for the periods. The flexibility of using different quasiorders yields more pruning when searching for a counterexample. In this paper, we push the envelope further by using an unbounded number of quasiorders: one for the stems and a family of quasiorders for the periods each of them depending on a distinct stem. We use the acronym FORQ, which stands for *family of right quasiorders*, to refer to these quasiorders. Using FORQs leads to significant algorithmic differences compared to the two quasiorders approaches. More precisely, the algorithms with two quasiorders [13,22] compute exactly two fixpoints (one for the stems and one for the periods) independently whereas the FORQ-based algorithm that we present computes two fixpoints for the stems and unboundedly many fixpoints for the periods (depending on the number of stems that belong to the first two fixpoints). Even though we lose the stem/period independence and we compute more fixpoints, in practice, the use of FORQs scales up better than the approaches based on one or two quasiorders.

We formalize the notion of FORQ by relaxing and generalizing the notion of *family of right congruences* introduced by Maler and Staiger [30] to advance the theory of recognizability of ω-regular languages and, in particular, questions related to minimal-state automata. Recently, families of right congruences have been used in other contexts like the learning of ω-regular languages (see [9] and references therein) and B¨uchi automata complementation [26].

Below, we describe how our contributions are organized:


#### **2 Preliminaries**

**Languages.** Let Σ be a finite and non-empty *alphabet*. We write Σ<sup>∗</sup> to refer to the set of finite words over Σ and we write ε to denote the empty word. Given u ∈ Σ∗, we denote by |u| the length of u. In particular |ε| = 0. We also define Σ<sup>+</sup> - <sup>Σ</sup><sup>∗</sup> \ {ε}, and <sup>Σ</sup>∇<sup>n</sup> - {u ∈ Σ<sup>∗</sup> | |u| ∇ n} with ∇ ∈ {≤, ≥}, hence Σ<sup>∗</sup> = Σ≥<sup>0</sup>, Σ<sup>+</sup> = Σ≥<sup>1</sup>. We write Σ<sup>ω</sup> to refer to the set of infinite words over <sup>Σ</sup>. An infinite word <sup>μ</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup> is said to be *ultimately periodic* if it admits a decomposition <sup>μ</sup> <sup>=</sup> uv<sup>ω</sup> with <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> (called a *stem*) and <sup>v</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup> (called a *period*). We fix an alphabet Σ throughout the paper.

**Order Theory.** Let E be a set of elements and be a binary relation over E. The relation is said to be a *quasiorder* when it is *reflexive* and *transitive*. Given a subset X of E, we define its *upward closure* with respect to the quasiorder - by -X - {<sup>e</sup> <sup>∈</sup> <sup>E</sup> | ∃<sup>x</sup> <sup>∈</sup> X, x e}. Given two subsets X, Y ⊆ E the set Y is said to be a *basis* for X with respect to -, denoted B-(Y,X), whenever Y ⊆ X and -X = -Y . The quasiorder is a *well-quasiorder* iff for each set X ⊆ E there exists a finite set Y ⊆ E such that B-(Y,X). This property on bases is also known as the *finite basis property*. Other equivalent definitions of well-quasiorders can be found in the literature [27], we will use the followings:

1. For every {ei}<sup>i</sup>∈<sup>N</sup> <sup>∈</sup> <sup>E</sup><sup>N</sup> there exists i, j <sup>∈</sup> <sup>N</sup> with i<j such that <sup>e</sup><sup>i</sup> e<sup>j</sup> . 2. No sequence {Xi}<sup>i</sup>∈<sup>N</sup> <sup>∈</sup> <sup>℘</sup>(E)<sup>N</sup> is such that -X<sup>1</sup> -X<sup>2</sup> ... holds.<sup>1</sup>

**Automata.** A (nondeterministic) *B¨uchi automaton* B (BA for short) is a tuple (Q, q<sup>I</sup> , Δ, F) where Q is a finite set of states including q<sup>I</sup> , the initial state, Δ ⊆ Q × Σ × Q is the transition relation, and, F ⊆ Q is the set of accepting states. We lift <sup>Δ</sup> to finite words as expected. We prefer to write <sup>B</sup>: <sup>q</sup><sup>1</sup> <sup>u</sup>−−→ <sup>q</sup><sup>2</sup> instead of (q1, u, q2) <sup>∈</sup> <sup>Δ</sup>. In addition, we write <sup>B</sup>: <sup>q</sup><sup>1</sup> <sup>u</sup>−−→<sup>F</sup> <sup>q</sup><sup>2</sup> when there exists a state <sup>q</sup><sup>F</sup> <sup>∈</sup> <sup>F</sup> and two words <sup>u</sup>1, u<sup>2</sup> such that <sup>B</sup>: <sup>q</sup><sup>1</sup> <sup>u</sup><sup>1</sup> −−→ <sup>q</sup><sup>F</sup> <sup>u</sup><sup>2</sup> −−→ <sup>q</sup>2, and <sup>u</sup> <sup>=</sup> <sup>u</sup>1u2.

A run <sup>π</sup> of <sup>B</sup> over <sup>μ</sup> <sup>=</sup> <sup>a</sup>0a<sup>1</sup> ··· ∈ <sup>Σ</sup><sup>ω</sup> is a function <sup>π</sup> : <sup>N</sup> <sup>→</sup> <sup>Q</sup> such that <sup>π</sup>(0) = <sup>q</sup><sup>I</sup> and for all position <sup>i</sup> <sup>∈</sup> <sup>N</sup>, we have that <sup>B</sup>: <sup>π</sup>(i) <sup>a</sup><sup>i</sup> −−→ <sup>π</sup>(<sup>i</sup> + 1). A run is said to be *accepting* if <sup>π</sup>(i) <sup>∈</sup> <sup>F</sup> for infinitely many values of <sup>i</sup> <sup>∈</sup> <sup>N</sup>. The language L(B) of words *recognized* by B is the set of ω-words for which B admits an accepting run. A language L is ω-*regular* if it is recognized by some BA.

<sup>1</sup> The notation ℘(E) denotes the set of all subsets of E.

#### **3 Foundations of Our Approach**

Let A - (P, p<sup>I</sup> , ΔA, FA) be a B¨uchi automaton and M be an ω-regular language. The main idea behind our approach is to compute a finite subset T<sup>A</sup> of ultimately periodic words of L(A) such that:

$$T\_{\mathcal{A}} \subseteq M \iff L(\mathcal{A}) \subseteq M \; . \tag{\dagger}$$

Then L(A) ⊆ M holds iff each of the finitely many words of T<sup>A</sup> belongs to M which is tested via membership queries.

First we observe that such a subset always exists: if the inclusion holds take T<sup>A</sup> to be any finite subset of L(A) (empty set included); else take T<sup>A</sup> to contain some ultimately periodic word that is a counterexample to inclusion. In what follows, we will show that a finite subset T<sup>A</sup> satisfying (†) can be computed by using an ordering to prune the ultimately periodic words of L(A). We will obtain such an ordering using a family of right quasiorders, a notion introduced below.

**Definition 1 (FORQ).** *A family of right quasiorders is a pair* <sup>&</sup>lt;∼, {∼u}<sup>u</sup>∈Σ<sup>∗</sup> *where* <sup>&</sup>lt;<sup>∼</sup> <sup>⊆</sup> <sup>Σ</sup><sup>∗</sup> <sup>×</sup> <sup>Σ</sup><sup>∗</sup> *is a right-monotonic*<sup>2</sup> *quasiorder as well as every* ∼<sup>u</sup> <sup>⊆</sup> <sup>Σ</sup><sup>∗</sup> <sup>×</sup> <sup>Σ</sup><sup>∗</sup> *where* <sup>u</sup> <sup>∈</sup> <sup>Σ</sup>∗*. Additionally, for all* u, u <sup>∈</sup> <sup>Σ</sup>∗*, we require* <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> <sup>⇒</sup> ∼<sup>u</sup> <sup>⊆</sup> ∼<sup>u</sup> *called the* FORQ constraint*.*

First, we observe that the above definition uses separate orderings for stems and periods. The definition goes even further, the ordering used for periods is depending on stems so that a period may or may not be discarded depending on the stem under consideration. The FORQ constraint tells us that if the periods v and w compare for a stem u , that is <sup>v</sup> ∼<sup>u</sup> <sup>w</sup>, then they also compare for every stem u subsuming u , that is <sup>v</sup> ∼<sup>u</sup> <sup>w</sup> if <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> .

Expectedly, a FORQ needs to satisfy certain properties for T<sup>A</sup> to be finite, computable and for (†) to hold (in particular the left to right direction). The property of right-monotonicity of FORQs is needed so that we can iteratively compute T<sup>A</sup> via a fixpoint computation (see Sect. 5).

**Definition 2 (Suitable FORQ).** *A FORQ* F - <sup>&</sup>lt;∼, {∼u}<sup>u</sup>∈Σ<sup>∗</sup> *is said to be* finite *(resp.* decidable*) when* <sup>&</sup>lt;∼*, its converse* <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup>*, and* {∼u} *for all* <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> *are all well-quasiorders (resp. computable). Given* <sup>L</sup> <sup>⊆</sup> <sup>Σ</sup><sup>ω</sup>*,* <sup>F</sup> *is said to* preserve <sup>L</sup> *when for all* u, <sup>u</sup><sup>ˆ</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> *and all* v, <sup>v</sup><sup>ˆ</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup> *if* uv<sup>ω</sup> <sup>∈</sup> <sup>L</sup>*,* <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>ˆ*,* <sup>v</sup> ∼u<sup>ˆ</sup> <sup>v</sup><sup>ˆ</sup> *and* <sup>u</sup><sup>ˆ</sup> <sup>v</sup><sup>ˆ</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup><sup>ˆ</sup> *then* <sup>u</sup>ˆvˆ<sup>ω</sup> <sup>∈</sup> <sup>L</sup>*. Finally,* <sup>F</sup> *is said to be* <sup>L</sup>-suitable *(for inclusion) if it is finite, decidable and preserves* L*.*

Intuitively, the "well" property on the quasiorders ensures finiteness of TA. The preservation property ensures completeness: a counterexample to L(A) ⊆ M can only be discarded (that is, not included in TA) if it is subsumed by another ultimately periodic word in T<sup>A</sup> that is also a counterexample to inclusion.

Before defining T<sup>A</sup> we introduce for each state p ∈ P the sets of words

Stem<sup>p</sup> - {<sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> | A: <sup>p</sup><sup>I</sup> <sup>u</sup>−−→ <sup>p</sup>} and Per<sup>p</sup> -{<sup>v</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup> | A: <sup>p</sup> <sup>v</sup>−−→ <sup>p</sup>} .

<sup>2</sup> A quasiorder on Σ<sup>∗</sup> is *right-monotonic* when uv implies u wv w for all w ∈ Σ∗.

The set Stem<sup>p</sup> is the set of stems of <sup>L</sup>(A) that reach state <sup>p</sup> in <sup>A</sup> while the set Per<sup>p</sup> is the set of periods read by a cycle of <sup>A</sup> on state <sup>p</sup>.

Given a M-suitable FORQ F - <sup>&</sup>lt;∼, {∼u}u∈Σ<sup>∗</sup> , we let

$$T\_{\mathcal{A}} \triangleq \left\{ uv^{\omega} \mid \exists s \in F\_{\mathcal{A}} \colon u \in U\_s, v \in V\_s^w \text{ for some } w \in W\_s \text{ with } u \mathop{\triangleleft}\_{\sim} w \right\} \quad (\ddagger)$$

where for all <sup>p</sup> <sup>∈</sup> <sup>P</sup>, the set <sup>U</sup><sup>p</sup> is a basis of Stem<sup>p</sup> with respect to <sup>&</sup>lt;∼, that is B< ∼(Up, Stemp) holds. Moreover <sup>B</sup><sup>&</sup>lt; <sup>∼</sup>−<sup>1</sup> (Wp, Stemp) holds and <sup>B</sup>∼<sup>w</sup> (<sup>V</sup> <sup>w</sup> <sup>p</sup> , Perp) holds for all <sup>w</sup> <sup>∈</sup> <sup>W</sup>p. Note that the quasiorder ∼<sup>w</sup> used to prune the periods of Per<sup>p</sup> depends on a maximal w.r.t. <sup>&</sup>lt;<sup>∼</sup> stem <sup>w</sup> of Stem<sup>p</sup> since <sup>w</sup> belongs to the basis <sup>W</sup><sup>p</sup> for <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup>. The correctness argument for choosing ∼<sup>w</sup> essentially relies on the FORQ constraint as the proof of (†) given below shows. In Sect. 8 we will show, that when <sup>w</sup> is not "maximal" the quasiorder ∼<sup>w</sup> yields a set <sup>T</sup><sup>A</sup> for which (†) does not hold.

Furthermore, we conclude from the finite basis property of the quasiorders of <sup>F</sup> that <sup>U</sup>p, <sup>W</sup><sup>p</sup> and {<sup>V</sup> <sup>w</sup> <sup>p</sup> }<sup>w</sup>∈Σ<sup>∗</sup> are finite for all p ∈ P, hence T<sup>A</sup> is a finite subset of ultimately periodic words of L(A). Next we prove the equivalence (†). The proof crucially relies on the preservation property of F which allows discarding candidate counterexamples without loosing completeness, that is, if inclusion does not hold a counterexample will be returned.

*Proof (of* (†)*).* Consider Ultim<sup>A</sup> - {uv<sup>ω</sup> | ∃<sup>s</sup> <sup>∈</sup> <sup>F</sup><sup>A</sup> : <sup>u</sup> <sup>∈</sup> Stem<sup>s</sup>, v <sup>∈</sup> Per<sup>s</sup>, uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>}. It is easy to show that Ultim<sup>A</sup> <sup>=</sup> {uv<sup>ω</sup> | ∃<sup>s</sup> <sup>∈</sup> <sup>F</sup><sup>A</sup> : <sup>u</sup> <sup>∈</sup> Stem<sup>s</sup>, v <sup>∈</sup> Per<sup>s</sup>} (same definition as Ultim<sup>A</sup> but without the constraint uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>) by reasoning on properties of well-quasi orders.<sup>3</sup> It is well-known that ω-regular language inclusion holds if and only if it holds for ultimately periodic words. Formally <sup>L</sup>(A) <sup>⊆</sup> <sup>M</sup> holds if and only if Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup> holds. Therefore, to prove (†), we show that <sup>T</sup><sup>A</sup> <sup>⊆</sup> <sup>M</sup> <sup>⇔</sup> Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup>.

To prove the implication Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup> <sup>⇒</sup> <sup>T</sup><sup>A</sup> <sup>⊆</sup> <sup>M</sup> we start by taking a word uv<sup>ω</sup> <sup>∈</sup> <sup>T</sup><sup>A</sup> such that, by definition (‡), <sup>u</sup> <sup>∈</sup> <sup>U</sup><sup>s</sup> and <sup>v</sup> <sup>∈</sup> <sup>V</sup> <sup>w</sup> <sup>s</sup> for some s ∈ F<sup>A</sup> and w ∈ Ws. We conclude from B<sup>&</sup>lt; <sup>∼</sup>(Us, Stem<sup>s</sup>) and <sup>B</sup>∼<sup>w</sup> (<sup>V</sup> <sup>w</sup> <sup>s</sup> , Per<sup>s</sup>) that <sup>u</sup> <sup>∈</sup> <sup>U</sup><sup>s</sup> <sup>⊆</sup> Stem<sup>s</sup> and <sup>v</sup> <sup>∈</sup> <sup>V</sup> <sup>w</sup> <sup>s</sup> <sup>⊆</sup> Per<sup>s</sup>. Thus, we find that uv<sup>ω</sup> <sup>∈</sup> Ultim<sup>A</sup> hence the assumption Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup> shows that uv<sup>ω</sup> <sup>∈</sup> <sup>M</sup> which proves the implication.

Next, we prove that <sup>T</sup><sup>A</sup> <sup>⊆</sup> <sup>M</sup> <sup>⇒</sup> Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup> holds as well. Let uv<sup>ω</sup> <sup>∈</sup> Ultim<sup>A</sup>, i.e., such that there exists <sup>s</sup> <sup>∈</sup> <sup>F</sup><sup>A</sup> for which <sup>u</sup> <sup>∈</sup> Stem<sup>s</sup> and <sup>v</sup> <sup>∈</sup> Per<sup>s</sup>, satisfying uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>. Since <sup>u</sup> <sup>∈</sup> Stem<sup>s</sup> and <sup>v</sup> <sup>∈</sup> Per<sup>s</sup>, there exist <sup>u</sup><sup>0</sup> <sup>∈</sup> <sup>U</sup>s, <sup>w</sup><sup>0</sup> <sup>∈</sup> <sup>W</sup><sup>s</sup> and <sup>v</sup><sup>0</sup> <sup>∈</sup> <sup>V</sup> <sup>w</sup><sup>0</sup> <sup>s</sup> such that <sup>u</sup><sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup><sup>0</sup> and <sup>v</sup><sup>0</sup> ∼<sup>w</sup><sup>0</sup> <sup>v</sup> thanks to the finite basis property. By definition we have u0v<sup>ω</sup> <sup>0</sup> <sup>∈</sup> <sup>T</sup><sup>A</sup> and thus we find that <sup>u</sup>0v<sup>ω</sup> <sup>0</sup> ∈ M since <sup>T</sup><sup>A</sup> <sup>⊆</sup> <sup>M</sup>. Next since <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup>0, the FORQ constraint shows that ∼<sup>w</sup><sup>0</sup> <sup>⊆</sup> ∼<sup>u</sup> which, in turn, implies that <sup>v</sup><sup>0</sup> ∼<sup>u</sup> <sup>v</sup> holds. Finally, we deduce from <sup>u</sup>0v<sup>ω</sup> <sup>0</sup> <sup>∈</sup> <sup>M</sup>, <sup>u</sup><sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>, <sup>v</sup><sup>0</sup> ∼<sup>u</sup> <sup>v</sup>, uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> and the preservation of <sup>M</sup> by the FORQ <sup>F</sup> that uv<sup>ω</sup> <sup>∈</sup> <sup>M</sup>. We thus obtain that Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup> and we are done.

<sup>3</sup> The case <sup>⊆</sup> is trivial. For the case <sup>⊇</sup>, let uv<sup>ω</sup> with <sup>u</sup> <sup>∈</sup> Stem<sup>s</sup> and <sup>v</sup> <sup>∈</sup> Pers. If uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> then we are done for otherwise consider the sequence {uv<sup>i</sup> }<sup>i</sup>∈<sup>N</sup>. Since <sup>&</sup>lt;<sup>∼</sup> −1 is a well-quasiorder, there exists x, y <sup>∈</sup> <sup>N</sup> such that x<y and uv<sup>x</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup> uv<sup>y</sup> (viz. uv<sup>y</sup> <sup>&</sup>lt;<sup>∼</sup> uv<sup>x</sup>). Therefore we have (uv<sup>x</sup>)(v<sup>y</sup>−<sup>x</sup>) <sup>ω</sup> <sup>=</sup> uv<sup>ω</sup>, (uv<sup>x</sup>) <sup>∈</sup> Stems, (v<sup>y</sup>−<sup>x</sup>) <sup>∈</sup> Pers, and (uv<sup>x</sup>)(v<sup>y</sup>−<sup>x</sup>) <sup>&</sup>lt;<sup>∼</sup> (uv<sup>x</sup>), hence uv<sup>ω</sup> <sup>∈</sup> UltimA.

*Example 3.* To gain more insights about our approach consider the BAs of Fig. 1 for which we want to check whether L(A) ⊆ L(B) holds. From the description of <sup>A</sup> it is routine to check that Stemp<sup>I</sup> <sup>=</sup> <sup>Σ</sup><sup>∗</sup> and Perp<sup>I</sup> <sup>=</sup> <sup>Σ</sup><sup>+</sup>. Let us assume the existence<sup>4</sup> of <sup>&</sup>lt;<sup>∼</sup> (hence <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup>), ∼<sup>ε</sup> and ∼aa such that <sup>a</sup> <sup>&</sup>lt;<sup>∼</sup> aa holds and so does B< ∼({ε, a}, Σ∗), <sup>B</sup><sup>&</sup>lt; <sup>∼</sup>−<sup>1</sup> ({ε, aa}, Σ∗), <sup>B</sup>∼<sup>ε</sup> ({b}, Σ<sup>+</sup>) and <sup>B</sup>∼aa ({a}, Σ<sup>+</sup>). In addition, we set Up<sup>I</sup> = {ε, a} since B<sup>&</sup>lt; <sup>∼</sup>({ε, a}, Σ∗) and <sup>W</sup>p<sup>I</sup> <sup>=</sup> {ε, aa} since B< ∼−<sup>1</sup> ({ε, aa}, Σ∗). Moreover <sup>V</sup> <sup>ε</sup> <sup>p</sup><sup>I</sup> <sup>=</sup> {b} since <sup>B</sup>∼<sup>ε</sup> ({b}, Σ<sup>+</sup>), and <sup>V</sup> aa <sup>p</sup><sup>I</sup> = {a} since <sup>B</sup>∼aa ({a}, Σ<sup>+</sup>). Next by definition (‡) of <sup>T</sup><sup>A</sup> and from <sup>a</sup> <sup>&</sup>lt;<sup>∼</sup> aa we deduce that <sup>T</sup><sup>A</sup> <sup>=</sup> {ε(b)<sup>ω</sup>, a(a)<sup>ω</sup>}. Finally, we conclude from (†) and <sup>a</sup><sup>ω</sup> <sup>∈</sup> <sup>T</sup><sup>A</sup> that <sup>a</sup><sup>ω</sup> <sup>∈</sup> <sup>L</sup>(A) (since <sup>T</sup><sup>A</sup> <sup>⊆</sup> <sup>L</sup>(A)) hence that <sup>L</sup>(A) <sup>L</sup>(B) because <sup>a</sup><sup>ω</sup> <sup>∈</sup>/ <sup>L</sup>(B). By checking membership of the two ultimately periodic words of T<sup>A</sup> into L(B) we thus have shown that L(A) ⊆ L(B) does not hold.

In the example above we did not detail how the FORQ was obtained let alone how to compute the finite bases. We fill that gap in the next two sections: we define FORQs based on the underlying structure of a given BA in Sect. 4 and show they are suitable; and we give an effective computation of the bases hence our FORQ-based inclusion algorithm in Sect. 5.

#### **4 Defining FORQs from the Structure of an Automaton**

In this section we introduce a type of FORQs called structural FORQs such that given a BA B the structural FORQ induced by B is L(B)-suitable.

**Definition 4.** *Let* B - (Q, q<sup>I</sup> , ΔB, FB) *be a BA. The structural FORQ of* B *is the pair* <sup>&</sup>lt;<sup>∼</sup> <sup>B</sup>, {∼<sup>B</sup> <sup>u</sup> }<sup>u</sup>∈Σ<sup>∗</sup> *where the quasiorders are defined by:*

$$\begin{aligned} u\_1 \mathop{\leqslant}\nolimits^{\mathcal{B}} u\_2 &\mathop{\mathop{\rm dg}\nolimits^{\triangle}}\nolimits \mathbf{Tgt}\_{\mathcal{B}}(u\_1) \subseteq \mathbf{Tgt}\_{\mathcal{B}}(u\_2) \\ v\_1 \mathop{\mathop{\rm dg}\nolimits^{\mathcal{B}}}\nolimits\_u &\mathop{\rm g}\nolimits^{\triangle} \mathbf{Cxt}\_{\mathcal{B}}(\mathbf{Tgt}\_{\mathcal{B}}(u), v\_1) \subseteq \mathbf{Cxt}\_{\mathcal{B}}(\mathbf{Tgt}\_{\mathcal{B}}(u), v\_2) \end{aligned}$$

*with* Tgt<sup>B</sup> : <sup>℘</sup>(Q) <sup>×</sup> <sup>Σ</sup><sup>∗</sup> <sup>→</sup> <sup>℘</sup>(Q) *and* Cxt<sup>B</sup> : <sup>℘</sup>(Q) <sup>×</sup> <sup>Σ</sup><sup>∗</sup> <sup>→</sup> <sup>℘</sup>(Q<sup>2</sup> × {⊥, })

$$\begin{aligned} \mathsf{Tgt}\_{\mathcal{B}}(u) & \triangleq \{ q' \in Q \mid \mathcal{B} \colon q\_I \\underline{u} \stackrel{u}{\longrightarrow} q' \} \\ \mathsf{Czt}\_{\mathcal{B}}(X, v) & \triangleq \{ (q, q', k) \mid q \in X, \mathcal{B} \colon q \stackrel{v}{\longrightarrow} q', (k = \top \Rightarrow \mathcal{B} \colon q \stackrel{v}{\longrightarrow}\_F q') \} \end{aligned}$$

Given <sup>u</sup> <sup>∈</sup> <sup>Σ</sup>∗, the set Tgt<sup>B</sup>(u) contains states that <sup>u</sup> can "target" from the initial state q<sup>I</sup> . A "context" (q, q , k) returned by Cxt<sup>B</sup>, consists in a source state q ∈ Q, a sink state q ∈ Q and a boolean k ∈ {, ⊥} that keeps track whether an accepting state is visited. Note that, having ⊥ as last component of a context does *not* mean that no accepting state is visited. When it is clear from the context, we often omit the subscript <sup>B</sup> from Tgt<sup>B</sup> and Cxt<sup>B</sup>. Analogously, we omit the BA from the structural FORQ quasiorders when there is no ambiguity.

**Lemma 5.** *Given a BA* <sup>B</sup>*, the pair* <sup>&</sup>lt;<sup>∼</sup> <sup>B</sup>, {∼<sup>B</sup> <sup>u</sup> }<sup>u</sup>∈Σ<sup>∗</sup> *of Definition 4 is a FORQ.*

<sup>4</sup> The definition of the orderings, needed to compute the bases, are given in Example 6.

**Fig. 1.** B¨uchi automata A and B over the alphabet Σ = {a, b}.

*Proof.* Let B - (Q, q<sup>I</sup> , ΔB, FB) be a BA, we start by proving that the FORQ constraint holds: <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>B</sup> <sup>u</sup> <sup>=</sup><sup>⇒</sup> ∼<sup>B</sup> <sup>u</sup> <sup>⊆</sup> ∼<sup>B</sup> <sup>u</sup> . First, we observe that, for all Y ⊆ <sup>X</sup> <sup>⊆</sup> <sup>Q</sup> and all v, v <sup>∈</sup> <sup>Σ</sup>∗, we have that Cxt(X, v) <sup>⊆</sup> Cxt(X, v ) <sup>⇒</sup> Cxt(Y,v) <sup>⊆</sup> Cxt(Y,v ). Consider u, u <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> such that <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>B</sup> u and v, v ∈ Σ<sup>∗</sup> such that <sup>v</sup> ∼<sup>B</sup> <sup>u</sup> v . Let X - Tgt(u) and <sup>X</sup> - Tgt(u ), we have that X ⊆ X following u <∼ <sup>B</sup> u . Next, we conclude from <sup>v</sup> ∼<sup>B</sup> <sup>u</sup> <sup>v</sup> that Cxt(X , v) <sup>⊆</sup> Cxt(X , v ), hence that Cxt(X, v) <sup>⊆</sup> Cxt(X, v ) by the above reasoning using X ⊆ X , and finally that <sup>v</sup> ∼<sup>B</sup> <sup>u</sup> v .

For the right monotonicity, Definition <sup>4</sup> shows that if Tgt(u) <sup>⊆</sup> Tgt(v) then Tgt(ua) <sup>⊆</sup> Tgt(va), hence we have <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>v</sup> implies ua <sup>&</sup>lt;<sup>∼</sup> va for all <sup>a</sup> <sup>∈</sup> <sup>Σ</sup>. The reasoning with the other quasiorders and Cxt proceeds analogously.

*Example 6.* Consider the BA <sup>B</sup> of Fig. <sup>1</sup> and let <sup>&</sup>lt;∼, {∼u}<sup>u</sup>∈Σ<sup>∗</sup> be its structural FORQ. More precisely, we have Tgt(ε) = {q<sup>I</sup> }; Tgt(a) = Tgt(b) = {q1}; and Tgt(u) = {q1, q2} for all <sup>u</sup> <sup>∈</sup> <sup>Σ</sup>≥<sup>2</sup>. In particular we conclude from <sup>u</sup><sup>1</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup><sup>2</sup> ⇐⇒ Tgt(u1) <sup>⊆</sup> Tgt(u2) that <sup>a</sup> <sup>&</sup>lt;<sup>∼</sup> aa, <sup>a</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>b</sup> and <sup>b</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>a</sup>; <sup>ε</sup> and <sup>a</sup> are incomparable; and so are <sup>ε</sup> and aa. Since Tgt has only three distinct outputs, the set {∼u}<sup>u</sup>∈Σ<sup>∗</sup> contains three distinct quasiorders.

1. <sup>v</sup><sup>1</sup> ∼<sup>ε</sup> <sup>v</sup><sup>2</sup> ⇐⇒ Cxt({q<sup>I</sup> }, v1) <sup>⊆</sup> Cxt({q<sup>I</sup> }, v2) where – Cxt({q<sup>I</sup> }, ε) = {(q<sup>I</sup> , q<sup>I</sup> , <sup>⊥</sup>)} – Cxt({q<sup>I</sup> }, a) = Cxt({q<sup>I</sup> }, b) = {(q<sup>I</sup> , q1, <sup>⊥</sup>)} – Cxt({q<sup>I</sup> }, v) = {(q<sup>I</sup> , q1, <sup>⊥</sup>),(q<sup>I</sup> , q2, <sup>⊥</sup>),(q<sup>I</sup> , q2, )} for all <sup>v</sup> <sup>∈</sup> <sup>Σ</sup>≥<sup>2</sup> 2. <sup>v</sup><sup>1</sup> ∼<sup>a</sup> <sup>v</sup><sup>2</sup> ⇐⇒ <sup>v</sup><sup>1</sup> ∼<sup>b</sup> <sup>v</sup><sup>2</sup> ⇐⇒ Cxt({q1}, v1) <sup>⊆</sup> Cxt({q1}, v2) where – Cxt({q1}, ε) = {(q1, q1, <sup>⊥</sup>)} – Cxt({q1}, v) = {(q1, q1, <sup>⊥</sup>),(q1, q2, <sup>⊥</sup>),(q1, q2, )} for all <sup>v</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup> 3. <sup>v</sup><sup>1</sup> ∼<sup>u</sup><sup>1</sup> <sup>v</sup><sup>2</sup> ⇐⇒ <sup>v</sup><sup>1</sup> ∼<sup>u</sup><sup>2</sup> <sup>v</sup><sup>2</sup> ⇐⇒ Cxt({q1, q2}, v1) <sup>⊆</sup> Cxt({q1, q2}, v2) for all <sup>u</sup>1, u<sup>2</sup> <sup>∈</sup> <sup>Σ</sup>≥<sup>2</sup> where

– Cxt({q1, q2}, ε) = {(q1, q1, <sup>⊥</sup>),(q2, q2, <sup>⊥</sup>),(q2, q2, )}

	- all <sup>v</sup> ∈ {b}<sup>+</sup>

With the above definitions the reader is invited to check the following predicates B< ∼({ε, a}, Σ∗), <sup>B</sup><sup>&</sup>lt; ∼({ε, b}, Σ∗), <sup>B</sup><sup>&</sup>lt; <sup>∼</sup>−<sup>1</sup> ({ε, aa}, Σ∗), <sup>B</sup>∼<sup>ε</sup> ({b}, Σ<sup>+</sup>), <sup>B</sup>∼<sup>a</sup> ({b}, Σ<sup>+</sup>) and <sup>B</sup>∼aa ({a}, Σ<sup>+</sup>). Also observe that none of the above finite bases contains comparable words for the ordering thereof. We also encourage the reader to revisit Example 3.

As prescribed in Sect. 3, we show that for every BA B its structural FORQ is L(B)-suitable, namely it is finite, decidable and preserves L(B).

## **Proposition 7.** *Given a BA* B*, its structural FORQ is* L(B)*-suitable.*

*Proof.* Let B - (Q, q<sup>I</sup> , ΔB, FB) be a BA and F - <sup>&</sup>lt;∼, {∼u}u∈Σ<sup>∗</sup> be its structural FORQ. The finiteness proof of F is trivial since Q is finite and so is the proof of decidability by Definition 4. For the preservation, given u0v<sup>ω</sup> <sup>0</sup> ∈ L(B), we show that for all <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> and all <sup>v</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup> such that uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> and <sup>u</sup><sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> and <sup>v</sup><sup>0</sup> ∼<sup>u</sup> <sup>v</sup> then uv<sup>ω</sup> <sup>∈</sup> <sup>L</sup>(B) holds. Let a run <sup>π</sup><sup>0</sup> <sup>q</sup><sup>I</sup> <sup>u</sup><sup>0</sup> −−→ <sup>q</sup><sup>0</sup> <sup>v</sup><sup>0</sup> −−→ <sup>q</sup><sup>1</sup> <sup>v</sup><sup>0</sup> −−→ <sup>q</sup><sup>2</sup> ... of <sup>B</sup> over <sup>u</sup>0v<sup>ω</sup> <sup>0</sup> which is accepting. Stated equivalently, we have <sup>q</sup><sup>0</sup> <sup>∈</sup> Tgt(u0) and (qi, qi+1, xi) <sup>∈</sup> Cxt(Tgt(u0v<sup>i</sup> <sup>0</sup>), v0) for every <sup>i</sup> <sup>∈</sup> <sup>N</sup> with the additional constraint that x<sup>i</sup> = holds infinitely often.

We will show that <sup>B</sup> has an accepting run over uv<sup>ω</sup> by showing that <sup>q</sup><sup>0</sup> <sup>∈</sup> Tgt(u) holds; (qi, qi+1, xi) <sup>∈</sup> Cxt(Tgt(uv<sup>i</sup> ), v) holds for every <sup>i</sup> <sup>∈</sup> <sup>N</sup>; and <sup>x</sup><sup>i</sup> <sup>=</sup> holds infinitely often. Since <sup>u</sup><sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> and <sup>q</sup><sup>0</sup> <sup>∈</sup> Tgt(u0) we find that <sup>q</sup><sup>0</sup> <sup>∈</sup> Tgt(u) by definition of <sup>&</sup>lt;∼. Next we show the remaining constraints by induction. The induction hypothesis states that for all 0 ≤ n we have (qn, qn+1, xn) <sup>∈</sup> Cxt(Tgt(uv<sup>n</sup>), v). For the base case (<sup>n</sup> = 0) we have to show that (q0, q1, x0) <sup>∈</sup> Cxt(Tgt(u), v). We conclude from (q0, q1, x0) <sup>∈</sup> Cxt(Tgt(u), v0), <sup>v</sup><sup>0</sup> ∼<sup>u</sup> <sup>v</sup> and the definition of ∼<sup>u</sup> that Cxt(Tgt(u), v0) <sup>⊆</sup> Cxt(Tgt(u), v) and finally that (q0, q1, x0) <sup>∈</sup> Cxt(Tgt(u), v). For the inductive case, assume (qn, qn+1, xn) <sup>∈</sup> Cxt(Tgt(uv<sup>n</sup>), v). The definition of context shows that <sup>q</sup>n+1 <sup>∈</sup> Tgt(uv<sup>n</sup>+1). It takes an easy an induction to show that uv<sup>n</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> for all <sup>n</sup> using uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> and right-monotonicity of <sup>&</sup>lt;∼. We conclude from uv<sup>n</sup>+1 <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>, the definition of <sup>&</sup>lt;<sup>∼</sup> and <sup>q</sup>n+1 <sup>∈</sup> Tgt(uv<sup>n</sup>+1) that <sup>q</sup>n+1 <sup>∈</sup> Tgt(u) also holds, hence that (qn+1, qn+2, xn+1) <sup>∈</sup> Cxt(Tgt(u), v0) following the definition of contexts and that of <sup>π</sup>0. Next, we find that (qn+1, qn+2, xn+1) <sup>∈</sup> Cxt(Tgt(u), v) following a reasoning analogous to the base case, this time starting with (qn+1, qn+2, xn+1) <sup>∈</sup> Cxt(Tgt(u), v0)). Finally, <sup>q</sup>n+1 <sup>∈</sup> Tgt(uv<sup>n</sup>+1) implies that (qn+1, qn+2, xn+1) <sup>∈</sup> Cxt(Tgt(uv<sup>n</sup>+1), v). We have thus shown that <sup>q</sup><sup>0</sup> <sup>∈</sup> Tgt(u) and (qi, qi+1, xi) <sup>∈</sup> Cxt(Tgt(uv<sup>i</sup> ), v) for every <sup>i</sup> <sup>∈</sup> <sup>N</sup> with the additional constraint that x<sup>i</sup> = holds infinitely often and we are done.

#### **5 A FORQ-Based Inclusion Algorithm**

As announced at the end of Sect. 3 it remains, in order to formulate our FORQbased algorithm deciding whether L(A) ⊆ M holds, to give an effective computation for the bases defining TA. We start with a fixpoint characterization of the stems and periods of BAs using the function Rcat<sup>A</sup> : <sup>℘</sup>(Σ∗)|<sup>P</sup> <sup>|</sup> <sup>→</sup> <sup>℘</sup>(Σ∗)|<sup>P</sup> <sup>|</sup> :

$$\mathsf{Rcat}\_{\mathcal{A}}(\vec{X}).p \triangleq \vec{X}.p \cup \{wa \in \Sigma^\* \mid w \in \vec{X}.p', a \in \Sigma, \mathcal{A} \colon p' \\_{a} \underset{-}{\mid p} \}$$

where S.<sup>p</sup> denotes the <sup>p</sup>-th element of the vector S <sup>∈</sup> <sup>℘</sup>(Σ∗)|<sup>P</sup> <sup>|</sup> . In Fig. 2, the repeat/until loops at lines 4 and 5 compute iteratively subsets of the stems of A, while the loop at line 10 computes iteratively subsets of the periods of A. The following lemma formalizes the above intuition.

**Input:** B¨uchi automaton A - (P, p<sup>I</sup> , ΔA, FA) **Input:** <sup>ω</sup>-regular language <sup>M</sup> with procedure deciding uv<sup>ω</sup> <sup>∈</sup> <sup>M</sup> given u, v **Input:** M-suitable FORQ F - <∼, {∼u}<sup>u</sup>∈Σ<sup>∗</sup> **Output:** Returns ok if L(A) ⊆ M and ko otherwise **1 Function: <sup>2</sup> let** U<sup>0</sup> <sup>∈</sup> <sup>℘</sup>(Σ∗) <sup>|</sup><sup>P</sup> <sup>|</sup> **as** U0*.*<sup>p</sup> - <sup>∅</sup> **with** <sup>p</sup> <sup>=</sup> <sup>p</sup><sup>I</sup> **and** U0.<sup>p</sup><sup>I</sup> - {ε} **<sup>3</sup>** W := U := U<sup>0</sup> **<sup>4</sup> repeat** W := RcatA(W ) **until** <sup>B</sup><∼−<sup>1</sup> (W *.*p, RcatA(W )*.*p) *for all* <sup>p</sup> <sup>∈</sup> <sup>P</sup> **<sup>5</sup> repeat** U := RcatA(U ) **until** <sup>B</sup><∼(U *.*p, RcatA(U )*.*p) *for all* <sup>p</sup> <sup>∈</sup> <sup>P</sup> **<sup>6</sup> for each** s ∈ F<sup>A</sup> **do <sup>7</sup> let** V <sup>s</sup> <sup>1</sup> ∈ ℘(Σ∗) <sup>|</sup><sup>P</sup> <sup>|</sup> **as** V <sup>s</sup> <sup>1</sup> *.*<sup>p</sup> - {a ∈ Σ | A: s <sup>a</sup> −−→ <sup>p</sup>} **with** <sup>p</sup> <sup>∈</sup> <sup>P</sup> **<sup>8</sup> for each** <sup>w</sup> <sup>∈</sup> W *.*<sup>s</sup> **do <sup>9</sup>** V <sup>s</sup> := V <sup>s</sup> 1 **<sup>10</sup> repeat** V <sup>s</sup> := RcatA(V <sup>s</sup>) **until** <sup>B</sup>∼<sup>w</sup> (V <sup>s</sup>*.*p, RcatA(V <sup>s</sup>)*.*p) *for all* <sup>p</sup> <sup>∈</sup> <sup>P</sup> **<sup>11</sup> for each** <sup>v</sup> <sup>∈</sup> V <sup>s</sup>*.*<sup>s</sup> **do <sup>12</sup> for each** <sup>u</sup> <sup>∈</sup> U *.*<sup>s</sup> **such that** <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup> **do <sup>13</sup> if** uv<sup>ω</sup> <sup>∈</sup>/ <sup>M</sup> **then return** ko **14 return** ok

**Fig. 2.** FORQ-based algorithm

**Lemma 8.** *Consider* U<sup>0</sup> *and* V <sup>s</sup> <sup>1</sup> *(with* s ∈ FA*) in the FORQ-based algorithm. The following holds for all* <sup>n</sup> <sup>∈</sup> <sup>N</sup>*:*

Rcat<sup>n</sup> <sup>A</sup>(U0)*.*<sup>p</sup> <sup>=</sup> Stem<sup>p</sup>∩Σ≤<sup>n</sup> *for all* <sup>p</sup> <sup>∈</sup> <sup>P</sup>*, and* Rcat<sup>n</sup> A(V <sup>s</sup> <sup>1</sup> )*.*<sup>s</sup> <sup>=</sup> Per<sup>s</sup>∩Σ≤n+1 .

Prior to proving the correctness of the algorithm of Fig. 2 we need the following result which is key for establishing the correctness of the repeat/until loop conditions of lines 4, 5, and 10.

**Lemma 9.** *Let be a right-monotonic quasiorder over* Σ∗*. Given* A - (P, p<sup>I</sup> , ΔA, FA) *and* S, S <sup>∈</sup> <sup>℘</sup>(Σ∗)|<sup>P</sup> <sup>|</sup> *, if* B-(S *.*p, S*.*p) *holds for all* <sup>p</sup> <sup>∈</sup> <sup>P</sup> *then* B-(RcatA(S )*.*p, Rcat<sup>A</sup>(S)*.*p) *holds for all* <sup>p</sup> <sup>∈</sup> <sup>P</sup>*.*

*Proof.* Consider <sup>w</sup> <sup>∈</sup> Rcat<sup>A</sup>(S).<sup>p</sup> where <sup>p</sup> <sup>∈</sup> <sup>P</sup>, we show that there exists <sup>w</sup> <sup>∈</sup> RcatA(S ).<sup>p</sup> such that <sup>w</sup> w. Assume that B-(S .p, S.p) holds for all <sup>p</sup> <sup>∈</sup> <sup>P</sup>. In particular, for all <sup>w</sup><sup>1</sup> <sup>∈</sup> S.p, there exists <sup>w</sup> <sup>1</sup> <sup>∈</sup> S .<sup>p</sup> such that <sup>w</sup> <sup>1</sup> w1. In the case where <sup>w</sup><sup>1</sup> <sup>∈</sup> Rcat<sup>A</sup>(S).<sup>p</sup> \ S.p, by definition of Rcat<sup>A</sup> <sup>w</sup><sup>1</sup> is of the form <sup>w</sup>2<sup>a</sup> for some <sup>a</sup> <sup>∈</sup> <sup>Σ</sup> and some <sup>w</sup><sup>2</sup> <sup>∈</sup> S.p<sup>ˆ</sup> such that <sup>A</sup>: ˆ<sup>p</sup> <sup>a</sup>−−→ <sup>p</sup>. Since <sup>B</sup>-(S .p, <sup>ˆ</sup> S.pˆ) and <sup>w</sup><sup>2</sup> <sup>∈</sup> S.pˆ, there exists <sup>w</sup><sup>3</sup> <sup>∈</sup> S .p<sup>ˆ</sup> such that <sup>w</sup><sup>3</sup> w2. We deduce that w3aw2a holds, hence w3aw<sup>1</sup> holds as well from the right-monotonicity of -. Furthermore <sup>w</sup>3<sup>a</sup> <sup>∈</sup> Rcat<sup>A</sup>(S ).<sup>p</sup> by definition of Rcat<sup>A</sup> and since <sup>A</sup>: ˆ<sup>p</sup> <sup>a</sup>−−→ <sup>p</sup>. Finally, we conclude that B-(RcatA(S ), Rcat<sup>A</sup>(S)) holds.

#### **Theorem 10.** *The FORQ-based algorithm decides the inclusion of BAs.*

*Proof.* We first show that every loop of the algorithm eventually terminates. First, we conclude from the definition of Rcat<sup>A</sup> and the initializations (lines <sup>3</sup> and 9) of each repeat/until loop (lines 4, 5, and 10) that each component of each vector holds a finite set of words. Observe that the halting conditions of the repeat/until loops are effectively computable since every quasiorder of F is decidable and because, in order to decide B-(Y,X) where X, Y are finite sets and is decidable, it suffices to check that Y ⊆ X and that for every x ∈ X there exists <sup>y</sup> <sup>∈</sup> <sup>Y</sup> such that <sup>y</sup> x. Next, we conclude from the fact that all the quasiorders of F used in the repeat/until loops are all well-quasiorders that there is no infinite sequence {Xi}<sup>i</sup>∈<sup>N</sup> such that -X<sup>1</sup> -X<sup>2</sup> ... Since B-(Y,X) is equivalent to Y ⊆ X ∧ -X ⊆ -<sup>Y</sup> and since each time Rcat<sup>A</sup> updates a component its upward closure after the update includes the one before, we find that every repeat/until loop must terminate after finitely many iterations.

Next, we show that when the repeat/until loop of line 5 halts, B< ∼(U .p, Stem<sup>p</sup>) holds for all <sup>p</sup> <sup>∈</sup> <sup>P</sup>. It takes an easy induction on <sup>n</sup> together with Lemma 9 to show that if B< ∼(Rcat<sup>n</sup>+1 <sup>A</sup> (U0).p, Rcat<sup>n</sup> <sup>A</sup>(U0).p) holds for all <sup>p</sup> <sup>∈</sup> <sup>P</sup> then B< ∼(Rcat<sup>n</sup> <sup>A</sup>(U0).p, Rcat<sup>m</sup> <sup>A</sup> (U0).p) holds for all m>n. Hence Lemma <sup>8</sup> shows that B< ∼(Rcat<sup>k</sup> <sup>A</sup>(U0).p, Stem<sup>p</sup>) holds for all <sup>p</sup> <sup>∈</sup> <sup>P</sup> where <sup>k</sup> is the number of iterations of the repeat/until loop implying B< ∼(U .p, Stem<sup>p</sup>) holds when the loop of line 5 halts.

An analogue reasoning shows that B< ∼−<sup>1</sup> (W .p, Stem<sup>p</sup>) holds for all <sup>p</sup> <sup>∈</sup> <sup>P</sup>, as well as <sup>B</sup>∼<sup>w</sup> (V <sup>s</sup>.s, Per<sup>s</sup>) holds for all <sup>w</sup> <sup>∈</sup> W .<sup>s</sup> and all <sup>s</sup> <sup>∈</sup> <sup>F</sup><sup>A</sup> upon termination of the loops of lines 4 and 10.

To conclude, we observe that each time a membership query is performed at line 13, the ultimately periodic word uv<sup>ω</sup> belongs to <sup>T</sup><sup>A</sup> defined by (‡). This is ensured since u ∈ B<sup>&</sup>lt; ∼(U .s, Stem<sup>s</sup>), <sup>w</sup> <sup>∈</sup> <sup>B</sup><sup>&</sup>lt; ∼−<sup>1</sup> (W .s, Stem<sup>s</sup>), <sup>v</sup> <sup>∈</sup> <sup>B</sup>∼<sup>w</sup> (V <sup>s</sup>.s, Per<sup>s</sup>) for some <sup>s</sup> <sup>∈</sup> <sup>F</sup><sup>A</sup> and, thanks to the test at line 12, the comparison <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup> holds.

*Remark 11.* The correctness of the FORQ-based algorithm still holds when, after every ":=" assignment (at lines 3, 4, 5, 9 and 10), we remove from the variable content zero or more subsumed words for the corresponding ordering. The effect of removing zero or more subsumed words from a variable can be achieved by replacing assignments like, for instance, U := Rcat<sup>A</sup>(U ) at line <sup>5</sup> with U := Rcat<sup>A</sup>(U );U := U<sup>r</sup> where U<sup>r</sup> satisfies <sup>B</sup><sup>&</sup>lt; ∼(Ur.p,U .p) for all <sup>p</sup> <sup>∈</sup> <sup>P</sup>. The correctness of the previous modification follows from Lemma 9. Therefore, the sets obtained by discarding subsumed words during computations still satisfy the basis predicates of T<sup>A</sup> given at (‡).

It is worth pointing out that the correctness arguments developed above, do not depend on the specifics of the structural FORQs. The FORQ-based algorithm is sound as long as we provide a suitable FORQ. Next we study the algorithmic complexity of the algorithm of Fig. 2.

#### **6 Complexity of the Structural FORQ-Based Algorithm**

In this Section, we establish an upper bound on the runtime of the algorithm of Fig. 2 when the input FORQ is the structural FORQ induced by a BA B. Let n<sup>A</sup> and n<sup>B</sup> be respectively the number of states in the BA A and B. We start by bounding the number of iterations in the repeat/until loops. In each repeat/until loop, each component of the vector holds a finite set of words the upward closure of which grows (for ⊆) over time and when all the upward closures stabilize the loop terminates. In the worst case, an iteration of the repeat/until loop adds exactly one word to some component of the vector which keeps the halting condition falsified (the upward closure strictly increases). Therefore a component of the vector cannot be updated more than 2<sup>n</sup><sup>B</sup> times for otherwise its upward closure has stabilized. We thus find that the total number of iterations is bounded from above by <sup>n</sup><sup>A</sup> · <sup>2</sup><sup>n</sup><sup>B</sup> for the loops computing U and W . Using an analogous reasoning we conclude that each component of the V vector has no more than 2(2nB2) elements and the total number of iterations is upper-bounded by <sup>n</sup><sup>A</sup> · <sup>2</sup>(2nB2). To infer an upper bound on the runtime of each repeat/until loop we also need to multiply the above expressions by a factor |Σ| since the number of concatenations in Rcat depends on the size of the alphabet.

Next, we derive an upper bound on the number of membership queries performed at line 13. The number of iterations of the loops of lines 6, 8, 10, 11 and <sup>12</sup> is <sup>n</sup>A, 2<sup>n</sup><sup>B</sup> , <sup>n</sup><sup>A</sup> · <sup>2</sup>(2nB2) , 2(2nB2) and 2<sup>n</sup><sup>B</sup> , respectively. Since all loops are nested, we multiply these bounds to end up with n<sup>2</sup> <sup>A</sup> · <sup>2</sup>O(n<sup>2</sup> <sup>B</sup>) as an upper bound on the number of membership queries. The runtime for each ultimately periodic word membership query (with a stem, a period and B as input) is upper bounded by an expression polynomial in the size <sup>n</sup><sup>B</sup> of <sup>B</sup>, 2<sup>n</sup><sup>B</sup> for the length of the stem and 2(2nB2) for the length of the period.

We conclude from the above that the runtime of the algorithm of Fig. 2 is at most <sup>|</sup>Σ| · <sup>n</sup><sup>2</sup> <sup>A</sup> · <sup>2</sup>O(n<sup>2</sup> <sup>B</sup>).

#### **7 Implementation and Experiments**

We implemented the FORQ-based algorithm of Fig. 2 instantiated by the structural FORQ in a tool called Forklift [2]. In this section, we provide algorithmic details about Forklift and then analyze how it behaves in practice (Sect. 7.1). **Data Structures.** Comparing two words given a structural FORQ requires to compute the corresponding sets of target for stems (Tgt), and sets of context for periods (Cxt). A na¨ıve implementation would be to compute Tgt and Cxt every time a comparison is needed. We avoid to compute this information over and over again by storing each stem together with its Tgt set and each period together with its Cxt set.

Moreover, the function Rcat inserts new words in the input vector by concatenating a letter on the right to some words already in the vector. In our implementation, we do not recompute the associated set of targets nor context for the newly computed word from scratch. For all stem u ∈ Σ<sup>∗</sup> and all letter <sup>a</sup> <sup>∈</sup> <sup>Σ</sup>, the set of states Tgt(ua) can be computed from Tgt(u) thanks to the following equality essentially stating that Tgt() can be computed inductively:

Tgt(ua) = - <sup>q</sup> <sup>∈</sup> <sup>Q</sup> <sup>|</sup> <sup>q</sup> <sup>∈</sup> Tgt(u), <sup>B</sup>: <sup>q</sup> <sup>a</sup>−−→ <sup>q</sup> .

Analogously, for all period <sup>v</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup>, all <sup>X</sup> <sup>⊆</sup> <sup>Q</sup> and all <sup>a</sup> <sup>∈</sup> <sup>Σ</sup>, the set of contexts Cxt(X, va) can be computed from Cxt(X, v) thanks to the following equality:

$$\mathsf{Cxt}(X,va) = \left\{ (q\_0, q, k) \in Q^2 \times \{ \perp, \top \} \; \middle| \; \begin{array}{l} (q\_0, q', k') \in \mathsf{Cxt}(X, v), \mathcal{B} \colon q' \\underline{\,} \,\, q \\\ (k = \bot \lor k' = \top \lor \mathcal{B} \colon q' \,\, \underline{\,}\_F q) \end{array} \right\}.$$

Intuitively Cxt can be computed inductively as we did for Tgt. The first part of the condition defines how new context are obtained by appending a transition to the right of an existing context while the second part defines the bit of information keeping record of whether an accepting state was visited.

**Bases, Frontier and Membership Test.** We stated in Remark 11 that the correctness of the FORQ-based algorithm is preserved when removing, from the computed sets, zero or more subsumed words for the corresponding ordering. In Forklift, we remove all the subsumed words from all the sets we compute which, intuitively, means each computed set is a basis that contains as few words as possible. To remove subsumed words we leverage the target or context sets kept along with the words. It is worth pointing out that the least fixpoint computations at lines 4, 5, and 10 are implemented using a frontier. Finally, the ultimately periodic word membership procedure is implemented as a classical depth-first search as described in textbooks [17, Chapter 13.1.1].

**Technical Details.** Forklift, a na¨ıve prototype implemented by a single person over several weeks, implements the algorithm of Fig. 2 with the structural FORQ in less than 1 000 lines of Java code. One of the design goals of our tool was to have simple code that could be easily integrated in other tools. Therefore, our implementation relies solely on a few standard packages from the Java SE Platform (notably collections such as HashSet or HashMap).

#### **7.1 Experimental Evaluation**

**Benchmarks.** Our evaluation uses benchmarks stemming from various application domains including benchmarks from theorem proving, software verification, and from previous work on the ω-regular language inclusion problem. In this section, a *benchmark* means an ordered pair of BAs such that the "left"/"right" BAs refer, resp., to the automata on the left/right of the inclusion sign. The BAs of the Pecan [31] benchmarks encode sets of solutions of predicates, hence a logical implication between predicates reduces to a language inclusion problem between BAs. The benchmarks correspond to theorems of type ∀x, ∃y, P(x) =⇒ Q(y) about Sturmian words [21]. We collected 60 benchmarks from Pecan for which inclusion holds, where the BAs have alphabets of up to 256 symbols and have up to 21 395 states.

The second collection of benchmarks stems from software verification. The Ultimate Automizer (UA) [19,20] benchmarks encode termination problems for programs where the left BA models a program and the right BA its termination proof. Overall, we collected 600 benchmarks from UA for which inclusion holds for all but one benchmark. The BAs have alphabets of up to 13 173 symbols and are as large as 6 972 states.

The RABIT benchmarks are BAs modeling mutual exclusion algorithms [8], where in each benchmark one BA is the result of translating a set of guarded commands defining the protocol while the other BA translates a modified set of guarded commands, typically obtained by randomly weakening or strengthening one guard. The resulting BAs are on a binary alphabet and are as large as 7 963 states. Inclusion holds for 9 out of the 14 benchmarks.

All the benchmarks are publicly available on GitHub [12]. We used all the benchmarks we collected, that is, we discarded no benchmarks.

**Tools.** We compared Forklift with the following tools: SPOT 2.10.3, GOAL (20200822), RABIT 2.5.0, ROLL 1.0, and BAIT 0.1.


techniques to decide inclusion. ROLL is written in Java and is publicly available [5].

**BAIT** [13] which shares authors with the authors of the present paper, implements a Ramsey-based algorithm with the antichain heuristics where two quasiorders (one for the stems and the other for the periods) are used to discard candidate counterexamples as described in Sect. 1. BAIT is written in Java and is publicly available [1].

As far as we can tell all the above implementations, including Forklift, are sequential except for RABIT which, using the -jf option, performs some computations in a separate thread.

**Experimental Setup.** We ran our experiments on a server with 24 GB of RAM, 2 Xeon E5640 2.6 GHz CPUs and Debian Stretch 64-bit. We used openJDK 11.0.12 2021-07-20 when compiling Java code and ran the JVM with default options. For RABIT, BAIT and Forklift the execution time is computed using timers internal to their implementations. For ROLL, GOAL and SPOT the execution time is given by the "real" value of the time(1) command. We preprocessed the benchmarks passed to Forklift and BAIT with a reduction of the set of final states of the "left" BA that does not alter the language it recognizes. This preprocessing aims to minimize the number of iterations of the loop at line 6 of Fig. 2 over the set of final states. It is carried out by GOAL using the acc -min command. Internally, GOAL uses a polynomial time algorithm that relies on computing strongly connected components. The time taken by this preprocessing is negligible.

**Plots.** We use survival plots for displaying our experimental results in Fig. 3. Let us recall how to obtain them for a family of benchmarks {pi}<sup>n</sup> <sup>i</sup>=1: (1) run the tool on each benchmark p<sup>i</sup> and store its runtime ti; (2) sort the ti's in increasing order and discard pairs corresponding to abnormal program termination like time out or memory out; (3) plot the points (t1, 1),(t<sup>1</sup> +t2, 2),. . . , and in general ( <sup>k</sup> <sup>i</sup>=1 ti, k); (4) repeat for each tool under evaluation.

Survival plots are effective at comparing how tools scale up on benchmarks: the further right and the flatter a plot goes, the better the tool thereof scales up. Also the closer to the x-axis a plot is, the less time the tool needs to solve the benchmarks.

**Analysis.** It is clear from Fig. 3a and 3b that Forklift scales up best on both the Pecan and UA benchmarks. Forklift's scalability is particularly evident on the PECAN benchmarks of Fig. 3a where its curve is the flattest and no other tool finishes on all benchmarks. Note that, in Fig. 3b, the plot for SPOT is missing because we did not succeed into translating the UA benchmarks in the input format of SPOT. On the UA benchmarks, Forklift, BAIT and GOAL scale up well and we expect SPOT to scale up at least equally well. On the other hand, RABIT and ROLL scaled up poorly on these benchmarks.

On the RABIT benchmarks at Fig. 3c both Forklift and SPOT terminate 13 out of 14 times; BAIT terminates 9 out of 14 times; and GOAL, ROLL and

**Fig. 3.** Survival plot with a logarithmic y axis and linear x axis. Each benchmark has a timeout value of 12 h. Parts of the plots left out for clarity. A point is plotted for abscissa value x and tool r iff r returns with an answer for x benchmarks. All the failures of BAIT and the one of Forklift are memory out.

RABIT terminate all the times. We claim that the RABIT benchmarks can all be solved efficiently by leveraging simulation relations which Forklift does not use let alone compute. Next, we justify this claim. First observe at Fig. 3c how GOAL is doing noticeably better than GOAL<sup>−</sup> while we have the opposite situation for the Pecan benchmarks Fig. 3a and no noticeable difference for the UA benchmarks Fig. 3b. Furthermore observe how ROLL and RABIT, which both leverage simulation relations in one way or another, scale up well on the RABIT benchmarks but scale up poorly on the PECAN and UA benchmarks.

The reduced RABIT benchmarks at Fig. 3d are obtained by pre-processing every BA of every RABIT benchmark with the simulation-based reduction operation of SPOT given by autfilt --high --ba. This preprocessing reduces the state space of the BAs by more than 90% in some cases. The reduction significantly improves how Forklift scales up (it now terminates on all benchmarks) while it has less impact on RABIT, ROLL and SPOT which, as we said above, already leverage simulation relation internally. It is also worth noting that GOAL has a regression (from 14/14 before the reduction to 13/14).

Overall Forklift, even though it is a prototype implementation, is the tool that returns most often (673/674). Its unique failure disappears after a preprocessing using simulation relations of the two BAs. The Forklift curve for the Pecan benchmarks shows Forklift scales up best.

Our conclusion from the empirical evaluation is that, in practice Forklift is competitive compared to the state-of-the-art in terms of scalability. Moreover the behavior of the FORQ-based algorithm in practice is far from its worst case exponential runtime.

#### **8 Discussions**

This section provides information that we consider of interest although not essential for the correctness of our algorithm or its evaluation.

**Origin of FORQs.** Our definition of FORQ and their suitability property (in particular the language preservation) are directly inspired from the definitions related to families of right congruences introduced by Maler and Staiger in 1993 [28] (revised in 2008 [30]). We now explain how our definition of FORQs generalizes and relaxes previous definitions [30, Definitions 5 and 6].

First we explain why the FORQ constraint does not appear in the setting of families of right congruences. In the context of congruences, relations are symmetric and thus, the FORQ constraint reduces to <sup>u</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup> <sup>⇒</sup> ∼<sup>u</sup> <sup>=</sup> ∼u. Therefore the FORQ constraint trivially holds if the set {∼u}<sup>u</sup>∈Σ<sup>∗</sup> is quotiented by the congruence relation <sup>&</sup>lt;∼, which is the case in the definition [29, Definition 5].

Second, we point that the condition <sup>v</sup> ∼<sup>u</sup> <sup>v</sup> <sup>⇒</sup> uv <sup>&</sup>lt;<sup>∼</sup> uv which appears in the definition for right families of congruences [30, Definition 5] is not needed in our setting. Nevertheless, this condition enables an improvement of the FORQ-based algorithm that we describe next.

**Less Membership Queries.** We put forward a property of structural FORQs allowing us to reduce the number of membership queries performed by Forklift. Hereafter, we refer to the *picky constraint* as the property of a FORQ stating <sup>v</sup> ∼<sup>u</sup> <sup>v</sup> <sup>⇒</sup> uv <sup>&</sup>lt;<sup>∼</sup> uv where u, v, v <sup>∈</sup> <sup>Σ</sup>∗. We first show how thanks to the picky constraint we can reduce the number of candidate counterexamples in the FORQ-based algorithm and then, we show that every structural FORQ satisfies the picky constraint.

In the algorithm of Fig. 2, periods are taken in a basis for the ordering ∼<sup>w</sup> where <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> belongs to a finite basis for the ordering <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup>. The only restriction on w is that of being comparable to the stem u, as ensured by the test at line 12. The following lemma formalizes the fact that we could consider a stronger restriction.

**Lemma 12.** *Let* <sup>&</sup>lt;<sup>∼</sup> *be a quasiorder over* <sup>Σ</sup><sup>∗</sup> *such that* <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup> *is a right-monotonic well-quasiorder. Let* S, S ⊆ Σ<sup>∗</sup> *be such that* B<sup>&</sup>lt; ∼−<sup>1</sup> (S , S) *and* S *contains no two distinct comparable words. For all* <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> *and* <sup>v</sup> <sup>∈</sup> <sup>Σ</sup><sup>+</sup> *such that* <sup>u</sup> <sup>∈</sup> <sup>S</sup> *and* {wv <sup>|</sup> <sup>w</sup> <sup>∈</sup> <sup>S</sup>} ⊆ <sup>S</sup>*, there exists* <sup>w</sup>˚ <sup>∈</sup> <sup>S</sup> *such that* uv<sup>i</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup>˚ *and* wv ˚ <sup>j</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup>˚ *for some* i, j <sup>∈</sup> <sup>N</sup> \ {0}*.*

As in Sect. 3, we show that the equivalence (†) holds but this time for an alternative definition of T<sup>A</sup> we provide next. Given a M-suitable FORQ F - <sup>&</sup>lt;∼, {∼u}<sup>u</sup>∈Σ<sup>∗</sup> , let

$$\hat{T}\_{\mathcal{A}} \triangleq \{ uv^{\omega} \mid \exists s \in F\_{\mathcal{A}} \colon u \in U\_s, v \in V\_s^w \text{ for some } w \in W\_s \text{ with } u \lesssim\_{\sim} w, wv \lesssim\_{\sim} w \}$$

where for all <sup>p</sup> <sup>∈</sup> <sup>P</sup> the sets <sup>U</sup>p, <sup>W</sup><sup>p</sup> and {<sup>V</sup> <sup>w</sup> <sup>p</sup> }<sup>w</sup>∈Σ<sup>∗</sup> such that B<sup>&</sup>lt; ∼(Up, Stem<sup>p</sup>), B< <sup>∼</sup>−<sup>1</sup> (Wp, Stem<sup>p</sup>) and <sup>B</sup>∼<sup>w</sup> (<sup>V</sup> <sup>w</sup> <sup>p</sup> , Per<sup>p</sup>) for all <sup>w</sup> <sup>∈</sup> <sup>Σ</sup>∗. Since <sup>T</sup><sup>ˆ</sup> <sup>A</sup> ⊆ T<sup>A</sup> by definition, it suffices to prove the implication Tˆ <sup>A</sup> <sup>⊆</sup> <sup>M</sup> <sup>⇒</sup> Ultim<sup>A</sup> <sup>⊆</sup> <sup>M</sup>. Let uv<sup>ω</sup> <sup>∈</sup> Ultim<sup>A</sup>, i.e., such that there exists <sup>s</sup> <sup>∈</sup> <sup>F</sup><sup>A</sup> for which <sup>u</sup> <sup>∈</sup> Stem<sup>s</sup> and <sup>v</sup> <sup>∈</sup> Per<sup>s</sup>, satisfying uv <sup>&</sup>lt;<sup>∼</sup> <sup>u</sup>. In the context of Lemma 12, taking <sup>S</sup> - Stem<sup>s</sup> and S - W<sup>s</sup> fulfills the requirements u ∈ S and {wv | w ∈ S} ⊆ S. We can thus apply the lemma and ensure the existence of some <sup>w</sup><sup>0</sup> <sup>∈</sup> <sup>W</sup><sup>s</sup> satisfying uv<sup>i</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup><sup>0</sup> and <sup>w</sup>0v<sup>j</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup><sup>0</sup> for some i, j <sup>∈</sup> <sup>N</sup> \ {0}. Since uv<sup>i</sup> <sup>∈</sup> Stem<sup>s</sup> and <sup>v</sup><sup>j</sup> <sup>∈</sup> Per<sup>s</sup> we find that there exist <sup>u</sup><sup>0</sup> <sup>∈</sup> <sup>U</sup><sup>s</sup> and <sup>v</sup><sup>0</sup> <sup>∈</sup> <sup>V</sup> <sup>w</sup><sup>0</sup> <sup>s</sup> such that <sup>u</sup><sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> uv<sup>i</sup> and <sup>v</sup><sup>0</sup> ∼<sup>w</sup><sup>0</sup> <sup>v</sup><sup>j</sup> thanks to the finite basis property. We conclude from above that <sup>v</sup><sup>0</sup> ∼<sup>w</sup><sup>0</sup> <sup>v</sup><sup>j</sup> , hence that <sup>w</sup>0v<sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup>0v<sup>j</sup> by the picky condition, and finally that <sup>w</sup>0v<sup>0</sup> <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup><sup>0</sup> by Lemma <sup>12</sup> and transitivity. By definition u0v<sup>ω</sup> <sup>0</sup> <sup>∈</sup> <sup>T</sup><sup>ˆ</sup> <sup>A</sup> and the proof continues as the one in Sect. 3 for TA.

To summarize, if the considered FORQ fulfills the picky constraint then the algorithm of Fig. 2 remains correct when discarding the periods v at line 11 such that wv <sup>&</sup>lt;<sup>∼</sup> <sup>w</sup>. Observe that discarding one period <sup>v</sup> possibly means skipping several membership queries (u1v<sup>ω</sup>, u2v<sup>ω</sup>,...). As proved below, the picky constraint holds for all structural FORQs.

**Lemma 13.** *Let* B - (Q, q<sup>I</sup> , ΔB, FB) *be a BA and* F - <sup>&</sup>lt;<sup>∼</sup> <sup>B</sup>, {∼<sup>B</sup> <sup>u</sup> }<sup>u</sup>∈Σ<sup>∗</sup> *its structural FORQ. For all* <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> *and all* v, v <sup>∈</sup> <sup>Σ</sup><sup>+</sup> *if* <sup>v</sup> ∼<sup>B</sup> <sup>u</sup> <sup>v</sup> *then* uv <sup>&</sup>lt;<sup>∼</sup> <sup>B</sup> uv *.*

*Proof.* For all <sup>q</sup> <sup>∈</sup> Tgt(uv), there exists <sup>q</sup> <sup>∈</sup> <sup>Q</sup> such that <sup>B</sup>: <sup>q</sup><sup>I</sup> <sup>u</sup>−−→ <sup>q</sup> <sup>v</sup>−−→ <sup>q</sup> . Hence (q, q , <sup>⊥</sup>) <sup>∈</sup> Cxt(Tgt(u), v). In fact (q, q , <sup>⊥</sup>) <sup>∈</sup> Cxt(Tgt(u), v ) holds as well since <sup>v</sup> ∼<sup>B</sup> <sup>u</sup> v . We deduce from the definition of Cxt that <sup>B</sup>: <sup>q</sup><sup>I</sup> <sup>u</sup>−−→ <sup>q</sup> <sup>v</sup> −−→ <sup>q</sup> which implies <sup>q</sup> <sup>∈</sup> Tgt(uv ). Thus Tgt(uv) <sup>⊆</sup> Tgt(uv ), i.e., uv <sup>&</sup>lt;<sup>∼</sup> B uv .

We emphasize that this reduction of the number of membership queries was not included in our experimental evaluation since (1) the proof of correctness is simpler and (2) Forklift already scales up well without this optimization. We leave for future work the precise effect of such optimization.

**Why a Basis for** <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup> **is Computed?** Taking periods in a basis for the ordering ∼<sup>w</sup> where <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> is picked in a basis for the ordering <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup> may seem unnatural. In fact, the language preservation property of FORQs even suggests that an algorithm without computing a basis for <sup>&</sup>lt;<sup>∼</sup> <sup>−</sup><sup>1</sup> may exist. Here, we show that taking periods in a basis for the ordering ∼<sup>u</sup> where <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> is picked in a basis for the ordering <sup>&</sup>lt;<sup>∼</sup> is not correct. More precisely, redefining <sup>T</sup><sup>A</sup> as

$$\tilde{T}\_{\mathcal{A}} \triangleq \{ uv^{\omega} \mid \exists s \in F\_{\mathcal{A}} \colon u \in U\_s, v \in V\_s^u \}$$

where for all p ∈ P we have that B<sup>&</sup>lt; <sup>∼</sup>(Up, Stem<sup>p</sup>) and <sup>B</sup>∼<sup>w</sup> (<sup>V</sup> <sup>w</sup> <sup>p</sup> , Per<sup>p</sup>) for all w ∈ Σ∗, leads to an *incorrect* algorithm because the equivalence (†) given by T˜ <sup>A</sup> ⊆ M ⇐⇒ L(A) ⊆ M no longer holds as shown below in Example 14.

*Example 14.* Consider the BAs given by Fig. 1. We have that <sup>L</sup>(A) <sup>L</sup>(B) and, in Example 3, we have argued that <sup>T</sup><sup>A</sup> <sup>=</sup> {ε(b)<sup>ω</sup>, a(a)<sup>ω</sup>} contains the ultimately periodic a<sup>ω</sup> which is a counterexample to inclusion. Recall from Example 3 and 6 that we can set U<sup>p</sup><sup>I</sup> = {ε, a} since B<sup>&</sup>lt; ∼({ε, a}, Σ∗), and <sup>V</sup> <sup>a</sup> <sup>p</sup><sup>I</sup> = V <sup>ε</sup> <sup>p</sup><sup>I</sup> = {b} since <sup>B</sup>∼<sup>a</sup> ({b}, Σ<sup>+</sup>) and <sup>B</sup>∼<sup>ε</sup> ({b}, Σ<sup>+</sup>). We conclude from the above definition that T˜ <sup>A</sup> <sup>=</sup> {ε(b)<sup>ω</sup>, a(b)<sup>ω</sup>}, hence that <sup>T</sup>˜ <sup>A</sup> ⊆ L(B) which contradicts (†) since <sup>L</sup>(A) <sup>L</sup>(B).

#### **9 Conclusion and Future Work**

We presented a novel approach to tackle in practice the language inclusion problem between B¨uchi automata. Our antichain heuristics is driven by the notion of FORQs that extends the notion of family of right congruences introduced in the nineties by Maler and Staiger [29]. We expect the notion of FORQs to have impact beyond the inclusion problem, e.g. in learning [9] and complementation [26]. A significant difference of our inclusion algorithm compared to other algorithms which rely on antichain heuristics, is the increased number of fixpoint computations that, counterintuitively, yield better scalability. Indeed our prototype Forklift, which implements the FORQ-based algorithm, scales up well on benchmarks taken from real applications in verification and theorem proving.

In the future we want to increase further the search pruning capabilities of FORQs by enhancing them with simulation relations. We also plan to study whether FORQs can be extended to other settings like ω-visibly pushdown languages.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## Sound Automation of Magic Wands

Thibault Dardinier1(B) , Gaurav Parthasarathy<sup>1</sup>, Noé Weeks<sup>2</sup>, Peter Müller<sup>1</sup> , and Alexander J. Summers<sup>3</sup>

<sup>1</sup> Department of Computer Science, ETH Zurich, Zurich, Switzerland {thibault.dardinier,gaurav.parthasarathy,peter.mueller}@inf.ethz.ch

<sup>2</sup> École Normale Supérieure, Paris, France

noe.weeks@ens.psl.eu <sup>3</sup> University of British Columbia, Vancouver, Canada alex.summers@ubc.ca

Abstract. The magic wand −∗ (also called separating implication) is a separation logic connective commonly used to specify properties of partial data structures, for instance during iterative traversals. A *footprint* of a magic wand formula A −∗ B is a state that, combined with any state in which A holds, yields a state in which B holds. The key challenge of proving a magic wand (also called *packaging* a wand) is to find such a footprint. Existing package algorithms either have a high annotation overhead or, as we show in this paper, are unsound.

We present a formal framework that precisely characterises a wide design space of possible package algorithms applicable to a large class of separation logics. We prove in Isabelle/HOL that our formal framework is sound and complete, and use it to develop a novel package algorithm that offers competitive automation and is sound. Moreover, we present a novel, restricted definition of wands and prove in Isabelle/HOL that it is possible to soundly combine fractions of such wands, which is not the case for arbitrary wands. We have implemented our techniques for the Viper language, and demonstrate that they are effective in practice.

#### 1 Introduction

Separation logic [38] (SL hereafter) is a program logic that has been widely used to prove complex properties of heap-manipulating programs. The two main logical connectives that enable such reasoning are the *separating conjunction* ∗ and the *separating implication* (more commonly known as the *magic wand*) −∗, in combination with *resource assertions* which represent e.g. exclusive ownership of (and permission to access) particular heap locations. The separating conjunction expresses that two assertions prescribe ownership of disjoint parts of the heap, useful, for instance, to reason about aliasing or race conditions. More precisely, the assertion A ∗ B holds in a program state σ if and only if σ can be split into two *compatible* program states σ*<sup>A</sup>* and σ*<sup>B</sup>* such that A and B hold in σ*<sup>A</sup>* and σ*B*, respectively. In SL, heaps of program states are *partial* maps from locations to values; their domains represent heap locations exclusively owned. Two program states are compatible if (the domains of) their heaps are disjoint.

Intuitively, a magic wand A−∗B can be used to express the difference between the heap locations that B and A provide permission to access. The magic wand is useful, for instance, to specify partial data structures, where B specifies the entire data structure and A specifies a part that is missing [33,41]. A −∗ B holds in a state σ*w*, if and only if for *any* program state σ*<sup>A</sup>* in which A holds and that is compatible with σ*w*, B holds in the state obtained by combining the heaps of σ*<sup>A</sup>* and σ*w*. Thus, if A ∗ (A −∗ B) holds in a state, then so does B, analogously to the *modus ponens* inference rule in propositional logic.

The magic wand has been shown to enable or greatly simplify proofs in many different cases [1,9,20,21,28,33,41,42]. For instance, Yang [42] uses the magic wand to prove the Schorr-Waite graph marking algorithm. Dodds *et al.* [20] employ the wand to specify synchronisation barriers for deterministic parallelism. Examples using magic wands to specify partial data structures include tracking ongoing traversals of a data structure [33,41], where the left-hand side of the wand specifies the part of the data structure yet to be traversed, or for specifying protocols that enforce orderly modification of data structures [21,25,28] (e.g. the protocol governing Java iterators). More recently, wands have been used for formal reasoning about borrowed references in the Rust programming language, which employs an ownership type system to ensure memory safety [1]. Magic wands concisely represent the *remainder* of a data structure from which a borrowed reference was taken, as well as reflecting back modifications to the part accessible via the reference. For example, consider a struct Point (represented by a SL predicate Point) with two fields x and y of type i32 (represented by the SL predicate i32). A Rust method that takes as input a Point p and returns a borrow of its field <sup>x</sup> is specified with the postcondition int32(x) ∗ (int32(x) −∗ Point(p)), thus enabling the caller to regain ownership of the entire data structure Point(p).

The complexity of SL proofs has given rise to a variety of automatic SL verifiers that reduce the required proof effort. Given the usefulness of magic wands, it is important that such verifiers also provide automatic support for wands. However, reasoning about a magic wand requires reasoning about *all* states in which the left-hand side holds, which is challenging. It has been shown that a separation logic even without the separating conjunction (but with the magic wand) is as expressive as a variant of second-order logic and, thus, undecidable [6].

Two different approaches [3,39] that provide partially-automated support are implemented in the verifiers Viper [34] and VerCors [2]. However, the approach implemented in VerCors [3] incurs significant annotation overhead, and the approach in Viper [39] suffers from a fundamental, previously undiscovered flaw that renders the approach unsound. Both approaches require user-provided *package operations* to direct the verifier's proof search. *Packaging* a wand A−∗B expresses that the verifier should prove and subsequently record A−∗B. To package A−∗B the verifier must split the current state into two compatible states σ and σ*<sup>w</sup>* such that A −∗ B holds in σ*w*. We call σ*<sup>w</sup>* a *footprint* of the wand. After successfully packaging a wand, the verifier must disallow changes to σ*<sup>w</sup>* to preserve the wand's validity: the verifier *packages the footprint into the wand*.

The key challenge for supporting magic wands in automatic verifiers is to define a *package algorithm* that packages a wand. In VerCors's package algorithm [3], a user must manually specify a footprint for the wand and the algorithm checks whether the wand holds in the specified footprint. This leads to a lot of annotation overhead. Viper's current package algorithm [39] reduces this overhead significantly by automatically inferring a suitable footprint. Unfortunately, as we show in this paper, Viper's current algorithm has a fundamental flaw that causes the algorithm to infer an *incorrect* footprint in certain cases, which may lead to unsound reasoning. We will explain the fundamental flaw in Sect. 2; it illustrates the subtlety of supporting this important connective.

Approach and Contributions. In this paper, we present a formal foundation for sound package algorithms, and we implement a novel such algorithm based on these foundations. Our algorithm requires the same annotation overhead as the prior, flawed Viper algorithm, which is (to our knowledge) the most automatic existing approach. We introduce a formal framework expressed via a novel *package logic* that defines the design space for package algorithms. The soundness of a package algorithm can be justified by showing that the algorithm finds a proof in our package logic. The design space for package algorithms is large since there are various aspects that affect how one expresses the algorithm including (1) which footprint an algorithm infers or checks (there are often multiple options, see Sect. 3), (2) the state model (which differs between different SL verifiers), and (3) restricted definitions of wands (for instance, to ensure each wand has a unique minimal footprint). Our package logic deals with (1) by capturing all sound derivations for the same wand. To deal with (2) and (3), our logic is parametric along multiple dimensions. For instance, the state model can be any separation algebra to support different SL extensions (e.g. fractional permissions [4]).

Our logic also supports parameters to restrict the allowed footprints for wands in systematic ways. Such restrictions are useful, for instance, in a logic supporting *fractional permissions*. Fractional permissions permit splitting ownership/resources into shared fragments which typically permit read access to the underlying data. However, as we show in Sect. 4, fractional parts of general magic wands cannot always be soundly recombined. Existing solutions for other connectives impose side conditions to enable sound recombinations [29], which are often hard to check automatically. We instead introduce a novel restriction of magic wands to avoid such side conditions and develop a corresponding second package algorithm again based on the formal framework provided by our package logic. We make the following contributions:


– We implement sound package algorithms for both the standard and the restricted wand in the Viper verifier and justify their soundness directly via our package logic (Sect. 5). We evaluate both algorithms on the Viper test suite. Our evaluation shows that (1) our algorithms perform similarly well to prior work and correctly reject examples where prior work is unsound, and (2) our restricted wand definition is expressive enough for most examples.

Our Isabelle formalisation and the implementation of our new package algorithm are publicly available [13–15]. Further details are available in our accompanying technical report (TR hereafter) [16].

#### 2 Background and Motivation

In this section, we present the necessary background for this paper. We use *implicit dynamic frames* [40] to represent SL assertions, since both existing automatic verifiers that support wands (VerCors and Viper) are based on it. There is a known strong correspondence between SL and implicit dynamic frames [36].

#### 2.1 Implicit Dynamic Frames

Just like SL assertions, implicit dynamic frames (IDF hereafter) assertions specify not only value information, but also *permissions* to heap locations that are allowed to be accessed. To justify dereferencing a heap location, the corresponding permission is required, ensuring memory safety. IDF assertions specify permissions to locations and value information separately. An assertion acc(x.val) (an *accessibility predicate*) denotes permission to the heap *location* x.val, while x.val = v expresses that x.val contains *value* v. The separating conjunction in IDF enforces disjointness (formally: acts multiplicatively) with respect to resource assertions such as accessibility predicates; in particular, if acc(x.val) ∗ acc(y.val) holds in a state, then <sup>x</sup> and <sup>y</sup> must be different (analogously to SL).

The main difference between IDF and SL is that SL does not allow general heap-dependent expressions such as x.val = v or x.left.right [40] to be specified separately from the permissions to the heap locations they depend on. The IDF assertion acc(x.val)∗x.val = v must be expressed in SL via the *points-to assertion* x.val → <sup>v</sup>, which also conveys exclusive permission to the location x.val. IDF supports heap dependent expressions within *self-framing* assertions: those which require permissions to all the heap locations on whose values they depend (e.g. acc(x.val) ∗ x.val = v is self-framing but x.val = v is not) [40].

#### 2.2 A Typical Example Using Magic Wands

Figure 1 shows a variation of an example from the VerifyThis competition [22]. The method leftLeaf iteratively computes the leftmost leaf of a binary tree (package and apply operations, shown in blue, should be ignored for now). The

Fig. 1. The code on the left finds the leftmost leaf of a binary tree and includes specifications to prove memory safety. The predicate describing the permissions of a tree is defined on the right. The loop invariant uses a wand to summarise the permissions of the input tree excluding the tree not yet traversed. The blue operations are ghost operations to guide the verifier; we omit those specific to predicates. The package requires further hints in existing approaches, see App. J of the TR [16]. (Color figure online)

pre- and postconditions of leftLeaf are both Tree(x), which is a *predicate instance* used to specify all permissions to the fields of the tree rooted at x (the recursive definition of this predicate is on the right of Fig. 1). Proving this specification amounts to proving that leftLeaf is memory-safe and that the permissions to the input tree are preserved, enabling further calls on the same tree.

The key challenge when verifying leftLeaf is specifying an appropriate loop invariant. The loop invariant must track the permissions to the subtree rooted at y that still needs to be traversed, since otherwise dereferencing y.left in the loop body is not allowed. Additionally, the invariant must track all of the remaining permissions in the input tree rooted at x (the permissions to the nodes already traversed and others unreachable from y), since otherwise the postcondition cannot be satisfied. The former can be easily expressed with Tree(y). The latter can be elegantly achieved with a magic wand Tree(y) −∗ Tree(x). This wand promises Tree(x) if one combines the wand with Tree(y). That is, the wand represents (at least) the difference between the permissions making up the two trees. Using SL's modus-ponens-like inference rule (directed by the apply operation on line 13, explained next), one can show that the loop invariant entails the postcondition.

#### 2.3 Wand Ghost Operations

Automatic SL verifiers such as GRASShopper [37], VeriFast [24], VerCors, and Viper generally represent permissions owned by a program state in two ways: by recording predicate instances (such as Tree(x) in Fig. 1) and *direct* permissions to heap locations. Magic wand instances provide a third way to represent permissions and are recorded analogously. Verifiers that support them require two

wand-specific *ghost operations*, which instruct the verifiers when to prove a wand and when to apply a recorded wand instance using SL's modus-ponens-like rule.

A package ghost operation expresses that a verifier should prove a new wand instance in the current state and report an error if the proof attempt fails. To prove a new wand instance, the verifier must split the current state into two states σ and σ*<sup>w</sup>* such that the wand holds in the *footprint* state σ*w*; on success, permissions in the footprint are effectively exchanged for the resulting magic wand instance. We call a procedure that selects a footprint by splitting the current state a *package algorithm*. On lines 5 and 10 of Fig. 1, new wands are packaged to establish and preserve the invariant, respectively.

The apply operation *applies* a wand A−∗B using SL's modus-ponens-like rule if the verifier records a wand instance of A −∗ B and A holds in the current state (and otherwise fails), exchanging these for the assertion B. The apply operation is directly justified by the wand's semantics: Combining a wand's footprint with *any* state in which A holds is guaranteed to yield a state in which B holds. For the apply operation on line 13 of Fig. 1, the verifier removes the applied wand instance and Tree(y), in exchange for the predicate instance Tree(x).

#### 2.4 The Footprint Inference Attempt (FIA)

Package algorithms differ in how a footprint for the specified magic wand is selected. In VerCors [3], the user must manually provide the footprint and the algorithm checks whether the specified footprint is correct. In Viper's current approach [39], a footprint is inferred. We explain and compare to the latter approach since it is the more automatic of the two; hereafter, we refer to its package algorithm as *the Footprint Inference Attempt* (*FIA*). Inferring a correct footprint is challenging due the complexity of the wand connective. In particular, we have discovered that, in certain cases, the FIA infers *incorrect* footprints, leading to unsound reasoning<sup>1</sup>. The goal of this subsection is to understand the FIA's key ideas, which our solution will build on, and why it is unsound.

In general, there may be multiple valid footprints for a magic wand A −∗ B. The FIA attempts to infer a footprint which is as close as possible to the *difference* between the permissions required by B and A, taking as few permissions as possible while aiming for a footprint compatible with A (so that the resulting wand can be later applied) [39]. That is, the FIA includes only permissions in the footprint it infers that are specified by B *and not* guaranteed by A.

For a wand A−∗B, the FIA constructs an arbitrary state σ*<sup>A</sup>* that satisfies A (representing σ*<sup>A</sup>* symbolically). Then, the FIA tries to construct a state σ*<sup>B</sup>* in which B holds by taking permissions (and copying corresponding heap values) from σ*<sup>A</sup>* if possible and the current state otherwise. If this algorithm succeeds, the (implicit) inferred footprint consists of the permissions that were taken from the current state. The FIA constructs σ*<sup>B</sup>* by iterating over the permissions and logical constraints in B. For each permission, the FIA checks whether σ*<sup>A</sup>* owns the permission. If so,

<sup>1</sup> This unsoundness might not be observable in restricted logics, but it is in Viper (see App. B of the TR [16]) and the rich logics supported by existing verification tools.

the FIA adds the permission to σ*<sup>B</sup>* and removes the permission from σ*A*. Otherwise, the FIA removes the permission from the current state or fails if the current state does not have the permission. For each logical constraint, the FIA checks that the constraint holds in σ*<sup>B</sup>* as constructed so far. We show an example of the FIA correctly packaging a wand in App. A of the TR [16].

Unsoundness of the FIA. We have discovered that for some wands A−∗B, the FIA determines an *incorrect* footprint for the magic wand. This unsoundness can arise when the FIA performs a case split on the content of the arbitrary state σ*<sup>A</sup>* satisfying A. In such situations, the FIA infers a footprint for each case *separately*, making use of properties that hold in that case. For certain wands, this leads to different footprints being selected for each case, while *none* of the inferred footprints can be used to justify B in *all* cases, i.e. for *all* states σ*<sup>A</sup>* that satisfy A. As a result, the packaged wand does *not* hold in any of the inferred footprints, which can make verification unsound, as we illustrate below.

The wand w := acc(x.f)∗(x.f = y∨x.f = z)−∗acc(x.f)∗acc(x.f.g) illustrates the problem. For this wand, every state σ*<sup>A</sup>* satisfying the left-hand side must have permission to x.f. However x.f may either point to <sup>y</sup> or <sup>z</sup>. If x.f points to <sup>y</sup> in σ*A*, then to justify the right-hand side's second conjunct, the footprint must contain permission to y.g. Analogously, if x.f points to <sup>z</sup> in σ*A*, then the footprint must contain permission to z.g. The wand's semantics requires a footprint to justify the wand's right-hand side for all states in which the left-hand side holds, and thus, a correct footprint must be able to justify *both* cases. Hence, the footprint must have permission to *both* y.g and z.g. However, the FIA's inferred footprint is in effect the disjunction of these two permissions.

Packaging the above wand w using the FIA leads to unsound reasoning. After the incorrect package described above in a state with permission to x.f, y.g, and z.g, the assertion acc(x.f)∗(acc(y.g)∨acc(z.g))∗w can be proved since the FIA removes permission to either y.g or z.g from the current state, but not both. However, this assertion does not actually hold! According to the semantics of wands, w's footprint must include permission to x.f or permission to both y.g and z.g, which implies that the assertion acc(x.f) ∗ (acc(y.g) ∨ acc(z.g)) ∗ w is equivalent to false.

The unsoundness of the FIA shows the subtlety and challenge of developing sound package algorithms. Algorithms that soundly infer a single footprint for all states in which the wand's left-hand side holds must be more involved than the FIA. Ensuring their soundness requires a *formal* framework to construct them and justify their correctness. We introduce such a framework in the next section.

#### 3 A Logical Framework for Packaging Wands

In this section, we present a new logical framework that defines the design space for (sound) package algorithms. The core of this framework is our *package logic*, which defines the space of potential algorithmic choices of a footprint for a particular magic wand. Successfully packaging a wand in a given state is (as we will show) equivalent to finding a derivation in our package logic, and any actual

package algorithm must correspond to a proof search in our logic (if it is sound). In particular, we provide soundness (Theorem 1) and completeness (Theorem 2) results for our logic. We define a specific package algorithm with this logic at its foundation, inspired by the FIA package algorithm [39] (described in Sect. 2.4) but amending its unsoundness, resulting in (to the best of our knowledge) the first sound and relatively automatic package algorithm.

All definitions and results in this section have been fully mechanised [13] in Isabelle/HOL. Our mechanised definitions are parametric with the underlying verification logic in various senses: the underlying separation algebra is a parameter, the syntax of assertions is defined in a way which allows simple extension with different base cases and connectives, and the semantics of magic wands itself can be restricted if only particular kinds of footprint are desired in practice. As a specific example of the latter parameter, in Sect. 4 we define a novel restriction of magic wand footprints which guarantees better properties in combination with certain usages of fractional permissions; this is seamlessly supported by the general package logic presented here. Nonetheless, to simplify the exposition of this section, we will assume that any magic wand footprint satisfying the connective's standard semantics is an acceptable result.

#### 3.1 Footprint Selection Strategies

As we explained in Sect. 1, there is a wide design space for package algorithms; in particular, many potential strategies for finding a magic wand's footprint exist and none is clearly optimal. Recall that a footprint is a state, and thus consists of permissions to certain heap locations as well as storing their corresponding values; for simplicity we identify a footprint by the permissions it contains.

For example, consider the following magic wand (using fractional permissions) acc(x.b, 1/2) −∗ acc(x.b, 1/2) ∗ (x.b ⇒ acc(x.f)). Suppose this magic wand is to be packaged in a state where full permissions to both x.b and x.f are held, and the value of x.b is currently false. Two valid potential footprints are:


There is no clear reason to prefer one choice over the other: different package algorithms (or manual choices) might choose either. Our package logic allows either choice along with any of many less optimal choices, such as taking both permissions. On the other hand, as motivated earlier in Sect. 3.1, our package logic must (and does) enforce that a single valid footprint is chosen for a wand that works for each and every potential state satisfying its left-hand side.

#### 3.2 Package Logic: Preliminaries

To capture different state models and flavours of separation logic, our package logic is parameterised by a separation algebra. For space reasons, we present here a simplified overview of this algebra, but all definitions (including our assertion semantics) are given in App. D of the TR [16] and have been mechanised. We consider a separation algebra [8,19] where Σ is the set of states, ⊕ : Σ ×Σ → Σ is a partial operation that is commutative and associative, and e ∈ Σ, which corresponds to the empty state, is a neutral element for ⊕. We write for the induced partial order of the resulting partial commutative monoid, and σ1#σ<sup>2</sup> iff σ1⊕σ<sup>2</sup> is defined (i.e. σ<sup>1</sup> and σ<sup>2</sup> are compatible). Finally, if σ<sup>2</sup> σ1, we define the subtraction σ<sup>2</sup> σ<sup>1</sup> to be the -largest state σ*<sup>r</sup>* such that σ<sup>2</sup> = σ<sup>1</sup> ⊕ σ*r*.

We define our package logic for an assertion language with the following grammar: A = A∗A | B⇒A | B, where A ranges over assertions and B over *semantic assertions*. To allow our package logic to be applied to a variety of underlying assertion logics, we distinguish only the two most-relevant connectives: the separating conjunction and an implication (for expressing conditional assertions). To support additional constructs of the assertion logic, the third type of assertion we consider is a *semantic assertion*, i.e. a function from Σ to Booleans. This third type can be instantiated to represent logical assertions that do not match the first two cases. In particular, assertions such as x.f = 5, acc(x.f), abstract predicates (such as Tree(x)) or magic wands can be represented as semantic assertions. This core assertion language can also be easily extended with native support for e.g. the logical conjunction and disjunction connectives; we explain in App. E of the TR [16] how to extend the rules of the logic accordingly.

#### 3.3 The Package Logic

We define our package logic to prescribe the design space of algorithms for deciding how, in an initial state σ0, to select a valid footprint (or fail) for a magic wand A−∗B. The aim is to infer states σ*<sup>w</sup>* and σ<sup>1</sup> that partition σ<sup>0</sup> (i.e. σ<sup>0</sup> = σ<sup>1</sup> ⊕σ*w*) such that σ*<sup>w</sup>* is a valid footprint for A−∗B (when combined with any compatible state satisfying A, the resulting state satisfies B). In particular, all permissions (and logical facts) required by the assertion B must either come from the footprint or be guaranteed to be provided by any compatible state satisfying A.

Recall from Sect. 2.4 that the mistake underlying the FIA approach ultimately resulted from allowing multiple different footprints to be selected conditionally on a state satisfying A, rather than a single footprint which works for all such states. Our package logic addresses this concern by defining judgements in terms of the *set* of all states satisfying A; whenever *any* of these tracked states is insufficient to provide a permission required by B, our logic will force this permission to be added *in general* to the wand's footprint (taken from the current state).

A *witness set* S is a set of pairs of states (σ*A*, σ*B*); conceptually, the first represents the state available for trying to prove B *in addition* to the current state; this is initially a state satisfying the wand's left-hand side A. The second represents the state assembled (so-far) to attempt to satisfy the right-hand side B. We write S<sup>1</sup> for the set of first elements of all pairs in a witness set S. A *context* Δ is a pair (σ, S) of a state and a witness set; here, σ represents the (asyet unused remainder of the) current state in which the wand is being packaged.

The basic idea behind a derivation in our logic is to show how to assemble a witness set in which *all* second elements are states satisfying B, via some combinations of: (1) moving a part of the first element of a pair in the witness set into the second, and (2) moving a part of the outer state σ into *all* first elements of the pairs (this becomes a part of the wand's footprint). The actual judgements of the logic are a little more complex, to correctly record any hypotheses (called *path-conditions*) that result from deconstructing conditional assertions in B.

Configurations and Reductions. A *configuration* represents a current objective in our package logic: the part of the wand's right-hand side still to be satisfied as well as the current state of a footprint computation. A configuration is a triple B, *pc*,(σ, S), where B is an assertion, pc is a *path condition* (a function from Σ to Booleans), and (σ, S) is a context. Conceptually, B is the assertion still to be satisfied, *pc* represents hypotheses we are currently working under, and the context (σ, S) tracks the current state and witness set, as described above.

A *reduction* is a judgement B, *pc*,(σ0, S0) - (σ1, S1), representing the achievement of the objective described via the configuration on the left, resulting in the final context on the right; σ<sup>1</sup> is the new version of the outer state (and becomes the new current state after the package operation); whatever was removed from the initial outer state is implicitly the selected footprint state σ*w*. If a reduction is derivable in our package logic, this footprint σ*<sup>w</sup>* guarantees that for all (σ*A*, σ*B*) ∈ S0, if (σ*<sup>A</sup>* ⊕ σ*B*)#σ*w*, then σ*<sup>A</sup>* ⊕ σ*<sup>w</sup>* satisfies *pc* ⇒ B. The condition (σ*<sup>A</sup>* ⊕ σ*B*)#σ*<sup>w</sup>* ensures that the pair (σ*A*, σ*B*) actually corresponds to a state in which the wand can be applied given the chosen footprint σ*w*, as we explain later. The package logic defines the steps an algorithm may take to achieve this goal.

We represent packaging a wand A −∗ B in state σ<sup>0</sup> by the derivation of a reduction B, λσ. ,(σ0, {(σ*A*, e) | σ*<sup>A</sup>* |= A}) - (σ1, S1), for some state σ<sup>1</sup> and witness set S1. The path condition is initially true (we are not yet under any hypotheses). The initial witness set contains all pairs of a state σ*<sup>A</sup>* that satisfies A and the empty state e, to which a successful reduction will add permissions in order to satisfy B<sup>2</sup>. An actual algorithm need not explicitly compute this (possibly infinite) set, but can instead track it symbolically. If the algorithm finds a derivation of this reduction, it has proven that the difference between σ<sup>0</sup> and σ<sup>1</sup> is a valid footprint of the wand A−∗B, since the logic is sound (Theorem 1 below).

Rules. Figure 2 presents the four rules of our logic, defining (via derivable reductions) how a configuration can be reduced to a context. There is a rule for each type of assertion B: *Implication* for an implication, *Star* for a separating conjunction, and *Atom* for a semantic assertion. The logic also includes the rule *Extract*, which represents a choice to extract permissions from the outer state and adds

<sup>2</sup> If <sup>B</sup> is intuitionistic, this can be simplified to only the -*minimal* states that satisfy A. B is intuitionistic [38] iff, if B holds in a state σ, then B holds in any state σ- such that σ-σ. In intuitionistic SL or in IDF, all assertions are intuitionistic.

$$\frac{\langle A, \lambda \sigma, pc(\sigma) \wedge b(\sigma), \Delta \rangle \leadsto \Delta'}{\langle b \Rightarrow A, pc, \Delta \rangle \leadsto \Delta'} \operatorname{Implication} \qquad \frac{\langle A\_1, pc, \Delta\_0 \rangle \leadsto \Delta\_1}{\langle A\_2, pc, \Delta\_1 \rangle \leadsto \Delta\_2} \operatorname{Str} \mathsf{T}$$

$$\begin{array}{c} \forall (\sigma\_A, \sigma\_B) \in S.pc(\sigma\_A) \Longrightarrow \sigma\_A \succeq \mathit{choice}(\sigma\_A, \sigma\_B) \land \mathcal{B}(\mathit{choice}(\sigma\_A, \sigma\_B))\\ S\_\top = \{ (\sigma\_A \ominus \mathit{choice}(\sigma\_A, \sigma\_B), \sigma\_B \oplus \mathit{choice}(\sigma\_A, \sigma\_B)) | (\sigma\_A, \sigma\_B) \in S \land pc(\sigma\_A) \} \\ S\_\bot = \{ (\sigma\_A, \sigma\_B) | (\sigma\_A, \sigma\_B) \in S \land \neg pc(\sigma\_A) \} \\ \hline \end{array}$$
 
$$\begin{array}{c} \mathit{\bf{S}}\_\bot \mbox{\bf{\mathbin{\bf{\varepsilon}}}}(\mathit{\bf{\varepsilon}}, \mathit{\bf{\varepsilon}}, \mathit{\bf{\varepsilon}}) \{ \mathit{\bf{\varepsilon}}\_\bot, \sigma\_B \} \in S \land \neg pc(\sigma\_A) \} \\ \hline \end{array} \begin{array}{c} \mathit{\bf{\varepsilon}} \mbox{\bf{\varepsilon}} \mbox{\bf{\varepsilon}} \} \end{array}$$

$$\begin{array}{ll} \sigma\_{0} = \sigma\_{1} \oplus \sigma\_{w} & \mathsf{stable}(\sigma\_{w}) \quad \langle A, pc, (\sigma\_{1}, S\_{1}) \rangle \leadsto \Delta\\ S\_{1} = \{ (\sigma\_{A} \oplus \sigma\_{w}, \sigma\_{B}) | (\sigma\_{A}, \sigma\_{B}) \in S\_{0} \land (\sigma\_{A} \oplus \sigma\_{B}) \# \sigma\_{w} \} & \textit{Extract} \end{array}$$

Fig. 2. Rules of the package logic.

them to all pairs of states in the witness set. In the following, we informally write *reducing an assertion* to refer to the process of deriving (in the logic) that the relevant configuration containing this assertion reduces to some context.

To reduce an implication B ⇒ A, the rule *Implication* conjoins the hypothesis B with the previous path condition, leaving A to be reduced. Informally, this expresses that satisfying pc ⇒ (b ⇒ A) is equivalent to satisfying (pc ∧ b) ⇒ A.

For a separating conjunction A<sup>1</sup> ∗ A2, the *Star* rule expresses that both A<sup>1</sup> and A<sup>2</sup> must be reduced, in order to reduce A<sup>1</sup> ∗ A2; permissions used in the reduction of the first conjunct must not be used again, which is reflected by the threading-through of the intermediate context Δ1. 3

The *Atom* rule specifies how to prove that all states in S<sup>1</sup> (where S is the witness set) satisfy the assertion pc ⇒ B. To understand the premises, consider a pair (σ*A*, σ*B*) ∈ S. If σ*<sup>A</sup>* does not satisfy the path condition, i.e. ¬pc(σ*A*), then σ*<sup>A</sup> does not* have to justify B, and thus the pair (σ*A*, σ*B*) is left unchanged; this case corresponds to the set S⊥. Conversely, if σ*<sup>A</sup>* satisfies the path condition, i.e. pc(σ*A*), then σ*<sup>A</sup>* must satisfy B, and the corresponding permissions must be transferred from σ*<sup>A</sup>* to σ*B*. Since some assertions may be satisfied in different ways, such as disjunctions, the algorithm has a choice in how to satisfy B, which might be different for each pair (σ*A*, σ*B*). This choice is represented by *choice*(σ*A*, σ*B*), which must satisfy B and be smaller or equal to σ*A*. We update the witness set by transferring *choice*(σ*A*, σ*B*) from σ*<sup>A</sup>* to σ*B*. This second case corresponds to the set S. Note that the *Atom* rule can be applied only if σ*<sup>A</sup>* satisfies B, for all pairs (σ*A*, σ*B*) ∈ S such that pc(σ*A*). If not, a package algorithm must either first extract more permissions from the outer state with the *Extract* rule, or fail.

The *Extract* rule (applicable at any step of a derivation), expresses that we can extract permissions (the state<sup>4</sup> σ*w*) from the outer state σ0, and combine

<sup>3</sup> The order in the premises is unimportant since <sup>A</sup><sup>1</sup> <sup>∗</sup> <sup>A</sup><sup>2</sup> and <sup>A</sup><sup>2</sup> <sup>∗</sup> <sup>A</sup><sup>1</sup> are equivalent. <sup>4</sup> We explain formally in App. D of the TR [16] the notion of a stable state, which is a technicality of our general state model; in standard SL, all states are stable.

them with the first element of each pair of states in the witness set. Note that (σ*A*, σ*B*) is removed from the witness set if σ*<sup>A</sup>* ⊕σ*<sup>B</sup>* is not compatible with σ*w*. In such cases, adding σ*<sup>w</sup>* to σ*<sup>A</sup>* would create a pair in the witness set representing a state in which the wand cannot be applied. Consequently, there is no need to establish the right-hand side of the wand for this pair and our logic correspondingly removes it. Finally, the rule requires that we reduce the assertion A in the new context.

A package algorithm's strategy is mostly reflected by how it uses the *Extract* rule. To package acc(x.b, 1/2) −∗ acc(x.b, 1/2) ∗ (x.b ⇒ acc(x.f)) from Sect. 3.1 one algorithm might use this rule to extract permission to x.f; another might use it to extract permission to x.b (if x.b had value false in the original state).

Example of a Derivation. Let us now illustrate how these rules can be used to package the wand from Sect. 3.1, w := acc(x.f)∗(x.f = y∨x.f = z)−∗acc(x.f)∗ acc(x.f.g). We omit the path condition since it is always the trivial condition (λσ. ). Assume that the outer state σ<sup>0</sup> is the addition of σ*yz*, a state that contains permission to y.g and z.g, and σ1. S<sup>0</sup> := {(σ*A*, e) | σ*<sup>A</sup>* ∈ Σ ∧ σ*<sup>A</sup>* |= acc(x.f) ∗ (x.f = y ∨ x.f = z)} is the initial witness set. We show below a part of a proof that acc(x.f) \* acc(x.f.g),(σ0, S0) - (σ1, S3) is correct, and thus that σ*yz* is a correct footprint of the wand w (since σ<sup>0</sup> = σ<sup>1</sup> ⊕ σ*yz*):

$$\begin{array}{c} \cdots \\ \hline \langle \mathtt{acc}(\mathtt{x}.\mathtt{t},\mathtt{0},\mathtt{S}\_{0}) \rangle \leadsto (\sigma\_{0},S\_{1}) \\ \hline \end{array} Atom \quad \begin{array}{c} \begin{array}{c} \cdots \\ \hline \langle \mathtt{acc}(\mathtt{x}.\mathtt{t},\mathtt{g}),(\sigma\_{1},S\_{2}) \rangle \leadsto (\sigma\_{1},S\_{3}) \\ \hline \langle \mathtt{acc}(\mathtt{x}.\mathtt{t},\mathtt{g}),(\sigma\_{0},S\_{1}) \rangle \leadsto (\sigma\_{1},S\_{3}) \\ \hline \end{array} \operatorname{Extract} t \\ \hline \end{array} \\ Extract \begin{array}{c} \begin{array}{c} \mathtt{Att} \\ \mathtt{Extract} \\ \mathtt{Set} \\ \mathtt{Set} \\ \mathtt{Set} \\ \end{array} \\ \hline \end{array} \\ Stract \begin{array}{c} \mathtt{Att} \\ \mathtt{Set} \\ \mathtt{Set} \\ \mathtt{Set} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Set} \\ \mathtt{Set} \\ \mathtt{Set} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Set} \\ \mathtt{Set} \\ \mathtt{Set} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Max} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Max} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Max} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Max} \\ \end{array} \\ Start \begin{array}{c} \mathtt{Max} \\ \mathtt{Max} \\ \mathtt{Max} \\ \end{array} \\ Start \end{array}$$

This derivation, which reflects the package algorithm that we will describe in Sect. 3.5, can be read from bottom to top and from left to right. Using the rule *Star* , we split the assertion into its two conjuncts, acc(x.f) (on the left) and acc(x.f.g) (on the right). We then handle acc(x.f) using the rule *Atom*. acc(x.f) holds in the first element of each pair of S0, since any state that satisfies the wand's left-hand side owns x.f. Therefore, we use the rule *Atom* with a *choice* function that always chooses the relevant state with exactly full permission to x.f. S<sup>1</sup> is the updated witness set where this permission to x.f has been transferred from the first to the second element of each pair of states. Next, we handle acc(x.f.g). We cannot do this directly using the rule *Atom* from S1. We know that, for each (σ*A*, σ*B*) ∈ S1, x.f.g evaluated in σ*<sup>A</sup>* is either <sup>y</sup> or <sup>z</sup>, but σ*<sup>A</sup>* owns neither y.g nor z.g. So, we transfer the permissions to both y.g and z.g from the outer state σ<sup>0</sup> to all states of S<sup>1</sup> <sup>1</sup> , using the rule *Extract*, which results in the context (σ1, S2); † represents the three other premises of the rule, namely σ<sup>0</sup> = σ*yz* ⊕ σ1, stable(σ*yz*), and S2's definition. Finally, we apply the rule *Atom* to prove acc(x.f.g),(σ1, S2) - (σ1, S3), where the *choice* function chooses for each pair the corresponding state that contains full permission to x.f.g.

#### 3.4 Soundness and Completeness

We write B, pc, Δ - Δ to express that a reduction can be derived in the logic. As explained above, the goal of a package algorithm is to find a derivation of B,λ\_. ,(σ, {(σ*A*, e) | σ*<sup>A</sup>* ∈ S*A*}) - (σ , S ). If it succeeds, then the difference between σ and σ is a valid footprint of A −∗ B, since our package logic is sound. In particular, we have proven the following soundness result in Isabelle/HOL:

#### Theorem 1 *Soundness. Let* B *be a well-formed*<sup>5</sup> *assertion. If*

	- *(a)* B *is intuitionistic*
	- *(b) For all* (σ*A*, σ*B*) ∈ S *,* σ*<sup>A</sup> contains no permission (i.e.* σ*<sup>A</sup>* ⊕ σ*<sup>A</sup>* = σ*A)*

*then there exists a stable state* σ*<sup>w</sup> s.t.* σ = σ ⊕ σ*<sup>w</sup> and* σ*<sup>w</sup> is a footprint of* A −∗ B*.*

The third premise shows that, in an intuitionistic SL or in IDF, the correspondence between a derivation in the logic and a valid footprint of a wand is straightforward (case (a)). However, in classical SL, one must additionally check that all permissions in the witness set have been consumed (case (b)).

We have also proved in Isabelle/HOL that our package logic is complete, i.e. *any* valid footprint can be computed via a derivation in our package logic:

Theorem 2 *Completeness. Let* B *be a well-formed (see footnote 5) assertion. If* σ*<sup>w</sup> is a stable footprint of* A−∗B*, and* σ = σ ⊕σ*w, then there exists a witness set* S *such that* B,λ*\_*. ,(σ, {(σ*A*, e) | σ*<sup>A</sup>* ∈ S*A*}) - (σ , S )*.*

#### 3.5 A Sound Package Algorithm

We now describe an automatic package algorithm that corresponds to a proof search strategy in our package logic, and which is thus sound. To convey the main ideas, consider packaging a wand of the shape A −∗ B<sup>1</sup> ∗ ... ∗ B*n*. <sup>6</sup> Our algorithm traverses the assertion B<sup>1</sup> ∗ ... ∗ B*<sup>n</sup>* from left to right, similarly to the FIA approach; this traversal is justified by repeated applications of the rule *Star*. Assume at some point during this traversal that the current context is (σ0, S). When we encounter the assertion B*i*, we have two possible cases:

1. All states <sup>σ</sup>*<sup>A</sup>* <sup>∈</sup> <sup>S</sup><sup>1</sup> satisfy <sup>B</sup>*i*, which means that the permissions (or values) required by B*<sup>i</sup>* are provided by the left-hand side of the wand. In this case, for each pair (σ*A*, σ*B*) ∈ S, we transfer permissions (and the corresponding values) to satisfy B*<sup>i</sup>* from σ*<sup>A</sup>* to σ*B*, using the rule *Atom*. Note that the transferred permissions might be different for each pair (σ*A*, σ*B*). This gives us a new witness set S , while the outer state σ<sup>0</sup> is left unchanged. We must then handle the next assertion B*i*+1 in the context (σ0, S ).

<sup>5</sup> We formally define well-formedness in App. D of the TR [16]. Intuitively, a well-formed assertion roughly corresponds to a self-framing assertion as defined in Sect. 2.1.

<sup>6</sup> In App. I of the TR [16], we also show how our package algorithm handles implications.

2. There is at least one pair (σ*A*, σ*B*) ∈ S such that B*<sup>i</sup>* does not hold in σ*A*. In this case, the algorithm fails if combining the permissions (and values) contained in the outer state with each <sup>σ</sup>*<sup>A</sup>* <sup>∈</sup> <sup>S</sup><sup>1</sup> is not sufficient to satisfy <sup>B</sup>*i*. Otherwise, we apply the rule *Extract* to transfer permissions from the outer state σ<sup>0</sup> to each state σ*<sup>A</sup>* in S<sup>1</sup> such that B*<sup>i</sup>* holds in σ*A*. This gives us a new context (σ 0, S ). We can now apply the first case with the context (σ 0, S ).

#### 4 Using the Logic with Combinable Wands

Extending SL with fractional permissions [4] is well-known to be useful for reasoning about heap-manipulating concurrent programs with shared state. In this setting, permission amounts are generalised to fractions 0 ≤ p ≤ 1. Reading a heap location is permitted if p > 0, and writing if p = 1, which permits concurrent reads and ensures exclusive writes. The assertion acc(x.f, p) holds in a state that has *at least* p permission to x.f. A permission amount p + q to a heap location x.f can be split into a permission amount p and a permission amount q, i.e. acc(x.f, p + q) |= acc(x.f, p) \* acc(x.f, q), and these two permissions can be recombined, i.e. acc(x.f, p) \* acc(x.f, q) |= acc(x.f, p + q).

This concept has been generalised [5,7,17,23,29] to *fractional assertions* A*<sup>p</sup>*, representing a fraction p of A. A*<sup>p</sup>* holds in a state σ iff there exists a state σ*<sup>A</sup>* in which A holds and σ is obtained from σ*<sup>A</sup>* by multiplying all permission amounts held by <sup>p</sup> [7,29]; in this case, we write <sup>σ</sup> <sup>=</sup> <sup>p</sup> · <sup>σ</sup>*A*. For example, acc(x.f)*<sup>p</sup>* <sup>≡</sup> acc(x.f, p), and Tree(x)*<sup>p</sup>* (where Tree is the predicate defined in Fig. 1) expresses p permission to all nodes of the tree rooted in x.

Using fractional assertions, one might specify a function find, which searches a binary tree and yields a subtree whose root contains key key, as follows [7]: { Tree(x)*<sup>p</sup>* } find(x, key) { <sup>λ</sup>ret.(Tree(ret) <sup>∗</sup> (Tree(ret) −∗ Tree(x)))*<sup>p</sup>* }, in which ret corresponds to the return value of find. This postcondition is similar to the loop invariant in Fig. 1, except that it needs only a fraction p of Tree(x). A number of automatic SL verifiers, such as Caper [18], Chalice [31], VerCors [2], VeriFast [24], and Viper [34], support fractional assertions in some form.

Combinable Assertions. While it is always possible to split an assertion A*<sup>p</sup>*+*<sup>q</sup>* into <sup>A</sup>*<sup>p</sup>* <sup>∗</sup>A*<sup>q</sup>*, recombining <sup>A</sup>*<sup>p</sup>* <sup>∗</sup>A*<sup>q</sup>* into <sup>A</sup>*<sup>p</sup>*+*<sup>q</sup>* is sound only under some conditions, for example [29] if A is *precise* (in the usual SL sense [38]). We say that A is *combinable* iff the entailment <sup>A</sup>*<sup>p</sup>* <sup>∗</sup> <sup>A</sup>*<sup>q</sup>* <sup>|</sup><sup>=</sup> <sup>A</sup>*<sup>p</sup>*+*<sup>q</sup>* holds for any two positive fractions p and q such that p + q ≤ 1. As an example, acc(x.f) is combinable, but acc(x.f)∨acc(x.g) is not because a state containing half permission to both x.f and x.g satisfies (acc(x.f) <sup>∨</sup> acc(x.g))<sup>0</sup>*.*<sup>5</sup> <sup>∗</sup> (acc(x.f) <sup>∨</sup> acc(x.g))<sup>0</sup>*.*<sup>5</sup>, but not acc(x.f)∨acc(x.g). Combinable assertions are particularly useful to reason about concurrent programs, for instance, to combine the postconditions of parallel branches when they terminate [7].

However, a magic wand is in general *not* combinable, as we show below. This is problematic for SL verifiers; they cannot soundly combine wands, nor predicates that could possibly contain wands in their bodies. One way to prevent

the latter is to forbid magic wands in predicate bodies entirely, but this limits the common usage of predicates to abstract over general assertions in specifications [35]. Another solution is to disallow combining fractional instances of a predicate if its body contains a wand, which means requiring additional annotations to "taint" such predicates transitively. This is overly restrictive for wands which are actually combinable and complicates reasoning about abstract predicate families [35].

To address this issue, we propose a novel restriction of the wand, called *combinable wand* (we use *standard wand* to refer to the usual, unrestricted connective). Unlike standard wands in general, a combinable wand is always combinable if its right-hand side is combinable. Thus, by only using combinable wands instead of standard wands, all assertions in logics such as those employed by Ver-Cors and Viper can be made combinable without any of the other aforementioned restrictions regarding predicates. Section 5 shows that the restriction combinable wands impose is sufficiently weak for practical purposes. Finally, footprints of combinable wands can be automatically inferred by package algorithms built on our package logic. All results in this section have been proven in Isabelle/HOL.

Standard Wands are Not Combinable in General. Even if B is combinable, the standard wand A −∗ B is, in general, not. As an example, the wand <sup>w</sup> := acc(x.f, 1/2) −∗ acc(x.g) is not combinable, because <sup>w</sup>0*.*<sup>5</sup> <sup>∗</sup> <sup>w</sup>0*.*<sup>5</sup> |<sup>=</sup> <sup>w</sup>. To see this, consider two states σ*<sup>f</sup>* and σ*g*, containing full permissions to only x.f and x.g, respectively. Both states are valid footprints of w, i.e. σ*<sup>f</sup>* |= w (because σ*<sup>f</sup>* is incompatible with all states that satisfy the left-hand side) and σ*<sup>g</sup>* |= w (because <sup>σ</sup>*<sup>g</sup>* entails the right-hand side). Thus, by definition, <sup>0</sup>.<sup>5</sup> · <sup>σ</sup>*<sup>f</sup>* <sup>|</sup><sup>=</sup> <sup>w</sup>0*.*<sup>5</sup> and <sup>0</sup>.<sup>5</sup> · <sup>σ</sup>*<sup>g</sup>* <sup>|</sup><sup>=</sup> <sup>w</sup>0*.*<sup>5</sup>. However, <sup>0</sup>.<sup>5</sup> · <sup>σ</sup>*<sup>f</sup>* <sup>⊕</sup> <sup>0</sup>.<sup>5</sup> · <sup>σ</sup>*g*, i.e. a state with half permission to both x.f and x.g, is *not* a valid footprint of <sup>w</sup>, and thus <sup>w</sup>0*.*<sup>5</sup> <sup>∗</sup> <sup>w</sup>0*.*<sup>5</sup> |<sup>=</sup> <sup>w</sup>.

Intuitively, w is not combinable because one of its footprints, σ*<sup>f</sup>* , is incompatible with the left-hand side of the wand, but becomes compatible when the footprint is scaled down to a fraction. After scaling, the wand no longer holds trivially, and the footprint does not necessarily establish the right-hand side.

To make this intuition more precise, we introduce the notion of *scalable footprints*. For a state σ, we define *scaled*(σ) to be the set of copies of σ multiplied by any fraction 0 < α ≤ 1, i.e. *scaled*(σ) := {α · σ | 0 < α ≤ 1}. A footprint σ*<sup>w</sup>* is *scalable w.r.t. a state* σ*<sup>A</sup>* iff either (1) σ*<sup>A</sup>* is compatible with *all* states from *scaled*(σ*w*), or (2) σ*<sup>A</sup>* is compatible with *no* state in *scaled*(σ*w*). A footprint is *scalable for a wand* A −∗ B iff it is scalable w.r.t. all states that satisfy A. Intuitively, this means that the footprint does not "jump" between satisfying the wand trivially and having to satisfy the right-hand side. In the above example, σ*<sup>g</sup>* is a scalable footprint for w, but σ*<sup>f</sup>* is not.

Making Wands Combinable. The previous paragraphs show that, even if B is combinable, the standard wand A−∗B is in general not combinable because it can be satisfied by non-scalable footprints. Therefore, we define a novel restricted interpretation for wands that *forces* footprints to be scalable, in the following sense. The restricted interpretation of a wand accepts all scalable footprints, and transforms non-scalable footprints before checking whether they actually satisfy the wand. We call a wand with this restricted interpretation a *combinable wand*, and write A −∗*<sup>c</sup>* B to differentiate it from the standard wand A −∗ B.

For standard wands, *any* state σ*<sup>w</sup>* is a footprint of A−∗B iff, for all states σ*<sup>A</sup>* that satisfy A, σ*A*#σ*<sup>w</sup>* ⇒ σ*<sup>A</sup>* ⊕ σ*<sup>w</sup>* |= B. We obtain the definition of combinable wands by replacing σ*<sup>w</sup>* with a (possibly smaller) state R(σ*A*, σ*w*) that is scalable w.r.t. σ*A*. R(σ*A*, σ*w*) is defined as σ*<sup>w</sup>* if *no* state in *scaled*(σ*w*) is compatible with any σ*A*; in that case, condition (2) of scalable footprints holds for R(σ*A*, σ*w*) w.r.t. σ*A*. Otherwise, R(σ*A*, σ*w*) is obtained by removing just enough permissions from σ*<sup>w</sup>* to ensure that *all* states in *scaled*(R(σ*A*, σ*w*)) are compatible with σ*A*, which ensures that condition (1) holds for R(σ*A*, σ*w*) w.r.t. σ*A*.

To formally define R(σ*A*, σ*w*), we fix a concrete separation algebra (formally defined in App. G of the TR [16]), whose states are pairs (π, h) of a *permission mask* π, which maps heap locations to fractional permissions, and a partial heap h, which maps heap locations to values.

Definition 1. *Let* (π*A*, h*A*) *and* (π*w*, h*w*) *be two states, and let* π *<sup>w</sup> be the permission mask such that* ∀l. π *<sup>w</sup>*(l) = min(π*w*(l), 1 − π*A*(l))*. Then*

$$\mathcal{R}((\pi\_A, h\_A), (\pi\_w, h\_w)) = \begin{cases} (\pi\_w, h\_w) & \text{if } \forall \sigma \in scaled((\pi\_w, h\_w)) . \neg(\pi\_A, h\_A) \# \sigma \\ (\pi'\_w, h\_w) & \text{otherwise} \end{cases}$$

*The* combinable wand A −∗*<sup>c</sup>* B *is then interpreted as follows:*

σ*<sup>w</sup>* |= A −∗*<sup>c</sup>* B ⇐⇒ (∀σ*A*. σ*<sup>A</sup>* |= A ∧ σ*A*#R(σ*A*, σ*w*) =⇒ σ*<sup>A</sup>* ⊕ R(σ*A*, σ*w*) |= B)

The following theorem (proved in Isabelle/HOL) shows some key properties of combinable wands.

Theorem 3. *Let* B *be an intuitionistic assertion.*


Property 1 expresses that combinable wands constructed from combinable assertions are combinable, which enables verification methodologies underlying tools such as VerCors and Viper to support flexible combinations of wands and predicates (as motivated at the start of this section). Property 2 implies that A∗(A−∗*c*B) |= B, that is, combinable wands can be applied like standard wands. Property 3 states that combinable wands pose no restrictions if the left-hand side is binary, that is, if it can be expressed without fractional permissions (formally defined in App. G of the TR [16]). For example, the predicate Tree(x) from Fig. 1 is binary, which implies that the wands Tree(y) −∗*<sup>c</sup>* Tree(x) and Tree(y) −∗ Tree(x) are equivalent. This property is an important reason for why combinable wands are expressive enough for practical purposes, as we further evidence in Sect. 5.



Footprints of combinable wands can be automatically inferred by algorithms built on our package logic. We explain (along with examples) in App. H of the TR [16] how to lift the package logic presented in Sect. 3 to handle alternative definitions of allowable footprints such as the restrictions imposed by Definition 1.

#### 5 Evaluation

We have implemented package algorithms for the standard wands and combinable wands in a custom branch of Viper's [34] verification condition generator (VCG). Both are based on the package logic described in Sect. 3, adapted to the fractional permission setting. Both algorithms automate the proof search strategy outlined in Sect. 3.5. Viper's VCG translates Viper programs to Boogie [32] programs. It uses a total-heap semantics of IDF [36], where Viper states include a heap and a permission mask (tracking fractional permission amounts). The heap and mask are represented in Boogie as maps; we also represent witness sets as Boogie maps.

We evaluate our implementations of the package algorithms on Viper's test suite and compare them to Viper's implementation of the FIA as presented in Sect. 2.4. Our key findings are that our algorithms (1) enable the verification of almost all correct package operations. (2) correctly report package operations that are supposed to fail (in contrast to the FIA), and (3) have an acceptable performance overhead compared to the FIA. Moreover, interpreting wands as combinable wands as explained in Sect. 4 has only a minor effect on the results, but correctly rejects attempts to package a non-combinable wand. This finding suggests that verifiers could improve their expressiveness by allowing flexible combinations of wands and predicates with only a minor completeness penalty.

For our evaluation, we considered all 85 files in the test suite for Viper's VCG with at least one package operation. From these 85 files, we removed 29 files containing features that our implementation does not yet support. 28 of these 29 files require proof scripts to guide the footprint inference, which are orthogonal to the concerns of this paper (see App. J of the TR [16] for details).

Table 1 gives an overview of our results. These confirm that our algorithms for standard and combinable wands (S-Alg and C-Alg) do not produce false negatives, that is, are sound. In contrast, the FIA does verify an incorrect program (which is similar to the example in Sect. 2.4). While this is only a single unsound

example, it is worth emphasing that (a) it comes from the pre-existing test suite of the tool itself, (b) the unsoundness was not known of until our work, and (c) soundness issues in a program verifier are critical to address; we show how to achieve this.

Compared with the FIA, our implementation reports a handful of false positives (spurious errors). For S-Alg, 3 out of 5 false positives are caused by missing features of our implementation (such as remembering a subset of the permissions that are inside predicate instances when manipulating predicates); these features could be straightforwardly added in the future. The other 2 false positives are caused by S-Alg's strategy. In one, the only potential footprint prevents the wand from ever being applied; although technically a false positive, it seems useful to reject the wand and alert the user. The other case is due to a coarse-grained heuristic applied by S-Alg that can be improved.

C-Alg reports the expected result in 48 benchmarks. Importantly, it correctly rejects one wand that indeed does not hold as a combinable wand. 5 of the 8 false positives are identical to those for S-Alg. In the other three benchmarks, the wands still do hold as combinable wands, but further extensions to C-Alg are required to handle them due to technical challenges regarding predicate instances. Once these extensions have been implemented, C-Alg will be as precise as S-Alg, indicating that comparable program verifiers could switch to combinable wands to simply enable sound, flexible combinations with predicates.

To evaluate performance, we ran each of the three implementations 5 times on each of the 56 benchmarks on a Lenovo T480 with 32 GB of RAM and a i7- 8550U 1.8 GhZ CPU, running on Windows 10. We removed the slowest and fastest time, and then took the mean of the remaining 3 runs. The FIA takes between 1 and 11 seconds per benchmark. On average, S-Alg is 21% slower than the FIA. For 46 of the 56 examples, the increase is less than 30%, and for 3 examples S-Alg is between a factor 2 and 3.4 slower. The overhead is most likely due to the increased complexity of our algorithms, which track more states explicitly and require more quantified axioms in the Boogie encoding. C-Alg is on average 10% slower than S-Alg. We consider the performance overhead of our algorithms to be acceptable, especially since wands occur much more frequently in our benchmarks than in average Viper projects, as judged by existing tests and examples. More representative projects will, thus, incur a much smaller slow-down.

#### 6 Related Work

VerCors [2] and Viper [34] are to the best of our knowledge the only automatic SL verifiers that support magic wands. Both employ package and apply ghost operations. VerCors' package algorithm requires a user to manually specify a footprint whereas Viper infers footprints using the FIA, which is unsound as we show in Sect. 2.4. Our package algorithm is as automatic as the FIA but is sound.

Lee and Park [30] develop a sound and complete proof system for SL including the magic wand. Moreover, they derive a decision procedure from their completeness proof for propositional SL. However, more expressive versions of SL (that

include e.g. predicates and quantifiers) are undecidable [6] and so this decision procedure cannot be directly applied in the logics employed by program verifiers.

Chang *et al.* [11] define a shape analysis that derives magic wands A −∗ B of a restricted form (A and B cannot contain general imprecise assertions); our package logic does not impose such restrictions, which rule out some useful kinds of wands. For example, A may be a data structure with a read-only part expressed via existentially-quantified fractional permissions or A may contain the necessary permission to invoke a method, which may be an arbitrary assertion. In follow-up work, Chang and Rival [10] present a restricted "inductive" magic wand. Footprints of inductive wands are expressed via a finite unrolling of an inductive predicate defining B until the permissions in A are revealed. Such wands are useful to reason about data structures with back-pointers such as doubly-linked lists.

Iris [26] provides a custom proof mode [27] for interactive SL proofs in Coq [12]. Separation logics expressed in Iris support wands and are more expressive than those of automatic SL verifiers at the cost of requiring more user guidance. Packaging a wand in the proof mode requires manually specifying a footprint and proving that the footprint is correct. While tactics can be used in principle to automate parts of this process, there are no specific tactics to infer footprints.

Fractional assertions have been used in various forms [5,7,17,23,29]. Le and Hobor [29] allow combining two fractional assertions A*<sup>p</sup>* and A*<sup>q</sup>* only if A is *precise* in the SL sense (i.e. A describes the contents of the heaps in which it holds precisely). To avoid requiring A to be precise, Brotherston *et al.* [7] introduce *nominal labels* for assertions. If an assertion is split into two fractional assertions, then the same fresh label can be associated with both parts to indicate that they were split from the same assertion.

Two fractional assertions with the same label can be combined. However, this solution has not been implemented and does not deal with packaging wands. Our solution also avoids requiring that an assertion is precise and allows combining assertions even if they were not split from the same assertion. Instead of introducing labels, we introduce a light restriction that ensures that wands are always combinable. As a result, assertions containing combinable wands but no other potentially imprecise connectives (such as disjunction) are combinable. In particular, all assertions employed in verifiers such as VerCors and Viper can be made combinable thanks to our work.

#### 7 Conclusion

We presented a package logic that precisely characterises sound package algorithms for automated reasoning about magic wands. Based on this logic, we developed a novel package algorithm that is inspired by an existing approach, but is sound. Moreover, we identified a sufficient criterion for wands to be combinable, such that they can be used flexibly in logics with fractional permissions, and presented a package algorithm for combinable wands. We implemented our

solutions in Viper and demonstrated their practical usefulness. The soundness and completeness of our package logic, as well as key properties of combinable wands are all proved in Isabelle/HOL. As future work, we plan to extend the implementation of the two package algorithms described in Sect. 5 by porting various features of the pre-existing FIA implementation. Moreover, we will use our package logic to develop another algorithm for Viper's symbolic-execution verifier.

Acknowledgement. This work was partially funded by the Swiss National Science Foundation (SNSF) under Grant No. 197065.

#### References


Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Divide-and-Conquer Determinization of B¨uchi Automata Based on SCC Decomposition**

Yong Li<sup>1</sup> , Andrea Turrini1,2 , Weizhi Feng1,3 , Moshe Y. Vardi<sup>4</sup> , and Lijun Zhang1,2,3(B)

<sup>1</sup> State Key Laboratory of Computer Science, Institute of Software,

Chinese Academy of Sciences, Beijing, China zhanglj@ios.ac.cn <sup>2</sup> Institute of Intelligent Software, Guangzhou, China <sup>3</sup> University of Chinese Academy of Sciences, Beijing, China

<sup>4</sup> Rice University, Houston, USA

**Abstract.** The determinization of a nondeterministic B¨uchi automaton (NBA) is a fundamental construction of automata theory, with applications to probabilistic verification and reactive synthesis. The standard determinization constructions, such as the ones based on the Safra-Piterman's approach, work on the whole NBA. In this work we propose a divide-and-conquer determinization approach. To this end, we first classify the strongly connected components (SCCs) of the given NBA as inherently weak, deterministic accepting, and nondeterministic accepting. We then present how to determinize each type of SCC *independently* from the others; this results in an easier handling of the determinization algorithm that takes advantage of the structure of that SCC. Once all SCCs have been determinized, we show how to compose them so to obtain the final equivalent deterministic Emerson-Lei automaton, which can be converted into a deterministic Rabin automaton without blowup of states and transitions. We implement our algorithm in our tool COLA and empirically evaluate COLA with the state-of-the-art tools Spot and Owl on a large set of benchmarks from the literature. The experimental results show that our prototype COLA outperforms Spot and Owl regarding the number of states and transitions.

#### **1 Introduction**

Nondeterministic B¨uchi automata (NBAs) [6] are finite automata accepting infinite words; they are a simple and popular formalism used in model checking to represent reactive and non-terminating systems and their specifications, characterized by ω-regular languages [2]. Due to their nondeterminism, however, there are situations in which NBAs are not suitable, so deterministic automata are required, as it happens in probabilistic verification [2] and reactive synthesis from logical specifications [34]. Consequently, translating NBAs into equivalent deterministic ω-automata (that is, deterministic automata accepting the same

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 152–173, 2022. https://doi.org/10.1007/978-3-031-13188-2\_8

ω-regular language) is a necessary operation for solving these problems. While there exists a direct translation from linear temporal logic (LTL) to deterministic ω-automata [15], not all problems of interests can be formalized by LTL formulas, since LTL cannot express the full class of ω-regular properties [42]. For instance, we have to use Linear Dynamic Logic (LDL) [11,41] instead of LTL to express the ω-regular property "the train will arrive in every odd minute". To the best of our knowledge, we still need to go through the determinization of NBAs for LDL to obtain deterministic ω-automata. Therefore, NBA determinization is very important in verifying the whole class of ω-regular properties.

The determinization of NBAs is a fundamental problem in automata theory that has been actively studied for decades. For the determinization of nondeterministic automata accepting finite words, it suffices to use a subset construction [20]. Determinization constructions for NBAs are, however, much more involved since the simple subset construction is not sufficient [36]. Safra [36] gave the first determinization construction for NBAs with the optimal complexity 2<sup>O</sup>(<sup>n</sup> log <sup>n</sup>) , here n is the number of states of the input NBA; Michel [30] then gave a lower bound n! for determinizing NBAs. Safra's construction has been further optimized by Piterman [33] to O((n!)<sup>2</sup>) [38], resulting in the widely known Safra-Piterman's construction. The Safra-Piterman's construction is rather challenging, while still being the most practical way for B¨uchi complementation [40]. Research on determinization since then either aims at developing alternative Safraless constructions [18,21,28] or further tightening the upper and lower bounds of the NBA determinization [9,26,39,43].

In this paper, we focus on the *practical* aspects of B¨uchi determinization. All works on determinization mentioned above focus on translating NBAs to either deterministic Rabin or deterministic parity automata. According to [37], the more relaxed an acceptance condition is, the more succinct a finite automaton can be, regarding the number of states. In view of this, we consider the translation of NBAs to deterministic *Emerson-Lei* automata (DELAs) [13,37] whose acceptance condition is an arbitrary Boolean combination of sets of transitions to be seen finitely or infinitely often, the most generic acceptance condition for a deterministic automaton. We consider here transition-based automata rather than the usual state-based automata since the former can be more succinct [12].

The B¨uchi determinization algorithms available in literature operate on the *whole* NBA structure at once, which does not scale well in practice due to the complex structure and the big size of the input NBA. In this work we apply a *divide-and-conquer* methodology to B¨uchi determinization. We propose a determinization algorithm for NBAs to DELAs based on their strongly connected components (SCCs) decomposition. We first classify the SCCs of the given NBA into three types: *inherently weak*, in which either all cycles do not visit accepting transitions or all must visit accepting transitions; *deterministic accepting* and *nondeterministic accepting*, which contain an accepting transition and are deterministic or nondeterministic, respectively. We show how to divide the whole B¨uchi determinization problem into the determinization for each type of SCCs *independently*, in which the determinization for an SCC takes advantage of the structure of that SCC. Then we show how to compose the results of the local determinization for each type of SCCs, leading to the final equivalent DELA. An extensive experimental evaluation confirms that the divide-and-conquer approach pays off also for the determinization of the whole NBA.

**Contributions.** First, we propose a *divide-and-conquer* determinization algorithm for NBAs, which takes advantage of the structure of different types of SCCs and determinizes SCCs independently. Our construction builds an equivalent DELA that can be converted into a deterministic Rabin automaton without blowing up states and transitions (cf. Theorem 2). To the best of our knowledge, we propose the *first* determinization algorithm that constructs a DELA from an NBA. Second, we show that there exists a family of NBAs for which our algorithm gives a DELA of size 2<sup>n</sup>+2 while classical works construct a DPA of size at least n! (cf. Theorem 3). Third, we implement our algorithm in our tool COLA and evaluate it with the state-of-the-art tools Spot [12] and Owl [23] on a large set of benchmarks from the literature. The experiments show that COLA outperforms Spot and Owl regarding the number of states and transitions. Finally, we remark that the determinization complexity for some classes of NBAs can be exponentially better than the known ones (cf. Corollary 1).

#### **2 Preliminaries**

Let Σ be a given alphabet, i.e., a finite set of letters. A transition-based Emerson-Lei automaton can be seen as a generalization of other types of ω-automata, like B¨uchi, Rabin or parity. Formally, it is defined in the HOA format [1] as follows:

**Definition 1.** *A* nondeterministic Emerson-Lei automaton *(NELA) is a tuple* <sup>A</sup> = (Q, ι, δ, Γk, <sup>p</sup>,Acc)*, where* <sup>Q</sup> *is a finite set of* states*;* <sup>ι</sup> <sup>∈</sup> <sup>Q</sup> *is the* initial state*;* <sup>δ</sup> <sup>⊆</sup> <sup>Q</sup> <sup>×</sup> <sup>Σ</sup> <sup>×</sup> <sup>Q</sup> *is a* transition relation*;* <sup>Γ</sup><sup>k</sup> <sup>=</sup> {0, <sup>1</sup>, ··· , k}*, where* <sup>k</sup> <sup>∈</sup> <sup>N</sup>*, is a set of* colors*;* <sup>p</sup>: <sup>δ</sup> <sup>→</sup> <sup>2</sup><sup>Γ</sup><sup>k</sup> *is a* coloring function *for transitions; and* Acc *is an* acceptance formula *over* <sup>Γ</sup><sup>k</sup> *given by the following grammar, where* <sup>x</sup> <sup>∈</sup> <sup>Γ</sup>k*:*

$$\alpha := \mathtt{tt} \mid \mathtt{ff} \mid \mathsf{Fin}(x) \mid \mathsf{Inf}(x) \mid \alpha \lor \alpha \mid \alpha \land \alpha.$$

We remark that the colors in Γ<sup>k</sup> are not required to be all used in Acc. We call a NELA a *deterministic* Emerson-Lei automaton (DELA) if for each <sup>q</sup> <sup>∈</sup> <sup>Q</sup> and <sup>a</sup> <sup>∈</sup> <sup>Σ</sup>, there is at most one <sup>q</sup> <sup>∈</sup> <sup>Q</sup> such that (q, a, q ) <sup>∈</sup> <sup>δ</sup>.

In the remainder of the paper, we consider <sup>δ</sup> also as a function <sup>δ</sup> : <sup>Q</sup> <sup>×</sup> <sup>Σ</sup> <sup>→</sup> <sup>2</sup><sup>Q</sup> such that <sup>q</sup> <sup>∈</sup> <sup>δ</sup>(q, a) whenever (q, a, q ) <sup>∈</sup> <sup>δ</sup>; we also write <sup>q</sup> <sup>a</sup>−→ <sup>q</sup> for (q, a, q ) <sup>∈</sup> <sup>δ</sup> and we extend it to words <sup>u</sup> <sup>=</sup> <sup>u</sup>0u<sup>1</sup> ··· <sup>u</sup><sup>n</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> in the natural way, i.e., <sup>q</sup> <sup>u</sup>−→ <sup>q</sup> <sup>=</sup> <sup>q</sup> <sup>u</sup>[0] −→ <sup>q</sup><sup>1</sup> u[1] −→ · · · <sup>u</sup>[n] −→ <sup>q</sup> , where σ[i] denotes the element s<sup>i</sup> of the sequence of elements <sup>σ</sup> <sup>=</sup> <sup>s</sup>0s1s<sup>2</sup> ··· at position <sup>i</sup>. We assume without loss of generality that each automaton is *complete*, i.e., for each state <sup>q</sup> <sup>∈</sup> <sup>Q</sup> and letter <sup>a</sup> <sup>∈</sup> <sup>Σ</sup>, we have <sup>δ</sup>(q, a) <sup>=</sup> <sup>∅</sup>. If it is not complete, we make it complete by adding a fresh state <sup>q</sup><sup>⊥</sup> <sup>∈</sup>/ <sup>Q</sup> and redirecting all missing transitions to it.

<sup>A</sup> *run* of <sup>A</sup> over an <sup>ω</sup>-word <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup> is an infinite sequence of states <sup>ρ</sup> such that <sup>ρ</sup>[0] = <sup>ι</sup>, and for each <sup>i</sup> <sup>∈</sup> <sup>N</sup>, (ρ[i], w[i], ρ[<sup>i</sup> + 1]) <sup>∈</sup> <sup>δ</sup>.

The *language* <sup>L</sup>(A) of <sup>A</sup> is the set of words accepted by <sup>A</sup>, i.e., the set of words <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup> such that there exists a run <sup>ρ</sup> of <sup>A</sup> over <sup>w</sup> such that <sup>p</sup>(*inf* (ρ)) <sup>|</sup><sup>=</sup> Acc, where *inf* (ρ) = { (q, a, q ) <sup>∈</sup> <sup>δ</sup> | ∀<sup>i</sup> <sup>∈</sup> <sup>N</sup>.∃j > i.(ρ[j], w[j], ρ[<sup>j</sup> + 1]) = (q, a, q ) } and the satisfaction relation <sup>|</sup>= is defined recursively as follows: given <sup>M</sup> <sup>⊆</sup> <sup>Γ</sup>k,

$$\begin{aligned} M &= \mathtt{tt}, \quad M \mid = \mathtt{Fin}(x) \text{ iff } x \notin M, \quad M \mid = \alpha\_1 \lor \alpha\_2 \text{ iff } M \mid = \alpha\_1 \text{ or } M \mid = \alpha\_2, \\\ M &\not\equiv \mathtt{ff}, \quad M \mid = \mathtt{Inf}(x) \text{ iff } x \in M, \quad M \mid = \alpha\_1 \land \alpha\_2 \text{ iff } M \mid = \alpha\_1 \text{ and } M \mid = \alpha\_2. \end{aligned}$$

Intuitively, a run ρ over w is accepting if the set of colors (induced by p) that occur infinitely often in ρ satisfies the acceptance formula Acc. Here Fin(x) specifies that the color x only appears for finitely many times while Inf(x) requires the color x to be seen infinitely often.

The more common types of ω-automata, such as B¨uchi, parity and Rabin can be treated as Emerson-Lei automata with the following acceptance formulas.

## **Definition 2.** *A NELA* <sup>A</sup> = (Q, ι, δ, Γk, <sup>p</sup>,Acc) *is said to be*


When the NELA <sup>A</sup> = (Q, ι, δ, Γk, <sup>p</sup>,Acc) is a nondeterministic BA (NBA), we just write <sup>A</sup> as (Q, ι, δ, F) where <sup>F</sup> is the set of accepting transitions. We call a set <sup>C</sup> <sup>⊆</sup> <sup>Q</sup> <sup>a</sup> *strongly connected component* (SCC) of <sup>A</sup> if for every pair of states q, q <sup>∈</sup> <sup>C</sup>, we have that <sup>q</sup> <sup>u</sup>−→ <sup>q</sup> for some <sup>u</sup> <sup>∈</sup> <sup>Σ</sup><sup>∗</sup> and <sup>q</sup> <sup>v</sup> −→ <sup>q</sup> for some <sup>v</sup> <sup>∈</sup> <sup>Σ</sup>∗, i.e., <sup>q</sup> and <sup>q</sup> can be reached by each other; by default, each state <sup>q</sup> <sup>∈</sup> <sup>Q</sup> reaches itself. C is a *maximal* SCC if it is *not* a proper subset of another SCC. *All* SCCs considered in the work are maximal. We call an SCC C *accepting* if there is a transition (q, a, q ) <sup>∈</sup> (<sup>C</sup> <sup>×</sup><sup>Σ</sup> <sup>×</sup>C)∩<sup>F</sup> and *nonaccepting* otherwise. We say that an SCC <sup>C</sup> is *reachable* from an SCC <sup>C</sup> if there exist <sup>q</sup> <sup>∈</sup> <sup>C</sup> and <sup>q</sup> <sup>∈</sup> <sup>C</sup> such that <sup>q</sup> <sup>u</sup>−→ <sup>q</sup> for some <sup>u</sup> <sup>∈</sup> <sup>Σ</sup>∗. An SCC <sup>C</sup> is *inherently weak* if either every cycle going through the C-states visits at least one accepting transition or none of the cycles visits an accepting transition. We say that an SCC C is *deterministic* if for every state <sup>q</sup> <sup>∈</sup> <sup>C</sup> and <sup>a</sup> <sup>∈</sup> <sup>Σ</sup>, we have <sup>|</sup>δ(q, a) <sup>∩</sup> <sup>C</sup>| ≤ 1. Note that a state q in a deterministic SCC C can have multiple successors for a letter a, but at most one successor remains in C.

Figure 1 shows an example of NBA we will use for our examples in the remainder of the paper; we depict the accepting transitions with a double arrow. Clearly, inside each SCC, depicted as a box, each state can be reached by any other state, and the SCCs are maximal. The SCC {q2, q3} is inherently weak and accepting, since every cycle takes an accepting transition; the SCC {q6} is also inherently weak, but nonaccepting, since every cycle never takes an accepting transition.

**Fig. 1.** An example of NBA.

The remaining two SCCs, i.e., {q0, q1} and {q4, q5}, are not inherently weak, since some cycle takes accepting transitions (like the cycle <sup>q</sup><sup>0</sup> <sup>a</sup>−→ <sup>q</sup>0) while others do not (like the cycle q<sup>0</sup> b −→ <sup>q</sup>0). Both SCCs contain an accepting transition, so they are accepting; the SCC {q0, q1} is clearly nondeterministic, while the SCC {q4, q5} is deterministic. Note that from <sup>q</sup><sup>5</sup> we have two transitions labelled by b, but only the transition q<sup>5</sup> b −→ <sup>q</sup><sup>4</sup> remains inside the SCC, while the other transition q<sup>5</sup> b −→ <sup>q</sup><sup>6</sup> leaves the SCC, so the SCC is still deterministic.

The following proposition is well known and is often used in prior works.

**Proposition 1.** *Let* <sup>A</sup> *be an NBA and* <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup>*. A run of* <sup>A</sup> *over* <sup>w</sup> *will eventually stay in an SCC. Moreover, if* <sup>w</sup> <sup>∈</sup> <sup>L</sup>(A)*, every accepting run of* <sup>A</sup> *over* <sup>w</sup> *will eventually stay in an accepting SCC.*

Proposition 1 is the key ingredient of our algorithm: it allows us to determinize the SCCs independently as <sup>L</sup>(A) is the union of the words whose runs stay in each accepting SCCs. In the remainder of the paper, we first present a translation from an NBA <sup>A</sup> to a DELA <sup>A</sup><sup>E</sup> based on the SCC decomposition of <sup>A</sup>. The obtained DELA <sup>A</sup><sup>E</sup> in fact can be converted to a *deterministic* Rabin automaton (DRA) <sup>A</sup><sup>R</sup> without blowing up states and transitions, i.e., we can just convert the coloring function and the acceptance formula of <sup>A</sup><sup>E</sup> to DRAs.

#### **3 Determinization Algorithms of SCCs**

Determinizing each SCC of A independently is not straightforward since it may be reached from the initial state only after reading a nonempty finite word; moreover, there can be words of different length leading to the SCC, entering through different states. To keep track of the different arrivals in an SCC at different times, we make use of run DAGs [24], that are a means to organize the runs of <sup>A</sup> over a word <sup>w</sup>. In this section, we first recall the concept of run DAGs and then describe how to determinize SCCs with their help.

**Definition 3.** *Let* <sup>A</sup> = (Q, ι, δ, F) *be an NBA and* <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup> *be a word. The run DAG* <sup>G</sup>A,w <sup>=</sup> V,E *of* <sup>A</sup> *over* <sup>w</sup> *is defined as follows: the set of vertices* <sup>V</sup> <sup>⊆</sup> <sup>Q</sup>×<sup>N</sup> *is defined as* <sup>V</sup> <sup>=</sup> <sup>l</sup>≥<sup>0</sup>(V<sup>l</sup>×{l}) *where* <sup>V</sup><sup>0</sup> <sup>=</sup> {ι} *and* <sup>V</sup><sup>l</sup>+1 <sup>=</sup> <sup>δ</sup>(Vl, w[l]) *for every* <sup>l</sup> <sup>∈</sup> <sup>N</sup>*; there is an edge* ( q,l, <sup>q</sup> , l ) <sup>∈</sup> <sup>E</sup> *if* <sup>l</sup> <sup>=</sup> <sup>l</sup>+1 *and* <sup>q</sup> <sup>∈</sup> <sup>δ</sup>(q, w[l])*.*

Intuitively, a state q at a level may occur in several runs and only one vertex is needed to represent it, i.e., the vertex q, who is said to be on level . Note that by definition, there are at most <sup>|</sup>Q<sup>|</sup> vertices on each level. An edge ( q,l, <sup>q</sup> , l + 1) is an <sup>F</sup>*-edge* if (q, w[l], q ) <sup>∈</sup> <sup>F</sup>. An infinite sequence of vertices <sup>γ</sup> <sup>=</sup> <sup>q</sup>0, <sup>0</sup> q1, <sup>1</sup>··· is called an <sup>ω</sup>*-branch* of <sup>G</sup>A,w if <sup>q</sup><sup>0</sup> <sup>=</sup> <sup>ι</sup> and for each <sup>∈</sup> <sup>N</sup>, we have ( <sup>q</sup>, , <sup>q</sup>+1, + 1) <sup>∈</sup> <sup>E</sup>. We can observe that there is a bijection between the set of runs of <sup>A</sup> on <sup>w</sup> and the set of <sup>ω</sup>-branches in <sup>G</sup>A,w. In fact, to a run <sup>ρ</sup> <sup>=</sup> <sup>q</sup>0q<sup>1</sup> ··· of <sup>A</sup> over <sup>w</sup> corresponds the <sup>ω</sup>-branch ˆ<sup>ρ</sup> <sup>=</sup> <sup>q</sup>0, <sup>0</sup> q1, <sup>1</sup>··· and, symmetrically, to an <sup>ω</sup>-branch <sup>γ</sup> <sup>=</sup> <sup>q</sup>0, <sup>0</sup> q1, <sup>1</sup>··· corresponds the run <sup>γ</sup><sup>ˆ</sup> <sup>=</sup> <sup>q</sup>0q<sup>1</sup> ··· . Thus <sup>w</sup> is accepted by <sup>A</sup> if and only if there exists an <sup>ω</sup>-branch in <sup>G</sup>A,w that takes <sup>F</sup>-edges infinitely often.

In the remainder of this section, we will introduce the algorithms for computing the successors of the current states inside different types of SCCs, with the help of run DAGs. We fix an NBA <sup>A</sup> = (Q, ι, δ, F) and a word <sup>w</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup>. We let <sup>Q</sup> <sup>=</sup> {q1,...,q<sup>n</sup>} and apply a total order on Q such that q<sup>i</sup> q<sup>j</sup> if i<j. Let <sup>S</sup> <sup>⊆</sup> <sup>Q</sup>, <sup>∈</sup> <sup>N</sup>, be the set of states reached at the level in the run DAG GA,w; we assume that this sequence <sup>S</sup>0, ··· , S, ··· is available as a *global* variable during the computations of every SCC where <sup>S</sup><sup>0</sup> <sup>=</sup> {ι} and <sup>S</sup>+1 <sup>=</sup> <sup>δ</sup>(S, w[ ]).

When determinizing the given NBA A, we classify its SCCs into three types, namely inherently weak SCCs (IWCs), deterministic-accepting SCCs (DACs) and nondeterministic-accepting SCCs (NACs). We assume that all DACs and NACs are *not* inherently weak, otherwise they will be classified as IWCs.

In our determinization construction, every level in GA,w corresponds to a state in our constructed DELA <sup>A</sup><sup>E</sup> while reading the <sup>ω</sup>-word <sup>w</sup>. Let <sup>m</sup> be the state of <sup>A</sup><sup>E</sup> at level . The computation of the successor m+1 of m for the letter w[ ] will be divided into the successor computation for states in IWCs, DACs and NACs independently. Then the successor m+1 is just the Cartesian product of these successors. In the remainder of this section, we present how to compute the successors for the states in each type of SCCs.

#### **3.1 Successor Computation Inside IWCs**

As we have seen, <sup>G</sup>A,w contains all runs of <sup>A</sup> over <sup>w</sup>, including those within DACs and NACs. Since we want to compute the successor only for IWCs, we focus on the states inside the IWCs and ignore other states in DACs and NACs. Let <sup>W</sup> be the set of states in all IWCs and WA <sup>⊆</sup> <sup>W</sup> be the set of states in all accepting IWCs.

For the run DAG <sup>G</sup>A,w, we use a pair of sets of states (P, O) <sup>∈</sup> <sup>2</sup><sup>W</sup> <sup>×</sup> <sup>2</sup>WA to represent the set of IWC states reached in <sup>G</sup>A,w at level . The set P is used to keep track of the states in W reached at level , while O, inspired by the breakpoint construction used in [31], keeps only the states reached in WA, that is, it is used to track the runs that stay in accepting IWCs. Since by definition each cycle inside an accepting IWC must visit an accepting transition, for each run tracked by O we do not need to remember whether we have taken an accepting transition: it suffices to know whether the run is still inside some accepting IWC or whether the run has left them.

We now show how to compute the sets (P, O) along w. For level 0, we simply set <sup>P</sup><sup>0</sup> <sup>=</sup> {ι} ∩ <sup>W</sup> and <sup>O</sup><sup>0</sup> <sup>=</sup> <sup>∅</sup>. For the other levels, given (P, O) at level <sup>∈</sup> <sup>N</sup>, the encoding (P+1, O+1) for the next level + 1 is defined as follows:

– <sup>P</sup>+1 <sup>=</sup> <sup>S</sup>+1 <sup>∩</sup>W, i.e., <sup>P</sup>+1 keeps track of the <sup>W</sup>-states reached at level + 1; – if <sup>O</sup> <sup>=</sup> <sup>∅</sup>, then <sup>O</sup>+1 <sup>=</sup> <sup>δ</sup>(O, w[ ]) <sup>∩</sup> WA, otherwise <sup>O</sup>+1 <sup>=</sup> <sup>P</sup>+1 <sup>∩</sup> WA.

Intuitively, the O-set keeps track of the runs that stay in the accepting IWCs. So if <sup>O</sup> <sup>=</sup> <sup>∅</sup>, then <sup>O</sup>+1 maintains the runs remaining in some accepting IWC; otherwise, <sup>O</sup> <sup>=</sup> <sup>∅</sup> means that at level all runs seen so far in the accepting IWCs have left them, so we can just start to track the new runs that entered the accepting IWCs but were not tracked yet.

2 - 3 4 On the right we show the fragment of the run DAG GA,a<sup>ω</sup> for the NBA A shown in Fig. 1 and its IWCs; we have <sup>W</sup> <sup>=</sup> {q2, q3, q6} and WA <sup>=</sup> {q2, q3}. The set <sup>P</sup> contains all states q at level ; the set O contains the underlined ones. As a concrete application of the construction given above, from <sup>P</sup><sup>3</sup> <sup>=</sup> {q2, q3} and <sup>O</sup><sup>3</sup> <sup>=</sup> <sup>δ</sup>(O2, a) <sup>∩</sup> WA <sup>=</sup> {q3}, at level 4 we get <sup>P</sup><sup>4</sup> <sup>=</sup> {q2, q3, q6} and <sup>O</sup><sup>4</sup> <sup>=</sup> <sup>δ</sup>(O3, a) <sup>∩</sup> WA <sup>=</sup> {q2}.

It is not difficult to see that checking whether w is accepted reduces to check whether the number of empty O-sets is finite. We assign color 1 to the transition from

(P, O) to (P+1, O+1) via w[ ] if <sup>O</sup> <sup>=</sup> <sup>∅</sup>, otherwise we assign color 2. Lemma <sup>1</sup> formalizes the relation between accepting runs staying in accepting IWCs and the colors we get from our construction.

**Lemma 1.** *(1) There exists an accepting run of* <sup>A</sup> *over* <sup>w</sup> *eventually staying in an accepting IWC if and only if we receive color* 1 *finitely many times when constructing the sequence* (P0, O0)···(P, O)··· *while reading* <sup>w</sup>*. (2) The number of possible* (P, O) *pairs is at most* 3|W<sup>|</sup> *.*

The proof idea is trivial: an accepting run ρ that stays in an accepting IWC will make the O-set contain ρ forever and we always get color 2 from some point on. A possible pair (P, O) can be seen as choosing a state from W, which can be from <sup>W</sup>\P, <sup>P</sup> <sup>∩</sup><sup>O</sup> and <sup>P</sup> \O, respectively. It thus gives at most 3|W<sup>|</sup> possibilities.

To ease the construction for the whole NBA A, we make the above computation of successors available as a function weakSucc, which takes as input a pair of sets (P, O) and a letter a, and returns the successor (P , O ) and the corresponding color <sup>c</sup> ∈ {1, <sup>2</sup>} for the transition ((P, O), a,(P , O )).

The construction we gave above works on all IWCs at the same time; considering IWCs separately does not improve the resulting complexity. If there are two accepting IWCs with n<sup>1</sup> and n<sup>2</sup> states, respectively, then the number of possible (P, O) pairs for the two IWCs is 3<sup>n</sup><sup>1</sup> and 3<sup>n</sup><sup>2</sup> , respectively. When combining the pairs for each IWC together, the resulting number of pairs in the Cartesian product is 3<sup>n</sup><sup>1</sup> <sup>×</sup> <sup>3</sup><sup>n</sup><sup>2</sup> = 3<sup>n</sup>1+n<sup>2</sup> , which is the same as considering them together. On the other hand, for each accepting IWC, we need to use two colors, so we need 2 ·<sup>i</sup> colors in total for <sup>i</sup> accepting IWCs, instead of just two colors by operating on all IWCs together. Hence, we prefer to work on all IWCs at once.

#### **3.2 Successor Computation Inside DACs**

In contrast to IWCs, we do not work on all DACs at once but we process each DAC separately. This is because there may be nondeterminism between DACs: a run in a DAC may branch into multiple runs that jump to different DACs, which requires us to resort to a Safra-Piterman's construction [33,36] when considering all DACs at once. Working on each DAC separately, instead, allows us to take advantage of the internal determinism: for a given DAC D, the transition relation <sup>δ</sup> inside <sup>D</sup>, denoted as <sup>δ</sup><sup>D</sup> = (<sup>D</sup> <sup>×</sup> <sup>Σ</sup> <sup>×</sup> <sup>D</sup>) <sup>∩</sup> <sup>δ</sup>, is now deterministic.

Although every run ρ entering D can have only one successor in D, ρ may just leave D while new runs can enter D, which makes it difficult to check whether there exists an accepting run that remains trapped into D. In order to identify accepting runs staying in D, we identify the following two rules for distinguishing runs that come to D by means of *unique* labelling numbers: **(1)** the runs already in D have precedence over newly entering runs, thus the latter get assigned a higher number. In practice, the labelling keeps track of the relative order of entering D, thus the lower the labelling value is, the earlier the run came to D; **(2)** when two runs in D merge, we only keep the run that came to D earlier, i.e., the run with lower number. If two runs enter D at the same time, we let them enter according to the total state order for their respective entry states.

We use a level-labelling function <sup>g</sup> : <sup>D</sup> → {1, ··· , <sup>2</sup>·|D|}∪ {∞} to encode the set of D-states reached at level of the run DAG <sup>G</sup>A,w. Here we use <sup>g</sup>(q) = <sup>∞</sup> to indicate that the state <sup>q</sup> <sup>∈</sup> <sup>D</sup> is not reached by <sup>A</sup> at level .

At level 0, we set <sup>g</sup>0(q) = <sup>∞</sup> for every state <sup>q</sup> <sup>∈</sup> <sup>D</sup> \ {ι}, and <sup>g</sup>0(ι) = 1 if <sup>ι</sup> <sup>∈</sup> <sup>D</sup>. Note that the SCC that <sup>ι</sup> resides in can be an IWC, a DAC or a NAC.

For a given level-labelling function <sup>g</sup>, we will make { <sup>q</sup> <sup>∈</sup> <sup>D</sup> <sup>|</sup> <sup>g</sup>(q) <sup>=</sup> ∞ } <sup>=</sup> <sup>S</sup> <sup>∩</sup> <sup>D</sup> hold, i.e., tracing correctly the set of <sup>D</sup>-states reached by <sup>A</sup> at level ; we denote the set <sup>g</sup>(D) \ {∞} by <sup>β</sup>(g), so <sup>β</sup>(g) is the set of unique labelling numbers at level . By the construction given below about how to generate g+1 from g on reading w[ ], we ensure that <sup>β</sup>(g) ⊆ {1, ··· , <sup>2</sup> · |D|} for all <sup>∈</sup> <sup>N</sup>.

We now present how to compute the successor level-labelling function g+1 of g on letter w[ ]. The states reached by <sup>A</sup> at level + 1, i.e., <sup>S</sup>+1 <sup>∩</sup>D, may come from two sources: some state may come from states not in D via transitions in <sup>δ</sup> \ <sup>δ</sup>D; some other via <sup>δ</sup><sup>D</sup> from states in <sup>S</sup> <sup>∩</sup> <sup>D</sup>. In order to generate <sup>g</sup>+1, we first compute an intermediate level-labelling function g +1 as follows.

1. To obey Rule **(2)**, for every state <sup>q</sup> <sup>∈</sup> <sup>δ</sup>D(S <sup>∩</sup> <sup>D</sup>, w[ ]), we set

$$\mathfrak{g}'\_{\ell+1}(q') = \min \{ \mathfrak{g}\_{\ell}(q) \mid q \in S\_{\ell} \cap \mathbf{D} \land \delta\_D(q, w[\ell]) = q' \}.$$

That is, when two runs merge, we only keep the run with the lower labelling number, i.e., the run entered in D earlier.

2. To respect Rule **(1)**, we set g +1(q ) = <sup>|</sup>D|+<sup>i</sup> for the <sup>i</sup>-th newly entered state <sup>q</sup> <sup>∈</sup> (S+1 <sup>∩</sup> <sup>D</sup>) \ <sup>δ</sup>D(S <sup>∩</sup> <sup>D</sup>, w[ ]) and the states q are ordered by the total order of the states. Since every state in <sup>δ</sup>D(S <sup>∩</sup> <sup>D</sup>, w[ ]) is on a run that already entered D, its labelling has already been determined by the case 1.

It is easy to observe that in order to compute the transition relation between two consecutive levels, we only need to know the labelling at the previous level. More precisely, we do not have to know the exact labelling numbers, since it suffices to know their relative order. Therefore, we can compress the level-labelling g +1 to <sup>g</sup>+1 as follows. Let ord: <sup>β</sup>(g +1) → {1, ··· , <sup>|</sup>β(g +1)|} be the function that maps each labelling value in β(g +1) to its relative position once the values in β(g +1) have been sorted in ascending order. For instance, if <sup>β</sup>(g +1) = {2, <sup>4</sup>, <sup>7</sup>}, then ord <sup>=</sup> {<sup>2</sup> → <sup>1</sup>, <sup>4</sup> → <sup>2</sup>, <sup>7</sup> → <sup>3</sup>}. Then we set <sup>g</sup>+1(q) = ord(g +1(q)) for each <sup>q</sup> <sup>∈</sup> <sup>S</sup>+1 <sup>∩</sup> <sup>D</sup>, and <sup>g</sup>+1(q ) = <sup>∞</sup> for each <sup>q</sup> <sup>∈</sup> <sup>D</sup> \ <sup>S</sup>+1. In this way, all level-labelling functions <sup>g</sup> we use are such that <sup>β</sup>(g) ⊆ {1, ··· , <sup>|</sup>D|}.

The intuition behind the use of these level-labelling functions is that, if we always see a labelling number h in the intermediate level-labelling g for all <sup>≥</sup> <sup>k</sup> after some level k, we know that there is a run that eventually stays in D and is eventually always labelled with h. To check whether this run also visits infinitely many accepting transitions, we will color every transition e = (g, w[ ], g+1). To decide what color to assign to e, we first identify which runs have merged with others or got out of D (corresponding to *bad* events and *odd* colors) and which runs still continue to stay in D and take an accepting transition (corresponding to *good* events and *even* colors).

The bad events correspond to the discontinuation of labelling values between g and g +1, defined as <sup>B</sup>(e) = <sup>β</sup>(g) \ <sup>β</sup>(g +1). Intuitively, if a labelling value k exists in the set B(e), then the run ρ associated with labelling k merged with a run with lower labelling value k < k, or ρ left the DAC D. The good events correspond to the occurrence of accepting transitions in some runs, whose labelling we collect into <sup>G</sup>(e) = { <sup>k</sup> <sup>∈</sup> <sup>β</sup>(g) | ∃(q, w[ ], q ) <sup>∈</sup> F.g(q) = <sup>g</sup> +1(q ) = <sup>k</sup> <sup>=</sup> ∞ }. In practice, a labelling value <sup>k</sup> in <sup>G</sup>(e) indicates that we have seen a run with labelling k that visits an accepting transition. We then let B(e) = <sup>B</sup>(e) ∪ {|D<sup>|</sup> + 1} and <sup>G</sup>(e) = <sup>G</sup>(e) ∪ {|D<sup>|</sup> + 1} where the value <sup>|</sup>D<sup>|</sup> + 1 is used to indicate that no bad (i.e., no run merged or left the DAC) or no good (i.e., no run took an accepting transition) events happened, respectively.

In order to declare a sequence of labelling functions as accepting, we want the good events to happen infinitely often and bad events to happen only finitely often, when the runs with bad events have a labelling number lower than that of the runs with good events. So we assign the color <sup>c</sup> = min{<sup>2</sup> · min <sup>B</sup>(e) <sup>−</sup> <sup>1</sup>, <sup>2</sup> · min <sup>G</sup>(e)} to the transition <sup>e</sup>. Since the labelling numbers are in {1, ··· , <sup>|</sup>D|}, we have that <sup>c</sup> ∈ {1, ··· , <sup>2</sup> · |D<sup>|</sup> + 1}. The intuition why we assign colors in this way is given as the proof idea of the following lemma.

**Lemma 2.** *(1) An accepting run of* <sup>A</sup> *over* <sup>w</sup> *eventually stays in the DAC* <sup>D</sup> *if and only if the minimal color* c *we receive infinitely often is even. (2) The number of possible labelling functions* <sup>g</sup> *is at most* <sup>3</sup> · |D|!*.*

The proof idea is as follows: an accepting run ρ on the word w that stays in D will have stable labelling number, say <sup>k</sup> <sup>≥</sup> 1, after some level since the labelling value cannot increase by construction and is finite. So all runs on w that have labelling values lower than k will not leave D: if they would leave or just merge with other runs, their labelling value vanishes, so ord would decrease the value for ρ. This implies that the color we receive afterwards infinitely often is either 1) an odd color larger than 2k, due to vanishing runs with value at least k + 1 or simply because no bad or good events occur, or 2) an even color at most 2k, depending on whether there is some run with value smaller than ρ also taking accepting transitions. Thus the minimum color occurring infinitely often is even. The number of labelling functions g is bounded by |D<sup>|</sup> <sup>i</sup>=0 |D<sup>|</sup> i · <sup>i</sup>! <sup>≤</sup> <sup>3</sup> · |D|!.

The fragment of the DAG GA,a<sup>ω</sup> shown on the right is relative to the only DAC <sup>D</sup> <sup>=</sup> {q4, q5}. The value of g (q), g(q) and the corresponding ord is given by the mapping near each state q; as a concrete application of the construction given above, consider how to get <sup>g</sup><sup>4</sup> from <sup>g</sup>3, defined as <sup>g</sup>3(q4) = 1 and <sup>g</sup>3(q5) = <sup>∞</sup>: since <sup>q</sup><sup>5</sup> <sup>∈</sup> <sup>δ</sup>D(S<sup>3</sup> <sup>∩</sup> <sup>D</sup>, a), according to case <sup>1</sup> we define g <sup>4</sup>(q5) = 1 because q<sup>5</sup> = δD(q4, a) and g3(q4) = 1;

3 - 4 5 q4, 3 → 1 q5, 1 → 1 q4, 3 → 2 q4, 1 → 1 q5, 2 → 2

since <sup>q</sup><sup>4</sup> <sup>∈</sup> (S<sup>4</sup> <sup>∩</sup> <sup>D</sup>) \ <sup>δ</sup>D(S<sup>3</sup> <sup>∩</sup> <sup>D</sup>, a), then case <sup>2</sup> applies, so <sup>g</sup> <sup>4</sup>(q4) = 3. The function ord is ord = [1 → <sup>1</sup>, <sup>3</sup> → 2], thus we get <sup>g</sup>4(q4) = 2 and <sup>g</sup>4(q5) = 1. As bad/good sets for the transition <sup>e</sup> <sup>=</sup> <sup>g</sup><sup>3</sup> <sup>a</sup>−→ <sup>g</sup>4, we have <sup>B</sup>(e) = ∅∪{3} while <sup>G</sup>(e) = {1}∪{3}, so the resulting color is 2.

Again, we make the above computation of successors available as a function detSucc, which takes as input the DAC D, a labelling g and a letter a, and returns the successor labelling <sup>g</sup> and the color <sup>c</sup> ∈ {1, ··· , <sup>2</sup> · |D<sup>|</sup> + 1}.

#### **3.3 Successor Computation Inside NACs**

The computation of the successor inside a NAC is more involved since runs can branch, so it is more difficult to check whether there exists an accepting run. To identify accepting runs, researchers usually follow the Safra-Piterman's idea [33,36] to give the runs that take more accepting transitions the precedence over other runs that join them. We now present how to compute labelling functions encoding this idea for NACs, instead of the whole NBA. Differently to the previous case about DACs, the labelling functions we use here use lists of numbers, instead of single numbers, to keep track of the branching, merging and new incoming runs. This can be seen as a generalization of the numbered brackets used in [35] to represent ordinary Safra-Piterman's trees. Differently from this construction, in our setting the main challenge we have to consider is how to manage correctly the newly entering runs, which are simply not occurring in [35] since there the whole NBA is considered. The fact that runs can merge, instead, is a common aspect, while the fact that a run ρ leaves the current NAC can be treated similarly to dying out runs in [35]. Below we assume that N is a given NAC; we denote by <sup>δ</sup><sup>N</sup> = (<sup>N</sup> <sup>×</sup> <sup>Σ</sup> <sup>×</sup> <sup>N</sup>) <sup>∩</sup> <sup>δ</sup> the transition function <sup>δ</sup> inside <sup>N</sup>.

To manage the branching and merging of runs of <sup>A</sup> over <sup>w</sup> inside a NAC, and to keep track of the accepting transitions taken so far, we use level-labelling functions as for the DAC case. For a given NAC N, the functions we use have lists of natural numbers as codomain; more precisely, let L<sup>N</sup> be the set of lists taking value in the set {1, ··· , <sup>2</sup> · |N|}, where a list is a finite sequence of values in ascending order. Given two lists [v1, ··· , vk] and [v <sup>1</sup>, ··· , v k-], we say that [v1, ··· , vk] is a prefix of [v <sup>1</sup>, ··· , v k- ] if 1 <sup>≤</sup> <sup>k</sup> <sup>≤</sup> <sup>k</sup> and for each 1 <sup>≤</sup> <sup>j</sup> <sup>≤</sup> <sup>k</sup>, we have v<sup>j</sup> = v <sup>j</sup> . Note that the empty list *is not* a prefix of any list. Given two lists [v1, ··· , vk] and [v <sup>1</sup>, ··· , v k- ], we denote by [v1, ··· , vk][v <sup>1</sup>, ··· , v k- ] their concatenation, that is the list [v1, ··· , vk, v <sup>1</sup>, ··· , v k- ]. Moreover, we define a total order on lists as follows: given two lists [v1, ··· , vk] and [v <sup>1</sup>, ··· , v k- ], we order them by padding the shorter of the two with ∞ in the rear, so to make them of the same length, and then by comparing them by the usual lexicographic order. This means, for instance, that the empty list [] is the largest list and that [1, 3, 5] is smaller than [1, 3] but larger than [1, 2]. The lists help to keep track of the branching history from their prefixes, such as [1, 2] is branched from [1].

As done for DACs, we use a level-labelling function <sup>t</sup> : <sup>N</sup> → L<sup>N</sup> to encode the set of <sup>N</sup>-states reached in the run DAG <sup>G</sup>A,w at level . We denote by β(t) the set of non-empty lists in the image of <sup>t</sup>, that is, <sup>β</sup>(t) = { <sup>t</sup>(q) <sup>|</sup> <sup>q</sup> <sup>∈</sup> <sup>N</sup> <sup>∧</sup>t(q) = [] }. We use the empty list [] for the states in N that do not occur in the vertexes of <sup>G</sup>A,w at level , so <sup>β</sup>(t) contains only lists associated with states that <sup>A</sup> is currently located at. Similarly to the other types of SCCs, at level 0, we set <sup>t</sup>0(ι) = [1] if <sup>ι</sup> <sup>∈</sup> <sup>N</sup>, and <sup>t</sup>0(q) = [] for each state <sup>q</sup> <sup>∈</sup> <sup>N</sup> \ {ι}.

To define the transition from t to t+1 through the letter w[ ], we use again an intermediate level-labelling function t +1 that we construct step by step as follows. We start with t +1(q) = [] for each <sup>q</sup> <sup>∈</sup> <sup>N</sup> and with the set of unused numbers <sup>U</sup> <sup>=</sup> { <sup>u</sup> <sup>≥</sup> <sup>1</sup> <sup>|</sup> u /<sup>∈</sup> <sup>β</sup>(t) }, i.e., the numbers not used in <sup>β</sup>(t).

1. For every state <sup>q</sup> <sup>∈</sup> <sup>δ</sup><sup>N</sup> (S∩N, w[ ]), let P<sup>q</sup>- <sup>=</sup> { <sup>q</sup> <sup>∈</sup> <sup>S</sup>∩<sup>N</sup> <sup>|</sup> (q, w[ ], q ) <sup>∈</sup> <sup>δ</sup><sup>N</sup> } be the set of currently reached predecessors of q , and C<sup>q</sup>- <sup>=</sup> <sup>∅</sup>. For each <sup>q</sup> <sup>∈</sup> P<sup>q</sup>- , if (q, w[ ], q ) <sup>∈</sup> <sup>F</sup>, then we add <sup>t</sup>(q)[u] to <sup>C</sup><sup>q</sup>- , where u = minU, and we remove u from U, so that each number in U is used only once; otherwise, for (q, w[ ], q ) <sup>∈</sup> <sup>δ</sup><sup>N</sup> \F, we add <sup>t</sup>(q) to <sup>C</sup><sup>q</sup>- . Lastly, we set t +1(q ) = min C<sup>q</sup>- , where the minimum is taken according to the list order.

Intuitively, if a run ρ can branch into two kinds of runs, some via accepting transitions and some others via nonaccepting transitions at level + 1, then we let those from nonaccepting transitions inherit the labelling from ρ, i.e., t(ρ[ ]); for the runs taking accepting transitions we create a new labelling t(ρ[ ])[u]. In this way, the latter get precedence over the former. Moreover, if a run ρ has received multiple labelling values, collected in C<sup>ρ</sup>[+1], then it will keep the smallest one, by t +1(ρ[ + 1]) = min C<sup>ρ</sup>[+1].

2. For each state <sup>q</sup> <sup>∈</sup> (S+1 <sup>∩</sup> <sup>N</sup>) \ <sup>δ</sup><sup>N</sup> (S <sup>∩</sup> <sup>N</sup>, w[ ]) taken according to the state order -, we first set t +1(q )=[u], where u = minU, and then we remove u from U, so we do not reuse the same values. That is, we give the newly entered runs lower precedence than those already in N, by means of the larger list [u].

We now need to prune the lists in β(t +1) and recognize good and bad events. Similarly to DACs, a bad event means that a run has left N or has been merged with runs with smaller labelling, which is indicated by a discontinuation of a labelling between β(t) and β(t +1). For the transition <sup>e</sup> = (t, w[ ],t+1) we are constructing, to recognize bad events, we put into the set B(e) the number <sup>|</sup>N<sup>|</sup> + 1 and all numbers in <sup>β</sup>(t) that have disappeared in <sup>β</sup>(<sup>t</sup> +1), that is, <sup>B</sup>(e) = {|N<sup>|</sup> + 1}∪{ <sup>v</sup> <sup>∈</sup> <sup>N</sup> <sup>|</sup> <sup>v</sup> occurs in <sup>β</sup>(t) but not in <sup>β</sup>(<sup>t</sup> +1) }.

Differently from the good events for DACs, which require to visit an accepting transition, we need all runs branched from a run to visit an accepting transition, which is indicated by the fact that there are no states labelled by t +1 with some list <sup>l</sup> <sup>∈</sup> <sup>β</sup>(t) but there are extensions of <sup>l</sup> associated with some state. To recognize good events, let <sup>G</sup>(e) = {|N<sup>|</sup> + 1} and <sup>t</sup> +1 be another intermediate labelling function. For each <sup>q</sup> <sup>∈</sup> <sup>S</sup>+1 <sup>∩</sup> <sup>N</sup>, consider the list <sup>t</sup> +1(q ): if for each prefix [v1, ··· <sup>v</sup>k] of <sup>t</sup> +1(q ) we have [v1, ··· <sup>v</sup>k] <sup>∈</sup> <sup>β</sup>(<sup>t</sup> +1), then we set t +1(q ) = t +1(q ). Otherwise, let [v1, ··· <sup>v</sup>k¯] <sup>∈</sup>/ <sup>β</sup>(<sup>t</sup> +1) be the shortest prefix of t +1(q ) not in β(t +1); we set t +1(q )=[v1, ··· <sup>v</sup>k¯] and add <sup>v</sup>k¯ to <sup>G</sup>(e). Setting <sup>t</sup> +1(q ) = [v1, ··· <sup>v</sup>k¯] in fact corresponds, in the Safra's construction [36], to the removal of all children of a node N for which the union of the states in the children is equal to the states in N. Lastly, similarly to the DAC case, we set t+1(q) = ord(t +1(q)) for each <sup>q</sup> <sup>∈</sup> <sup>S</sup>+1 <sup>∩</sup> <sup>N</sup> and <sup>t</sup>+1(q ) = [] for each <sup>q</sup> <sup>∈</sup> <sup>N</sup> \ <sup>S</sup>+1, where ord([v1, ··· , vk]) = [ord(v1), ··· , ord(vk)]. Regarding the color to assign to the transition <sup>e</sup>, we just assign the color <sup>c</sup> = min{<sup>2</sup> · min <sup>G</sup>(e), <sup>2</sup> · min <sup>B</sup>(e) <sup>−</sup> <sup>1</sup>}.

**Lemma 3.** *(1) An accepting run of* <sup>A</sup> *over* <sup>w</sup> *eventually stays in the NAC* <sup>N</sup> *if and only if the minimal color* c *we receive infinitely often is even. (2) The number of possible labelling functions* <sup>t</sup> *is at most* <sup>2</sup> · (|N|!)<sup>2</sup>*.*

Similarly to DACs, also for NACs we have handled each NAC independently. The reason for this is that this potentially reduces the complexity of the single cases: assume that we have two NACs N<sup>1</sup> and N2. If we apply the Safra-Piterman's construction directly to <sup>N</sup><sup>1</sup> <sup>∪</sup> <sup>N</sup>2, we might incur in the worst-case complexity 2 · ((|N1<sup>|</sup> <sup>+</sup> <sup>|</sup>N2|)!)<sup>2</sup>, as mentioned in the introduction. However, if we determinize them separately, then the worst complexity for each NAC N<sup>i</sup> is <sup>2</sup> ·(Ni!)<sup>2</sup>, for an overall 4 ·(|N1|! · |N2|!)<sup>2</sup>, much smaller than 2 ·((|N1<sup>|</sup> <sup>+</sup> <sup>|</sup>N2|)!)<sup>2</sup>.

As usual, we make the above construction available as a function nondetSucc, which takes as input the NAC N, a labelling t and a letter a, and returns the successor labelling t and the corresponding color <sup>c</sup> ∈ {1, ··· , <sup>2</sup> · |N<sup>|</sup> + 1}.

0 - 1 2 q0, [1] q0, [1, 2] q1, [1] q0, [1] q1, [1] Similarly to the constructions for other SCCs, we show on the right the fragment of run DAG GA,a<sup>ω</sup> for the NAC <sup>N</sup> <sup>=</sup> {q0, q1}, with <sup>q</sup><sup>0</sup> q1. The construction of t<sup>1</sup> is easy, so consider its <sup>a</sup>-successor <sup>t</sup>2: we start with <sup>U</sup> <sup>=</sup> {3, <sup>4</sup>, ···}; for <sup>q</sup>0, we have <sup>P</sup><sup>q</sup><sup>0</sup> <sup>=</sup> {q0, q1} and <sup>C</sup><sup>q</sup><sup>0</sup> <sup>=</sup> {[1, <sup>2</sup>, 3], [1]}, hence t <sup>2</sup>(q0) = [1, <sup>2</sup>, 3]. For <sup>q</sup>1, we get <sup>P</sup><sup>q</sup><sup>1</sup> <sup>=</sup> {q0} and <sup>C</sup><sup>q</sup><sup>1</sup> <sup>=</sup> {[1, 2]}, so <sup>t</sup> <sup>2</sup>(q1) = [1, 2]. Thus, for e = (t1, w[1],t2), we have <sup>B</sup>(e) = {3} while <sup>G</sup>(e) = {1, <sup>3</sup>}, since both lists in β(t <sup>2</sup>) = {[1, 2], [1, <sup>2</sup>, 3]} are missing the prefix [1], so we get <sup>t</sup>2(q0) = <sup>t</sup>2(q1) = [1] and color c = 2.

#### **4 Determinization of NBAs to DELAs**

In this section, we fix an NBA <sup>A</sup> = (Q, ι, δ, F) with <sup>n</sup> <sup>=</sup> <sup>|</sup>Q<sup>|</sup> states and we show how to construct an equivalent DELA <sup>A</sup><sup>E</sup> = (QE, ιE, δE, Γ <sup>E</sup>, <sup>p</sup>E,Acc<sup>E</sup>), by using the algorithms developed in the previous section. We assume that A has {D1, ··· , <sup>D</sup>d} as set of DACs and {N1, ··· , <sup>N</sup>k} as set of NACs.

When computing the successor for each type of SCCs while reading a word w, we just need to know the set S of states reached at the current level and the letter <sup>a</sup> <sup>∈</sup> <sup>Σ</sup> to read. We can ignore the actual level , since if S = S- , then their successors under the same letter will be the same. As mentioned before, every state of <sup>A</sup><sup>E</sup> corresponds to a level of <sup>G</sup>A,w. We call a state of <sup>A</sup><sup>E</sup> <sup>a</sup> *macrostate* and a run of <sup>A</sup><sup>E</sup> <sup>a</sup> *macrorun*, to distinguish them from those of <sup>A</sup>.

**Macrostates** Q<sup>E</sup>**.** Each macrostate consists of the pair (P, O) for encoding the states in IWCs, a labelling function <sup>g</sup><sup>i</sup> : <sup>D</sup><sup>i</sup> → {1, ··· , <sup>|</sup>D<sup>i</sup> |} ∪ {∞} for the states of each DAC D<sup>i</sup> and a labelling function t <sup>j</sup> : <sup>N</sup><sup>j</sup> → LN<sup>j</sup> for each NAC <sup>N</sup><sup>j</sup> , without the explicit level number. The initial macrostate ι <sup>E</sup> of <sup>A</sup><sup>E</sup> is the encoding of level 0, defined as the set {(P0, O0)}∪{ <sup>g</sup><sup>i</sup> <sup>0</sup> <sup>|</sup> <sup>D</sup><sup>i</sup> is a DAC }∪{ t j <sup>0</sup> <sup>|</sup> <sup>N</sup><sup>j</sup> is a NAC }, where each encoding for the different types of SCCs is the one for level 0.

We note that ι must be present in one type of SCCs. In particular, if ι is a transient state, then {ι} is classified as an IWC.

**Transition Function** <sup>δ</sup><sup>E</sup>**.** Let <sup>m</sup> be the current macrostate in <sup>Q</sup><sup>E</sup> and <sup>a</sup> <sup>∈</sup> <sup>Σ</sup> be the letter to read. Then we define m = δ<sup>E</sup>(m, a) as follows.


Note that the set <sup>S</sup> of the current states of <sup>A</sup> used by the different successor functions is implicitly given by the sets <sup>P</sup>, { <sup>q</sup> <sup>∈</sup> <sup>D</sup><sup>i</sup> <sup>|</sup> <sup>g</sup><sup>i</sup> (q) <sup>=</sup> ∞ } for each DAC <sup>D</sup><sup>i</sup> and { <sup>q</sup> <sup>∈</sup> <sup>N</sup><sup>j</sup> <sup>|</sup> <sup>t</sup> <sup>j</sup> (q) = [] } for each NAC <sup>N</sup><sup>j</sup> in the current macrostate <sup>m</sup>.

**Color Set** Γ <sup>E</sup> **and Coloring Function** p<sup>E</sup>. From the constructions given in Sect. 3, we have two colors from the IWCs, 2 · |D<sup>i</sup> <sup>|</sup> + 1 colors for each DAC <sup>D</sup><sup>i</sup> , and 2 · |N<sup>j</sup> <sup>|</sup> + 1 colors for each NAC <sup>N</sup><sup>j</sup> , yielding a total of at most 3 · |Q<sup>|</sup> colors. Thus we set <sup>Γ</sup> <sup>E</sup> <sup>=</sup> {0, <sup>1</sup>, ··· , <sup>3</sup> · |Q|} with color 0 not being actually used.

Regarding the color to assign to each transition, we need to ensure that the colors returned by the single SCCs are treated separately, so we transpose them. For a transition e = (m, a, m ) <sup>∈</sup> <sup>δ</sup><sup>E</sup>, we define the coloring function <sup>p</sup><sup>E</sup> as follows.


Intuitively, we make the colors returned for each SCC not overlap with those of other SCCs without changing their relative order. In this way, we can still independently check whether there exists an accepting run staying in an SCC.

**Acceptance Formula** AccE**.** We now define the acceptance AccE, which is basically the *disjunction* of the acceptance formula for each different types of SCCs, after transposing them. Regarding the IWCs, we trivially define Acc<sup>E</sup> <sup>W</sup> <sup>=</sup> Fin(1), since this is the acceptance formula for IWCs; as said before, color 0 is not used.

For DACs and NACs, the definition is more involved. For instance, regarding the DAC D<sup>i</sup> , we know that all returned colors are inside {1, ··· , <sup>2</sup> · |D<sup>i</sup> | + 1}. According to Lemma 2, an accepting run eventually stays in D<sup>i</sup> if and only if the minimum color that we receive infinitely often is even. Thus, the acceptance formula for the above lemma is parity(|D<sup>i</sup> |) = -<sup>|</sup>Di<sup>|</sup> <sup>c</sup>=1( c <sup>j</sup>=1 Fin(2<sup>j</sup> <sup>−</sup> 1) <sup>∧</sup> Inf(2c)). Let b<sup>i</sup> = 2+ h<i(2 · |D<sup>h</sup><sup>|</sup> + 1) be the base number for the colors of <sup>D</sup><sup>i</sup> , which is also the number of colors already used by IWCs and the DACs D<sup>h</sup> with h<i. Since we have added the base number b<sup>i</sup> to every color of D<sup>i</sup> , we then have the acceptance formula Acc<sup>E</sup> <sup>D</sup><sup>i</sup> = -<sup>|</sup>Di<sup>|</sup> <sup>c</sup>=1( c <sup>j</sup>=1 Fin(2<sup>j</sup> <sup>−</sup> 1 + <sup>b</sup>i) <sup>∧</sup> Inf(2<sup>c</sup> <sup>+</sup> <sup>b</sup>i)).

For each NAC <sup>N</sup><sup>j</sup> , the colors we receive are in {1, ··· , <sup>2</sup> · |N<sup>j</sup> <sup>|</sup> + 1}. Let b<sup>j</sup> =2+ <sup>1</sup>≤h≤<sup>d</sup>(2 · |D<sup>h</sup><sup>|</sup> + 1) + h<j (2 · |N<sup>j</sup> <sup>|</sup> + 1) be the base number for <sup>N</sup><sup>j</sup> . Similarly to the DAC case, for each NAC N<sup>j</sup> , we let Acc<sup>E</sup> <sup>N</sup><sup>j</sup> = -<sup>|</sup>N<sup>j</sup> <sup>|</sup> <sup>c</sup>=1( c <sup>i</sup>=1 Fin(2i<sup>−</sup> 1 + <sup>b</sup><sup>j</sup> ) <sup>∧</sup> Inf(2<sup>c</sup> <sup>+</sup> <sup>b</sup><sup>j</sup> )).

The acceptance formula for <sup>A</sup><sup>E</sup> is Acc<sup>E</sup> <sup>=</sup> Acc<sup>E</sup> <sup>W</sup> ∨ d <sup>i</sup>=1 Acc<sup>E</sup> <sup>D</sup><sup>i</sup> ∨ k <sup>j</sup>=1 Acc<sup>E</sup> <sup>N</sup><sup>j</sup> .

Consider again the NBA A given in Fig. 1 and its various SCCs. As acceptance formula for the constructed DELA, it is the disjunction of the formulas Acc<sup>E</sup> <sup>W</sup> <sup>=</sup> Fin(1); Acc<sup>E</sup> <sup>D</sup> = -2 <sup>c</sup>=1( c <sup>j</sup>=1 Fin(2<sup>j</sup> <sup>−</sup> 1 + 2) <sup>∧</sup> Inf(2<sup>c</sup> + 2)), since the base number for D is 2; and Acc<sup>E</sup> <sup>N</sup> = -2 <sup>c</sup>=1( c <sup>i</sup>=1 Fin(2<sup>i</sup> <sup>−</sup> 1 + 7) <sup>∧</sup> Inf(2<sup>c</sup> + 7)), since 7 is the base number for N.

The construction given in this section is correct, as stated by Theorem 1.

**Theorem 1.** *Given an NBA* <sup>A</sup> *with* <sup>n</sup> <sup>=</sup> <sup>|</sup>Q<sup>|</sup> *states, let* <sup>A</sup><sup>E</sup> *be the DELA constructed by our method. Then (1)* <sup>L</sup>(A<sup>E</sup>) = <sup>L</sup>(A) *and (2)* <sup>A</sup><sup>E</sup> *has at most* <sup>3</sup>|W<sup>|</sup> · <sup>d</sup> <sup>i</sup>=1 <sup>3</sup> · |D<sup>i</sup> |! · <sup>k</sup> <sup>j</sup>=1 <sup>2</sup> · (|N<sup>i</sup> |!)2 *macrostates and* 3n + 1 *colors.*

Obviously, if <sup>d</sup> <sup>=</sup> <sup>k</sup> = 0, <sup>A</sup> is a weak BA [32]. If <sup>k</sup> = 0, <sup>A</sup> is an elevator BA, a new class of BAs recently introduced in [19] which have only IWCs and DACs, a strict superset of semi-deterministic BAs (SDBAs) [10]. SDBAs will behave *deterministically* after seeing acceptance transitions. An elevator BA that is not an SDBA can be obtained from the NBA <sup>A</sup> shown in Fig. <sup>1</sup> by setting <sup>q</sup><sup>2</sup> as initial state and by removing all states and transitions relative to the NAC.

It is known that the lower bound for determinizing SDBAs is n! [14,27]. Then the determinization complexity of weak BAs and elevator BAs can be easily improved exponentially as follows.

**Corollary 1.** *(1) Given a weak B¨uchi automaton* <sup>A</sup> *with* <sup>n</sup> <sup>=</sup> <sup>|</sup>Q<sup>|</sup> *states, the DELA constructed by our algorithm has at most* 3<sup>n</sup> *macrostates. (2) Given an* *elevator B¨uchi automaton* <sup>A</sup> *with* <sup>n</sup> <sup>=</sup> <sup>|</sup>Q<sup>|</sup> *states, our algorithm constructs a DELA with* Θ(n!) *macrostates; it is asymptotically optimal.*

The upper bound for determinizing weak BAs is already known [5]. Elevator BAs are, to the best of our knowledge, the *largest* subclass of NBAs known so far to have determinization complexity Θ(n!).

The acceptance formula for an SCC can be seen as a parity acceptance formula with colors being shifted to different ranges. A parity automaton can be converted into a Rabin one without blow-up of states and transitions [16]. Since Acc<sup>E</sup> is a disjunction of parity acceptance formulas, Theorem 2 then follows.

**Theorem 2.** *Let* <sup>A</sup><sup>E</sup> *be the constructed DELA for the given NBA* <sup>A</sup>*. Then* <sup>A</sup><sup>E</sup> *can be converted into a DRA* <sup>A</sup><sup>R</sup> *without blow-up of states and transitions.*

**Translation to Deterministic Parity Automata (DPAs).** We note that there is an *optimal* translation from a DRA to a DPA described in [7], implemented in Spot via the function acd transform [8].

#### **5 Empirical Evaluation**

To analyze the effectiveness of our Divide-and-Conquer determinization construction proposed in Sect. 3, we implemented it in our tool COLA, which is built on top of Spot [12]. The source code of COLA is publicly available from https://github.com/liyong31/COLA. We compared COLA with the official versions of Spot [12] (2.10.2) and Owl [23] (21.0). Spot implements the algorithm described in [35], a variant of [33] for transition-based NBAs, while Owl implements the algorithms described in [28,29], both constructing DPAs as result. To make the comparison fair, we let all tools generate DPAs, so we used the command autfilt --deterministic --parity=min\ even -F file.hoa to call Spot and owl nbadet -i file.hoa to call Owl. Recall that we use the function acd transform [8] from Spot for obtaining DPAs from our DRAs. The tools above also implement optimizations for reducing the size of the output DPA, like simulation and state merging [29], or stutter invariance [22] (except for Owl); we use the default settings for all tools. We performed our experiments on a desktop machine equipped with 16GB of RAM and a 3.6 GHz Intel Core i7-4790 CPU. We used BenchExec<sup>1</sup> [3] to trace and constrain the tools' executions: we allowed each execution to use a single core and 12 GB of memory, and imposed a timeout of 10 min. We used Spot to verify the results generated by three tools and found only outputs equivalent to the inputs.

As benchmarks, we considered all NBAs in the HOA format [1] available in the automata-benchmarks repository.<sup>2</sup> We have pre-filtered them with autfilt to exclude all deterministic cases and to have nondeterministic BAs, obtaining in total 15,913 automata coming from different sources in literature.

The artifact with tools, benchmarks, and scripts to run the experiments and generate the plots is available at [25].

<sup>1</sup> https://github.com/sosy-lab/benchexec/.

<sup>2</sup> https://github.com/ondrik/automata-benchmarks/.

**Fig. 2.** The cactus plot for the determinization of NBAs from automata-benchmarks.

**Fig. 3.** States comparison for the determinization of NBAs from automatabenchmarks. (Color figure online)

In Fig. 2 we show a cactus plot reporting how many input automata have been determinized by each tool, over time. As we can see, COLA works better than Spot, with COLA solving in total 15,903 cases and Spot 15,862 cases, with Owl solving in total 15,749 cases and taking more time to solve as many instances as COLA and Spot. From the plot given in Fig. 2 we see that COLA is already very competitive with respect to its performance.

In Fig. 3 we show the number of states of the generated DPAs. In the plot we indicate with the bold dashed line the maximum number of states of the automata produced by either of the two tools, and we place a mark on the upper or right border of the plot to indicate that one tool has generated an automaton with that size while the other tool just failed. The color of each mark represents how many instances have been mapped to the corresponding point. As the plots show, Spot and COLA generate automata with similar size, with COLA being more likely to generate smaller automata, in particular for larger outputs. Owl, instead, very frequently generates automata larger than COLA. In fact, on the 15,710 cases solved by all tools, on average COLA generated 44 states, Spot 65, and Owl 87. If we compare COLA with just one tool at a time, on the 15,854 cases solved by both COLA and Spot, we have 125 states for COLA and 246 for Spot; on the 15,749 cases solved by both COLA and

**Fig. 4.** Acceptance sets comparison for the determinization of NBAs from automatabenchmarks. (Color figure online)

**Table 1.** Pearson correlation coefficients for the automata-benchmarks experiments.


Owl, we have 45 states for COLA and 88 for Owl. A similar situation occurs for the number of transitions, so we omit it.

Lastly, in Fig. 4 we compare the number of acceptance sets (i.e., the colors in Definition 1) of the generated DPAs; more precisely, we consider the integer value occurring in the mandatory Acceptance: INT acceptance-cond header item of the HOA format [1], which can be 0 for the automata with all or none accepting transitions. From the plots we can see that COLA generates more frequently DPAs with a number of colors that is no more than the number used by Spot, as indicated by the yellow/red marks on (10,394 cases) or above (5,495 cases) the diagonal. Only in very few cases COLA generates DPAs with more colors than Spot (22 cases), as indicated by the few blue/greenish marks below the diagonal. Regarding Owl, however, from the plot we can clearly see that COLA uses almost always (15,840 cases) fewer colors than Owl; the only exception is for the mark at (0, 0) representing 63 cases.

The number and sizes of SCCs influence the performance of COLA, so we provide some statistics about the correlation between these and the runtime and size of the generated DPA. By combining the execution statistics with the input SCCs and states, we get the Pearson correlation coefficients shown in Table 1. Here the larger the number in a cell is, the stronger the positive correlation between the element that the row and the column represent. From these coefficients we can say that there is a quite strong positive correlation between the number of states and of SCCs and the running time, but not for the average SCC size; regarding the output states, the situation is similar but much weaker.

We also considered a second set of benchmarks – 644 NBAs generated by Spot's ltl2tgba on the LTL formulas considered in [23], as available in the

**Fig. 5.** The family of NBAs *A<sup>n</sup>* with Σ = *{*0, 1, *···* , n*}*.

Owl's repository at https://gitlab.lrz.de/i7/owl. The outcomes for these benchmarks are similar, but a bit better for COLA, to the ones for automatabenchmarks, so we do not present them in detail.

#### **6 Related Work**

To the best of our knowledge, our determinization construction is the *first* algorithm that determinizes SCCs *independently* while taking advantage of different structures of SCCs, which is the main difference between our algorithm and existing works. We illustrate other minor differences below.

Different types of SCCs, like DACs and IWCs, are also taken with special care in [29] as in our work, modulo the handling details. However, the work [29] does not treat them independently as the labelling numbers in those SCCs still have relative order with those in other SCCs. Thus their algorithm can be exponentially worse than ours (cf. Theorem 3) and performs not as well as ours in practice; see the comparison with Owl in Sect. 5. The determinization algorithm given in [14] for SDBAs is a special case of the one presented in [35] for NBAs, which gives precedence to the deterministic runs seeing accepting transitions earlier, while we give precedence to runs that enter DACs earlier. More importantly, the algorithm from [14] does not work when there is nondeterminism between DACs, while our algorithm overcomes this by considering DACs separately and by ignoring runs going to other SCCs.

Current works for determinization of general NBAs, such as [18,21,28,35,36, 38] can all be interpreted as different flavours of the Safra-Piterman based algorithm. Our determinization of NACs is also based on Safra-trees and inspired by Spot, except that we may have newly arriving states from other SCCs while other works only need to consider the successors from the current states in the Safra-tree. The modular approach for determinizing B¨uchi automata given in [17] builds on reduced split trees [21] and can construct the deterministic automaton with a given tree-width. The algorithm constructs the final deterministic automaton by running in parallel the NBA for all possible tree-widths, rather than working on SCCs independently as we do in this work.

Compared to the algorithms operating on the whole NBA, our algorithm can be exponentially better on the family of NBAs shown in Fig. 5, as formalized in Theorem 3; we can encounter some variation of this family of NBAs when working with fairness properties. The intuition is that we take care of the DACs {qi}<sup>n</sup> <sup>i</sup>=1 independently, so for each of them we have only two choices: either the run is in the DAC, or it is not in the DAC; resulting in a single exponential number of combinations. Existing works [14,21,28,33,35,36] order the runs entering the DACs based on when they visit accepting transitions, in which every order corresponds to a permutation of {q1, ··· , q<sup>n</sup>}.

**Theorem 3.** *There exists a family of NBAs* <sup>A</sup><sup>n</sup> *with* <sup>n</sup> + 2 *states for which the algorithms in [14,21,28,33,35,36] give a DPA with at least* n! *macrostates while ours gives a DELA with at most* 2<sup>n</sup>+2 *macrostates.*

In practice, for each NBA <sup>A</sup><sup>n</sup>, <sup>n</sup> <sup>≥</sup> 3, COLA produces a DELA/DPA with <sup>n</sup> macrostates, while both Spot and Owl give a DPA with n! + 1 macrostates.

## **7 Conclusion and Future Work**

We proposed a divide-and-conquer determinization construction for NBAs that takes advantage of the structure of different types of SCCs and determinizes them independently. In particular, our construction can be exponentially better than classical works on a family of NBAs. Experiments showed that our algorithm outperforms the state-of-the-art implementations regarding the number of states and transitions on a large set of benchmarks. To summarize, our divide-andconquer determinization construction is very *practical*, being a good complement to existing theoretical approaches.

Our divide-and-conquer approach for NBAs can also be applied to the complementation problems of NBAs. By Proposition 1, <sup>w</sup> is not accepted by <sup>A</sup> if and only if there are no accepting runs staying in an SCC. Thus we can construct a *generalized* B¨uchi automaton with a conjunction of Inf(i) as the acceptance formula to accept the complement language <sup>Σ</sup><sup>ω</sup> \ <sup>L</sup>(A) of <sup>A</sup>; the generalized B¨uchi automaton in fact takes the *intersection* of the complement language of each type of SCCs. For complementing IWCs, we use the same construction as determinization except that the acceptance formula will be Inf(1). For complementing DACs, we can borrow the idea of NCSB complementation construction [4] which complements SDBAs in time 4<sup>n</sup>. For complementing NACs, we just adapt the *slice-based* complementation [21] of general NBAs. We leave the details of this divide-and-conquer complementation construction for NBAs as future work.

**Acknowledgement.** We thank the anonymous reviewers for their valuable suggestions to this paper. This work is supported in part by the National Natural Science Foundation of China (Grant Nos. 62102407 and 61836005), NSF grants IIS-1527668, CCF-1704883, IIS-1830549, CNS-2016656, DoD MURI grant N00014-20-1-2787, and an award from the Maryland Procurement Office.

This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 101008233.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **From Spot 2.0 to Spot 2.10: What's New?**

Alexandre Duret-Lutz1(B) , Etienne Renault<sup>1</sup> , Maximilien Colange<sup>2</sup> , Florian Renkin<sup>1</sup> , Alexandre Gbaguidi Aisse<sup>2</sup>, Philipp Schlehuber-Caissier<sup>1</sup> , Thomas Medioni<sup>2</sup>, Antoine Martin<sup>1</sup> , J´erˆome Dubois<sup>1</sup>, Cl´ement Gillard<sup>2</sup>,

and Henrich Lauko<sup>2</sup>

<sup>1</sup> LRDE, EPITA, Le Kremlin-Bicˆetre, France {adl,renault,frenkin,philipp, amartin,jdubois}@lrde.epita.fr <sup>2</sup> Bicˆetre, France

**Abstract.** Spot is a C**++**17 library for LTL and ω-automata manipulation, with command-line utilities, and Python bindings. This paper summarizes its evolution over the past six years, since the release of Spot 2.0, which was the first version to support ω-automata with arbitrary acceptance conditions, and the last version presented at a conference. Since then, Spot has been extended with several features such as acceptance transformations, alternating automata, games, LTL synthesis, and more. We also shed some lights on the data-structure used to store automata. **Artifact:** https://zenodo.org/record/6521395.

#### **1 Availability, Purpose, and Evolution**

Spot is a library for LTL and ω-automata manipulation, distributed under a GPLv3 license. Its source code is available from https://spot.lrde.epita.fr/. We provide packages for some Linux distributions like Debian and Fedora, but other packages can also be found for Conda-Forge [17] (for Linux & Darwin), Arch Linux, FreeBSD...

Spot can be used via three interfaces: a C**++**17 library, a set of commandline tools that give easy access to many features of the library, and Python bindings, that makes prototyping and interactive work very attractive. Our web site now contains many examples of how to perform some tasks using these three interfaces, and we have a public mailing list for questions.

In our last tool paper [21], Spot 2.0 had just converted from being a library for working on Transition-based Generalized B¨uchi Automata and had become a library supporting ω-automata with arbitrary Emerson-Lei [22,41] acceptance conditions, as enabled by the development of the HOA format [5].

In the HOA format, transitions can carry multiple colors, and acceptance conditions are expressed as a positive Boolean formulas over atoms like Fin(i) or Inf(i) that tell if a color should be seen finitely or infinitely often for a run to be accepting. Table 1 gives some examples.

M. Colange, A. Gbaguidi Aisse, T. Medioni, C. Gillard and H. Lauko—Previously at LRDE.

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 174–187, 2022. https://doi.org/10.1007/978-3-031-13188-2\_9


**Table 1.** Acceptance formulas corresponding to classical names.

While Spot 2.0 was able to read automata with arbitrary acceptance conditions, not all of its algorithms were able to support such a generality. For instance testing an automaton for emptiness or finding an accepting word, would only work on automata with "Fin-less" acceptance conditions. For other conditions, Spot 2.0 would rely on a procedure called remove fin() to convert automata with arbitrary acceptance conditions into "Fin-less" acceptance conditions [9]. This was ultimately fixed by developing a generic emptiness check [6]. Additionally the support for arbitrary acceptance conditions has allowed us to implement many useful algorithms; the most recent being the Alternating Cycle Decomposition [15,16] a powerful data structure with many applications (conversion to parity acceptance, degeneralization, typeness checks...)<sup>1</sup>.

There have been 56 releases of Spot since version 2.0, but only 10 of these are major releases. Releases are numbered 2.x.y where y is updated for minor upgrades that mostly fix bugs, and x is updated for major release that add new features. (The leading 2 would be incremented in case of a serious redesign of the API.) Table 2 summarizes the highlights of the various releases in chronological order. Not appearing in this list are many micro-optimizations and usability improvements that Spot has accumulated over the years.

#### **2 Use-cases of Spot, and Related Tools**

As it is a library, there are many ways to use Spot. We are mostly aware of such uses via citations<sup>2</sup>. Historical and frequent uses-cases are to use Spot for translating LTL formulas to automata (Winners of the sequential LTL and parallel LTL tracks of RERS'19 challenge [26] both used Spot to translate the properties into automata, many competitors on the Model Checking Contest [28] also use

<sup>1</sup> https://spot.lrde.epita.fr/ipynb/zlktree.html.

<sup>2</sup> Our previous tool paper [21] has over 250 citations according to Google scholar.

**Table 2.** Milestones in the history of Spot.


Spot this way), or to use it as a research/development toolbox, since it provides helper tools for generation of random formulas/automata, verification of LTLto-automata translation, simplifications, syntax conversions, etc. Nowadays, the algorithms for ω-automata implemented in Spot are often used as baseline for studying better algorithms [e.g., 18,25,32,33], but we also see some new applications built on top of ω-automata algorithms from Spot [e.g., 12,13].

The projects that have the largest intersections of features with Spot seem to be GOAL [43] and Owl [30]. These are two Java-based frameworks that deal with similar objects and provide a range of algorithms. Owl and Spot share a similar and traditional Unix view of the command-line experience, where multiple commands are expected to be chained with pipes, and they both communicate smoothly via the HOA format [5]. GOAL is centered on a graphical interface in which the user can edit automata, and apply algorithms listed in menu entries. Using GOAL from the command-line is possible by writing short scripts in a custom language.

As far as interfacing goes, the most important feature of Spot is probably that it exposes its algorithms and data structures in Python. Beside being usable as a glue language between various tools, this allows us (1) to leverage Python's ecosystem and (2) to quickly prototype new algorithms in Python.

#### **3 Automata Representation**

In this section and the next three, we focuses on how the storage of automata evolved to support alternation, games, and Mealy machines.

The main automaton class of Spot is called twa graph and inherits from the twa class. The letters twa stand for *Transition-based* <sup>ω</sup>-Automaton.

The class twa implements an abstract interface that allows on-the-fly exploration of an automaton similar to what had been present in Spot from the start: essentially, one can query the initial state, and query the transitions leaving any known state. In particular, before exploring the state-space of a twa, it is unknown how many states are reachable. Various subclasses of twa are provided in Spot, for instance to represent the state-space of Promela or Divine models [21]. Users may create subclasses, for instance to create a Kripke structure on-the-fly.<sup>3</sup>

The class twa graph, introduced in Spot 2.0, implements an explicit, graphbased, representation of an automaton, in which states and edges are designated by integers. This makes for a much simpler interface<sup>4</sup> and usually simplifies the data structures used in algorithms (since states and edges can be used as indices in arrays). The data structure is best illustrated by using the show storage() method of the Python bindings, as shown by Fig. 1. A twa graph is stored as two C**++** vectors: a vector of states, and a vector of edges. For each state, the first vector stores two edge numbers: succ is the first outgoing edge, and succ tail is the last one. These number are indices into the edge vector, which stores five pieces of information per edge. Four of them are related to the identity of the

<sup>3</sup> As demonstrated by https://spot.lrde.epita.fr/tut51.html.

<sup>4</sup> Contrast on-the-fly and explicit APIs at https://spot.lrde.epita.fr/tut50.html.

**Fig. 1.** Internal representation of a twa graph as two vectors.

edge: src, dst, cond, acc are respectively the source, destination, guard, and color sets of the edge. The remaining field, next succ gives the next outgoing edge, effectively creating a linked list of all edges leaving a given state. There is no edge 0: this value is used as terminator for such lists. Outgoing edges of the same state are not necessarily adjacent in that structure. When a new edge is added to the automaton, it is simply appended to the edge vector, and the succ tail field of the state is used to update the previous end of the list.

To iterate over successors of state 1 in C**++** or Python, one can ignore the above linked list implementation and write one of the following loops:


The twa graph::out methods simply returns a lightweight temporary object which can be iterated upon using iterators that will follow the linked list. Then the object e is effectively a reference to a column of the edge vector.

As seen on Fig. 1, the automaton additionally stores an initial state (Spot only supports a single initial state), a number of colors (num sets), an acceptance condition, a list of atomic propositions (Spot only supports alphabets of the form 2*AP* ), and 10 fields storing structural properties of the automaton.

These property fields have only three possible values: they default to *maybe*, but can be set to *no* or *yes* by algorithms that work on the automaton. They can also be read and written in the HOA format. For instance if prop universal is set to *yes*, it means that automaton does not have any existential choice (a.k.a. non-determinism). Spot's *is deterministic()* algorithm can return in constant time if prop universal is known, otherwise it will inspect the automaton and set that property before returning, so that the next call to *is deterministic()* will be instantaneous. Some algorithms know how to take advantage of any hint they get from those properties: for instance the product() of two automata is optimized to use fewer colors when one of the arguments is known to be weak (i.e., in an SCC all transitions have the same colors).

Note that algorithms that modify an automaton in place have to remember to update those properties. This has caused a couple of bugs over the years.

#### **4 Introduction of Alternating Automata**

Support for alternating ω-automata, as defined in the HOA format, was added to Spot in version 2.3 without introducing a new class. Rather, the twa graph class was extended to support alternation in such a way that existing algorithms would not require any modification to continue working on automata without universal branching. This was done by reserving the sign bit of the destination state number of each transition to signal universal branching.

Figure 2 shows an example of Alternating automaton (top-left) with co-B¨uchi acceptance. In many works on alternating automata, it is conventional to not represent accepting sinks, and instead have transition without destination. The top-right picture shows that Spot has a rendering option to hide accepting sinks.

The bottom of the figure shows that the automaton has prop state acc set, which means that the automaton is meant to be interpreted as using state-based acceptance. Colors are still stored on edges internally, but all edges leaving a state have the same colors. Seeing that the condition is co-B¨uchi (Fin(0)), the display code automatically switched to the convention of using double-circles for rejecting states.

Destinations with the sign bit set are called *universal destination groups* and appear as pink in the figure. There are two groups here: ~0 and ~3. The complement of these numbers can be used as indices in the dests vector, that actually store the destination groups. At the given index, one can read the size n of the destination group, followed by the state number of the n destinations.

Algorithms that work on alternating automata need to be able to iterate over all destinations of an edge. The process of checking the sign bit of the destination to decide if its a group, and to iterate on that group is hidden by the univ dests() method:

}

**Fig. 2.** Internal representation of alternating automata.

```
for(auto& e: aut->out(1)) {
 // use e.cond, e.acc, e.src
 for(unsigned d:aut->univ_dests(e))
  // use d
                                     for e in aut.out(1):
                                        # use e.cond, e.acc, e.src
                                        for d in aut.univ_dests(e):
                                           # use d
```
Note that this code works on non-universal branches as well: if e.dst is unsigned, univ dests(e) will simply iterate on that unique value.

Spot has two alternation removal procedures. One is an on-the-fly implementation of the Breakpoint construction [37] which transforms an n-state alternating B¨uchi automaton into a non-alternating B¨uchi automaton with at most 3*<sup>n</sup>* states. For very weak alternating automata, it is known that a powerset-based procedure can produce a transition-based generalized B¨uchi automaton with 2*<sup>n</sup>* states [23]; in fact that algorithm even works on *ordered* automata [11], i.e., alternating automata where the only rejecting cycles are self-loops. The second alternation removal procedure of Spot is a mix between these two procedures but does not work on the fly: it takes a *weak* automaton as input, and uses the break-point construction on rejectings SCCs that have more than one state, and uses the powerset construction for other SCCs.

#### **5 Extending Automata via Named Properties**

Spot's automata have a mechanism to attach arbitrary data to automata, called *named properties*. (This is similar to the notion of attributes in the R language.) An object can be attached to the automaton with:

```
aut->set_named_prop("property-name", new mytype(...));
```
and later retrieved with:

mytype\* data = aut->get\_named\_prop<mytype>("property-name");

Ensuring that mytype is the correct type for the retrieved property is the programmer's responsibility.

Spot has grown a list of many such properties over time.<sup>5</sup> For instance automaton-name stores a string that would be displayed as the name of the automaton. The highlight-edges and highlight-states properties can be used to color edges and states. The state-names is a vector of strings that gives a name to each state, etc. While those examples are mostly related to the graphical rendering of the automata, some algorithms store useful byproducts as properties. For instance the product() algorithm will define a product-states named property that store a vector of pairs of the original states.

These named properties are sometimes used to provide additional semantics to the automaton, for instance to obtain a game or a Mealy machine.

#### **6 Games, Mealy Machines, and LTL Synthesis**

The application of Spot to LTL synthesis was introduced in Spot 2.5 in the form of the ltlsynt tool [35], but the inner workings of this tool were progressively redesigned and publicly exposed until version 2.10.

An automaton can now be turned into a game by attaching the state-player property to it.<sup>6</sup> Only two-player games are supported, so state-player should be a std::vector<bool>. Currently, Spot has solvers for safety games and for games with *parity max odd* acceptance, but we plan to at least generalize the latter to any kind of parity condition. Once a game has been solved, it contains two new named properties: state-winner (a std::vector<bool> indexed by state numbers indicating the player winning in each state), and strategy (a std::vector<unsigned> that gives for each state the edge that its owner should follow to win).

<sup>5</sup> https://spot.lrde.epita.fr/concepts.html#named-properties.

<sup>6</sup> https://spot.lrde.epita.fr/tut40.html illustrates how a game can be used to decide if a state simulates another one.

**Fig. 3.** (top) Solving a game to display the strategy. States with green borders are winning for player 1, who wants to satisfy the acceptance condition, by following the green arrows. States with red color are winning for player 0, who wants to fail the acceptance condition, by following the red arrows. (bottom) Conversion of the winning strategy to a Mealy machine and then an AIGER circuit. (Color figure online)

Figure <sup>3</sup> shows an example of game generated by ltlsynt, and how we can display the winning strategy once the game is solved. The winning strategy can be extracted and converted into a Mealy machine, which is just an automaton that uses the synthesis-output property to specify which atomic propositions belong to the output. Such a Mealy machine can then be encoded into an ANDinverter graph, and saved into the AIGER format [7]. Here L0 represents a latch, i.e., one bit of memory, that stores the previous value of a so that the circuit can output b if and only if a is true in the present and in the previous step.

#### **7 Online Application for LTL Formulas**

The Python ecosystem makes it easy to develop web interfaces for convenient access to a subset of features of Spot. For instance Fig. 4 shows screenshots of a web application built using a React frontend, and running Spot on the server. It can transform LTL formulas into automata, can display many properties of a formula (membership to the Manna & Pnueli hierarchy [34], Safety/Liveness classification [2], Rabin and Streett indices [14], stutter-invariance [36]), or simply compare two formulas using a Venn diagram.

**Fig. 4.** A web application, built on top of Spot. https://spot.lrde.epita.fr/app/

This application has been found to be useful for teaching about LTL and its relation with automata, but is also a helpful research tool.

#### **8 Shortcomings and One Future Direction**

While Spot has been used for many applications, there are two recurrent issues: they are related to the types used for some fields of the edge vector (see Figs. 1– 2). By default, the set of colors that labels an edge (the acc field) is stored as a 32-bit bit-vector, the transition label (cond, a formula over 2*AP* ), is stored as a BDD identified by a unique 32-bit integer, and the other three fields (src, dst, next succ) are all 32-bit integers. One edge therefore takes 20 bytes.

While limiting the number of states to 32-bit integers has never been a problem so far, the limit of 32 colors can be hit easily. Spot 2.6 added a compile-time option to enlarge the number of supported colors to any multiple of 32; this evidently has a memory cost (and therefore also a runtime cost) as the acc field will be larger for each edge. However this constraint generally means that all the algorithms we implement try to be "color-efficient", i.e., to not introduce useless colors. For instance while the product of an automaton with x colors and an automaton with y colors is usually an automaton with x + y colors, the product() implementation will output fewer colors in presence of a *weak* automaton.

The use of BDDs as edge labels causes another type of issues. Spot uses a customized version of the BuDDy library, with additional functions, and several optimizations (more compact BDD nodes for better cache friendliness, most operations have been rewritten to be recursion-free). However BuDDy is inherently not thread safe, because of its global unicity table and caches. This prevents us from doing any kind of parallel processing on automata. A long term plan is to introduce a new class twacube that represent an automaton in which edges are cubes (i.e., conjunctions of literals) represented using two bit-vectors. Such a class was experimentally introduced in Spot 2.10 and is currently used in some parallel emptiness check procedures [38].

## **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Complementing B¨uchi Automata with Ranker**

Vojtˇech Havlena , Ondˇrej Leng´al(B) , and Barbora Smahl´ <sup>ˇ</sup> ıkov´a

Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic ihavlena@fit.vut.cz, *{*lengal,xsmahl00*}*@vut.cz

**Abstract.** We present the tool Ranker for complementing B¨uchi automata (BAs). Ranker builds on our previous optimizations of rankbased BA complementation and pushes them even further using numerous heuristics to produce even smaller automata. Moreover, it contains novel optimizations of specialized constructions for complementing (i) inherently weak automata and (ii) semi-deterministic automata, all delivered in a robust tool. The optimizations significantly improve the usability of Ranker, as shown in an extensive experimental evaluation with real-world benchmarks, where Ranker produced in the majority of cases a strictly smaller complement than other state-of-the-art tools.

#### **1 Introduction**

B¨uchi automata (BA) complementation is an essential operation in the toolbox of automata theory, logic, and formal methods. It has many applications, e.g., implementing negation in decision procedures of some logics (such as the monadic second-order logic S1S [1,2], the temporal logics EPTL and QPTL [3], or the first-order logic over Sturmian words [4]), proving termination of programs [5–7], or model checking of temporal properties [8]. BA complementation also serves as the foundation stone of algorithms for checking inclusion and equivalence of ω-regular languages. In all applications of BAs, the number of states of a BA affects the overall performance. The many uses of BA complementation, as well as the challenging theoretical nature of the problem, has incited researchers to develop a number of different approaches, e.g., *determinization-based* [9–11], *rank-based* [12–14], or *Ramsey-based* [1,15], some of them [14,16] producing BAs with the number of states asymptotically matching the lower bound (0.76n)<sup>n</sup> of Yan [17]. Despite their theoretical optimality, for many real-world cases the constructions create BAs with a lot of unnecessary states, so optimizations making the algorithms efficient in practice are needed.

We present Ranker, a robust tool for complementing (transitionbased) BAs. Ranker uses several complementation approaches based on properties of the input BA: it combines an optimization of the rank-based procedure developed in [18–20] with specialized (and further optimized) procedures for complementing semi-deterministic BAs [21], inherently weak BAs [22,23], and elevator BAs [19]. An extensive experimental evaluation on a wide range of automata occurring in practice shows that Ranker can obtain a smaller complement in the majority of cases compared to the other state-of-the-art tools.

*Contribution.* We describe a major improvement of Ranker [18,19], turning it from a prototype into a robust tool. We list the particular optimizations below.


All of these improvements are pushing the capabilities of Ranker, and also of practical BA complementation itself, much further.

#### **2 B¨uchi Automata**

*Words, Functions.* We fix a finite nonempty alphabet Σ and the first infinite ordinal ω = {0, 1,...}. An (infinite) word α is a function α: ω → Σ where the i-th symbol is denoted as αi. We abuse notation and sometimes represent α as an infinite sequence α = α0α<sup>1</sup> ... Σ<sup>ω</sup> denotes the set of all infinite words over Σ.

*B¨uchi Automata.* A (nondeterministic transition/state-based) *B¨uchi automaton* (BA) over Σ is a quintuple A = (Q, δ, I, Q<sup>F</sup> , δ<sup>F</sup> ) where Q is a finite set of *states*, <sup>δ</sup> : <sup>Q</sup> <sup>×</sup> <sup>Σ</sup> <sup>→</sup> <sup>2</sup><sup>Q</sup> is a *transition function*, <sup>I</sup> <sup>⊆</sup> <sup>Q</sup> is the sets of *initial* states, and Q<sup>F</sup> ⊆ Q and δ<sup>F</sup> ⊆ δ are the sets of *accepting states* and *accepting transitions* respectively. A is called deterministic if |I| ≤ 1 and |δ(q, a)| ≤ 1 for each q ∈ Q and <sup>a</sup> <sup>∈</sup> Σ. We sometimes treat <sup>δ</sup> as a set of transitions <sup>p</sup> <sup>a</sup> → q, for instance, we use p <sup>a</sup> → q ∈ δ to denote that q ∈ δ(p, a). Moreover, we extend δ to sets of states P ⊆ Q as δ(P, a) = - <sup>p</sup>∈<sup>P</sup> <sup>δ</sup>(p, a). The notation <sup>δ</sup><sup>|</sup> S for S ⊆ Q is used to denote the restriction of the transition function δ ∩ (S × Σ × S). Moreover, for q ∈ Q, we use A[q] to denote the automaton (Q, δ, {q}, Q<sup>F</sup> , δ<sup>F</sup> ).

A *run* of A from q ∈ Q on an input word α is an infinite sequence ρ: ω → Q that starts in q and respects δ, i.e., ρ<sup>0</sup> = q and ∀i ≥ 0: ρ<sup>i</sup> → <sup>α</sup>*<sup>i</sup>* <sup>ρ</sup>i+1 <sup>∈</sup> <sup>δ</sup>. Let infQ,δ(ρ) ⊆ Q∪δ denote the set of states and transitions occurring in ρ infinitely often. The run ρ is called *accepting* iff infQ,δ(ρ) ∩ (Q<sup>F</sup> ∪ δ<sup>F</sup> ) = ∅. A word α is *accepted by* A *from a state* q ∈ Q if A has an accepting run ρ on α from q, i.e., <sup>ρ</sup><sup>0</sup> <sup>=</sup> <sup>q</sup>. The set <sup>L</sup>A(q) = {<sup>α</sup> <sup>∈</sup> <sup>Σ</sup><sup>ω</sup> | A accepts <sup>α</sup> from <sup>q</sup>} is called the *language* of q (in A). Given a set of states R ⊆ Q, we define the language of R as LA(R) = - <sup>q</sup>∈<sup>R</sup> <sup>L</sup>A(q) and the language of <sup>A</sup> as <sup>L</sup>(A) = <sup>L</sup>A(I). If <sup>δ</sup><sup>F</sup> <sup>=</sup> <sup>∅</sup>, we call A *state-based* and if Q<sup>F</sup> = ∅, we call A *transition-based*.

A *co-B¨uchi automaton* (co-BA) C is the same as a BA except the definition of when a run is accepting: a run ρ of C is *accepting* iff infQ,δ(ρ)∩(Q<sup>F</sup> ∪δ<sup>F</sup> ) = ∅.

**Fig. 1.** Overview of the architecture of Ranker with the most important commandline options. Default settings are highlighted in blue. (Color figure online)

*Automata Types.* Let A = (Q, δ, I, Q<sup>F</sup> , δ<sup>F</sup> ) be a BA. C ⊆ Q is a *strongly connected component* (SCC) of A if for any pair of states q, q ∈ C it holds that q is reachable from q and q is reachable from q. C is *maximal* (MSCC) if it is not a proper subset of another SCC. An MSCC is *non-accepting* if it contains no accepting state and no accepting transition. We say that an SCC C is *inherently weak accepting* (IWA) iff *every cycle* in the transition diagram of A restricted to C contains an accepting state or an accepting transition. We say that an SCC <sup>C</sup> is *deterministic* iff (C, δ<sup>|</sup> <sup>C</sup> , <sup>∅</sup>, <sup>∅</sup>, <sup>∅</sup>) is deterministic. <sup>A</sup> is *inherently weak* (IW) if all its MSCCs are inherently weak accepting or nonaccepting, and *weak* if for states q, q that belong to the same SCC, q ∈ Q<sup>F</sup> iff q ∈ Q<sup>F</sup> . A is *semi-deterministic* (SDBA) if A[q] is deterministic for every <sup>q</sup> <sup>∈</sup> <sup>Q</sup><sup>F</sup> ∪ {<sup>p</sup> <sup>∈</sup> <sup>Q</sup> <sup>|</sup> <sup>s</sup> <sup>a</sup> → p ∈ δ<sup>F</sup> , s ∈ Q, a ∈ Σ}. Finally, A is called *elevator* if all its MSCCs are inherently weak accepting, deterministic, or non-accepting.

### **3 Architecture**

Ranker [24] is a publicly available command line tool, written in C++, implementing several approaches for complementation of (transition/state-based) B¨uchi automata. As an input, Ranker accepts BAs in the HOA [25] or the simpler ba [26] format. The architecture overview is shown in Fig. 1. An input automaton is first adjusted by various structural preprocessing steps to an intermediate equivalent automaton with a form suitable for a complementation procedure. Based on the intermediate automaton type, a concrete complementation procedure is used. The result of the complementation is subsequently polished by postprocessing steps, yielding an automaton on the output. In the following text, we provide details about the internal blocks of Ranker's architecture.

#### **3.1 Preprocessing and Postprocessing**

Before an input BA is sent to the complementation block itself, it is first transformed into a form most suitable for a concrete complementation technique. On top of that as a part of preprocessing, we identify structural features that are further used to enabling/disabling certain optimizations during the complementation. After the complementation, the resulting automaton is optionally reduced in a postprocessing step. Ranker provides several options of preprocessing/postprocessing that are discussed below.

*Preprocessing*. The following are the most important settings for preprocessing:

	- --preprocess=copyall: Every component is deelevated.
	- --preprocess=copyiwa: Only IWA components are deelevated.
	- --preprocess=copyheur: This option combines two modifications applied in sequence: (i) If the input BA is not IW and the rank bound estimation [19] of the BA is at least 5, then all MSCCs with an accepting state/transition are deelevated (the higher rank bound indicates a longer sequence of components, for which deelevation is likely to be benefical). (ii) If on all paths from all initial states of the intermediate BA, the first non-trivial MSCC is non-accepting, then we partially determinize the initial part of the BA (up to the first non-trivial MSCCs); this reduces sizes of macrostates obtained in rank-based complementation.

**Fig. 2.** Overview of complementation approaches used in Ranker.

a smaller rank bound, it is counter-productive to use techniques reducing the rank bound based on reasoning about the waiting part).

*Postprocessing*. After the complementation procedure finishes, Ranker removes useless states and optionally applies simulation reduction (--postprocess=red).

#### **3.2 Complementation Approaches**

Based on the automaton type, Ranker uses several approaches for complementation (cf. Fig. 2). These are, ordered by decreasing priority, the following:


#### **4 Optimizations of the Constructions**

In this section, we provide details about new optimizations of complementation of inherently weak and semi-deterministic automata implemented in Ranker. Proofs of their correctness can be found in the technical report [28].

#### **4.1 Macrostates Adjustment for Inherently Weak Automata**

For complementing IW automata, Ranker uses a method based on the Miyano-Hayashi construction (denoted as MiHay) [22]: In the first step, accepting states of an input IW BA A are saturated to obtain a language-equivalent weak automaton W = (Q, δ, I, Q<sup>F</sup> , ∅) (we remove accepting transitions because they do not provide any advantage for IW automata). In the second step, W is converted to the equivalent co-B¨uchi automaton C = (Q, δ, I, Q <sup>F</sup> = Q \ Q<sup>F</sup> , ∅) by swapping accepting and non-accepting states. Finally, the Miyano-Hayashi construction is used to obtain the complement (state-based) BA.

Our optimizations of the MiHay procedure are inspired by optimizations of the determinization algorithm for automata over finite words [29] and by saturation of macrostates in rank-based BA complementation procedure [20], where simulation relations are used to adjust macrostates in order to obtain a smaller automaton. We modify the original construction by introducing an *adjustment function* that modifies obtained macrostates, either to obtain *smaller* macrostates (for *pruning* strategy) or *larger* macrostates (for *saturating* strategy; the hope is that *more* original macrostates map to *the same* saturated macrostate). Formally, given a co-BA C and an *adjustment function* <sup>θ</sup> : 2<sup>Q</sup> <sup>→</sup> <sup>2</sup><sup>Q</sup>, the construction MiHay<sup>θ</sup> gives the (deterministic, state-based) BA MiHayθ(C)=(Q , δ , I , Q <sup>F</sup> , ∅), whose components are defined as follows:

$$\begin{array}{l} -\ \begin{array}{l} Q' = 2^{Q} \times 2^{Q}, \\ - \ \end{array} \\ - \ \begin{array}{l} I' = \{ (\theta(I), \theta(I) \mid Q'\_{F}) \}, \\ - \ \delta'((S, B), a) = (S', B') \text{ where} \\ \bullet \ S' = \theta(\delta(S, a)), \\ \bullet \ \text{and} \\ \* \ B' = S' \backslash Q'\_{F} \text{ if } B = \emptyset \text{ or} \\ \* \ B' = (\delta(B, a) \cap S') \backslash Q'\_{F} \text{ if } B \neq \emptyset, \text{ and} \\ - \ F' = 2^{Q} \times \{ \emptyset \}. \end{array}$$

Intuitively, the construction tracks in the S-component all runs over a word and uses the B-component to check that each of the runs sees infinitely many accepting states from Q <sup>F</sup> (by a cut-point construction). The original MiHay procedure can be obtained by using identity for the adjustment function, θ = id.

In the following, we use <sup>W</sup> *di* and <sup>C</sup> <sup>f</sup> to denote a *direct simulation* on W and a *fair simulation* on C respectively (see, e.g., [30] for more details; in particular, p <sup>C</sup> <sup>f</sup> q iff for every trace of C from state p over α with finitely many accepting states, there exists a trace from q with finitely many accepting states over α).

Let ⊆ Q×Q be a relation on the states of C defined as follows: p q iff (i) p <sup>C</sup> <sup>f</sup> q, (ii) q is reachable from p in C, and (iii) either p is not reachable from q in <sup>C</sup> or <sup>p</sup> <sup>=</sup> <sup>q</sup>. The two adjustment functions *pr* , *sat* : 2<sup>Q</sup> <sup>→</sup> <sup>2</sup><sup>Q</sup> are then defined for each S ⊆ Q as follows:

– *pruning*: *pr* (S) = S where S ⊆ S is the lexicographically smallest set (given a fixed ordering on Q) such that ∀q ∈ S∃q ∈ S : q q and – *saturating*: *sat*(S) = {p ∈ Q | ∃q ∈ Q: p <sup>C</sup> <sup>f</sup> q}.

Informally, *pr* removes simulation-smaller states and *sat* saturates a macrostate with all simulation-smaller states.<sup>1</sup> The correctness of the constructions is summarized by the following theorem:

**Theorem 1.** *For a co-BA* <sup>C</sup>*,* <sup>L</sup>(MiHay*sat*(C)) = <sup>L</sup>(MiHay*pr* (C)) = Σ<sup>ω</sup> \ L(C)*.*

In Ranker, we approximate a fair simulation <sup>C</sup> <sup>f</sup> by a direct simulation <sup>W</sup> *di* (which is easier to compute); the correctness holds due to the following lemma:

**Lemma 2.** *Let* W = (Q, δ, I, Q<sup>F</sup> , ∅) *be a weak BA and* C = (Q, δ, I, Q <sup>F</sup> = Q \ Q<sup>F</sup> , ∅) *be a co-BA. Then* <sup>W</sup> *di* ⊆ <sup>C</sup> f *.*

#### **4.2 NCSB-MaxRank Construction**

The structure of semi-deterministic BAs allows to use more efficient complementation techniques. From the point of view of rank-based complementation, the maximum rank of semi-deterministic automata can be bounded by 3. If a rankbased complementation procedure based on *tight rankings* (such as [18,19]) is used to complement an SDBA, it can suffer from having too many states due to the presence of the *waiting* part (intuitively, runs wait in the waiting part of the complement until they can see only tight rankings, then they jump to the *tight* part where they can accept, cf. [13,14,18] for more details). Furthermore, the information about ranks of individual runs may sometimes be more precise than necessary, which disables merging some runs. The NCSB construction [21] overcomes these issues by not considering the waiting part and keeping only rough information about the ranks. As a matter of fact, NCSB and the rank-based approach are not comparable due to tight-rankings and additional techniques restricting the ranking functions [18,19], taking into account structural properties of the automaton, which is why Ranker in the default setting tries both rank-based and NCSB-based procedures for complementing SDBAs.

An issue of the NCSB algorithm is a high degree of nondeterminism of the constructed BA (and therefore also a higher number of states). The NCSB-Lazy construction [7] improves the original algorithm with postponing the nondeterministic choices, which usually produces smaller results. Even the NCSB-Lazy construction may, however, suffer in some cases from generating too many successors. We propose an improvement of the original NCSB algorithm, inspired by the MaxRank construction in rank-based complementation from [18] (which

<sup>1</sup> It has been brought to our attention by Alexandre Duret-Lutz that a strategy similar to *pruning* with direct simulation has been implemented in Spot's [31] determinization and, moreover, generalized in [32] to also work in some cases *within* SCCs.

is inspired by [14, Section 4]), hence called the NCSB-MaxRank construction, reducing the number of successors of any macrostate and symbol to at most two.

Formally, for a given SDBA A = (Q1Q2, δ = δ1δ2δt,I,Q<sup>F</sup> , δ<sup>F</sup> ) where Q<sup>2</sup> are the states reachable from an accepting state or transition and Q<sup>1</sup> is the rest, δ<sup>1</sup> = δ|Q<sup>1</sup> , δ<sup>2</sup> = δ|Q<sup>2</sup> , and δ<sup>t</sup> is the transition function between Q<sup>1</sup> and Q2, we define NCSB-MaxRank(A)=(Q , I , δ , Q <sup>F</sup> , ∅) to be the (state-based) BA whose components are the following:

– <sup>Q</sup> <sup>=</sup> {(N, C, S, B) <sup>∈</sup> <sup>2</sup><sup>Q</sup><sup>1</sup> <sup>×</sup> <sup>2</sup><sup>Q</sup><sup>2</sup> <sup>×</sup> <sup>2</sup><sup>Q</sup>2\Q*<sup>F</sup>* <sup>×</sup> <sup>2</sup><sup>Q</sup><sup>2</sup> <sup>|</sup> <sup>B</sup> <sup>⊆</sup> <sup>C</sup>},

$$-\underset{\bullet}{I}' = \{ (Q\_1 \cap I, Q\_2 \cap I, \emptyset, Q\_2 \cap I) \},$$


Intuitively, NCSB-MaxRank provides at most two choices for each macrostate: either keep all states in B or move all states from B to S (if B contains no accepting state). If a word is not accepted by A, it will be safe to put all states from B to S at some point. The construction is in fact incomparable to the original NCSB algorithm [21] (in particular due to the condition C ⊆ δ2(C \Q<sup>F</sup> , a), which need not hold in NCSB-MaxRank). Correctness of the construction is given by the following theorem.

**Theorem 3.** *Let* <sup>A</sup> *be an SDBA. Then* <sup>L</sup>(*NCSB-*MaxRank(A)) = Σ<sup>ω</sup> \ L(A)*.*

#### **5 Experimental Evaluation**

We compared the improved version of Ranker presented in this paper with other state-of-the-art tools, namely, Goal [33] (implementing Piterman [10], Safra [9], and Fribourg [16]), Spot 2.9.3 [31] (implementing Redziejowski's algorithm [11]), Seminator 2 [34], LTL2dstar 0.5.4 [35], Roll [36], and the previous version of Ranker from [19], denoted as RankerOld. All tools were set to the mode where they output a state-based BA. The correctness of our implementation was tested using Spot's autcross on all of BAs from our benchmarks. The experimental evaluation was performed on a 64-bit GNU/Linux Debian workstation with an Intel(R) Xeon(R) CPU E5-2620 running at 2.40 GHz with 32 GiB of RAM, using a 5-minute timeout. Axes in plots are logarithmic. An artifact that allows reproduction of the results is available as [37].

**Fig. 3.** Evaluation of the effect of our optimizations for IW and SDBA automata.

*Datasets.* We use automata from the following three datasets: (i) random containing 11,000 BAs over a two letter alphabet used in [38], which were randomly generated via the Tabakov-Vardi approach [39], starting from 15 states and with various parameter settings; (ii) LTL with 1,721 BAs over larger alphabets (up to 128 symbols) used in [34], obtained from LTL formulae from literature (221) or randomly generated (1,500), (iii) Automizer containing 906 BAs over larger alphabets (up to 2<sup>35</sup> symbols) used in [7], which were obtained from the Ultimate Automizer tool (all benchmarks are available at [40]). Note that we included random in order to simulate applications that cannot easily generate BAs of one of the easier fragments (unlike, e.g., Ultimate Automizer, which generates in most cases SDBAs) and have thus, so far, not been seriously considered by the community due to the lack of practically efficient BA complementation approaches (e.g., the automata-based S1S decision procedure [1]). All automata were preprocessed using Spot's autfilt (using the {-}{-}high simplification level), and converted to the HOA format [25]. We also removed trivial one-state BAs. In the end, we were left with 4,533 (random, **blue** data points), 1,716 (LTL, **red** data points), and 906 (Automizer, **green** data points) automata. We use all to denote their union (7,155 BAs).

#### **5.1 Effect of the Proposed Optimizations**

In the first part of the experimental evaluation, we measured the effect of the proposed optimizations from Sect. 4 on the size of the generated state space, i.e., sizes of output automata without any postprocessing. This use case is motivated by language inclusion and equivalence checking, where the size of the generated state space directly affects the performance of the algorithm. We carried out the evaluation on LTL and Automizer benchmarks (we use both to denote their union) since most of the automata there are either IW or SDBAs.

**Fig. 4.** Comparison of the complement size obtained by Ranker, RankerOld, and Spot (horizontal and vertical dashed lines represent timeouts).

The first experiment compares the number of states generated by the original MiHay and by the macrostatespruning optimization MiHay*pr* from Sect. 4.1 on inherently weak BAs (948 BAs from LTL and 360 BAs from Automizer = 1,308 BAs). Note that we omit MiHay*sat* as it is overall worse than MiHay*pr* . The scatter plot is shown in Fig. 3a and statistics are in the top part of Table 1. We can clearly see **Table 1.** Effects of our optimizations for IW and SDBA automata. Sizes of output BAs are given as "both (LTL : Automizer)".


that the optimization works well, substantially decreasing both the mean and the median size of the output BAs.

The second experiment compares the size of the state space generated by NCSB-Lazy [7] and NCSB-MaxRank from Sect. 4.2 on 735 SDBAs (that are not IW) from LTL (328 BAs) and Automizer (407 BAs). We omit a comparison with the original NCSB [21] procedure, since NCSB-Lazy behaves overall better [7]. The results are in Fig. 3b and the bottom part of Table 1. Again, both the mean and the median are lower for NCSB-MaxRank. The scatter plot shows that the effect of the optimization is stronger when the generated state space is larger (for BAs where the output had ≥ 150 states, our optimization was never worse).

#### **5.2 Comparison with Other Tools**

In the second part of the experimental evaluation, we compared Ranker with other state-of-the-art tools for BA complementation. We measured how small output BAs we can obtain, therefore, we compared the number of states after

**Table 2.** Statistics for our experiments. The table compares the sizes of complement BAs obtained by Ranker and other approaches (after postprocessing). The **wins** and **losses** columns give the number of times when Ranker was strictly better and worse. The values are given for the three datasets as "all (random : LTL : Automizer)". Approaches in Goal are labelled with -.


reduction using autfilt (with the simplification level --high). The scatter plots in Fig. 4 compare the numbers of states of automata generated by Ranker, RankerOld, and Spot. Summarizing statistics are given in Table 2. The backoff strategy in Ranker was applied in 278 (264:1:13) cases.

First, observe that Ranker significantly outperforms RankerOld, especially in the much lower number of timeouts, which decreased by 65 % (moreover, 66 of the 158 timeouts were due to the timeout of autfilt in postprocessing). The higher mean of Ranker compared to RankerOld is also caused by less timeouts). From Table 2, we can also see that Ranker has the

**Table 3.** Run times of the tools [s] given as "all (random : LTL : Automizer)"


smallest mean and median (except Roll and RankerOld, but they have a much higher number of timeouts). Ranker has also the second lowest number of timeouts (Spot has the lowest). If we look at the number of **wins** and **loses**, we can see that Ranker in majority of cases produces a strictly smaller automaton compared to other tools. In Table 3, see that the run time of Ranker is comparable to the run times of other tools (much better than Goal and Roll, comparable with Seminator 2, and a bit worse than Spot and LTL2dstar).

**Acknowledgements.** We thank the anonymous reviewers for their useful remarks that helped us improve the quality of the paper, the artifact evaluation committee for their thorough testing of the artifact, and Alexandre Duret-Lutz for useful feedback on an earlier version of the paper. This work was supported by the Czech Ministry of Education, Youth and Sports project LL1908 of the ERC.CZ programme, the Czech Science Foundation project 20-07487S, and the FIT BUT internal project FIT-S-20- 6427.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Deductive Verification and Decision Procedures**

## **Even Faster Conflicts and Lazier Reductions for String Solvers**

<sup>1</sup> Department of Computer Science, Stanford University, Stanford, USA noetzli@cs.standford.edu

Andres N¨otzli1(B) , Andrew Reynolds<sup>2</sup> ,

<sup>2</sup> Department of Computer Science, The University of Iowa, Iowa City, USA <sup>3</sup> Universidade Federal de Minas Gerais, Belo Horizonte, Brasil

**Abstract.** In the past decade, satisfiability modulo theories (SMT) solvers have been extended to support the theory of strings and regular expressions. This theory has proven to be useful in a wide range of applications in academia and industry. To accommodate the expressive nature of string constraints used in those applications, string solvers use a multi-layered architecture where extended operators are reduced to a set of core operators. These reductions, however, are often costly to reason about. In this work, we propose new techniques for eagerly discovering conflicts based on equality reasoning and lazily avoiding reductions for certain extended functions based on lightweight reasoning. We present a strategy for integrating and scheduling these techniques in a CDCL(T) based theory solver for strings and regular expressions. We implement the techniques and the strategy in cvc5, a state-of-the-art SMT solver, and show that they lead to a significant performance improvement.

#### **1 Introduction**

Most software processes strings and, as a result, modern programming languages integrate rich functionality to represent and manipulate strings. The semantics of string-manipulating functions are often complex, which makes reasoning about them challenging. In recent years, researchers have proposed various approaches to tackle this challenge with dedicated solvers for string constraints [3,5,11,19,21], often as extensions of satisfiability modulo theories (SMT) solvers [10]. Dedicated solvers have been successfully used in a wide range of applications, including: finding or proving the absence of SQL injections and XSS vulnerabilities in web applications [30,32,35]; reasoning about access policies in cloud infrastructure [6,7,13]; and generating database tables from SQL queries for unit testing [34].

SMT solvers are frequently used as back ends for formal tools that reason about software or hardware. These tools typically produce a mix of easy and hard proof obligations that must be discharged by the solver. For many applications,

c The Author(s) 2022

This work was supported by a gift from Amazon Web Services.

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 205–226, 2022. https://doi.org/10.1007/978-3-031-13188-2\_11

it is crucial that the SMT solver responds quickly, and modern solvers are finely tuned to deliver the required performance. String solvers often stratify reasoning about constraints by combining different reasoning techniques rather than relying on a single, monolithic procedure. Specifically, it is common for a string solver to have a core procedure that processes only a basic language of string constraints with a minimal set of string operators. *Extended constraints*, containing additional operators, are supported by applying transformations that reduce them to combinations of basic constraints. Optimizations to this design have been explored in previous work, e.g., by simplifying extended string constraints based on the current *context* (i.e., the current set of asserted constraints) [29]. However, existing techniques still sometimes fall short for industrial applications, which continue to require richer languages of constraints while expecting the underlying solvers to remain efficient. To meet these needs, string solvers must have an even greater understanding of extended constraints and be equipped with fast procedures that leverage this knowledge.

In this work, we focus on CDCL(T)-based SMT solvers [26], where solving is done through the cooperation of a SAT solver and one or more theory solvers. The SAT solver is responsible for finding truth assignments M that satisfy the Boolean abstraction of the input formula, and the theory solvers are responsible for returning *conflict clauses* (disjunctions of literals that are valid in the theory T but are falsified by M) and, optionally, *lemmas* (selected clauses that are valid in T). The conflict clauses and lemmas from theory solvers are then added to the original input formula, and the process of finding a satisfying assignment M is repeated until no conflicts are detected, indicating that the input formula is satisfiable in T, or an unrecoverable conflict is derived, indicating that the input is unsatisfiable in T. Theory reasoning done while the SAT solver is constructing the assignment M is characterized as *eager*. Theory reasoning done after a full assignment has been computed is called *lazy*.

Inspired by real-world benchmarks, we propose new techniques for string solvers that make them more eager, and hence *faster*, in their discovery of conflicts and *lazier* in reducing constraints that are hard to handle such as, for instance, negated regular expression membership constraints. For the former, we extend the congruence closure [24] module at the heart of the string solver to perform selected theory-specific forms of reasoning including eager evaluation, reasoning based on inferred prefixes and suffixes, and (integer) arithmetic approximations (Sect. 3). For the latter, we introduce several new techniques for avoiding reductions involving extended string operators (Sects. 4 and 5). This set of techniques is particularly useful for satisfiable benchmarks, where it is possible to determine that a (candidate) model indeed satisfies the input formula without having to fully process extended constraints. We have designed these techniques to be compatible with most existing solving techniques for strings. In Sect. 6, we propose an extended strategy that describes the integration of the new techniques within an existing string solver.

In summary, our contributions are as follows:

– We describe new techniques for eagerly detecting conflicts based on an enriched congruence closure procedure for the theory of strings.


#### **1.1 Related Work**

As mentioned above, string solvers typically reduce the input constraints to a basic form. Common basic representations include finite automata [14,17,18,31, 33], bit-vectors [19], arrays [20], variations of word equations and length constraints [12,29,32,36], and hybrid approaches that combine word equations and bit-vector representations [23]. Our techniques for lazier reductions are primarily targeted at reductions to word equations, but our other techniques are more broadly applicable and could be used with any of the other basic representations.

In general, the theory of strings is undecidable [12], but modern solvers integrate a wide range of techniques to solve problems that appear in practice. One line of work has been exploring techniques that avoid reductions or make them more efficient. Reynolds et al. [29] describe an approach for lazily performing reductions after simplifying extended functions based on other constraints in the current context. In later work, Reynolds et al. [27] propose the use of aggressive rewriting to eliminate or simplify extended string constraints before performing reductions. In this work, we propose techniques that can be combined with that earlier work to perform reductions even more lazily. Reynolds et al. [28] also proposed a technique for improving the efficiency of reductions by introducing fewer fresh variables. Our approach is orthogonal to this work, because it further avoids reductions, but cannot avoid them entirely.

Both Reynolds et al. [28] and Backes et al. [7] reduce a fragment of regular expression constraints to extended string constraints. In contrast, our approach avoids reductions of certain regular membership constraints.

#### **2 Preliminaries**

We work in many-sorted first-order logic with equality and assume the reader is familiar with the notions of signature, term, literal, (quantified) formula, and free variable (see, e.g., [16]). We consider many-sorted signatures Σ, each containing a family of logical symbols ≈ for equality and interpreted as the identity relation, with input sort σ ˆ σ for all sorts σ in Σ. A Σ-interpretation is a Σ-structure that additionally assigns a value to each variable. A *theory* is a pair T = (Σ, **I**), in which Σ is a signature and **I** is a class of Σ-interpretations, the *models* of T. A Σ-formula ϕ is *satisfiable* (resp., *unsatisfiable*) *in* T if it is satisfied by

**Fig. 1.** Functions in signature of the theory of strings TS.

some (resp., no) interpretation in **I**. By convention and unless otherwise stated, we use letters x, y, z to denote variables and s, t to denote terms.

We consider an (extended) theory T<sup>S</sup> of strings whose signature Σ<sup>S</sup> is given in Fig. 1. We fix a totally ordered finite alphabet A of characters. The signature includes the sorts Str, Lan, Int, and Bool, denoting A<sup>∗</sup>, regular languages over A, integers, and Booleans respectively. The *core* signature is given on the first two lines. It includes the usual symbols of linear integer arithmetic, interpreted as expected. We will write t<sup>1</sup> t2, with P {ą, ă, ď}, as syntactic sugar for the equivalent inequality between t<sup>1</sup> and t<sup>2</sup> expressed using only ě. The core string symbols are given on the second line, and include a constant symbol, or *string constant*, for each word of A<sup>∗</sup> interpreted as that word; a variadic function symbol · ... · : Str ˆ ... ˆ Str → Str, interpreted as word concatenation; and a function symbol | | : Str → Int, interpreted as the word length function. In our examples, we will take a A to be the set of ASCII characters and denote string constants by double-quote-delimited string literals (as in "abc").

The four function symbols in the next two lines of Fig. 1 encode operations on strings that often occur in applications: a substring operator, a string containment predicate, an operation to find the position of one string in another, and one to replace a substring with another. We refer to these function symbols as *extended functions*. For details on the semantics of these operators, see for example [29].

The remainder of the signature covers regular expressions. It includes an infix binary predicate symbol P : Str ˆ Lan Ñ Bool, which denotes word membership in a given regular language. The remaining symbols are used to construct regular expressions. In particular, Σ denotes (the language of) all strings of length one; re(s) denotes the singleton language containing just the word denoted by s; rcon(R1, ··· , R*n*) denotes all strings that are a concatenation of strings denoted by the arguments; the Kleene star operator R<sup>∗</sup> denotes all strings that are obtained as the concatenation of zero or more repetitions of the strings denoted by R; inter(R1, ··· , R*n*) denotes the intersection of the languages denoted its arguments; and union(R1, ··· , R*n*) denotes the union of the languages denoted by its arguments. Finally, we include the class of all indexed regular expression symbols of the form range*<sup>c</sup>*1*,c*<sup>2</sup> where c<sup>1</sup> and c<sup>2</sup> are string constants of length one. We call this a *regular expression range* and interpret it

as the language containing all strings of length one that are between c<sup>1</sup> and c<sup>2</sup> (inclusive) in the ordering associated with A.

#### **3 Eager Equality-Based Conflicts for Strings**

We consider theory solvers for strings like those described by Liang et al. [21], which have at their core a congruence closure algorithm that determines whether a set of string constraints S is satisfiable in the empty theory (i.e., all function symbols, including string operations, are treated as uninterpreted). In this section, we describe two enhancements to such congruence closure algorithms, which can help detect theory-inconsistencies in S. We stress that our extended congruence closure is computed eagerly and *incrementally* as the SAT solver assigns truth values to string equalities. This enables the enhanced congruence closure algorithm to detect theory inconsistencies early, when the truth assignment is still only partially specified. We elaborate on how this enables eager backtracking in Sect. 6.

#### **3.1 Enhancing Congruence Closure with Evaluation**

The string solver implements a procedure to compute the congruence closure C(S) over the set S of currently asserted string equalities. Let T (S) be the set of all terms and subterms in S. Formally, C(S) is the set of all equalities between terms in T (S) that are entailed by the empty theory:

$$\mathcal{C}(\mathbb{S}) = \{ s \approx t \mid s, t \in \mathcal{T}(\mathbb{S}), \mathbb{S} \mid = s \approx t \}$$

The output of the procedure that computes C(S) can be represented as a set of *equivalence classes*, that is, a partition of T (S) where each block of the partition is a maximal set of equivalent terms. For each equivalence class, we designate a unique term in it as the *representative* for that class; if the class contains at least one constant term, then the representative must be one of them. We will denote by [t] the equivalence class of a term t induced by C(S). By a slight abuse of notation we will use [t] also to denote the representative of that class.

Computing the congruence closure C(S) allows the string solver to detect theory conflicts in the current context which occur when the context contains a disequality s ≈ t, where [s]=[t]. It also allows the string solver to propagate to the SAT solver entailed equalities that occur in the input formula but have not been explicitly asserted yet.

By default, congruence closure procedures effectively treat theory symbols as uninterpreted functions. Here, we propose a lightweight approach for injecting some theory-specific reasoning by *evaluating* string terms whenever possible. Specifically, for every term that is a function application f(t1,...,t*n*), where f is a string theory symbol, if the representatives [t1],..., [t*n*] are all constants, the enhanced congruence closure procedure adds the equality f(t1,...,t*n*) ≈ f([t1],..., [t*n*])↓ to C(S), where f([t1],..., [t*n*])↓ is the constant resulting from

the evaluation of f([t1],..., [t*n*]). Adding these equalities improves the ability of the congruence closure layer to detect more theory conflicts and propagations, as illustrated in the following example.

*Example 1.* Consider the constraints {<sup>y</sup> <sup>≈</sup> "b", z <sup>≈</sup> replace(x, y, "d"), x <sup>≈</sup> z, x <sup>≈</sup> "abc"}, where the term replace(x, y, "d") denotes the result of replacing the first occurrence of <sup>y</sup> in <sup>x</sup> by "d" if one exists. The congruence closure for this set of constraints determines the following equivalence classes, each with a constant representative:

$$\{\text{"\textbf{"}b"},y\}, \quad \{\text{"\textbf{"}d"}\}, \quad \{\text{"\textbf{"}abc"},x,z,\text{"}\text{\*\*place}(x,y,\text{"d"})\}\dots$$

This means that the term replace(x, y, "d") is equivalent to the concrete term replace("abc", "b", "d"). Evaluating the latter results in the constant "adc". Hence, the congruence closure procedure will add the equality replace(x, y, "d") <sup>≈</sup> "adc" to its input set of equalities and recompute the congruence closure. This will cause the third equivalence class in the list above to contain the (distinct) string constants "abc" and "adc", thus resulting in a conflict.

In our implementation, we must track explanations for inferred equalities for the purposes of reporting conflict clauses. In the above example, the equality replace(x, y, "d") <sup>≈</sup> "adc" is added to the congruence with the explanation <sup>x</sup> <sup>≈</sup> "abc" ^ <sup>y</sup> <sup>≈</sup> "b", which is then used in the standard technique for constructing explanations for congruence-closure-based reasoning [25].

We remark that enhancing congruence closure with evaluation is not specific to the theory of strings, and can be leveraged by other theory solvers based on congruence closure. Further exploration of this technique and its impact on other theories is left as future work.

#### **3.2 Tracking Properties of Equivalence Classes**

In addition to the use of evaluation, we enhance our congruence closure procedure with further information that can be used to discover conflicts eagerly based on string-specific reasoning. We describe two examples of this mechanism below.

First, we maintain a mapping Z from integer equivalence classes e to intervals of the form [, u], indicating concrete lower and upper bounds on the value that the terms in e can have. Open intervals are achieved by letting and u be ´8 and 8 respectively. The interval can be inferred using string-specific reasoning over the terms in e.

Second, we maintain a mapping S from string equivalence classes e to a pair of string constants (l1, l2) denoting the maximal known prefix l<sup>1</sup> and suffix l<sup>2</sup> of the value that the terms in e can have. For example, if e contains the term "abc" · <sup>x</sup> then <sup>l</sup><sup>1</sup> for <sup>e</sup> is, at least, "abc". When no prefix is known, <sup>l</sup><sup>1</sup> is the empty string. The suffix l<sup>2</sup> is handled similarly.

Figure 2 shows how the maps Z and S are updated when new equivalence classes are created (newEqc) and when equivalence classes are merged

$$t: \text{Int} \qquad \mathcal{Z}\begin{bmatrix} t \end{bmatrix} := \begin{cases} [n, n] & \text{if } t = n \\ [\ell\_{|s|}, u\_{|s|}] & \text{if } t = |s| \\ [-\infty, \infty] & \text{otherwise} \end{cases}$$

$$t: \text{Str} \qquad \mathcal{S}\begin{bmatrix} t \end{bmatrix} := \begin{cases} (t, t) & \text{if } t \text{ is a constant} \\ (l\_1, l\_2) & \text{if } t \text{ reduces to } l\_1 \cdot t' \cdot l\_2 \text{ with } l\_1, l\_2 \text{ constants} \end{cases}$$




**Fig. 2.** Methods for tracking intervals, prefixes, and suffixes for equivalence classes.

(mergeEqc), the two basic methods that are used when computing congruence closures. For the second method, a helper method (mergeEntry) is used to combine the contents of the entries in two maps. We assume without loss of generality that when mergeEqc is called on equivalence classes ([t1], [t2]), [t1] becomes the new representative for the merged class.

We now look at these methods in more detail. When a new equivalence class for term t is created, we look at the type of t. If t has integer type, there are three cases. If t is a numeral n, it is mapped to the interval [n, n]. If t is a length term of the form |s|, then we compute an interval [|*s*|, u|*s*|] where |*s*<sup>|</sup> (resp., u|*s*|) is a sound under-approximation (resp., over-approximation) of the length of s. We use the procedure described by Reynolds et al. [27] to compute these approximations. We use it because it is available, well-tested, and designed to be fast, but any sound approximation could be used. Otherwise, t is mapped to the open interval [´8, 8]. If t has string type, we consider two cases. If t is a string constant, its prefix and suffix are both set to t. If t can be normalized using a simple set of rewrite rules to a concatenation term of the form l<sup>1</sup> · t · l2, where l<sup>1</sup> and l<sup>2</sup> are string constants of maximal length and t is a non-constant term, then t is mapped to the pair (l1, l2). Note that the notation l<sup>1</sup> ·t · l<sup>2</sup> is meant to include the case where either l<sup>1</sup> or l<sup>2</sup> (or both) is the empty string.<sup>1</sup>

When two equivalence classes [t1] and [t2] are merged, first, if [t1] is and [t2] is a regular expression membership predicate x P R, then we may infer information about x, because x P R is now known to be true in the current context. We compute upper and lower bounds [|*R*|, u|*R*|] on the length of all strings that occur in R. We use fast approximate techniques for computing these bounds (e.g., sum the length of constant components of concatenations to infer lower bounds). Note that these techniques are context-independent and are solely based on the structure of R. We update the entry Z [x] based on this information. Similarly, we update the entry S [x] with information about the constant prefix and suffix of the regular expression R. On the other hand, when [t1] and [t2] are integer or string equivalence classes, we merge the entries for the appropriate mapping. We stress that the entry for [t1] is updated with the information from the entry for [t2] and not vice versa. This is because [t1] is the new representative of the merged equivalence class, and further merges may refer to it, while [t2] is subsequently unused.

When merging entries, we may determine that the constraints represented by the two entries are inconsistent, in which case we have found a conflict. For example, when merging integer equivalence classes, if the lower bound for one equivalence class is greater than the upper bound for the other, we raise a conflict. For string equivalence classes, a conflict is raised if the prefixes for the two equivalence classes are incompatible (i.e., neither is a prefix of the other) and similarly for suffixes. We write p<sup>1</sup> ∼*pre* p<sup>2</sup> (resp., s<sup>1</sup> ∼*suf* s2) to denote that p<sup>1</sup> is not a prefix of p<sup>2</sup> or vice versa (resp., s<sup>1</sup> is not a suffix of s<sup>2</sup> or vice versa), and max<sup>|</sup> <sup>|</sup> to denote the function returning the string constant having maximum length. If no conflict is raised, then the new entry E<sup>1</sup> is updated to contain the merged information: for integers, we take the maximal lower bound and minimal upper bound; and for strings, we take the prefix or suffix of maximal length.

In the context of CDCL(T), when the procedure raises a conflict, it is required to return a *conflict clause*, which in turn will cause the solver to backtrack. To make it possible to compute conflict clauses in the methods described above, each component of the entries for an equivalence class e in the two maps Z and S is additionally annotated with an explanation pair (t, ϕ), where t is a term in e and ϕ entails that t has the property represented by the component. This is maintained independently for each lower bound, upper bound, prefix and suffix. In most cases, this pair is of the form (t, ), where t is the source of the annotation. When inferring annotations from an asserted membership constraint x P R during mergeEqc above, their explanations are the pair (x, x P R). Explanations are updated when entries E<sup>1</sup> and E<sup>2</sup> are merged, where, e.g., the explanation for the lower bound is taken from E<sup>2</sup> when <sup>2</sup> ą 1. When

<sup>1</sup> It is possible to produce tighter prefixes and suffixes recursively—for instance for terms t<sup>1</sup> · t - · t<sup>2</sup> where the equivalence class of t<sup>1</sup> (resp., t2) is assigned a constant prefix (resp., suffix). However, in our experiments, this did not turn out to be worth the extra effort.

two entries are in conflict, the explanations are used to generate the conflict. For example, assuming two entries have explanations (t1, ϕ1) and (t2, ϕ2), we send the conflict clause ¬(t<sup>1</sup> ≈ t<sup>2</sup> ^ ϕ<sup>1</sup> ^ ϕ2). The equality t<sup>1</sup> ≈ t<sup>2</sup> may be further expanded using standard methods for explanations during congruence closure [25].

*Example 2.* Consider the constraints {<sup>x</sup> <sup>P</sup> rcon(re("a"), <sup>Σ</sup>∗,re("b")), z <sup>≈</sup> "bcd" · w, x ≈ z}. The state of the map S after processing each assertion is as follows:


When the first constraint <sup>x</sup> <sup>P</sup> rcon(re("a"), <sup>Σ</sup>∗,re("b")) is asserted, we construct the (Boolean) equivalence class for this constraint and merge it with []. Based on the mergeEqc method, we infer that the prefix and suffix for the string equivalence class [x] are "a" and "b" respectively, which are added to <sup>S</sup> to obtain <sup>S</sup><sup>1</sup> When the second constraint is asserted, we infer the prefix "bcd" for [z] and add it to S<sup>1</sup> to get S2; no suffix is inferred since we do not know the value of w. When the third constraint is asserted, the equivalence classes [x] and [z] merge. Since we have inferred that "a" is a prefix of [x] and "bcd" is a prefix of [z], we have a conflict, as these two strings do not have a common prefix. Our procedure will thus report a conflict containing the three constraints.

*Example 3.* Consider the constraints {|s<sup>|</sup> ≈ <sup>0</sup>, <sup>|</sup>"abc" · <sup>w</sup><sup>|</sup> ≈ <sup>0</sup>, x <sup>≈</sup> s, x <sup>≈</sup> "abc" · w}, where s is the term substr(y, 0, 2), which takes the substring of y at position 0 of length (at most) 2. The state of the map Z after processing each assertion is as follows:


When the first constraint |s| ≈ 0 is asserted, we construct the equivalence classes [0] and [|s|]. The former trivially has bounds [0, 0]. For the latter, we use the methods from [27] to infer lower and upper bounds for |s|. Note that every string has a lower length bound of 0. The upper bound for the length of substr(y, <sup>0</sup>, 2) can easily be inferred to be 2. Similarly, when <sup>|</sup>"abc" · <sup>w</sup><sup>|</sup> ≈ 0 is asserted, the equivalence class [|"abc" · <sup>w</sup>|] is created, whose length has a lower bound of 3 and no upper bound. After the latter two constraints are asserted, note that <sup>s</sup> becomes equal to "abc" · <sup>w</sup> by transitivity, and hence <sup>|</sup>s<sup>|</sup> is equal to <sup>|</sup>"abc" · <sup>w</sup><sup>|</sup> by congruence. When these two equivalence classes merge, we obtain

a conflict from their respective entries in Z, since the former has an upper bound of 2 and the latter has a lower bound of 3. Thus, our procedure returns the latter two constraints as a conflict.

#### **4 Model-Based Reductions for Strings**

The bottleneck for string solving often lies in reasoning about the reductions of extended string functions. Context-dependent simplification can greatly improve the scalability of string solvers for extended string constraints [29]. At a high level, this approach attempts to simplify extended terms based on information that holds in the current context, which can preempt the need for potentially expensive reasoning. In this work, we extend this strategy by additionally reasoning about candidate models.

First, we briefly review how extended string terms are reduced to more basic constructs. A *reduction formula* for term t is a formula ϕ ^ t ≈ k, where k is a fresh variable and ϕ is a formula over terms k, t1,...,t*<sup>n</sup>* that *characterizes* the meaning of t in the sense that a theory interpretation satisfies ϕ if and only if it satisfies t ≈ k. As a result, the formula ∃ k. (ϕ ^ t ≈ k) is valid in the theory, and hence its Skolemized version can be given to the SAT solver as a lemma. This effectively reduces the satisfiability of constraints of the form c[t] to the satisfiability of c[k] ∧ ϕ, where t has been replaced by k.

*Example 4.* Let <sup>t</sup> be the regular expression membership constraint <sup>x</sup> <sup>P</sup> re("a")∗. The formula (<sup>k</sup> <sup>≈</sup> (<sup>x</sup> <sup>≈</sup> \_ <sup>x</sup> <sup>P</sup> re("a") \_ <sup>ψ</sup>)) ^ <sup>t</sup> <sup>≈</sup> <sup>k</sup> where <sup>ψ</sup> is

<sup>∃</sup>k1k2k3. x <sup>≈</sup> <sup>k</sup><sup>1</sup> · <sup>k</sup><sup>2</sup> · <sup>k</sup><sup>3</sup> ^ <sup>k</sup><sup>1</sup> <sup>P</sup> re("a") ^ <sup>k</sup><sup>2</sup> <sup>P</sup> re("a") <sup>∗</sup> ^ <sup>k</sup><sup>3</sup> <sup>P</sup> re("a")

is a reduction for t.

Reductions like the one above can be expensive to reason about, since they may introduce fresh (possibly universally) quantified variables. Context-dependent simplifications can avoid these reductions in some cases.

Given a string term t of the form f(t1,...,t*n*), where f is an extended function, a *context-dependent simplification* is a formula of the form (t<sup>1</sup> ≈ s<sup>1</sup> ^ ... ^ t*<sup>n</sup>* ≈ s*n*) ñ t ≈ l where l is the constant value obtained by evaluating or rewriting f(s1,...,s*n*). Whenever possible, we use context-dependent simplifications for extended string terms, where t<sup>1</sup> ≈ s1,...,t*<sup>n</sup>* ≈ s*<sup>n</sup>* are equalities that hold in the current context. The same approach can be applied to regular expression memberships as well, where a membership constraint of the form x P R can be simplified to or ⊥ whenever x is inferred to be equal to a concrete string literal.

*Example 5.* Let <sup>t</sup> be as in the previous example. The formula <sup>x</sup> <sup>≈</sup> "b" <sup>ñ</sup> <sup>t</sup> ≈ ⊥ is a context-dependent simplification for t. 2

While context-dependent simplification eliminates some reductions, in this paper we propose making certain reductions even lazier by taking into account *candidate* models. If a candidate model can be built that already satisfies a constraint with extended terms, it is not necessary to reduce it.

To elaborate, existing procedures for strings [21] are able to construct candidate models M (or, more precisely, interpretations) for satisfiable sets of string constraints before reductions are considered by treating all (sub)terms headed by an extended function as fresh variables, and by ignoring regular expression membership constraints. A strategy for *model-based* reduction only considers reductions for t if the candidate model M is inconsistent with the semantics of t—something that can be easily checked by evaluating t in the model and verifying that the computed value coincides with the value that M assigns to t as a variable. This allows us to avoid reductions for cases where a candidate model is correctly guessed in the presence of extended functions and regular expression membership constraints. A concrete instantiation of this strategy is described in Sect. 6.

*Example 6.* Consider the constraints {<sup>x</sup> <sup>≈</sup> <sup>y</sup> · "c",¬<sup>x</sup> <sup>P</sup> rcon(Σ∗, re("j"), <sup>Σ</sup>∗)}. A model-based reduction strategy would first construct a candidate model that satisfies the first constraint, e.g., <sup>M</sup> <sup>=</sup> {<sup>x</sup> <sup>Ñ</sup> "abc", y <sup>Ñ</sup> "ab"}. It would then check whether the membership constraint <sup>x</sup> <sup>P</sup> rcon(Σ∗,re("j"), <sup>Σ</sup>∗) evaluates to false in <sup>M</sup>. This is indeed the case, since <sup>x</sup><sup>M</sup> <sup>=</sup> "abc", making <sup>M</sup> a model for the full set of constraints. Hence, the reduction for the regular membership constraint in this example can be avoided altogether.

#### **5 Fast Techniques for Regular Expression Inclusion**

As mentioned in Sect. 4, regular expression memberships are handled by a lazy reduction, which can be seen as a single-step unfolding. While model-based reductions can avoid some reductions, the remaining ones may still be expensive. In this section, we show another technique to avoid reductions, based on the observation that most regular expressions in real programs are relatively simple. We focus on those of the form rcon(R1,...,R*n*), where each R*<sup>i</sup>* corresponds to a fixed or arbitrary number of range or constant regular expressions. Such regular expressions are frequently used to match a string that is made up of multiple segments, each with a different alphabet. For this fragment of regular expressions, our procedure allows us to detect conflicts before unfolding and may additionally tell us which regular expression memberships are entailed by others, and hence can be discarded.

We use the notation L(R1) ⊆ L(R2) to denote that R<sup>1</sup> matches a subset of the strings matched by R2. The derivation rules in Fig. 3 can be used to implement a

<sup>2</sup> We omit from the implication the trivial antecedent re("a") <sup>∗</sup> ≈ re("a") ∗.

$$\begin{aligned} \mathsf{Funp}\frac{\mathsf{E}(\mathsf{u}^{\ast\mathsf{w}}) \subseteq \mathcal{L}(R^{\mathsf{x}})}{\mathcal{L}(R)} & \quad \mathsf{Start}\frac{\mathsf{Str}\mathsf{Ar}\overline{\mathcal{L}(R)} \subseteq \mathcal{L}(R^{\mathsf{x}})}{\mathcal{L}(R)} \\\\ \mathsf{All}\frac{\mathsf{E}(R) & \subseteq \mathcal{L}(\Sigma^{\mathsf{x}})}{\mathcal{L}(R)} & \quad \mathsf{Ref}\frac{\mathsf{P}\mathsf{H}\mathsf{I}\_{2}}{\mathcal{L}(R) \subseteq \mathcal{L}(R)} \\\\ \mathsf{Trans}\frac{\mathcal{L}(R\_{1}) & \subseteq \mathcal{L}(R\_{2}) & \mathcal{L}(R\_{3}) \subseteq \mathcal{L}(R\_{3})}{\mathcal{L}(R\_{1}) & \leq \mathcal{L}(R\_{3})} & \quad \mathsf{Config} \,\mathsf{Star}\frac{\mathcal{L}(R\_{1}) \subseteq \mathcal{L}(R\_{2})}{\mathcal{L}(R\_{1}^{\mathsf{x}}) \subseteq \mathcal{L}(R\_{2}^{\mathsf{x}})} \\\\ \mathsf{far}\,\frac{\mathsf{For each } x \in \mathcal{L}(R), \,|x| = 1 & \quad \mathsf{Range}\frac{c\_{1} \geqslant c\_{3} \quad c\_{2} \leqslant c\_{4}}{\mathcal{L}(\mathsf{range}\_{c\_{1}, c\_{2}}) \subseteq \mathcal{L}(\mathsf{range}\_{c\_{3}})} \,\,. \end{aligned}$$

$$\mathsf{Concat} \frac{\mathcal{L}(R\_1) \subseteq \mathcal{L}(R\_3) \quad \mathcal{L}(R\_2) \subseteq \mathcal{L}(R\_4)}{\mathcal{L}(\mathsf{rcon}(R\_1, R\_2)) \subseteq \mathcal{L}(\mathsf{rcon}(R\_3, R\_4))}$$

**Fig. 3.** Rules for deriving L(R1) ⊆ L(R2).

fast, incomplete procedure to prove L(R1) ⊆ L(R2). The procedure applies the rules bottom-up to build a derivation tree with L(R1) ⊆ L(R2) as the root. The statement is proven if a derivation tree is found where all leaves have no preconditions. For any given pair of regular expressions, the number of possible rule applications is finite, and whether a rule applies can be checked in polynomial time w.r.t. the number of elements in the regular expression concatenations.

The first four rules in Fig. 3 have no preconditions. A regular expression R matches zero or more occurrences of R and the rules Emp and Star use that fact to conclude that (the language generated by) R<sup>∗</sup> includes the empty string, corresponding to zero occurrences of R, and (the language generated by) R, corresponding to a single occurrence of R. The third rule, All, concludes that every R is included in Σ∗, which matches all strings. Finally, Refl captures the reflexivity of the regular expression inclusion relation. Regular expression inclusion is transitive, which is captured by Trans. Additionally, CongStar captures that applying the Kleene star to regular expressions preserves the inclusion relation. The next two rules are related to regular expressions that match single characters: Char concludes that if a regular expression matches only single characters then it is included in Σ, which matches all characters; Range compares the bounds of two ranges to determine if one is included in the other. Finally, the rule Concat splits regular expression concatenations into two parts and ensures that the parts on the right-hand side include the parts on the left-hand side. Note that the splits themselves can be concatenations, so there is a choice regarding how those concatenations are split into two parts. In the context of this rule, we treat regular expressions that match a single word as a concatenation of the individual letters of that word. For example, for <sup>L</sup>("abc") ⊆ L(rcon("ab", Σ)), we could choose the subgoal <sup>L</sup>("c") ⊆ L(Σ) after applying Concat.

Given a regular expression inclusion L(R1) ⊆ L(R2), the above procedure may potentially derive conflicts or propagate regular membership constraints, avoiding reducing them. A conflict can be derived from membership constraints x P R<sup>1</sup> and ¬y P R<sup>2</sup> if x ≈ y is entailed by the current context. Similarly, from x ≈ y being entailed and y P R<sup>1</sup> being asserted, we can propagate the regular membership constraint x P R2; and from x ≈ y and ¬y P R<sup>2</sup> we can propagate ¬x P R1.

*Example 7.* Consider the following theory literals:

$$x \in \mathsf{rcon}((\mathsf{range}\_{0,9})^\*, \Sigma^\*, \mathsf{"b"}, \Sigma^\*)\tag{1}$$

$$\neg x \in \mathsf{rcon}((\mathsf{range}\_{0,9})^\*, \Sigma^\*)\tag{2}$$

We can apply Concat, Refl, and All to the two regular expressions:

$$\mathsf{Concat} \frac{\mathsf{Refl}}{\mathcal{L}((\mathsf{range}\_{0,9})^{\*}) \subseteq \mathcal{L}((\mathsf{range}\_{0,9})^{\*})} \, \mathsf{All} \frac{\mathsf{All} \, \overline{\mathcal{L}(\mathsf{rcon}(\Sigma^{\*}, \mathsf{re}(\mathsf{"b"}), \Sigma^{\*})) \subseteq \mathcal{L}(\Sigma^{\*})}}{\mathcal{L}(\mathsf{rcon}((\mathsf{range}\_{0,9})^{\*}, \Sigma^{\*}, \mathsf{re}(\mathsf{"b"}), \Sigma^{\*})) \subseteq \mathcal{L}(\mathsf{rcon}((\mathsf{range}\_{0,9})^{\*}, \Sigma^{\*}))}$$

This allows us to derive a conflict, since the regular expression of the negative membership constraint in Eq. (2) includes the regular expression in the positive regular membership constraint in Eq. (1).

#### **6 An Extended Strategy for Strings in CDCL(***T* **)**

In this section, we summarize our overall strategy for solving string constraints that leverages the aforementioned techniques. This strategy integrates the techniques presented in this paper with existing techniques used in modern string solvers. In general, the techniques presented in this work are applicable to a wide range of solvers. The techniques from Sect. 3 can be combined with any string solver that computes the congruence closure of the constraints. Modelbased reductions are applicable to string solvers that can compute models and have the infrastructure to selectively refine/ignore certain constraints. Regular expression inclusion can be used in all string solvers.

Recall that in a CDCL(T)-based SMT solver, the theory solvers produce conflict clauses or lemmas based on the content of the current context, the truth assignment incrementally constructed by the SAT solver. In the following, we split the discussion between checks that are performed on partial assignments and checks that are performed on full assignments from the SAT solver.

*Checking Partial Assignments.* Recall that M is the assignment to literals chosen by the SAT solver. In our implementation, whenever the SAT solver adds a literal (¬)t ≈ s to M, that literal is immediately added to the congruence closure data structure of the appropriate theory.<sup>3</sup> This means that in a typical configuration,

<sup>3</sup> In our implementation, each theory locally maintains its own congruence closure data structure.

**Fig. 4.** Strings theory solver using context-dependent simplification, regular expression inclusion, and model-based reductions.

conflicts that are based purely on equality reasoning may be raised the moment M becomes unsatisfiable in the theory. This behavior makes the SMT solver faster, as it may backtrack without having to generate any further extension to M. The techniques in Sects. 3.1 and 3.2 increase the likelihood that such conflicts may be discovered eagerly based on evaluation, arithmetic approximations, and tracking prefixes and suffixes for string terms. Given that those techniques are executed every time the SAT solver assigns a value, it is imperative that they are inexpensive.

*Checking Full Assignments.* When a full assignment is generated by the SAT solver, each theory solver is called upon to do a *full effort* consistency check on the assignment M. We describe the strategy used for strings that incorporates reasoning about context-dependent simplification, regular expression inclusion, and model-based reductions.

Our approach checkFull is sketched in Fig. 4, which summarizes the behavior of our (extended) theory solver for strings to be used in the CDCL(T) loop. The method takes as input a set of string constraints S, which is the subset of the literals assigned by the SAT solver that belongs to the theory of strings. We assume the method is called when S is satisfiable in the empty theory, and is such that the techniques from Sect. 3 did not raise a conflict. It calls the subprocedure getRefineExt, which returns a set of formulas F. This set may contain a *conflict clause*, that is, a disjunction of literals that are false in S. If F is non-empty, these formulas are returned to the SAT solver. Otherwise, if F is empty, then the method returns SAT, indicating that S is satisfiable.

In the subprocedure getRefineExt, we first classify the extended terms t from S by adding them to (at most) one of three sets: the set of terms C to simplify based on the context, the set of terms E to reduce, and the set of terms E*<sup>m</sup>* to reduce if necessary based on a candidate model. This is done as follows. We first check if term t can be simplified based on the context, that is, if we can infer that its arguments are equivalent to terms s1,...,s*<sup>n</sup>* such that f(s1,...,s*n*) can be simplified to a constant c. In this case, t is added to C if it is not already entailed in S to be equal to c. Otherwise, if t is a regular expression membership x P R, then we check whether t is otherwise directly in conflict with another membership or can be discarded. The former holds when it is the case that x P R holds with negative polarity, there exists a term x that is entailed to be equal to x such that x P R is entailed to hold with positive polarity, and our regular expression inclusion test can prove that the language of R includes that of R . In this case, we know that we are in conflict since x cannot be both in R and not in R, and a conflict clause is returned. Otherwise, we may avoid reducing t if it is entailed by another membership x P R with the same polarity again where x is entailed equal to x. This may occur if the language of R includes R and the polarity of both memberships are positive, or if R includes R and the polarity of both memberships are negative. If none of these cases hold, then we add t to E if it is a positive membership, and E*<sup>m</sup>* otherwise. Here, the intuition is that *negative* memberships are both more expensive to reason about via reductions, and more likely to be satisfied by candidate models. All other extended terms are added to E, marking them to be reduced. Although not shown in the figure, if t is an application of string containment, then it is handled analogously to regular expression membership, noting that ctn(x, y) is equivalent to x P rcon(Σ∗,re(y), Σ∗).

Assuming the above classification, we run four steps in decreasing order of priority. First, if C is non-empty, we add the simplification formula for each t P C, where we write cd simplify(S, t) to denote the formula corresponding to the context-dependent simplification of t in S. Second, we run the core theory solver for strings, denoted by method getRefine, which we assume runs the rulebased procedure from [21]. For our purposes, we assume this method returns a (possibly empty) set of refinement lemmas or conflict clauses, which we denote F and return this set if it is non-empty. Otherwise, if our set E of terms to reduce is non-empty, we return the set of reduction formulas reduce(t) for all t P E. If none of these cases generated lemmas, then we construct a candidate model M for the abstraction of S, denoted α(S), which denotes a formula where all


**Table 1.** Number of solved problems per benchmark set for different configurations. Best results are in **bold**. All benchmarks ran with a timeout of 1200 s.

extended terms in S are replaced by fresh variables. Then, for each t P E*<sup>m</sup>* we check whether the constraint for t holds in the candidate model M. In particular, this is the case if S t ≈ t <sup>M</sup>. We return reduce(t) only for terms t for which this does not hold.

Notice that the model M serves only as a way of filtering our reductions. We do not apply context-dependent simplification based on the model, e.g., adding the lemma (t<sup>1</sup> ≈ t M <sup>1</sup> ^ ... ^ t*<sup>n</sup>* ≈ t <sup>M</sup>*<sup>n</sup>* ) ñ t ≈ f(t M <sup>1</sup> ,...,t<sup>M</sup>*<sup>n</sup>* )↓, as this would introduce an unbounded number of new literals t*<sup>i</sup>* ≈ t M *<sup>i</sup>* to the search.

#### **7 Evaluation**

We have implemented the strategy from Sect. 6 by extending cvc5, a CDCL(T) based state-of-the-art SMT solver that implements context-dependent simplifications [29], aggressive rewriting [27], and efficient reductions [28]. To evaluate our extension, we measure its performance on the 69,907 SMT-LIB benchmarks [9] that include the theory of strings<sup>4</sup> and on a set of 74 benchmarks which we have obtained from an industrial partner but are not allowed to make public. In this section, we present and discuss the results of that evaluation.

We test the performance impact of the four techniques presented in this paper: enhanced congruence closure (**v**), eager conflicts based on properties of equivalence classes (**e**), model-based reductions (**m**), and regular expression inclusion (**r**). We compare a configuration with all techniques enabled (**cvc5**) with configurations that disable individual techniques (prefixed with **cvc5-\***). To measure the combined impact, we additionally include a configuration that disables all

<sup>4</sup> We excluded one benchmark with a quantifier in the quantifier-free logic QF SLIA.

**Fig. 5.** Cactus plot of the number of solved benchmarks. All benchmarks ran with a timeout of 1200 s.

techniques presented in this paper, but otherwise uses all of cvc5's advanced techniques for strings (**cvc5-vmre**). Finally, as an additional reference point, we compare with another state-of-the-art solver, z3 Version 4.8.14 [15]. In our experience, z3 is the most stable, feature-complete competitor to cvc5's string solver. We omit a comparison with z3str4 [23] because it returned wrong answers at SMT-COMP 2021 [2] and there has not been a new release. Similarly, we omit a comparison with z3-Trau 1.1 [1] (the successor of Trau [4]), because we found it to be unsound in earlier work [28]. Finally, Ostrich 1.1 [14] requires inputs to be in the straight-line fragment [22], which is not the case for some of the benchmarks.

We ran all experiments on a cluster equipped with Intel Xeon E5-2620 v4 CPUs. We allocated one physical CPU core and 8 GB of RAM for each solverbenchmark pair and used a time limit of 1200 s, which is the same time limit used at SMT-COMP 2021. In the following presentation of the results, we omit the 59,050 benchmarks that are solved in less than a second by all solvers to emphasize non-trivial benchmarks. Table 1 lists the number of solved benchmarks for each benchmark family and configuration. Figure 5 shows a cactus plot of the number of solved instances for each configuration. The scatter plots in Fig. 6 compare the performance of **cvc5** with the other cvc5 configurations and z3. Each scatter plot shows the solving times of the two solvers for each benchmark and differentiates between satisfiable and unsatisfiable inputs.

Overall, all configurations of cvc5 significantly outperform z3, which is reflected in Fig. 5. The scatter plot Fig. 6f shows that while cvc5 outperforms z3, they also complement each other to a certain extent, which is not surprising given the complexity of the problem and the fact that the two code bases

**Fig. 6.** Scatter plots that compare the performance of cvc5 with the other configurations. The scatter plots differentiate between satisfiable and unsatisfiable benchmarks.

differ significantly. Overall, z3 solves 270 benchmarks that **cvc5-vmre** does not solve and 171 benchmarks that **cvc5** does not solve. Conversely, **cvc5** solves 1645 benchmarks that z3 does not solve. Between **cvc5** and **cvc5-vmre**, **cvc5** uniquely solves 309 benchmarks and **cvc5-vmre** 15 benchmarks. This suggests that our techniques help cvc5 solve some of the benchmarks that previously only z3 could solve, but that they also have a significant impact on benchmarks that z3 could not solve. Thus, adapting those techniques in z3 may be beneficial.

The PyEx benchmarks show the biggest difference in number of solved benchmarks across the techniques, with model-based reductions (**m**) solving 160 more benchmarks, significantly increasing the success rate for **cvc5**. Figure 6c indicates that primarily satisfiable benchmarks benefit from **m**. This is expected because the technique allows the solver to skip reductions if it guesses a correct model. Nevertheless, some unsatisfiable benchmarks are also solved noticeably faster due to **m**. This is possibly due to the technique resulting in a search that prioritizes reducing operators that are more likely to participate in conflicts.

Both the enhanced congruence closure (**v**) and the more eager conflicts (**e**) have a relatively low impact on the number of solved benchmarks. However, Figs. 6a and 6b show they significantly improve solving times on several benchmarks. This is expected because they allow the solver to detect conflicts more

eagerly, but the same or similar conflicts would have been found (later on) with existing techniques. Since the solving procedure does not fundamentally change, roughly the same benchmarks should be solved when adding these techniques, but potentially much faster.

Finally, the regular expression inclusion technique (**r**) has a low impact overall, since it is restricted to a specific fragment, but Fig. 6d shows it significantly improves solving time for a few benchmarks. The benchmarks come from the set of industrial problems and from the QGen set of benchmarks. While the technique does not always apply, we have found it to be very important for certain industrial problems. Moreover, the scatter plot shows that having the technique available has no negative effect, which allows such a specialized procedure to be always active in a modular solver.

#### **8 Conclusion**

We have presented new techniques that make conflict detection more eager and reductions lazier in CDCL(T)-based string solvers. Our evaluation shows that both classes of techniques significantly improve performance in the state-of-theart SMT solver cvc5 on SMT-LIB and industrial problems. As future work, we plan to generalize our eager equality-based conflict detection to leverage more sophisticated properties. We also plan to apply similar techniques to other congruence-closure-based theory solvers, such as those for the theory of finite sets and relations. The set of rules for proving regular expression inclusion was driven by empirical work on industrial benchmarks, but it could be expanded. We also plan to investigate further strategies for lazy reductions of other extended string terms that lead to bottlenecks in real-world applications.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Local Search for SMT on Linear Integer Arithmetic**

Shaowei Cai1,2(B) , Bohan Li1,2 , and Xindi Zhang1,2

<sup>1</sup> State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China {caisw,libh,zhangxd}@ios.ac.cn <sup>2</sup> School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China

**Abstract.** Satisfiability Modulo Linear Integer Arithmetic, SMT (LIA) for short, has significant applications in many domains. In this paper, we develop the first local search algorithm for SMT (LIA) by directly operating on variables, breaking through the traditional framework. We propose a local search framework by considering the distinctions between Boolean and integer variables. Moreover, we design a novel operator and scoring functions tailored for LIA, and propose a two-level operation selection heuristic. Putting these together, we develop a local search SMT (LIA) solver called LS-LIA. Experiments are carried out to evaluate LS-LIA on benchmarks from SMTLIB and two benchmark sets generated from job shop scheduling and data race detection. The results show that LS-LIA is competitive and complementary with state-of-the-art SMT solvers, and performs particularly well on those formulae with only integer variables. A simple sequential portfolio with Z3 improves the state-of-the-art on satisfiable benchmark sets of LIA and IDL benchmarks from SMT-LIB. LS-LIA also solves Job Shop Scheduling benchmarks substantially faster than traditional complete SMT solvers.

**Keywords:** SMT · Local Search · Linear Integer Arithmetic · Integer Difference Logic

#### **1 Introduction**

Satisfiability Modulo Theories (SMT) is the problem of deciding the satisfiability of a first order logic formula with respect to certain background theories. Inspired by the great success of propositional satisfiability (SAT) solving, SMT attempts to generalize the advances of satisfiability solvers from propositional logic to fragments of first order logic. Typical theories supported by SMT include the theories of integers, real numbers, lists, arrays and bit-vectors. The field of SMT

S. Cai, B. Li and X. Zhang—The authors are listed in alphabetical order, as they all contribute significantly.

has seen significant progress in the past two decades. SMT solvers have become important formal verification engines, with applications in various domains.

In this paper, we focus on the theory of *Linear Integer Arithmetic* (LIA), consisting of arithmetic atomic formulae in the form of - <sup>i</sup> aix<sup>i</sup> + c - 0, where -∈ {=, ≤}, c and ai's are rational numbers and xi's are integer variables. Moreover, we are also interested in a popular fragment of LIA, namely *Integer Difference Logic* (IDL), consisting of arithmetic atomic formulae to constrain the difference between pairs of integer variables in the form of a − b ≤ k, where a, b are integer variables and k is integer constant. The SMT problem with the background theory of LIA and IDL, is to determine the satisfiability of the Boolean combination of respective arithmetic atomic formulae and propositional variables, and referred to as SMT (LIA) and SMT (IDL).

SMT (LIA) is important in software verification and automated reasoning, since most programs use integer variables and perform arithmetic operation on them [35]. Specifically, SMT (LIA) has various applications in automated termination analysis [16], sequential equivalence checking [34], and state reachability checking under weak memory models [24]. SMT (IDL) has found applications in problems with timing-related constraints [17], such as hardware models with ordered data structures [23], stable models computing [30], and job shop scheduling [40].

Much effort has been devoted to solving SMT (LIA) and SMT (IDL). The most popular approach is the *lazy* approach [3,41], also known as DPLL(T) [38], which is a central development of SMT. Many DPLL(T) solvers have been developed for SMT (LIA) [7,19] and SMT (IDL) [31,37,47]. In this approach, the formula is abstracted into a Boolean formula by replacing arithmetic atomic formulae with fresh Boolean variables. A SAT solver is used to reason about the Boolean structure and solve the Boolean formula, while a theory solver receives assignments from the SAT solver and performs decision procedure to solve the conjunctions of atomic subformulae, including consistency checking of the assignments and theory-based deduction.

The effort in this approach is mainly devoted to producing more effective theory solvers. Simplex-based linear arithmetic solvers that can be integrated efficiently in the DPLL(T) framework were studied [19]. A simplex-based decision procedure that minimizes the sum of infeasibilities of constraints was proposed [32]. A theory solver made use of layering and several heuristics to achieve good performance [26]. A theory solver called SPASS-IQ was designed to efficiently handle unbounded problems [6,8]. According to recent SMT Competitions,<sup>1</sup> almost all state-of-the-art SMT (LIA) and SMT (IDL) solvers are based on the lazy approach, including MathSAT5 [15], CVC5 [2], Yices2 [21], Z3 [18], SMTInterpol [14] and SPASS-SATT [7].

The other approach is the *eager* approach, where the formula is reduced to an equi-satisfiable Boolean formula and then solved by a SAT solver. This approach works well for SMT (IDL). Typically, all intrinsic dependencies between integer variables are computed and encoded as Boolean constraints. Encoding to

<sup>1</sup> https://smt-comp.github.io/.

Boolean formula is done either by deriving adequate ranges for formula variables (a.k.a. small domain encoding) [9,39,45], or by deriving all possible transitivity constraints (a.k.a per-constraint encoding) [44]. A hybrid method combining the strengths of two encoding scheme showed robust performance [43].

Local search is an incomplete method which plays an important role in many combinatorial problems [28]. Local search algorithms move from solution to solution in the space of candidate solutions by applying local changes. It has been successfully applied to Boolean Satisfiability (SAT) problem [1,4,12,13,33] and is competitive with CDCL solvers on certain types of instances. However, very limited effort has been devoted to local search for SMT. The idea of integrating local search solvers with theory solvers has been explored before, where a local search SAT solver WalkSAT is used to solve the Boolean skeleton of the SMT formula [26]. A pure local search solver [22] was proposed to solve SMT on the theory of bit vectors directly on the theory level, by lifting the successful techniques in local search SAT solvers to the SMT level. In [36], a precise propagation based local search for SMT on the theory of bit vectors is proposed, by introducing a notion of essential inputs to lift the concept of controlling inputs from the bit-level to the word-level. We are not aware of any work on local search solvers for SMT on integer arithmetic theories.

This work, for the first time, develops a local search solver for SMT (LIA), which directly operates on both Boolean and integer variables, breaking through the traditional approaches. We propose a local search framework, which switches between two modes, namely Boolean mode and Integer mode. Each mode consists of consecutive operations of the same type (either Boolean or integer). Moreover, for the Integer mode, we propose a literal-level operator named *critical move* and a fine-grained scoring function named *distance score* which takes into account the *distance to truth* of literals and *distance to satisfaction* of clauses. A two-level heuristic is proposed to pick a *critical move* operation. By putting these together, we develop a local search solver for SMT (LIA) called LS-LIA.

Experiments are conducted to evaluate LS-LIA on 4 benchmarks, including QF LIA and QF IDL benchmarks from SMTLIB (excluding unsatisfiable instances),<sup>2</sup> instances encoded from job shop scheduling (JSP) and instances generated by data race detection system on a real world benchmark [29]. We compare our solver with state of the art SMT solvers including Z3, CVC5, Yices and MathSAT5. Experimental results show that LS-LIA is competitive and complementary with state-of-the-art SMT solvers. Particularly, LS-LIA is good at solving instances without Boolean variables, noting that a large portion in SMTLIB (81.1% for LIA and 44.1% for IDL) belongs to this type. A simple sequential portfolio with Z3 improves the state-of-the-art on satisfiable QF LIA and QF IDL benchmarks from SMT-LIB. LS-LIA also solves Job Shop Scheduling benchmarks substantially faster than traditional complete SMT solvers.

<sup>2</sup> http://www.smt-lib.org/.

#### **2 Preliminary**

**Definition 1.** *Linear Integer Arithmetic (LIA): Let* P = {p1, p2, ...pn} *be a set of propositional (Boolean) variables and* X = {x1, x2...xm} *be a set of integervalued variables. The linear integer arithmetic formulae are inductively defined.*


In the above definition, we note that with '≤' and '=', we other inequalities can also be expressed. Specifically, we can express n <sup>i</sup> aix<sup>i</sup> < k as n <sup>i</sup> aix<sup>i</sup> ≤ k − 1, n <sup>i</sup> aix<sup>i</sup> > k as ¬( n <sup>i</sup> (aixi) ≤ k), n <sup>i</sup> aix<sup>i</sup> ≥ k as n <sup>i</sup> (−aixi) ≤ (−k) and (n <sup>i</sup> aixi) = k as (n <sup>i</sup> aix<sup>i</sup> ≤ (k − 1) ∨ ¬( n <sup>i</sup> (aixi) ≤ k).

A popular fragment of linear integer arithmetic is call *Integer Difference Logic* (IDL), where the arithmetic atomic formulae are in the form of xi−x<sup>j</sup> k, where -∈ {=, ≤}, xi, x<sup>j</sup> ∈ X and k is constant.

*Example 1.* A typical SMT (LIA) formula F: (p<sup>1</sup> ∨(x<sup>1</sup> + 2x<sup>2</sup> ≤ 2))∧(p<sup>2</sup> ∨(3x<sup>3</sup> + 4x4+5x<sup>5</sup> = 2)∨(−x2−x<sup>3</sup> ≤ 3)), where X = {x1, x2, x3, x4, x5} and P = {p1, p2} are the sets of integer-valued and propositional variables respectively.

A literal is an atomic formula, or the negation of an atomic formula. A clause is the disjunction of a set of literals, and a formula in conjunctive normal form (CNF) is the conjunction of a set of clauses. For an SMT (LIA) formula F, an assignment α is a mapping X → Z and P → {f alse, true}, and α(x) denotes the value of a variable x under α. A *complete assignment* is a mapping which assigns to each variable a value. A literal is a true literal if it evaluates to true under the given assignment, and otherwise it is a false literal. A clause is satisf ied if it has at least one true literal, and f alsif ied if all literals in the clause are false. A complete assignment is a *solution* to an SMT (LIA) formula if it satisfies all the clauses.

When applying local search algorithms to solve a satisfiability problem, the search space consists of all complete assignments, each of which is a candidate solution. Typically, a local search algorithm starts from a complete assignment, and iteratively modifies the assignment by changing the value of one variable, to search for a satisfying assignment.

In local search, an *operator* defines how to modify the candidate solution. When an operator is instantiated by specifying the variable to operate, we obtain an *operation*. For example, a standard operator for SAT is *flip*, which modifies the current assignment by changing the value of a Boolean variable, and *flip*(x1) is an operation, where x<sup>1</sup> is a Boolean variable in the formula.

Given a formula F, the *cost* of an assignment α, denoted as cost(α), is the number of falsified clauses under α. In dynamic local search algorithms which use clause weighting techniques, however, cost(α) denotes the total weight of all falsified clauses under an assignment α. Given a formula and an assignment α, an operation op is said decreasing if cost(α ) < cost(α), where α is the resulting assignment by applying op to α.

**Algorithm 1:** Local Search of Mode X /\* X can be Integer or Boolean \*/ **while** *non impr steps* <sup>≤</sup> <sup>L</sup> <sup>×</sup> <sup>P</sup>*<sup>X</sup>* **do if** <sup>α</sup> *satisfies* <sup>F</sup> **then** return <sup>α</sup> **if** <sup>∃</sup> *decreasing* <sup>X</sup> *operations* **then** op := a decreasing X operation **if** *fail to find decreasing* X *operation* **then** update clause weights; op := an X operation from a random falsified clause containing X literals; perform op to modify α;

#### **3 A Local Search Framework for SMT (LIA)**

In this section, we introduce a local search framework for SMT (LIA), which switches between integer operations and Boolean operations.

**Fig. 1.** An SMT Local Search Framework

In the beginning, the algorithm generates a complete assignment α. Then, it iteratively modifies α by performing operations on variables. The algorithm terminates once α becomes a solution to the formula, and outputs "SATISFI-ABLE" as well as the solution. If the algorithm fails to find a solution within the pre-set time limit, it is cut off and outputs "UNKNOWN".

As depicted in Fig. 1, after the initialization, the algorithm works in two modes, namely Integer mode and Boolean mode. In each mode X (X is Integer or Boolean), an X operation is picked to modify α, where an X operation refers to an operation that works on a variable of data type X. The two modes switches to each other when the number of non-improving steps (denoted as non improve steps) of the current mode reaches a threshold. The threshold is set to L × P<sup>b</sup> for the Boolean mode and L × P<sup>i</sup> for the Integer mode, where P<sup>b</sup> and P<sup>i</sup> denote the proportion of Boolean and integer literals to all literals in falsified clauses, and L is a parameter. Note that non improve steps is set to 0 whenever entering a mode, and then in each following step, it increases by one if cost(α) ≥ cost<sup>∗</sup> in the current step, where cost<sup>∗</sup> is the cost of the best assignment visited before.

The intuitions of the two mode framework are as follows. When all variables of one type (either Boolean or integer) are fixed, the formula is reduced to a subformula that contains only variables of the other type. Thus, by consecutively performing X (X can be Boolean or Integer) operations in a certain period, the algorithm focuses on dealing with a subformula consisting of only X variables. The switching threshold is set as L × PX, as we consider that when X literals accounts for larger proportion of all literals in falsified clauses, more steps should be allocated for the corresponding mode.

#### **Local Search in One Mode**

No matter the mode in which the algorithm works, it adopts a general procedure as described in Algorithm 1. It prefers to pick a decreasing operation (according to some heuristic) if any. If the algorithm fails to find any decreasing operation, it updates clause weights by increasing the weights of falsified clauses, and then picks an X operation from a random falsified clause containing X literals. Note that we can always pick a falsified clause with X literals (line 7). This is because when the algorithm works in X mode, since *non impr steps* ≤ L×PX, we have P<sup>X</sup> > 0, and so there exists at least one falsified clause with X literals.

As for clause weighting, our algorithm employs the probabilistic version of the PAWS scheme [13,46]. When the clause weighting scheme is activated, the clause weights are updated as follows. With probability 1−sp, the weight of each falsified clause is increased by one, and with probability sp, for each satisfied clause whose weight is greater than 1, the weight is decreased by one.

#### **4 The Critical Move Operator and a Two-Level Heuristic**

In this section, we introduce key techniques in the Integer mode. We propose a novel operator called critical move, and also a two-level heuristic for choosing a critical move in the Integer mode.

A key and basic component of a local search algorithm is the operator. For handling Boolean variables, our algorithm adopts the typical local search operator for SAT, namely *flip*, which modifies the value of a Boolean variable to the opposite of its current value (from True to False, or from False to True). For handling integer variables, we propose a novel operator called *critical move* which works on the literal level.

#### **4.1 Critical Move**

Different from the Boolean operator, an integer operator has two parameters – besides the variable to operate, it also needs to consider the increment (may be positive or negative) on the value.

Let us first consider a simple operator, which motivates us to propose a literallevel operator. A simple integer operator is to modify the value of a variable a by a fixed increment inc, that is, α(a) := α(a) ± inc. The parameter inc needs fine tuning. If inc is too small, it may take many iterations before making any falsified literal become true. If inc is too big, the algorithm may even become problematic that it can never make some literals true and thus essentially unable to solve some formulae.

*Example 2.* Given a formula F : (b − a ≥ 3) ∧ (b − a ≤ 5) and the current assignment is α = {a = 0, b = 0}. If inc = 1, it needs at least 3 operations to satisfy the formula. If inc = 10, then the formula cannot be satisfied using operations of this type, as the value of b − a would be always a multiple of 10.

In fact, in order to avoid the case that some literals can never become true (when the inc is too big), the only acceptable value of inc is 1. The main reason accounting for such a drawback is that the above operator ignores the literallevel information. We propose a literal-level operator for integer variables called *critical move*, which is defined below.

**Definition 2.** *The critical move operator, denoted as* cm(x, )*, assigns an integer variable* x *to the threshold value making literal true, where is a falsified literal containing* x*. Specifically, for each of the four basic forms of the falsified literal , let* Δ = - <sup>i</sup> aix<sup>i</sup> − k*, an operation is described below:*


Given the above definition of the critical move, an issue with this operator is that it may stall on equalities, when there is no such variable with a<sup>i</sup> | Δ in . To address this issue, in this situation, we additionally employ a simple strategy pick a random variable in that literal and performs +1 or −1 to decrease |Δ|.

*Example 3.* Assume we are given two falsified literals l<sup>1</sup> : (2b − a ≤ −3) and l<sup>2</sup> : (5c−d+ 3a = 5), and the current assignment is α = {a = 0, b = 0, c = 0, d = 0}. Then cm(a, l1), cm(b, l1), cm(c, l2), and cm(d, l2) refers to assigning a to 3, assigning b to −2, assigning c to 1 and assigning d to −5 respectively. Note that there does not exist cm(a, l2), since 3 -−5.

An important property of the *cm* operator is that after the execution of a *cm* operation, the corresponding literal must be true. Therefore, by picking a falsified literal and performing a *cm* operation on it, we can make the literal become true.

The critical move operations are analogous to update operations in other linear arithmetic model searching procedures. For example, Simplex for DPLL(T) [20] also progresses through a sequence of candidate assignments by updating the assignment to a variable to satisfy its bound. The significant distinction of critical moves is only updating input variables and always updating by an integral amount, as we can see from Definition 2.

#### **4.2 A Two-Level Heuristic**

In this subsection, we propose a two-level heuristic for selecting a decreasing *cm* operation. We distinguish a special type of decreasing *cm* operations from others, and give a priority to such operations.

From the viewpoint of algorithm design, there is a major difference between *cm* and *flip* operations. A *flip* operation is decreasing only if the flipping variable appears in at least one falsified clause. For a cm(x, ) operation to be decreasing, the literal does not necessarily appear in any falsified clause. This is because integer variables are multi-valued, and a cm(x, ) operation that modifies the value of x would have impact on other literals with the same variable x.

*Example 4.* Given a formula F = c1∧c<sup>2</sup> = (a−b ≤ 0∨b−e ≤ −2)∧(b−d ≤ −1), suppose the current assignment is α = {a = 0, b = 0, d = 0, e = 0}, then c<sup>1</sup> is satisfied and c<sup>2</sup> is falsified. The operation op1 = cm(b, b − e ≤ −2) refers to assigning b to −2, and op2 = cm(b, b − d ≤ −1) refers to assigning b to −1. The literal of op1 does not appear in any falsified clause while the literal of op2 appears in a falsified clause c2. Both operations are decreasing, as either of them would make clause c<sup>2</sup> become satisfied without breaking any satisfied clause.

In order to find a decreasing *cm* operation whenever one exists, we need to scan all *cm* operations on false literals. That is, the candidate set of decreasing operations is D = {cm(x, )| is a false literal and x appears in }. If D = ∅, there is no decreasing *cm* operation. We propose to distinguish a special subset S ⊆ D from the rest of D, which is S = {cm(x, )| appears in at least one falsified clause and x appears in }. Note that any *cm* operation in S would make at least one falsified clause become satisfied. Based on this distinction, we propose a two-level selection heuristic as follows:


Besides improving the efficiency of picking a decreasing *cm* operation, there is an important intuition underlying this two-level heuristic. We prefer to pick a decreasing *cm* operation from S, because such operations are conflict driven, as any cm ∈ S would force a falsified clause become satisfied. This idea can be seen as a LIA version of focused local search for SAT, which has been the core idea of WalkSAT-family SAT solvers [1,4,42].

#### **5 Scoring Functions**

Local search algorithms employ scoring functions to guide the search. We introduce two scoring functions, which are used to compare different operations and guide the local search algorithm to pick an operation to execute in each step.

A perhaps most commonly used scoring function for SAT, denoted as *score*, measures the change on the cost of the assignment by flipping a variable. This scoring function indeed can be used to evaluate all types of operations as it only concerns the clauses state (satisfied or falsified). We also employ *score* in our algorithm, for both *flip* and *cm* operations. Formally, the *score* of an operation is defined as

$$score(op) = cost(\alpha) - cost(\alpha'),$$

where α is obtained from α by applying op. Note that, our algorithm employs a clause weighting scheme which associates a positive integer weight to each clause, and thus the cost of an assignment is the total weight of falsified clauses. It is easy to see that an operation op is decreasing if and only if score(op) > 0. Our algorithm prefers to choose the operation with greater score in the greedy mode, for both Boolean and integer operations.

For integer operations, we propose a more fine-grained scoring function, measuring the potential benefit about pushing a falsified literal towards the direction of becoming true. Firstly, we propose a property for literals to measure this merit.

**Definition 3.** *Given an assignment* α*, for an arithmetic literal* : - <sup>i</sup> aix<sup>i</sup> ≤ k*, its distance to truth is* dtt(, α) = max{ - <sup>i</sup> aiα(xi) − k, 0}*. For a Boolean literal and an arithmetic literal* : - <sup>i</sup> aix<sup>i</sup> = k*,* dtt(, α)=0 *if is true under* α *and* dtt(, α)=1 *otherwise.*

Suppose the current assignment is α, for an arithmetic literal : - <sup>i</sup> aix<sup>i</sup> ≤ k, if - i - aiα(xi) > k, then the literal is falsified, and its dtt is defined to be <sup>i</sup> aiα(xi)−k. In this case, if we decrease the value of x<sup>i</sup> with a<sup>i</sup> > 0, or increase the value of x<sup>i</sup> with a<sup>i</sup> < 0, the dtt of would decrease. When - <sup>i</sup> aiα(xi) ≤ k, the literal is true, and thus its dtt is defined to be 0.

The definition of dtt for arithmetic literals somehow resembles the violation function for constraint satisfaction problems [27], and the violation operator in the simplex with sum of infeasibilities for SMT [32]. In this work, we extend it to the clause level to measure the distance of a clause away from satisfaction in a fine-grained manner. Based on the concept of *distance to truth* of literals, we define a function to measure the distance of a clause away from satisfaction.

**Definition 4.** *Given an assignment* α*, the distance to satisfaction of a clause* c *is* dts(c, α) = min∈<sup>c</sup>{dtt(, α)}*.*

According to the definition, the dts is 0 for satisfied clause, since there is at least one satisfied literal with dtt = 0, while dts is positive for falsified clauses. It is desirable to lead the algorithm to decrease the dts of clauses. To this end, we propose a scoring function to measure the benefit of decreasing the sum of dts of all clauses. Additionally, the function takes into account the clause weights as the score function.

**Definition 5.** *Given an LIA formula* F*, the distance score of an operation* op *is defined as*

$$dscore(op) = \sum\_{c \in F} (dts(c, \alpha) - dts(c, \alpha')) \cdot w(c),$$

*where* α *and* α *denotes the assignment before and after performing* op*.*

For Boolean *flip* operations, dscore is equal to score. For integer operations, however, compared to the score function which only concerns the state (satisfied or falsified) transformations of clauses, dscore is more fine-grained, as it considers the dts of clauses, which are different among falsified clauses.

*Example 5.* Given a formula F = c1∧c3∧c<sup>3</sup> = (a−b ≤ −1)∧(a−c ≤ −5∨a−d ≤ −10)∧(b−c ≤ −5∨b−d ≤ −10). Suppose w(c1)=1, w(c2)=2, w(c3) = 3, and the current assignment is α = {a = 0, b = 0, c = 0, d = 0}, and thus all clauses are falsified. Consider two *cm* operations op1 = cm(a, a − b ≤ −1) and op2 = cm(b, a−b ≤ −1), which assign α(a) := −1 and α(b) := 1 respectively, leading to α and α respectively. Then score(op1) = score(op2) = 1, as they both make c<sup>1</sup> satisfied. Also, dts(c2, α) − dts(c2, α ) = 1, and dts(c3, α) − dts(c3, α) = −1, so dscore(op1) = (dts(c1, α)−dts(c1, α ))·w(c1)+(dts(c2, α)−dts(c2, α ))·w(c2) = 1 × 1+1 × 2 = 3 and dscore(op2) = −2 by similar calculation. Therefore, op1 is a better operation.

Since the computation of dscore has considerable overhead, this function is only used when there is no decreasing operation, as the number of candidate operations is limited here, and it is affordable to calculate their dscore.

#### **6 LS-LIA Algorithm**

Based on the ideas in previous sections, we develop a local search solver for SMT (LIA) called LS-LIA. As described in Sect. 3, after the initialization, the local search works in either Boolean or Integer mode to iteratively modify α until a given time limit is reached or α satisfies the formula F. This section is dedicated to the details of the initialization and the two modes of local search, as well as other optimization techniques.

**Initialization:** LS-LIA generates a complete assignment α, by assigning the variables one by one until all variables are assigned. All Boolean variables are assigned with True. As for integer variables xi, if it has upper bound ub and lower bound lb, that is, there exist unit clauses x<sup>i</sup> ≤ ub and x<sup>i</sup> ≥ lb, it is assigned with a random value in [lb, ub]. If x<sup>i</sup> only has upper(lower) bound, x<sup>i</sup> is assigned with ub(lb). Otherwise, if the variable is unbounded, it is assigned with 0.


#### **Algorithm 3:** Local Search of Integer Mode


**Boolean Mode (Algorithm 2):** If there exist decreasing *flip* operations, the algorithm selects such an operation with highest score.

If the algorithm fails to find any decreasing operation, it first updates clause weights according to the weighting scheme described in Sect. 3. Then, it picks a random falsified clause with Boolean literals and chooses a *flip* operation with greatest score.

**Integer Mode (Algorithm 3):** If there exist decreasing *cm* operations, the algorithm chooses a *cm* operation using the two-level heuristic: it first traverses falsified clauses to find a decreasing *cm* operation with greatest score (line 9); if no such operation exists, it searches for a decreasing *cm* operation via BMS heuristic (line 10) [10]. Specifically, it samples t *cm* operations (t is a parameter) from the false literals in satisfied clauses, and selects the decreasing one with greatest score.

If the algorithm fails to find any decreasing operation, it first updates clause weights similarly to the Boolean mode. Then, it picks a random falsified clause with Integer literals and chooses a *cm* operation with greatest dscore.

**Restart Mechanism:** The search is restarted when the number of falsified clauses has not decreased for M axNoImprove iterations, where M axNoImprove is a parameter.

**Forbidding Strategies.** Local search methods tend to be stuck in suboptimal regions. To address the cycle phenomenon (i.e. revisiting some search regions), we employ a popular forbidding strategies, called the tabu strategy [25]. After an operation is executed, the tabu strategy forbids the reverse operations in the following tt iterations, where tt is a parameter usually called *tabu tenure*. The tabu strategy can be directly applied in LS-LIA. (1) If a *flip* operation is performed to flip a Boolean variable, then the variable is forbidden to flip in the following tt iterations. (2) If a *cm* operation that increases (decreases, resp.) the value of an integer variable x is performed, then it is forbidden to decrease (increase, resp.) the value of x in the following tt iterations.

## **7 Experiments**

We carried out experiments to evaluate LS-LIA on 4 benchmarks, and compare it with state-of-the-art SMT solvers. Also, we combine LS-LIA with Z3 to obtain a sequential portfolio solver, which shows further improvement. Additionally, experiments are conducted to analyze the effectiveness of the proposed ideas.

#### **7.1 Experiment Preliminaries**

**Implementation:** LS-LIA is implemented in C++ and compiled by g++ with '−O3' option. There are 5 parameters in LS-LIA: L for switching phases, tt for the tabu scheme, M axNoImprove for restart, t (the number of samples) for the BMS heuristic and sp (the smoothing probability) for the PAWS scheme. The parameters are tuned according to suggestions from the literature and our preliminary experiments on 20% sampled instances, and are set as follows: L = 20, t = 45, tt =3+ rand(10), M axNoImprove = 500000 and sp = 0.0003 for all benchmarks.

**Competitors:** We compare LS-LIA with 4 state-of-the-art SMT solvers according to SMT-COMP 2021,<sup>3</sup> namely MathSAT5 (version 5.6.6), CVC5 (version 0.0.4), Yices2 (version 2.6.2), and Z3 (version 4.8.14), which are the union of the top 3 solvers (excluding portfolio solvers) of QF LIA and QF IDL tracks. The binaries of all competitors are downloaded from their websites.

**Benchmarks:** Our experiments are carried out with 4 benchmarks.


<sup>3</sup> https://smt-comp.github.io/2021.

<sup>4</sup> https://clc-gitlab.cs.uiowa.edu:2443/SMT-LIB-benchmarks/QF LIA.

<sup>5</sup> https://clc-gitlab.cs.uiowa.edu:2443/SMT-LIB-benchmarks/QF IDL.


Instances from SMTLIB-LIA and SMTLIB-IDL benchmarks are divided into two categories depending on whether it contains Boolean variables. From the viewpoint of algorithm design, there is a major difference between the operations on Boolean and integer variables. We observe that instances containing only integer variables takes up a large proportion, amount to 81.1% and 44.1%, in these two benchmarks.

**Experiment Setup:** All experiments are carried out on a server with Intel Xeon Platinum 8153 2.00 GHz and 2048G RAM under the system CentOS 7.9.2009. Each solver is executed one run with a cutoff time of 1200 s (as in the SMT-COMP) for each instance in SMTLIB-LIA, SMTLIB-IDL and JSP benchmarks, as they contain sufficient instances. For the RVPredict benchmark (15 instances), the competitors are also executed one run for each instance as they are exact solvers, while LS-LIA is performed 10 runs for each instance. "#inst" denotes the number of instances in each family. We compare the number of instances where an algorithm finds a model ("#solved"), as well as the run time. The bold value in table emphasizes the solver with greatest "#solved". For RVPredict, LS-LIA solves all instances with 100% success rate and we report the median, minimum and maximum run time among the 10 runs for each instance.

We uploaded our solver as well as JSP and RVPredict benchmarks (along with related information) in the anonymous Github repository.<sup>6</sup>

#### **7.2 Results on SMTLIB-LIA and SMTLIB-IDL Benchmarks**

**Results on SMTLIB-LIA (Table** 1 **and Fig.** 2**).** We organize the results into two categories: instances Without Boolean variables, and instances With Boolean variables. LS-LIA outperforms its competitors on the Without Boolean category, solving 2294 out of the 2385 instances. We also present the run time comparisons between LS-LIA and each competitor on the Without Boolean category of SMTLIB-LIA benchmark in Fig. 2. As for the With Boolean category,

<sup>6</sup> https://anonymous.4open.science/r/sls4lia/.


**Table 1.** Results on instances from SMTLIB-LIA.

the performance of LS-LIA is overall worse than its competitors, but still comparable. A possible explanation is that as local search SAT solvers, LS-LIA is not good at exploiting the relations among Boolean variables. Nevertheless, LS-LIA has obvious advantage in "tropical-matrix" and "arctic-matrix" instances, which are industrial instances from automated program termination analysis [16], showing its complementary performance compared to CDCL(T) solvers.

**Fig. 2.** Run time comparison on Without Boolean category of SMTLIB-LIA

**Results on SMTLIB-IDL Benchmark (Table** 2 **and Fig.** 3**).** Similar to the case for SMTLIB-LIA, our local search solver shows the best performance on IDL instances Without Boolean variables (solving 597 out of the 707 instances), which can be seen from Table 2 and Fig. 3. However, LS-LIA performs worse than its competitors on those With Boolean variables. Overall, LS-LIA cannot rival its competitors on this benchmark, but works particularly well on the instances without Boolean variables.

**Combination with Z3 and Summary on SMTLIB benchmarks (Table** 3**).** To confirm the complementarity of our local search solver with state of the art SMT solvers, we combine LS-LIA with Z3, by running Z3 with a time limit 600 s, and then LS-LIA from scratch with the remaining 600 s if Z3 fails to solve the instance. This wrapped solver can be regarded as a sequential portfolio solver, denoted as "Z3+LS".

We summarize the results of all solvers, including Z3+LS, on SMTLIB-LIA and SMTLIB-IDL benchmarks in Table 3. Among all single-engine solvers, Math-SAT5 solves the most instances of SMTLIB-LIA benchmark, while Z3 solves the most instances of SMTLIB-IDL benchmark. LS-LIA outperforms its competitors on instances Without Boolean variables, indicating that local search is an effective approach for solving SMT (LIA) instances with only integer variables.

Z3+LS solves more instances than any other solver on both benchmarks, confirming that LS-LIA and Z3 have complementary performance and their


**Table 2.** Results on instance from SMTLIB-IDL.

**Fig. 3.** Run time comparison on Without Boolean category of SMTLIB-IDL


**Table 3.** Summary results on SMTLIB-LIA and SMTLIB-IDL. Instances without and with Boolean variables are denoted by "no bool" and "with bool" respectively.

**Fig. 4.** Run time comparison on job shop scheduling instances.

combination pushes the state of the art in solving satisfiable instances of SMT (LIA). We also combined LS-LIA with Yices in the same manner, resulting in a wrapped solver called YicesLS [11], which won the Single-Query and Model-Validation Track on QF IDL in SMT-COMP 2021.

#### **7.3 Results on Job Shop Scheduling Benchmark**

LS-LIA significantly outperforms the competitors on the JSP benchmark. LS-LIA solves 74 instances, while MathSAT5, CVC5, Yices2, Z3 can only solve 27, 29, 49, 44 instances respectively. The run time comparison on the JSP benchmark are presented in Fig. 4, where the instances that both the competitors and LS-LIA cannot solve are excluded. LS-LIA shows dominating advantage over it competitors on these JSP instances.

#### **7.4 Results on RVPredict Benchmark**

Table 4 presents the results on satisfiable instances generated by running RVPredict [29] on Dacapo benchmark suite [5]. LS-LIA solves all the instances



consistently, and ranks second on this benchmark, only slower than Yices2. Particularly, on the 10 large instances RVPredict 1-10, LS-LIA is much faster than competitors except Yices2.

#### **7.5 Effectiveness of Proposed Strategies**

To analyze the effectiveness of the strategies in LS-LIA, we modify LS-LIA to obtain 5 alternative versions as follows.


We compare LS-LIA with these modified version on the SMTLIB-LIA and SMTLIB-IDL benchmarks. The runtime distribution of LS-LIA and its modified versions on the two benchmarks are presented in Fig. 5, confirming the effectiveness of the strategies.

**Fig. 5.** Run time distribution comparison

#### **8 Conclusion and Future Work**

We developed the first local search solver for SMT (LIA) and SMT (IDL), opening the local search direction for SMT on integer theories. Main features of our solver include a framework switching between Boolean and Integer modes, the critical move operator and a scoring function based on distance to satisfaction. Experiments show that our solver is competitive and complementary to stateof-the-art SMT solvers.

We would like to enhance our solver by improving the performance on instances with Boolean variables. Also, it is interesting to explore deep cooperation with DPLL(T) solvers.

**Acknowledgements.** This work is supported by NSFC Grant 62122078. We thank the reviewers of CAV 2022 for comments on improving the quality of the paper.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Reasoning About Data Trees Using CHCs**

Marco Faella<sup>1</sup> and Gennaro Parlato2(B)

<sup>1</sup> University of Naples Federico II, Naples, Italy m.faella@unina.it <sup>2</sup> University of Molise, Campobasso, Italy gennaro.parlato@unimol.it

**Abstract.** Reasoning about data structures requires powerful logics supporting the combination of structural and data properties. We define a new logic called Mso-D *(Monadic Second-Order logic with Data)* as an extension of standard Mso on trees with predicates of the desired data logic. We also define a new class of *symbolic data tree automata* (Sdtas) to deal with data trees using a simple machine. Mso-D and Sdtas are both Turing-powerful, and their high expressiveness is necessary to deal with interesting data structures. We cope with undecidability by encoding Sdta executions as a system of CHCs *(Constrained Horn Clauses)*, and solving the resulting system using off-the-shelf solvers. We also identify a fragment of Mso-D whose satisfiability can be effectively reduced to the emptiness problem for Sdtas. This fragment is very expressive since it allows us to characterize a variety of data trees from the literature, solving certain infinite-state games, etc. We implement this reduction in a prototype tool that combines an Mso decision procedure over trees (Mona) with a CHC engine (Z3), and use this tool to conduct several experiments, demonstrating the effectiveness of our approach across different problem domains.

#### **1 Introduction**

Reasoning about linear or tree-like data structures requires very expressive logics that allow combining structural and data properties. Logical characterizations of common data structures often impose restrictions on the structural part, which are intertwined with constraints on the data part. For example, in a *binary search tree* (Bst) the data values are organized in the form of a binary tree, where the numerical value associated with each node is greater than or equal to all the values stored in its left sub-tree and smaller than all those in its right sub-tree. Logical characterisations of data structures may also require the calculation of measures concerning parts of the structure such as size or height. Think of *redblack trees* (Rbt), a type of Bst with additional constraints, such as "every path from a given node to any of its descendant leaves goes through the same number

c The Author(s) 2022 S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 249–271, 2022. https://doi.org/10.1007/978-3-031-13188-2\_13

This work was partially supported by INDAM-GNCS 2020-2021, and by AWS 2021 Amazon Research Awards.

The authors dedicate this paper to their advisor Margherita Napoli.

of black nodes". Similarly, for AVL trees we need to impose that the heights of the sub-trees rooted in the children of any node differ by a maximum of one.

As a first contribution, we define a new logic called Mso-D *(Monadic Second-Order logic with Data)* as an extension of standard Mso on binary trees with data constraints. The Mso component of the logic allows us to express structural properties, while the data constraint component allows us to impose properties on the data associated with the nodes. Constraints on data are expressed by predicates from a desired data logic that is completely agnostic to the underlying tree structure. We connect the two components by means of uninterpreted functions that map each node of the tree to a data item. An example of an Mso-D formula that defines Bsts is:

$$\forall x. \forall y. \left( \left( path\_l(x, y) \to val(x) \ge val(y) \right) \land \left( path\_r(x, y) \to val(x) < val(y) \right) \right), \tag{1}$$

where x and y are first-order variables ranging over the set of nodes, *path*l(x, y) (resp., *path*r(x, y)) is an Mso formula expressing that "y is in the left (resp., right) sub-tree of x", and *val* is an uninterpreted function that maps each node of the tree to an integer.

As a second contribution, we define a new class of *symbolic data tree automata* (Sdtas) to recognize languages of data trees using a simple machine. Such automata perform a bottom-up computation starting from the leaves of the data tree. The state of an Sdta is represented by the value of a set of *state variables*, whereas the data trees recognized by the automaton carry another set of *alphabet variables*. The transitions of an Sdta are expressed by joint constraints over state and alphabet variables. For example, Bsts attach to each node a single alphabet variable, say *val*, holding the numerical value of that node. An Sdta recognizing Bsts will use additional state variables to check that the data tree is indeed a Bst. In this case, two state variables are sufficient to achieve this goal: one holding the minimum and one holding the maximum value stored in the sub-tree rooted in the current node. Similarly, Sdtas can be designed to recognize the classes of Rbts and AVL trees.

We have to deal with undecidable problems when reasoning about data trees using Mso-D or Sdtas, and this is unavoidable if we want to *(a)* handle trees with data from infinite domains, and *(b)* relate data from different nodes. These two features make Sdtas *Turing-powerful* since they can encode executions of two-counter machines. A similar argument holds for the satisfiability problem of Mso-D, since we can write a formula that allows us to relate data in consecutive nodes. By prohibiting the propagation of unbounded information between nodes, the decidability of relevant decision problems can be recovered (see [11,12] for an account on this). However, these features are both essential to deal with data structures such as Bsts, Rbts, AVL trees, Heaps, etc.

A way to cope with undecidability is to encode the executions of an Sdta as a system of CHCs (*Constrained Horn Clauses*) or, equivalently, as a CLP (*Constraint Logic Program*) [26], and solve the system using efficient off-theshelf tools. Systems of CHCs correspond to a restricted class of first-order logic, and are a versatile formalism for representing and solving a variety of program verification or model checking problems, including those regarding sequential, concurrent, and functional programs. Efficient algorithms have been proposed for solving systems of CHCs, often leveraging or generalizing techniques developed in the context of automatic program verification [2,21,22]. As a result, CHCs are often used as an intermediate representation in a variety of verification and synthesis tools [6,18,20,23,25,27,29,35]. Here, we follow a similar approach to solve the emptiness problem for Sdtas, and this offers several advantages. First, it provides a separation of concerns, allowing users of our framework to focus only on aspects related to the tree data structure at hand, while giving CHC solver developers a clean framework that can be instantiated using various model checking algorithms and specialized decision procedures. Furthermore, by expressing CHCs in the standard SMT-LIB language, one can take advantage of different CHC engines, whose performance keeps improving year-over-year, as witnessed by the *competition on constrained Horn clauses* CHC-COMP [19].

As a third contribution, we show several results linking the Mso-D satisfiability problem to the emptiness of Sdtas, and thus to the problem of solving a CHC system. A fundamental theorem for the class of regular (word or tree) languages states that a language is regular if and only if it is Mso-definable, i.e., definable by a closed formula (i.e., a sentence) of standard Mso [4,5,14,15,39,41]. Here, we show that if we allow Mso-D data predicates to talk only about the data of a single node, the satisfiability problem can be reduced to the emptiness of Sdtas. Furthermore, both decision problems are decidable in this case [11]. Moreover, we identify a larger undecidable syntactical fragment of Mso-D where the above reduction can still be performed. Namely, we give an effective reduction when the Mso-D formula is of the form <sup>∃</sup>*x*∀*<sup>y</sup>* . ϕ(*x*, *<sup>y</sup>*), where <sup>ϕ</sup> can contain additional quantifiers and each data constraint in ϕ is either unary, or accesses the data in a bounded neighborhood of the nodes referred to by *x* and at most one of the variables of *y*. We show that this fragment is very expressive as it allows us to characterize a variety of tree data structures from the literature, solve certain infinite-state games, and handle many other potential applications.

As a fourth and final contribution, we have implemented the reduction for the syntactic Mso-D fragment described above in a prototype tool that combines an Mso decision procedure over trees (Mona [28]) with a CHC engine (of the SMT solver Z3 [24,36]). Using this tool we have conducted several experiments to demonstrate the effectiveness and the practicality of our approach.

*Organization of the Paper.* The rest of the paper is organized as follows. Section 2 defines data trees, while Sect. 3 introduces the Mso-D logic. Section 4 deals with the definition of Sdtas, and Sect. 5 shows that the emptiness problem for Sdtas is undecidable in general but can be solved by off-the-shelf CHC engines. Section 6 shows a reduction from the Mso-D satisfiability problem to the emptiness of Sdtas, for the Mso-D fragment <sup>∃</sup>*x*∀*<sup>y</sup>* . ϕ(*x*, *<sup>y</sup>*). Section <sup>7</sup> describes our prototype implementation and summarises our experimental evaluation. Related work and concluding remarks can be found in Sect. 8 and 9, respectively.

#### **2 Data Trees**

Here we formally define *data trees*. We deal with trees that are finite in size and labeled with data from possibly infinite domains. We consider only binary trees (i.e., trees of arity 2) to keep notation to a minimum. However, the methods and approaches presented in the paper apply to any class of trees of fixed arity.

We will use N to denote the set of all natural numbers, Z to stand for the set of integers, and <sup>B</sup> to represent the set {0, <sup>1</sup>}. For a number <sup>n</sup> <sup>∈</sup> <sup>N</sup>, we write [n] to denote the interval {1,...,n}.

**Words.** An *alphabet* is a finite set of *symbols*. A *word* w over an alphabet Σ is a finite (possibly empty) sequence w = a1a<sup>2</sup> ...a<sup>n</sup> where a<sup>i</sup> ∈ Σ for i ∈ [n]. We denote with |w| the *length* of the sequence of symbols forming w. The *empty word*, denoted by , is the word formed by no symbol. We denote the set of all words over Σ by Σ∗. A *language* L over Σ is any subset of Σ∗. A *prefix* (resp. *suffix*) of a word w is either , or any sequence a<sup>1</sup> ...a<sup>j</sup> (resp., a<sup>j</sup> ...an), for some j ∈ [n]. Given two words a = a1a<sup>2</sup> ...a<sup>n</sup> and b = b1b<sup>2</sup> ...bm, their *concatenation* denoted ab, is the word a1a<sup>2</sup> ...anb1b<sup>2</sup> ...bm. Given a word w ∈ Σ<sup>∗</sup> and a language L ⊆ Σ∗, we define *Ext*(w, L) as the language of all words w such that ww is a word in L, i.e., *Ext*(w, L) = {w | ww ∈ L}.

**Trees.** A *binary tree* T, or simply a *tree*, is a finite and prefix-closed subset of {0, 1}<sup>∗</sup>. We call the elements of T *nodes*, and the node identified by the *root* of T. The *edge relation* is defined implicitly: for d ∈ {0, 1}, if v and vd are both nodes of T, then (v, vd) is an *edge* of T. Further, if d is 0 (resp., 1) we say that vd is the *left* (resp., *right*) *child* of v, and v is the *parent* of vd. A *leaf* is a node with no children, while an *internal node* is a node that is not a leaf. The *height* of T is max<sup>t</sup>∈<sup>T</sup> |t|. The *sub-tree* of T rooted at a given node t ∈ T is *Ext*(t, T). Further, *Ext*(t, T) is a *left* (resp., *right*) *sub-tree* of t if t = t 0 (resp., t = t 1), for some t ∈ T. The k-th *level* of a tree T consists of the sequence of all t ∈ T, with |t| = k, sorted in ascending lexicographic order. Further, the k-th *level* of T is filled *left to right* if it is a prefix of the k-th *level* of {0, 1}<sup>∗</sup>.

**Data Signatures.** Data signatures are like structured data types (a.k.a. *records*) in programming languages: a *data signature* S is a set of pairs {*id*<sup>i</sup> : *type*i}i=1...n. Common types of interest include bounded or unbounded integers (denoted by int and <sup>Z</sup>, resp.), floating point rationals and real numbers (float and <sup>R</sup>), the Boolean type B and the bit vectors of length k. If a signature contains a single field whose type is a finite alphabet Σ, we call that signature an *enumeration*. An *evaluation* ν of a data signature S is a map that associates each field name *id* in S with a value of the corresponding type, denoted by ν.*id*. We denote by L(S) the set of all evaluations of S, also called the *language* of S.

**Data Trees.** A *data tree* with data signature S, or an S-*tree*, is a pair (T,λ) where T is a tree and λ is a labelling function that maps each node t ∈ T into an evaluation of S, i.e., λ(t) ∈ L(S). Another way of looking at data trees is to think of them as a traditional tree data structure where the data λ(t) associated with each node t is structured. Thus, to simplify the notation when λ is clear from the context, we adopt a C-like notation to refer to the value of fields associated with tree nodes: if t is a tree node and *id* is a field of S, we write t.*id* as a shorthand for λ(t).*id*. If the data signature is an enumeration, we recover the traditional notion of Σ-labelled tree.

Many data structures from the literature can be seen as data trees. Below we give a high-level description of well-known data structures [9]. In addition to using them for motivating purposes, we will also use them as running examples.

*Example 1 (Binary Search Trees).* A Bst is a binary tree where each node stores a key taken from a totally ordered set, with the property that the key stored in each internal node is greater than or equal to all the keys stored in the node's left subtree, and smaller than those in its right subtree. Thus, an appropriate signature for data trees representing Bsts is {*val* : <sup>Z</sup>}.

*Example 2 (Red-black Trees).* An Rbt is a binary tree where each internal node stores a numerical value, satisfying the binary search tree property. The leaves do not contain keys or data and they represent a NIL pointer. Each node has a color (red or black), and the following properties hold: *(i)* every leaf is black, *(ii)* if a node is red then both its children are black, and *(iii)* every path from a given node to a descendent leaf contains the same number of black nodes.

Note that while the color is a piece of information stored in the node, the *black height* can instead be computed on demand. Thus, the signature for data trees representing Rbts may be {*val* : <sup>Z</sup>, *is black* : <sup>B</sup>}.

*Example 3 (Max-Heap).* A Max-Heap is a binary tree where each node stores a key taken from a totally ordered set, say <sup>Z</sup>, and can be described as an <sup>S</sup>tree (T,λ) where S is a data signature consisting of a single integer field, say {*key* : <sup>Z</sup>}, that obeys the following two constraints: (i) (*shape property*) <sup>T</sup> is almost complete, i.e., all its levels are complete, except the last one, that is filled from left to right; and (ii) (*heap property*) the value stored in each node is greater than or equal to the values stored in the node's children.

#### **3 Monadic Second-Order Logic with Data**

In this section, we introduce our Mso-D logic to express properties of data trees. We define Mso-D by extending the standard Monadic Second-Order logic on (enumeration) trees (a.k.a. Mso) with constraints on the Data.

Data constraints are formulas in first-order logic (Fol) with equality (here we use standard Fol syntax and semantics [34]). However, since data trees may involve different data types, we will consider formulas with many-sorted signatures as opposed to the classical unsorted version. Specifically, we deal with formulas of a many-sorted first-order theory D with sorts *data*1,..., *data*n. For each *data*i, we allow a theory D*data<sup>i</sup>* whose function symbols have type *data*<sup>n</sup> <sup>i</sup> <sup>→</sup> *data*<sup>i</sup> and whose relation symbols have type *data*<sup>m</sup> <sup>i</sup> <sup>→</sup> <sup>B</sup>, for some <sup>n</sup> and m. For example, each of these theories can be the theory of *arithmetic*, *reals*, *arrays*, etc. From now on, we may refer to <sup>D</sup> as the *data theory* of Mso-D.

We also introduce a finite set of *connecting function symbols*, denoted F, which we use to extend the Mso component of our logic with data. Let *nodes* be the sort of the Mso component. Then, each <sup>f</sup> ∈ F is an uninterpreted function symbol with type *nodes* → *data*i, for some i ∈ [n]. These functions allow us to model fields that we associate with each node in the tree. In particular, we say that f ∈ F models a field of an S-tree if f is also the name of a field in S. Otherwise, f may serve the purpose of endowing tree nodes with extra data fields without these being present in the labels of the data tree. This is a very useful feature for characterizing tree data structures, e.g., we can use *bh* ∈ F to logically characterize the *black height* of nodes in Rbts, even if that information is not part of the data signature of Rbts.

We are now ready to formally define the syntax of Mso-D over <sup>S</sup>-trees with data theory D and connecting functions F. We fix countable sets of propositional variables (denoted by p), first-order node variables (denoted by x, y, etc.), and node-set variables (denoted by X, Y, etc.). We assume that D includes relation symbols <sup>D</sup>rel. We also assume that the symbols in <sup>D</sup>, the variable names, and the symbols in <sup>F</sup> do not overlap. Since <sup>D</sup> is imported in Mso-D unchanged we do not report its definition here. The remaining components of the syntax of Mso-D(D, <sup>F</sup>, <sup>S</sup>) are defined by the following grammar:

$$\begin{aligned} \text{Node terms: } & t \stackrel{\text{def}}{=} x \mid t. left \mid t. right\\ \text{Formulas: } & \varphi \stackrel{\text{def}}{=} p \mid t\_1 = t\_2 \mid t \in X \mid \exists x. \,\varphi \mid \exists X. \,\varphi \mid \neg \varphi \mid \varphi \land \varphi\\ & \mid r(f\_1(t\_1), \dots, f\_k(t\_k)) \qquad r \in \mathcal{D}^{\text{rel}}, f\_1, \dots, f\_k \in \mathcal{F} \end{aligned}$$

where r and f1,...,f<sup>k</sup> are well-typed, i.e., there is an index i such that the type of r is *data*<sup>k</sup> <sup>i</sup> , and for every j ∈ [k], f<sup>j</sup> has type *nodes* → *data*i. We denote the set of all variables occurring in ϕ by *Var* (ϕ).

An *interpretation* of a formula <sup>ϕ</sup> is a pair (<sup>T</sup> <sup>λ</sup>,I), where <sup>T</sup> <sup>λ</sup> is an <sup>S</sup>-tree (T,λ), and I interprets the remaining symbols of the logic. We interpret the D-component of our theory as we would interpret D in isolation. Assume that <sup>D</sup> is the chosen interpretation of <sup>D</sup>, with underlying universes <sup>D</sup><sup>i</sup> for sort *data*i. Also, <sup>I</sup> maps each function symbol <sup>f</sup> in <sup>F</sup> with type *nodes* <sup>→</sup> *data*<sup>i</sup> to a concrete function <sup>I</sup>(f) : <sup>T</sup> <sup>→</sup> <sup>D</sup>i. Moreover, if <sup>f</sup> is also the name of a field in <sup>S</sup>, then we require that I(f) coincides with the value of the field f in each node of the tree, i.e., <sup>I</sup>(f)(v) = <sup>λ</sup>(v).f, for every <sup>v</sup> <sup>∈</sup> <sup>T</sup>. The satisfaction relation depends also on D, but we omit it here because we consider it fixed.

We interpret the variables in *Var* (ϕ) by mapping each of them into a subset of nodes of T with the following properties: (i) first-order variables are assigned singletons, and (ii) propositional variables are assigned either all nodes (encoding *true*) or no node at all (encoding *false*). For a set of nodes S ⊆ T and a (first- or second-order) variable <sup>α</sup> <sup>∈</sup> *Var* (ϕ), we denote by <sup>I</sup>[S/α] the function that maps α to S, and agrees with I on all the other variables. Node terms are interpreted as follows:

$$\mathbb{I}(t) = \begin{cases} \mathbb{I}(x) & \text{if } t = x \\ \mathbb{I}(s)0 & \text{if } t = s.left \text{ and } \mathbb{I}(s)0 \in T \\ \mathbb{I}(s)1 & \text{if } t = s.right \text{ and } \mathbb{I}(s)1 \in T \\ \mathbb{I}(s) & \text{otherwise.} \end{cases}$$

Notice that the .*left* and .*right* operators *stutter* on leaves, that is, for all leaves v it holds v = v.*left* = v.*right*.

The *satisfaction relation* <sup>T</sup> <sup>λ</sup>,<sup>I</sup> <sup>|</sup><sup>=</sup> <sup>ϕ</sup> is so defined.

$$\begin{array}{lll} T^{\lambda}, \mathbbm{1} \vDash p & \text{iff } \mathbb{I}(p) = T \\ T^{\lambda}, \mathbbm{1} \vDash t\_{1} = t\_{2} & \text{iff } \mathbb{I}(t\_{1}) = \mathbb{I}(t\_{2}) \\ T^{\lambda}, \mathbbm{1} \vDash t \in X & \text{iff } \mathbb{I}(t) \subseteq \mathbb{I}(X) \\ T^{\lambda}, \mathbbm{1} \vDash (f\_{1}(t\_{1}), \dotsc, f\_{k}(t\_{k})) \text{ iff } \mathbb{D}(r) (\mathbb{I}(f\_{1}) (\mathbb{I}(t\_{1})), \dotsc, \mathbb{I}(f\_{k}) (\mathbb{I}(t\_{k})) \text{ } \\ T^{\lambda}, \mathbbm{1} \vDash \varphi & \text{iff } T^{\lambda}, \mathbbm{1} \vDash \varphi \\ T^{\lambda}, \mathbbm{1} \vDash \varphi\_{2} & \text{iff } T^{\lambda}, \mathbbm{1} \vDash \varphi\_{1} \text{ and } T^{\lambda}, \mathbbm{1} \vDash \varphi\_{2} \\ T^{\lambda}, \mathbbm{1} \vDash \exists x. \varphi & \text{iff } \text{there exists } v \in T \text{ such that } T^{\lambda}, \mathbb{I}[\{v\}/x] = \varphi \\ T^{\lambda}, \mathbbm{1} \vDash \exists X. \varphi & \text{iff there exists } S \subseteq T \text{ such that } T^{\lambda}, \mathbb{I}[S/X] \mid \varphi \end{array}$$

We say that <sup>T</sup> <sup>λ</sup> *satisfies* <sup>ϕ</sup>, denoted <sup>T</sup> <sup>λ</sup> <sup>|</sup><sup>=</sup> <sup>ϕ</sup>, if there is an interpretation <sup>I</sup> such that <sup>T</sup> <sup>λ</sup>,<sup>I</sup> <sup>|</sup><sup>=</sup> <sup>ϕ</sup>. We define the language of trees satisfying an Mso-D sentence in the usual way. An Mso-D *sentence* is a formula with no free variables. We define the set of all <sup>S</sup>-trees <sup>T</sup> <sup>λ</sup> satisfying an Mso-D(D, <sup>F</sup>, <sup>S</sup>) sentence <sup>ϕ</sup> by <sup>L</sup>(ϕ), i.e., the set of all <sup>S</sup>-trees <sup>T</sup> <sup>λ</sup> such that <sup>T</sup> <sup>λ</sup> <sup>|</sup><sup>=</sup> <sup>ϕ</sup>. A language of trees <sup>L</sup> is Mso-D(D, <sup>F</sup>, <sup>S</sup>) *definable* if there exists an Mso-D(D, <sup>F</sup>, <sup>S</sup>) sentence <sup>ϕ</sup> such that L = L(ϕ).

We recover standard Mso when <sup>S</sup> is an enumeration and <sup>D</sup> includes a unary relation r<sup>a</sup> for each a ∈ Σ, whose interpretation is {a}.

*Undecidability of the Satisfiability Problem.* The *satisfiability problem* for a given Mso-D(D, <sup>F</sup>, <sup>S</sup>) sentence <sup>ϕ</sup> asks whether <sup>L</sup>(ϕ) is empty. Let <sup>D</sup> be the theory of linear integer arithmetic. It is easy to model an execution of any given 2 counter machine using a unary data tree whose signature has two fields of type N to model the counters, and an enumeration field to keep track of the current instruction. Each machine configuration is represented by a node, and we can impose constraints in our logic so that two consecutive nodes in the tree model a machine transition. Likewise, we can also express the property of a halting computation in our logic. Thus, the satisfiability of the Mso-D logic is undecidable, even though the underlying data logic D is decidable. Of course, by choosing a finite domain for the interpretation of the underlying data logic, we regain decidability in that the problem matches that of the standard Mso.

**Examples.** We now show various examples to illustrate the expressiveness of Mso-D. We will use the usual predefined abbreviations to denote the remaining propositional connectives (ϕ<sup>1</sup> ∨ ϕ<sup>2</sup> and ϕ<sup>1</sup> → ϕ2), the universal quantifier (∀α.ϕ def <sup>=</sup> ¬∃α .¬ϕ), <sup>x</sup> <sup>=</sup> <sup>y</sup> def = ¬(x = y), and the conditional expression (<sup>ϕ</sup> ? <sup>ϕ</sup><sup>1</sup> : <sup>ϕ</sup>2) def = (<sup>ϕ</sup> <sup>∧</sup> <sup>ϕ</sup>1) <sup>∨</sup> (¬<sup>ϕ</sup> <sup>∧</sup> <sup>ϕ</sup>2). Finally, the following standard Mso predicates will come in handy: *child*(x, y), *root*(x), *leaf* (x), and *path*(x, y).

*Example 4 (*Mso-D *Characterization of* Bst*s).* We define the characteristic property of Bsts on data trees with data signature {*val* : <sup>Z</sup>}. We first introduce the auxiliary predicate *path*l(x, y) (resp., *path*r(x, y)) with the meaning "y is in the left (resp., right) sub-tree of x". Using these predicates, we define the Mso-D sentence (1) that says that all values in the left sub-tree of a node x contain values that are smaller than the value in x, and similarly for the right sub-tree.

To demonstrate the use of connecting functions to model auxiliary node fields, we give an alternative way to characterize Bsts. We introduce two auxiliary connecting functions: *min* and *max* . We impose constraints to ensure that for each node x in the tree min(x) and max(x) are the minimum and maximum values of the sub-tree rooted in x, respectively. It is straightforward to see that we can impose the Bst property by relating the values in each node with *min* and *max* in their children as follows:

ψbst def <sup>=</sup> <sup>∀</sup>x . - x = x.*left* ? *min*(x) = *min*(x.*left*) ∧ *max* (x.*left*) ≤ *val* (x) : *min*(x) = *val* (x) ∧ x = x.*right*? *max* (x) = *max* (x.*right*) ∧ *min*(x.*right*) > *val* (x) : *max* (x) = *val* (x) .

*Example 5 (*Mso-D *characterization of* Rbt*s).* We can also express the defining properties of red-black trees as follows:


$$\forall x. \left( \neg is.black(x) \right) \rightarrow \left( is.black(x.left) \land is.black(x.right) \right).$$

*(c) Every path from a node to a leaf contains the same number of black nodes.* We encode this property as the consistency of the black height data field *bh*:

$$\begin{split} \forall x. \forall y. \left( \left( \left\{ \begin{array}{l} is\\_label(x)\wedge\\_child(y,x) \right\} \rightarrow (bh(y) = bh(x) = 1) \end{array} \right\} \right) \\ \land \left( \left( \neg is\\_label(x)\wedge\\_child(y,x) \right) \rightarrow (bh(y) = bh(x)) \right) \\ \land \left( \left( \left\{ leaf(x) \wedge\\_child(x) \right\} \rightarrow bb(x) = 1 \right) \right) \\ \land \left( \left( \left\{ leaf(x) \wedge\neg is\\_black\_$$

*Extended Models.* Since we are going to build automata corresponding to formulas with free variables, it is convenient to encode the variable interpretation in the tree itself, by expanding the data signature with an extra Boolean flag for each free variable. The flag corresponding to a free variable will be set to 1 in the node(s) that belong to the interpretation of that variable. In detail, for a given interpretation (<sup>T</sup> <sup>λ</sup>,I), assume that *Var* (ϕ) = {α1,...,αn}, we can define an *extended tree* (T,λE) with data signature <sup>S</sup><sup>E</sup> <sup>=</sup> S∪{a1,...,a<sup>n</sup> : <sup>B</sup>}, where <sup>λ</sup>E(u)(ai) = 1 iff <sup>u</sup> <sup>∈</sup> <sup>I</sup>(αi). Conversely, from an extended tree we can extract the corresponding variable interpretation. For such an extended tree we can write T <sup>λ</sup>*<sup>E</sup>* <sup>|</sup><sup>=</sup> <sup>ϕ</sup> without mentioning <sup>I</sup>.

#### **4 Symbolic Data-Tree Automata**

In this section, we define a new class of tree automata called *Symbolic Data-Tree Automata*. They generalize traditional bottom-up finite tree automata as they work with data trees. Furthermore, they are symbolic because the alphabet and set of states are defined using evaluations of data signatures, and its transition function is defined through constraints<sup>1</sup> involving states and alphabet.

**Definition 1** *(*Symbolic Data-Tree Automata*). A* symbolic data-tree automaton*, or* Sdta *for short,* <sup>A</sup> *is a tuple* (S<sup>Σ</sup>, <sup>S</sup><sup>Q</sup>, ψ<sup>F</sup> , Ψ<sup>Δ</sup>) *where:*


ψ*lr* (ql, qr, σ, q), ψl(ql, σ, q), ψr(qr, σ, q), ψ*leaf* (σ, q),

*where* <sup>q</sup>l, qr*, and* <sup>q</sup> *are variables of type* <sup>S</sup><sup>Q</sup>*, and* <sup>σ</sup> *is of type* <sup>S</sup><sup>Σ</sup>*.*

<sup>A</sup> *accepts* <sup>S</sup><sup>Σ</sup>*-trees. A tree* (T,λ) *is* accepted *by* <sup>A</sup> *if there is a total function* π : T → Q *such that for every node* t ∈ T *the following holds:*


*The* language *of* <sup>A</sup>*, denoted* <sup>L</sup>(A)*, is the class of all* <sup>S</sup><sup>Σ</sup>*-trees accepted by* <sup>A</sup>*.*

We recover standard tree automata when both data signatures <sup>S</sup><sup>Σ</sup> and <sup>S</sup><sup>Q</sup> are enumerations. In that case, we call A an *enumeration tree automaton* and we denote it as (Σ, Q, F, Δ), where <sup>Σ</sup> <sup>=</sup> <sup>L</sup>(S<sup>Σ</sup>), <sup>Q</sup> <sup>=</sup> <sup>L</sup>(S<sup>Q</sup>), and so on.

<sup>1</sup> We use the term *constraint* to denote a generic predicate *con*(x1,...,x*k*) in which the type of variable x*<sup>i</sup>* is some data signature S*i*. We deliberately leave the definition of the constraints unspecified, and specify them only when it is necessary to do so.

*Example 6* (*Symbolic Data-Tree Automaton for Max Heap*). We define an Sdta A*bmh* where L(A*bmh* ) is the set of all max heaps. The state data signature of <sup>A</sup>*bmh* is {<sup>h</sup> : <sup>N</sup>, f : <sup>B</sup>, *val* : <sup>N</sup>}. We use the <sup>h</sup> field to store the height of the sub-tree rooted in the node, the f field to store whether the sub-tree rooted in the node is complete with the last level completely filled, and *val* stores the node's data value. The transition data constraints are as follows:

$$\begin{split} \psi\_{\mathit{laf}}(\sigma,q) \stackrel{\scriptstyle \mathsf{at}}{=} q.h &= 1 \land q.f \land q.val = \sigma.val \\ \psi\_{\mathit{l}}(q\_{l},\sigma,q) \stackrel{\scriptstyle \mathsf{at}}{=} q.h &= 1 \land q.h = 2 \land \neg q.f \land \sigma.val \ge \sigma\_{l}.val \land q.val = \sigma.val \\ \psi\_{\mathit{lr}}(q\_{l},q\_{r},\sigma,q) \stackrel{\scriptstyle \mathsf{at}}{=} & \ (q.h-1 \le q\_{r}.h \le q.h) \land \ \{q.f \leftrightarrow (q.f \land q\_{r}.f \land q.h = q\_{r}.h)\} \\ & \qquad \land \left(\neg q.f \to (q\_{r}.h < q\_{l}.h \land q\_{r}.f)\right) \land \left(\neg q\_{r}.f \to (q\_{r}.h \land q\_{l}.f)\right) \\ & \qquad \land \ \sigma.val \ge q.val \land \sigma.val \ge q\_{r}.val \land q.val = \sigma.val. \end{split}$$

For each leaf, we set the height field <sup>h</sup> to 1, the field <sup>f</sup> to true, and copy the label of the node into the state field *val*. Note that all sub-trees of a complete tree are still complete trees. Thus, if a node has only the left child, this child must be a leaf, and we set the parent node's fields accordingly. A node with only the right child leads to a violation of the shape property, thus ψr(qr, σ, q) def = *false*. Finally, we consider the case where the node has both children. Here ψ*lr* constrains the fields of the state data signature to guarantee their invariants. Specifically, the first two lines enforce the shape property while the last line enforces the heap property and copies the value of the label into the state field *val*. To conclude, we define ψ<sup>F</sup> as a tautology. However, if, for example, we wanted to accept only max heaps of height at least 100 we could have defined <sup>ψ</sup><sup>F</sup> (q) def = (q.h <sup>≥</sup> 100).

## **5 Solving the Emptiness Problem for** Sdta**s**

The *emptiness problem* for Sdtas consists in determining whether the tree language recognized by a given Sdta <sup>A</sup> is empty, i.e., whether <sup>L</sup>(A) is empty. We first prove that the emptiness problem for Sdtas is undecidable, and then show that it can be reduced to the satisfiability of a system of constrained Horn clauses (CHCs), for which increasingly efficient off-the-shelf semi-procedures exist.

It is well known that the emptiness problem for tree automata is decidable [8]. However, as explained for Mso-D in Sect. 3, as soon as the state data signature involves an unbounded data domain (such as integers or reals) and basic arithmetics (e.g., increment and test for zero), the emptiness problem becomes undecidable. Thus, we have the following.

#### **Theorem 1.** *The emptiness problem for* Sdta*s is undecidable.*

We cope with this negative result by providing a reduction to the satisfiability of a system of CHCs, when the transition constraints of the automaton are defined through quantifier-free first-order logic formulas.

**Constrained Horn Clauses.** We fix a set R of uninterpreted fixed-arity relation symbols, which represent the unknowns in the system. A *Constrained Horn Clause*, or CHC for short, is a formula of the form H ← C ∧B<sup>1</sup> ∧···∧B<sup>n</sup> where:


A finite set H of CHCs is a *system*, and it corresponds to the first-order formula obtained by putting all its CHCs in conjunction. We assume that the semantics of constraints is given a priori as a structure. A system H with relation symbols R is *satisfiable* if there is an interpretation to each predicate in R that makes all clauses in H valid.

It is a well-known result from constraint logic programming that every system of CHCs H has a unique minimal model that can be computed as the fixed-point of an operator derived by the clauses of H [16,26]. This property, which allows us to use a fixed-point semantics for CHC systems, to justify the correctness of the reduction defined below (i.e., Theorem 2).

**Reduction.** We give a linear time reduction from the emptiness problem for Sdtas to the satisfiability of systems of CHCs. Let A = (S<sup>Σ</sup>, <sup>S</sup><sup>Q</sup>, ψ<sup>F</sup> , Ψ<sup>Δ</sup>) be an Sdta with Ψ<sup>Δ</sup> = (ψ*lr* , ψl, ψr, ψ*leaf* ), ql, q<sup>r</sup> and q be structured variables of type <sup>S</sup><sup>Q</sup>, <sup>σ</sup> be a struc-

$$\begin{aligned} h(q) &\leftarrow \psi\_{lr}(q\_l, q\_r, \sigma, q) \land h(q\_l) \land h(q\_r) \\ h(q) &\leftarrow \psi\_l(q\_l, \sigma, q) \land h(q\_l) \\ h(q) &\leftarrow \psi\_r(q\_r, \sigma, q) \land h(q\_r) \\ h(q) &\leftarrow \psi\_{leaf}(\sigma, q) \\ false &\leftarrow \psi^F(q) \land h(q) \end{aligned}$$

tured variable of type <sup>S</sup><sup>Σ</sup>, and <sup>h</sup>(q) be an uninterpreted predicate. We map <sup>A</sup> into the CHC system H<sup>A</sup> formed by the CHCs shown on the right.

**Theorem 2** *(*Emptiness*). Let* <sup>A</sup> *be an* Sdta*. Then,* <sup>L</sup>(A) *is empty if and only if* H<sup>A</sup> *is satisfiable.*

#### **6 From Logic to Automata**

In this section, we describe a reduction from the satisfiability problem of Mso-D to the emptiness problem of Sdtas, when the Mso-D formula ϕ is a sentence in the following form:

$$\varphi = \exists x\_1, \dots, x\_n. \forall y\_1, \dots, y\_m. \theta,\tag{2}$$

where each data constraint of the formula θ, say r(f1(t1),...,fk(tk)), satisfies one of the following:


Notice that θ may contain other quantifiers, but the additional quantified variables can occur only inside unary data constraints. Moreover, it is easy to see that this fragment is closed under positive Boolean combinations (i.e., conjunctions and disjunctions).

This fragment strictly includes the Mso logic with data defined in [11] for data words, which only allows unary data constraints. Below we show that the added expressivity can be used to define and verify properties of a variety of data structures, including those from Examples 4 and 5, and infinite-state games.

In our reduction, we first construct a standard finite-state tree automaton over a finite alphabet (Sect. 6.1), which we then convert to an Sdta (Sect. 6.2).

#### **6.1 Building the Enumerated Tree Automaton**

The first step in our reduction from Mso-D to Sdtas is to convert the Mso-D formula ϕ of type (2) into a formula ϕ in standard Mso by *abstracting away* all data constraints. We distinguish two types of data constraints. *Global constraints* refer only to the data of the existentially quantified variables xi; on a given data tree, once the interpretation of those variables is chosen, each global constraint is either *true* or *false*: it is a global property of the tree. *Local constraints*, instead, additionally refer to a variable, say z, that is not one of {x1,...,xn}; even if the interpretation of {x1,...,xn} is fixed, the truth of such constraints depends on the interpretation of z. Accordingly, we replace each data constraint in θ, say r(f1(t1),...,fk(tk)), as follows:


Besides the above substitutions, in the resulting Mso formula we leave variables x1,...,x<sup>n</sup> free, so that the models of the formula will carry the interpretation of those variables as extra bits in the node labels (recall the discussion on extended models in Sect. 3). We thus obtain the following Mso formula:

$$
\varphi' \stackrel{\text{at}}{=} \forall y\_1, \dots, y\_m. \theta'. \tag{3}
$$

Since ϕ has no data constraints, we can take its data signature to be empty.

*Example 7.* Consider the formula ψbst from Example 4 that defines Bsts using auxiliary data *min* and *max* . Since it uses a single universal quantifier, it belongs to the syntactic fragment (2). For the sake of simplicity, consider a stronger formula ψ bst forcing internal nodes to have two children (a.k.a. a *full* Bst):

$$\psi\_{\text{bst}}' \stackrel{\text{def}}{=} \psi\_{\text{bst}} \land \forall y . full . tree(y), \quad \text{where}$$

$$full . tree(y) \stackrel{\text{def}}{=} \{\neg leaf(y) \to (y. left \ne y \land y. right \ne y)\}.$$

Now, consider the following true property of full Bsts: the successor of an internal node is the left-most leaf in its right sub-tree. The following formula states the opposite of that property:

ψsucc def = ∃x1, x2, x<sup>3</sup> . *val*(x1) < *val*(x2) < *val*(x3) ∧ ¬*leaf* (x1) ∧ *leaf* (x3) ∧ *left only path*(x1.*right*, x3) .

It is easy to see that ψ bst ∧ ψsucc is equivalent to a formula ψ in our fragment: <sup>∃</sup>x1, x2, x<sup>3</sup> . <sup>∀</sup>y . - *val* (x1) < *val* (x2) < *val* (x3) ∧ ¬*leaf* (x1) ∧ *leaf* (x3)

$$\begin{aligned} &\wedge \text{left\text{\textquotedblleft}} \text{only\textquotedblright}(x\_1.right) \text{right}(x\_3) \wedge \text{left\text{\textquotedblleft}} \text{left\text{\textquotedblleft}} (y) \\ &\wedge \left\{ y \neq y.left \text{\textquotedblleft} \text{\textquotedblright} \, \text{\textquotedblleft} \, \text{\textquotedblright}(y) = \min(y.left) \wedge \max(y.left) < \text{val}(y) < \text{val}(y) \, : \, \min(y) = \text{val}(y) \right\} \\ &\wedge \left\{ y \neq y.right \, \text{right} \, \text{?} \, \max(y) = \max(y.right) \wedge \min(y.right) > \text{val}(y) \, : \, \max(y) = \text{val}(y) \right\}. \end{aligned}$$

The conversion outlined above turns ψ into the following Mso formula:

$$\forall y. \left(p\_1 \land \neg \mathit{left}(x\_1) \land \mathit{left}(x\_3) \land \mathit{right}\mathit{right}.\right) \land \mathit{right}\mathit{right}.\mathtt{with}\left(x\_1.\mathit{right}, x\_3\right) \land \mathit{right}\mathit{left}.\mathtt{tree}(y)$$

$$\land \left(y \neq y.\mathit{left}?\,y \in C\_1:y \in C\_2\right) \land \left(y \neq y.\mathit{right}?\,y \in C\_3:y \in C\_4\right),$$

where proposition p<sup>1</sup> corresponds to the global constraint *val*(x1) < *val*(x2) < *val*(x3), the second-order variable C<sup>1</sup> corresponds to the local constraint *min*(y) = *min*(y.*left*) ∧ *max* (y.*left*) < *val*(y), and variables C<sup>2</sup> – C<sup>4</sup> correspond to the other data constraints in ψbst.

We now apply the standard Mso construction to ϕ , leading to a bottomup finite-state tree automaton A<sup>ϕ</sup> on the alphabet <sup>Σ</sup> <sup>=</sup> {0, <sup>1</sup>}<sup>n</sup>+h+<sup>l</sup> , accepting all finite trees that represent interpretations satisfying ϕ . The alphabet is Σ because n + h + l is the total number of free variables in ϕ : n first-order variables xi, h propositional variables p<sup>i</sup> (corresponding to global constraints), and l second-order variables C<sup>i</sup> (corresponding to local constraints). We recall the formal statement of this construction below, for more details see [40] and [8].

**Theorem 3.** *For all* Mso *formulas* ϕ *on the empty data signature, with free first-order variables* x1,...,xn*, propositional variables* p1,...,ph*, and secondorder variables* C1,...,Cl*, there is a deterministic bottom-up tree automaton on the alphabet* {0, <sup>1</sup>}<sup>n</sup>+h+<sup>l</sup> *whose language consists of all extended trees* <sup>T</sup> *such that* T |= ϕ *.*

*Simplifying Assumptions.* To simplify the presentation of the following constructions, we make two simplifying assumptions. First, we assume that all terms appearing in data constraints are variables, and not composite terms like x.*left*.*right*. Dropping this assumption is technically simple and omitted due to space constraints. Second, we assume that all connecting functions f appearing in data constraints correspond to fields in S. Sentences that satisfy the second assumption have a unique interpretation I, because they have no free variables and the connecting functions must be interpreted as the functions extracting the corresponding field from each node. We discuss how to remove this restriction in Sect. 6.3.

We now establish a relation between Σ-trees accepted by A<sup>ϕ</sup>- , and data trees on the data signature S defined by ϕ. Denote by (a1,...,an, b1,...,bh, c1,...,cl) the generic element of Σ. Given a Σ-tree (T,σ) and a variable x<sup>i</sup> in ϕ , we define *node*(σ, xi) to be the unique node u ∈ T such that the a<sup>i</sup> component of σ(u) is 1. In words, the function *node* picks the position in the tree where the Σ-tree activates the bit ai.

**Definition 2.** *Consider an* Mso-D *sentence* ϕ *of the form* (2) *on the data signature* <sup>S</sup>*, and let* <sup>I</sup> *be its unique interpretation. We say that a* <sup>Σ</sup>*-tree* (T,σ) *and an* S*-tree* (T,λ) *are* consistent *iff for all nodes* u ∈ T *the following hold:*

*1. For all* i ∈ [h]*, let* r glb i f 1 <sup>i</sup> (α<sup>1</sup> <sup>i</sup> ),...,f<sup>j</sup>*<sup>i</sup>* <sup>i</sup> (α<sup>j</sup>*<sup>i</sup>* i ) *be the global constraint from* ϕ *corresponding to the propositional variable* p<sup>i</sup> *from* ϕ *. Recall that under the simplifying assumptions each* α<sup>j</sup> <sup>i</sup> *is one of* <sup>x</sup>1,...,xn*, and each* <sup>f</sup><sup>j</sup> <sup>i</sup> *is one of the names of the fields of* S*. Then,* σ(u)(bi)=1 *iff the following holds*

$$\mathbb{D}(r\_i^{\text{glb}}) \Big( \mathbb{I}(f\_i^1)(node(\sigma, \alpha\_i^1)), \dots, \mathbb{I}(f\_i^{j\_i})(node(\sigma, \alpha\_i^{j\_i})) \Big).$$

*2. For all* <sup>i</sup> <sup>∈</sup> [l]*, let* <sup>r</sup>loc i g1 <sup>i</sup> (β<sup>1</sup> <sup>i</sup> ),...,g<sup>k</sup>*<sup>i</sup>* <sup>i</sup> (β<sup>k</sup>*<sup>i</sup>* <sup>i</sup> ), gi(zi) *be the local constraint from* ϕ *corresponding to the second-order variable* C<sup>i</sup> *from* ϕ *. Recall that each* β<sup>j</sup> i *is one of* x1,...,xn*, and each* g<sup>j</sup> <sup>i</sup> *(as well as* gi*) is one of the names of the fields of* S*. Then,* σ(u)(ci)=1 *iff the following holds*

$$\mathbb{D}(r\_i^{\rm loc})\Big(\mathbb{I}(g\_i^1)(node(\sigma, \beta\_i^1)), \dots, \,\_r\mathbb{I}(g\_i^{k\_i})(node(\sigma, \beta\_i^{k\_i}))\,\,, \mathbb{I}(g\_i)(\lambda(u))\Big).$$

The following result states the fundamental relationship between ϕ and A<sup>ϕ</sup>-.

**Theorem 4.** *Let* ϕ *be an* Mso-D *sentence of the form* (2) *on the data signature* S*, and let* A<sup>ϕ</sup> *be the corresponding tree automaton described above. For all data trees* (T,λ) *with data signature* S*, the following are equivalent:*


#### **6.2 Building the Symbolic Data Tree Automaton**

We now convert the tree automaton from the previous section into an Sdta that accepts all and only the data trees satisfying the original Mso-D formula ϕ.

Intuitively, the Sdta mimics the behavior of the tree automaton, and in doing so, it enforces the data constraints contained in ϕ. The information about which constraints should be true and which should be false at every node is encoded in the alphabet <sup>Σ</sup> <sup>=</sup> {0, <sup>1</sup>}n+h+<sup>l</sup> of the tree automaton. In detail, if (a1,...,an, b1,...,bh, c1,...,cl) is a generic symbol from the alphabet, the bi's encode the truth value of the global constraints, and the ci's encode the truth value of the local constraints. However, the *data* on which to evaluate those constraints comes from different sources. The global constraints are evaluated only on the *guessed* data for the existentially quantified variables x1,...,xn, whereas the local constraints also access the data of the current node.

Finally, the a<sup>i</sup> component of the alphabet encodes the actual position of each x<sup>i</sup> in the current tree (i.e., a<sup>i</sup> is 1 only in the node that is the interpretation of xi). So, when a<sup>i</sup> = 1 the symbolic automaton checks that the guessed data evaluation for x<sup>i</sup> corresponds to the data in the current node.

Let A<sup>ϕ</sup>- = (Σ, Q, F, Δ) be the tree automaton from Sect. 6.1, we now define the Sdta <sup>A</sup><sup>ϕ</sup> = (S, <sup>S</sup><sup>Q</sup>, ψ<sup>F</sup> , Ψ<sup>Δ</sup>). First, notice that the alphabet data signature <sup>S</sup> coincides with that of the original Mso-D formula. We then set the state data signature <sup>S</sup><sup>Q</sup> to {*state* : <sup>Q</sup>}∪{*id*<sup>i</sup> : *type* <sup>|</sup> (*id* : *type*) ∈ S, i = 1 ...n}, i.e., <sup>S</sup><sup>Q</sup> contains an enumerated data field representing the state of the tree automaton A<sup>ϕ</sup>- , and n copies of each data field in S. These copies are used to store the guessed data evaluations for the existentially quantified variables x<sup>i</sup> from (2). For a symbolic state <sup>q</sup> <sup>∈</sup> <sup>L</sup>(S<sup>Q</sup>) and <sup>i</sup> <sup>∈</sup> [n], we denote by <sup>q</sup>[xi] the <sup>i</sup>-th projection of <sup>q</sup> on <sup>S</sup>, i.e., the evaluation that assigns to each field *id* in <sup>S</sup> the value q.*id*<sup>i</sup> . The acceptance constraint <sup>ψ</sup><sup>F</sup> (q) is simply defined as q.*state* <sup>∈</sup> <sup>F</sup>.

Regarding the transition constraints Ψ<sup>Δ</sup>, we will focus only on the case of nodes with two children, since the other cases are similar. Let (sl, sr, a, s) be a transition in A<sup>ϕ</sup>- , where a = (a1,...,an, b1,...,bh, c1,...,cl) ∈ Σ. We add the following implicant to the transition constraint ψ*lr* :

$$\begin{cases} \end{cases}\quad q\_l.state = s\_l \wedge q\_r.state = s\_r \wedge q.state = s\tag{4a}$$

$$\bigwedge\_{i \in [n]} \left( q[x\_i] = q[x\_i] \land q[x\_i] = q\_r[x\_i] \right) \quad \land \bigwedge\_{\{i \mid a\_i = 1\}} \left( q[x\_i] = \sigma \right) \tag{4b}$$

$$\wedge \bigwedge\_{i \in [h]} \left[ (b\_i = 1) \leftrightarrow \mathbb{D}(r\_i^{\text{glb}}) (q[\alpha\_i^1], f\_i^1, \dots, q[\alpha\_i^{j\_i}], f\_i^{j\_i}) \right] \tag{4c}$$

$$\begin{aligned} \{ \wedge \bigwedge\_{i \in [l]} \left[ (c\_i = 1) \leftrightarrow \mathbb{D}(r\_i^{\text{loc}}) (q[\beta\_i^1], g\_i^1, \dots, q[\beta\_i^{k\_i}], g\_i^{k\_i}, \sigma.g\_i) \right] \: \} \\ & \implies \quad \psi\_{lr}(q\_l, q\_r, \sigma, q) . \end{aligned} \tag{4d}$$

The above conjuncts can be explained as follows: (4a) mimics the state change in the discrete transition, the first part of (4b) states that the n copies of the data fields held by the symbolic automaton are uniform over the whole tree, the second part of (4b) additionally states that the i-th copy of the data fields coincides with the data σ in the unique node where the discrete automaton

**Fig. 1.** Architecture of the prototype implementation. Dashed transformations are performed manually, but could be automated by an Mso-D parser.

prescribes a<sup>i</sup> = 1, (4c) enforces the i-th global constraint r glb <sup>i</sup> in all nodes where the discrete automaton prescribes b<sup>i</sup> = 1, and finally (4d) enforces the local constraints when the c<sup>i</sup> component of the discrete alphabet is 1.

**Theorem 5.** *Let* <sup>ϕ</sup> *be an* Mso-D *sentence of the form* (2) *and let* <sup>A</sup><sup>ϕ</sup> *be the corresponding* Sdta *described above. We have* <sup>L</sup>(ϕ) = <sup>L</sup>(Aϕ)*.*

#### **6.3 Supporting Auxiliary Data**

So far, we have assumed that all connecting function symbols f appearing in the data constraints correspond to fields in S. In other words, all data constraints refer to data fields that are present in the trees. However, our logic also supports connecting function symbols that do not correspond to fields in the data signature. In that case, the interpretation is free to assign any value to f(u), for each node <sup>u</sup> in the data tree. Thus, the Sdta <sup>A</sup><sup>ϕ</sup> must accept a data tree if there exists an interpretation for those functions that satisfies the formula. To achieve this effect, let {fi}i=1...k be the set of connecting function symbols occurring in ϕ and not corresponding to data fields in S, where f<sup>i</sup> has type nodes → datai. Define the extended data signature

$$\mathcal{S}' = \mathcal{S} \cup \{ f\_i : data\_i \}\_{i=1\ldots k}.$$

We enrich the state data signature of A<sup>ϕ</sup> as follows:

$$\mathcal{S}^{Q} = \{ state : Q\} \cup \{ name^i : type \mid (name : type) \in \mathcal{S}', i = 0 \dots n\}.$$

Compared to the original definition from Sect. 6.2, we store an extra copy of the data fields, identified by index 0, representing the data in the current node. Moreover, all copies include the auxiliary data fields. It is straightforward to adapt the constraint Ψ<sup>Δ</sup> from Sect. 6.2 to support such auxiliary data fields.

#### **7 Implementation and Experiments**

We implemented a prototype toolchain supporting our framework as shown in Fig. 1. Instead of developing an Mso-D parser, we provide an Mso formula already in the form (3), and supply the data constraints and the data signature for the formula in a separate file, directly in the SMT-LIB format. Next, we convert the Mso formula into an equivalent tree automaton, and in turn into a system of CHCs (as described in Sect. 5). We used Z3 v4.8.10 (64bit for Windows 10) [36] as the CHC solver, and Mona v1.4 [28] as the Mso-to-automata translator. The only new piece of code required by this implementation is the converter from the Mona tree automaton format to CHCs in the SMT-LIB language, which is a simple one-to-one textual transformation. For the experiments, we used a dedicated machine with 16 GB of physical memory and an AMD Ryzen 7 2700X clocked at 3.7 Ghz, running Windows 10.

#### **7.1 Proving Properties of Tree Data Structures**

Consider the property of full Bsts described in Example 7, namely that the successor of an internal node is the left-most leaf in its right subtree. We submitted to our tool the conjunction ψ bst ∧ ψsucc, which would be satisfied only by a full Bst where the successor of an internal node is *not* the left-most leaf in its right subtree (property Successor). Once the formula is converted into a system of CHCs, the SMT solver proves satisfiability of the system (and hence, unsatisfiability of the original formula) in less than a second.


**Table 1.** Mso-D satisfiability experiments (Sect. 7).

For Rbts, we consider the property that there exists an internal node whose black height is less than half of its height (property BlackHeight). Our approach can prove that this property is unsatisfiable on Rbts in less than a second. Both experiments are summarized in Table 1.

#### **7.2 Solving an Infinite-State Game**

Our approach can be used to solve certain infinite-state games, such as the Cinderella-Stepmother game [1,3]. In this software synthesis benchmark, two players share n buckets, each holding up to c units of water. The buckets are positioned in a circle and are initially empty. The game is played in a discrete sequence of turns: when it is Cinderella's turn, she can empty two adjacent buckets. When it is the Stepmother's turn, she can pour water into any subset of buckets, for a total of 1 unit of water. If any of the buckets overflows, Stepmother wins. If the game continues forever with no overflows, Cinderella wins. It can be described as an infinite-state turn-based two-player game of infinite duration with a safety objective (for Cinderella). Notice how not only the game state-space is infinite, but so are the moves available to Stepmother at each step.

Given values for the parameters n and c, we build an Mso-D formula ϕn,c that is satisfiable if and only if Stepmother wins the game with those parameters. The formula holds true on finite trees representing winning strategies of Stepmother. In other words, a tree that satisfies ϕ tracks all possible game plays where Stepmother pours water according to a specific deterministic plan and Cinderella takes all possible moves. Due to space constraints, further details on the encoding are deferred to an extended version of this paper.

In our experiments, we fixed the number of buckets n to 5 and checked the satisfiability of ϕ5,c for various values of the capacity c. In Table 1, we denote by Stepmother(c) the formula ϕ5,c. Bodlaender et al. [3], among their comprehensive analysis of this game, show that for n = 5, the minimum capacity for which Cinderella wins the game is c = 2 (see Table 1 in [3]). Their proof for this case is manual. Other cases were settled with the help of an SMT solver, using invariants based on non-trivial insights on the reasonable strategies of Stepmother. On the contrary, our encoding based on Mso-D employs only the rules of the game, with no further constraints on the players' moves.

Our setup successfully solves the game for various values of the capacity. The time needed by the SMT solver is very uneven, ranging from three minutes to a maximum of almost two hours for c = 2. That is explained by the fact that c = 2 is the hardest case for Cinderella to win the game. Therefore, proving that property requires building a complex winning strategy for Cinderella. Such strategy is embedded in the proof of unsatisfiability, and extracting it would be an interesting exercise beyond the scope of the present paper. When the capacity moves away from the critical threshold in either direction, the solving time visibly decreases.

#### **8 Related Work**

Our work is related to many works in the literature in different ways. In addition to the works already mentioned in the introduction, here we focus on those that seem to be closest to the results presented in this paper.

*Automata on Infinite Alphabets.* Symbolic finite automata (SFAs) [43] and symbolic tree automata (STAs) [42] replace the traditional finite alphabet by a decidable theory of unary predicates over a possibly infinite domain. They predicate over data words and trees, but they do not support storing, comparing, or combining data from different positions in the model, as that leads quickly to undecidability. Symbolic register automata [10] extend SFAs by storing data values in a set of registers. They retain decidability of the emptiness problem by only allowing equality comparisons between registers and input data.

Recently, Shimoda et al. [38] introduced *symbolic automatic relations* (SARs) as a formalism to verify properties of recursive data structures. While both Mso-D and SARs rely on CHCs as a backend, they differ in motivation and purpose. SARs aim at encoding specific properties of interest in a way that reduces the verification effort of the underlying CHC solver, whereas Mso-D is intended to provide a high-level language that can be compiled into CHCs.

*Decidable Logics with Data Extensions.* In [11], D'Antoni et al. design an extension of WS1S on finite data sequences where data can be examined with arbitrary predicates from a decidable theory, similarly to the capabilities of SFAs. They develop custom representations and algorithms to efficiently solve the satisfiability problem by reducing it to the emptiness of SFAs. Colcombet et al. [7] study a decidable fragment of Mso with data equality, called *rigidly guarded MSO*∼, where data equality constraints of the type *val*(x) = *val*(y) can only be checked on a single y-position for each x-position. Constraint LTL [13] is another decidable logic for infinite data words, where data in different positions can be compared for equality and for order. Segoufin [37] provides a wider, albeit slightly outdated, perspective on decidable data logics and automata.

*Logics for Automated Reasoning About Heap-Manipulating Programs.* Similarly to Mso-D, Strand [32] is a logic that combines Mso on tree-like graphs with the theory of integers. Although Strand has a fragment that admits a decidable and efficient decision procedure, it is not sufficiently expressive to state properties of classic data structures such as the balancedness of a tree. Also it does not allow solving the Cinderella-Stepmother game. Dryad logic [33] is a quantifierfree logic supporting recursion on trees, that is deliberately undecidable but admits a sound, incomplete, and terminating validity procedure, based on *natural* proofs [30]. Dryad recursive definitions could be expressed by our Sdtas that uses the theories of integers and integer (multi)sets; vice versa, proof techniques developed for Dryad could be used to check the emptiness of (some) Sdtas.

*Infinite-State Games.* Many infinite-state reachability games like the Cinderella game of Sect. 7.2 can be encoded in Mso-D, including all the reachability games used in the experiments performed by Farzan and Kincaid [17]. In that paper, the authors present a fully automated but incomplete approach for the (undecidable) class of *linear arithmetic games*. Our approach is incomparable to theirs: on the one hand, the approach proposed in Sect. 7.2 does not extend to all linear arithmetic games, because it assumes that one player has a bounded number of moves; on the other hand, we could easily handle games whose transition relations is not limited to linear arithmetic. Another related approach is presented by Beyene et al. [1], who reduce infinite-state games to CHCs extended with existential quantifiers. Such existential quantifiers are handled with the help of user-provided templates.

#### **9 Conclusions and Future Directions**

We presented Mso-D and Sdtas as extensions of Mso on trees and finite-state tree automata, respectively, for the purpose of reasoning about data trees. We have shown that these are versatile and powerful models for reasoning about relevant problems, outside the realm of classical automata theory. We believe that the key idea, namely separating the structural properties of interest from the data constraints, makes it easier to reason about challenging problems.

Several future directions are interesting. First, we may want to investigate theoretical questions about Sdtas, such as closure properties, and whether we can reduce classical automata decision problems to solving a system of CHCs. In addition, it will be interesting to identify more expressive Mso-D fragments that can be reduced to the emptiness of Sdtas.

Secondly, we believe that our results have applications to other areas in verification. We have conducted preliminary studies defining extensions of LTL with data (LTL-D) and, by using the framework developed in this paper and closure properties of Sdtas, obtained LTL-D model checking algorithms for (recursive) programs using scalar variables. Our approach is limited to finite runs only, so it will also be interesting to see how we can extend it to infinite trees and games.

Finally, (enumeration) trees can be used to encode executions of different classes of automata, such as concurrent pushdown automata or concurrent queue systems. It will be interesting to see if our approach can help lift the results of [31] to the corresponding class of concurrent programs.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Verified Erasure Correction in Coq with MathComp and VST**

Joshua M. Cohen(B) , Qinshi Wang , and Andrew W. Appel

Princeton University, Princeton, NJ 08544, USA jmc16@princeton.edu

**Abstract.** Most methods of data transmission and storage are prone to errors, leading to data loss. Forward erasure correction (FEC) is a method to allow data to be recovered in the presence of errors by encoding the data with redundant parity information determined by an error-correcting code. There are dozens of classes of such codes, many based on sophisticated mathematics, making them difficult to verify using automated tools. In this paper, we present a formal, machine-checked proof of a C implementation of FEC based on Reed-Solomon coding. The C code has been actively used in network defenses for over 25 years, but the algorithm it implements was partially unpublished, and it uses certain optimizations whose correctness was unknown even to the code's authors. We use Coq's Mathematical Components library to prove the algorithm's correctness and the Verified Software Toolchain to prove that the C program correctly implements this algorithm, connecting both using a modular, wellencapsulated structure that could easily be used to verify a high-speed, hardware version of this FEC. This is the first end-to-end, formal proof of a real-world FEC implementation; we verified all previously unknown optimizations and found a latent bug in the code.

**Keywords:** Reed-Solomon coding · functional correctness verification · interactive theorem proving

#### **1 Introduction**

As part of a larger project of ensuring reliable networks, we are applying formal functional-correctness verification to network components: machine-checked proofs that C programs (and, eventually, P4 programs and FPGAs) satisfy their high-level functional specs. When attackers may gain access to the source code and analyze it for bugs and vulnerabilities, we want something stronger than software testing or conventional static analysis: we want a proof that the software works *no matter what input is provided, no matter how dastardly*. And we want a proof that the program works *correctly,* not merely that it does not crash.

One key to reliable networking is *forward erasure correction (FEC):* in a portion of the network in which packets are being lost, add extra parity packets that allow reconstruction of lost packets without retransmission. We use an FEC algorithm and C program that have been in active use for over 25 years. The program does many clever and not-so-clever things, and comments indicate that some parts are not fully trusted even by its original authors.

This FEC is a particularly intriguing target for verification because its highlevel correctness depends on fairly intricate mathematics—we must reason about polynomials, matrices, and finite fields. Meanwhile, the C implementation's correctness relies on C programming features and careful manipulation of pointers in memory. Thus, we need a tool that can reason at both of these levels. We use the Coq proof assistant, utilizing the Mathematical Components [10] (Math-Comp) library for the high-level reasoning and the Verified Software Toolchain [6] (VST) for the C program verification.<sup>1</sup> Our VST specs are written using *separation logic*, in which we specify precisely what memory is read from and written to as well as all external effects (I/O, system calls, etc.). This gives us a blanket containment property: the C function is guaranteed to only interact with the outside world (memory, OS, etc.) in ways stated in the spec.

#### *Contributions*


#### **1.1 Forward Erasure Correction**

When transmitting data over a noisy channel, one can use an *error-correcting code*—adding generalized "parity" bits, sending the data across the channel, and then decoding to recover the data if any errors occurred; this technique is known as *forward error correction*. In an *erasure code*, the locations of the missing data are known to the decoder; this allows correction of more errors.

FEC is useful in any network where non-congestion-related packet loss is frequent and retransmission is infeasible or expensive. For instance, wireless networks are especially prone to packet loss due to interference or jamming. More generally,

<sup>1</sup> Our Coq proofs and an appendix with expanded definitions, specs, and proofs can be found at github.com/verified-network-toolchain/Verified-FEC/tree/cav22.

errors in network devices due to firmware bugs, misconfiguration, or malware can lead to dropped packets. In these cases, retransmission with TCP is not desirable, because TCP will incorrectly interpret these losses as congestion, grinding the network to a halt. Similarly, applications such as video or audio streaming, often run over UDP, cannot handle retransmission without additional work; moreover, the latency of retransmitting is often too high. Thus, FEC continues to be important in ensuring network reliability.

The algorithm we consider is based on Reed-Solomon [24] coding; it groups the input bits into symbols representing elements of a finite field and interprets the data as a polynomial over this field. Reed-Solomon codes are particularly useful for correcting *burst* errors—errors that occur sequentially—since n + 1 consecutive bit errors can only affect at most 2 symbols of length n. Reed-Solomon decoders can be quite complex, both in theory and implementation; many mechanisms have been developed for this purpose. Nevertheless, these codes have been heavily used in applications such as CDs, DVDs, Blu-Ray disks, hard drives, and satellite communications [27].

In the early 1990's, there was a flurry of activity in Reed-Solomon erasure coding. McAuley described [18] and patented [19] a method for FEC based on Reed-Solomon coding for use in network transmission. Rabin [23] described an alternate technique for information dispersal, which was further developed by Preparata [22], Schwarz [25], and others, mainly for use in RAID storage systems; Plank [21] provides a tutorial and explanation. McAuley later wrote a C implementation of FEC for network packets based on this second technique with several further modifications. We will refer to the algorithm implemented by McAuley's C code as the Reed-Solomon Erasure (RSE) code.

Bellcore (now Peraton Labs) has employed this FEC algorithm (and implementation) successfully in numerous networking projects to support resilient communication, most recently in the DARPA EdgeCT program. McAuley's implementation includes many optimizations and modifications to the core algorithm, including some whose correctness was unknown to the code's authors (Sect. 5.2). It had one bug that we corrected (Sect. 6.6). We have produced a formal, machine-checked proof that this FEC implementation correctly recovers data in the presence of erasures—we proved the *algorithm* correct and proved that the *program* correctly implements it.

#### **1.2 Coq and VST**

We use the Coq interactive theorem prover, in which the user states and proves theorems in a higher-order dependently typed logic. These theorems are mechanically checked by the Coq kernel. Proofs can be (semi)automated by Coq's built-in tactics and by user-defined tactic programming.

Coq has been widely used in program verification and formalized mathematics. One particularly important verification effort is CompCert [15], an optimizing C compiler written and proved correct in Coq. That is, CompCert comes with a formal proof that the assembly code generated by the compiler preserves the semantics of the input C program.

VST is a program logic and set of proof automation tools that enables the verification of C programs in Coq. Using VST, one can write a specification for each C function, stating its preconditions (properties that must hold before the function is run) and postconditions (properties that must hold when the function finishes). These properties can involve both C-specific assertions (e.g., about the contents of memory) and arbitrary statements in Coq's logic. Then, using custom tactics and proof automation included with VST, the user can prove in Coq that the C function satisfies its specification.

VST's program logic is proved sound, with a machine-checked proof in Coq. When we prove that McAuley's RSE correctly reconstructs missing packets, the soundness proof guarantees that the assembly-language program generated by the CompCert C compiler really has that behavior. VST is formally proved sound for CompCert, but not for gcc or clang. VST is intended (and believed) to be sound for gcc/clang; its program logic has stricter rules than would be necessary only for soundness w.r.t. CompCert. For example, for signed integer arithmetic, where CompCert is (unfortunately) a refinement of C11 (CompCert wraps while C11 is u.b.), VST imposes the (more abstract) C11 spec. Thus, VST proofs about C programs also provide useful (though less foundational) assurance about programs compiled with other compilers.

While conventional separation logics have spatial conjuncts that are predicates just on memory resources, VST's separation logic has spatial conjunct predicates on both memory locations and the outside world, which one might affect by performing IO or making a system call [17, Section 3]. In our project, none of the VST funspecs mention the outside world in the precondition or postcondition; this means, like any Hoare triple in separation logic, that those functions can neither access nor modify that resource.

Proving that a C program satisfies a specification is quite challenging. We must prove low-level correctness properties (the program does not crash, all memory accesses are valid, etc.) and provide loop invariants and intermediate proofs to prove high-level properties (that the function satisfies its spec). Though VST's proof automation is able to hide some of this complexity, many parts must still be done manually. Dealing with heavily optimized C code that was never intended to be verified makes these tasks substantially more complicated.

Section 2 describes the RSE algorithm, which differs in several ways from the technique described by Rabin, Preparata, and Schwarz. Section 3 explains the different verification tasks, including defining a functional model of the algorithm and showing with VST that the C code implements this model. Section 4 describes the functional model, Sect. 5 discusses the verification of this functional model, including the proof that the algorithm correctly reconstructs missing packets, and Sect. 6 discusses the proofs about the C code. Section 7 and Sect. 8 give related and future work.

#### **2 The RSE Algorithm**

Like all Reed-Solomon codes, the algorithm treats input symbols as elements of a finite field and interprets the input sequence of words as the coefficients of a polynomial over this field. However, both the C implementation and the RSE algorithm are more naturally described using linear algebra and matrix operations.

Let D be the input data, which consists of k packets, each of length at most c bytes. If any packets are smaller, fill in the missing entries with zeroes so that D is a k × c matrix. Let h be the number of parity packets we wish to append. We will be able to reconstruct up to h total packet-drops.

Let k*max* and h*max* be (fixed) parameters such that k ≤ k*max* and h ≤ h*max*. Let n*max* = h*max* + k*max* (maximum number of packets per batch) and let F be a field such that |F| > n*max*.

#### **2.1 Initialization**

First, we generate a Vandermonde matrix of size h*max*×n*max*; that is, take n*max* distinct nonzero elements of F, denoted as α1, α2, ..., α*<sup>n</sup>max* , and generate the following matrix:

$$V = \begin{pmatrix} 1 & 1 & \dots & 1 \\ \alpha\_1 & \alpha\_2 & \dots & \alpha\_{n\_{max}} \\ \alpha\_1^2 & \alpha\_2^2 & \dots & \alpha\_{n\_{max}}^2 \\ \vdots & \vdots & \vdots & \vdots \\ \alpha\_1^{h\_{max}-1} \alpha\_2^{h\_{max}-1} & \dots & \alpha\_{n\_{max}}^{h\_{max}-1} \end{pmatrix}$$

Then, we run Gaussian elimination (see Sect. 4.2) on this matrix to get the row-reduced form, which consists of the identity matrix followed by W, the h*max* × k*max weight matrix* :

$$V \xrightarrow{\text{Gaussian elimination}} \left[I\_{h\_{max} \times h\_{max}} \, \middle| \, W \right]$$

#### **2.2 Encoding**

The encoder receives as input the data D, a k×c matrix. Let W be the submatrix of W consisting of the first h rows and the first k columns. The encoder computes P = W D, an h × c matrix. These are the parity packets that are sent (along with the original data) to the receiver.

$$W = \begin{array}{c} \begin{array}{c} k \\ \cdot \\ \cdot \\ \cdot \\ \end{array} \end{array} \begin{array}{c} \begin{array}{c} k \\ \cdot \\ \cdot \\ \cdot \\ \cdot \\ \cdot \\ \end{array} \end{array} \dots \begin{array}{c} \begin{array}{c} \\ \\ \\ \\ \cdot \\ \cdot \\ \cdot \\ \cdot \\ \end{array} \dots \begin{array}{c} P = W'D \\ \cdot \\ \end{array} \tag{1}$$

#### **2.3 Decoding**

The decoder is significantly more complicated. However, if no packets are lost, the decoder simply returns the first k packets; only if packets are dropped does the following algorithm need to be invoked. To give some intuition, we will first present the decoder for a special case before giving the full algorithm.

Since this is an erasure code, we know the locations of the missing packets; we also require that the total number of missing packets is at most h.

For a special case, suppose that the last h data packets were lost and all parity packets were received. We can think of the original data D as a block matrix consisting of D1, the (k − h) × c matrix of the received data, and D2, the h × c matrix of the lost data. Similarly, we can split the h × k matrix W (from the encoder) into W <sup>1</sup>, consisting of the first k −h columns of W , and W 2, consisting of the rest.

$$W = \begin{array}{c} \underbrace{k-h}\_{h} \\ \left[ \begin{array}{ccc} \overbrace{W\_1' & W\_2'}^{\prime} & \dots & \\ \vdots & \vdots & \ddots & \\ \end{array} \right] & D = \begin{array}{c} k-h \end{array} \Bigg[\overbrace{D\_1}^{c} \end{array} \Bigg[\overbrace{D\_1}^{c} \end{array}$$

$$P = W'D = W\_1'D\_1 + W\_2'D\_2$$

From this, the missing data D<sup>2</sup> can be computed using P, D1, and the parts of W , all of which are known:

$$D\_2 = (W\_2')^{-1}(P - W\_1'D\_1) \tag{2}$$

The general case is similar, but we need to define the relevant submatrices more carefully. Let *xh* be the number of missing data packets. We must have received at least *xh* parity packets (or else the total number of missing packets is more than h). Let P be the submatrix of P consisting of the first *xh* received parity packets. This time, we let W <sup>1</sup> be the *xh* × (k − *xh*) submatrix of W whose rows consist of the locations of the *xh* found parity packets and whose columns consist of the k − *xh* locations of the received data. Let W <sup>2</sup> be the *xh* × *xh* submatrix of W whose rows consist of the locations of the *xh* found parities and whose columns consist of the locations of the missing data. Finally, D<sup>1</sup> and D<sup>2</sup> are still defined such that D<sup>1</sup> contains the received rows and D<sup>2</sup> contains the missing rows. This time, these rows need not be contiguous. These definitions reduce to the previous ones in the special case considered above.

By the definitions of the above submatrices, Eq. 2 still holds (except that we replace P with P ), so we can find the missing data D<sup>2</sup> by computing (W <sup>2</sup>)−<sup>1</sup>(P <sup>−</sup> <sup>W</sup> <sup>1</sup>D1).

This decoder is only well defined if W <sup>2</sup> is invertible. W <sup>2</sup> is dynamically chosen based on the found parities and missing data, so we must show a stronger claim that any square submatrix up to size h × h of W is invertible. Proving this was the crucial step in the functional model verification, described in Sect. 5.1.

As noted in Sect. 1.1, this algorithm is a modified version of the technique described by Rabin, Preparata, Schwarz, and others. The main difference is the use of the static weight matrix in RSE; all the others assume that the Vandermonde matrix has dimensions h × (k + h) and exactly h packets are lost. Thus, their needed correctness property is weaker; it requires only that any h × h submatrix of W is invertible.

#### **3 Verification Structure**

The verification consists of two distinct tasks: we prove that the RSE algorithm is correct (i.e., the decoder recovers the original data in the presence of errors) and that the C program truly implements this algorithm. These two tasks are quite different; the first is purely mathematical and involves proofs about linear algebra, while the second involves implementation details and C-language verification conditions. To separate these tasks and make the proofs more modular, we define a *functional model*, a purely functional program written in Coq that implements the RSE algorithm. This functional model is inefficient but easy to reason about in Coq. Then we use VST to prove that the C program refines this functional model. Finally, we compose these two parts to produce a formal proof that the C implementation of this erasure code is correct.

Separating the functional specification and the VST proofs is a common paradigm; it has been used to verify SHA-256 hashing [4], HMAC-DRBG cryptographic random number generation [28], and floating-point numerical programming [5]. This approach provides a clear formal specification independent of any implementation; we can reuse the same functional model and its correctness proofs to verify another implementation of this algorithm (for instance, an FPGA version). It makes verification more flexible; we can prove further properties later simply by adding additional lemmas about the functional model. It makes the proofs shorter and clearer; we can tell which parts are needed for the core correctness proofs and which are implementation-specific. Finally, it permits a separation of expertise: the person who proves mathematical theorems about the functional model need not know anything about C programming or VST verification, and the person who proves C refinement in VST need not know why the functional model accomplishes the high-level goals.

Our functional model was written in Gallina, the functional programming language embedded in Coq, using the Mathematical Components (MathComp) library for formalized mathematics. MathComp contains definitions and theorems about groups, rings, fields, vector spaces, matrices, polynomials, graphs, and other mathematical objects.

In fact, we define two functional models—a high-level version uses Math-Comp's abstract and dependent types of matrices, polynomials, and the like, while a low-level version uses concrete types such as list (list byte), which VST can use to represent memory contents. Translating between these types is nontrivial (because of all the dependent types in MathComp), so we separate the type conversion proofs from both the high-level mathematical reasoning and the low-level VST refinement proof. This makes the proofs more modular and helps to improve the readability of the resulting formalization. The translation is largely mechanical and we expect that it could be automated; we focus on the high-level functional model and the VST refinement proofs.

#### **4 Functional Model**

#### **4.1 The Encoder and Decoder**

We translate Eq. 1 into the language of Coq/MathComp:

**Definition** encoder (h k c max h max n : nat) (Hh: h ≤max h) (Hk: k ≤max n) (weights : 'M[F] (max h, max n)) (input : 'M[F] (k, c)) := (mxsub (fun (x : 'I h) ⇒ widen ord Hh x) (fun (x : 'I k) ⇒ rev ord (widen ord Hk x)) weights) ∗m input.

'M[F] (x, y) denotes a matrix of size x × y over field F and <sup>∗</sup><sup>m</sup> denotes matrix multiplication. The type 'I n represents an ordinal, a natural number in the range [0, n−1]. The encoder takes in the parameters h, k, c, h*max*, and n*max* (all defined as in §2), the h*max* ×n*max* weight matrix, the k ×c data matrix, and proofs that h and k are bounded appropriately. mxsub creates a submatrix from an input matrix by selecting rows and columns via user-specified functions. widen ord is needed to handle some dependent type casting; it has no computational content and can be ignored. Finally, rev ord selects the "opposite" ordinal; for x : 'I k, rev ord x = k − x − 1. Therefore, this function selects the first h rows and the last k columns (in reverse order) of the weight matrix and multiplies this by the input. This differs from the algorithm in Sect. 2.2, which selects the first k columns. The overall algorithm's correctness is not affected as long as we choose the matrices W <sup>1</sup> and W <sup>2</sup> in the decoder to be consistent, but this change makes the model consistent with the C implementation (see Sect. 6.2).

The decoder (Eq. 2) can be similarly translated into MathComp; we omit the full definition, but note that we defined the decoder more generally than needed: it is defined over any field and over any Vandermonde matrix on distinct elements of that field.

#### **4.2 Gaussian Elimination**

Gaussian elimination, or row reduction, is a well known algorithm in linear algebra for solving systems of linear equations, finding matrix inverses, and calculating determinants. The C code includes an implementation of Gaussian elimination, used to row-reduce the Vandermonde matrix to produce the weight matrix and to invert W <sup>2</sup> in the decoder. Thus, we need to define a corresponding functional model.

Gaussian elimination proceeds by applying a sequence of elementary row operations—swapping two rows, multiplying a row by a scalar, and adding a scalar multiple of one row to another row—to a matrix until it is in row-echelon form, which for full-rank matrices (including all relevant matrices in this application) means that the left hand side becomes the identity matrix. Crucially, these row operations preserve invertibility because each corresponds to left multiplication by an (invertible) elementary matrix.

The order of the row operations may vary; Algorithm 1 describes one concrete implementation of Gaussian elimination (we use 0-indexing to be consistent with

#### Algorithm 1: Gaussian Elimination

```
On input A, an m × n matrix:
  r ← 0; c ← 0
  while r<m and c<n do
    if for all i such that r ≤ i<m, Ai,c = 0 then
      c ← c + 1
    else
      i ← the first index s.t. r ≤ i<m and Ai,c = 0
      Swap rows r and i
      For all 0 ≤ j<m, if Aj,c = 0, multiply row j by A−1
                                                         j,c
      For all 0 ≤ j<m, j = r, if Aj,c = 0, subtract row r from row j
      r ← r + 1; c ← c + 1
    end if
  end while
  for r = 0 to r = m − 1 do
    Let c be the index of the first nonzero entry in row r if one exists
    Multiply row r by A−1 r,c
  end for
```
MathComp). While translating this into MathComp is largely straightforward, it turns out that the C program does not actually implement Algorithm 1. Rather, rows are never swapped and at each iteration, all entries in column c must be nonzero.

The following excerpt from the C code, with the original comments, shows the error checks to ensure this condition. The code is mainly interesting for the error checks and comments, but we briefly detail how it works: the while guard value never changes; instead for current column k, the code iterates through rows w. The second conditional checks if matrix element (w, k) is nonzero for swapping (but returns an error because swapping is not implemented), while the first conditional breaks out of the loop with an error when w has reached the last row.

```
while (∗(q − k) == 0){ /∗ if zero ∗/
 if (++w == i max){
   return (FEC ERR TRANS FAILED); /∗ failed ∗/
 }
 if (∗(p + (w ∗ j max) + j max − 1 − k) != 0){
   /∗ swap rows ∗/
   return (FEC ERR TRANS SWAP NOT DONE); /∗ Not done yet! ∗/
 }
}
```
The "swap rows" and "Not done yet!" messages suggest that the authors intended to (eventually) implement the full algorithm. The error checks indicate that the authors were not sure if these errors could be triggered.

We will call this algorithm "Restricted" Gaussian elimination (Algorithm 2). Once again, defining this function in MathComp is not difficult, but proving that this limited form of Gaussian elimination suffices was a major part of the functional model verification (Sect. 5.2).

#### Algorithm 2: Restricted Gaussian Elimination

```
On input A, an m × n matrix:
  r ← 0
  while r<m do
    For all 0 ≤ j<m, if Aj,r = 0, return error
    For all 0 ≤ j<m, multiply row j by A−1
                                          j,r
    For all 0 ≤ j<m, j = r, subtract row r from row j
    r ← r + 1
  end while
  for r = 0 to r = m − 1 do
    Multiply row r by A−1 r,r
  end for
```
#### **4.3 Field Operations**

The encoder, decoder, and Gaussian elimination work over any field, but the C implementation uses the field GF(2<sup>8</sup>), which we must define. Mathematically, this field is isomorphic to F2[x]/(1 + x<sup>2</sup> + x<sup>3</sup> + x<sup>4</sup> + x<sup>8</sup>). That is, the elements of this field are polynomials of degree at most 7 with coefficients in F<sup>2</sup> (the field of two elements), and all operations are performed modulo 1 + x<sup>2</sup> + x<sup>3</sup> + x<sup>4</sup> + x<sup>8</sup>. The choice of F<sup>2</sup> is important; it allows us to represent polynomials as sequences of bits. Since the polynomials are of degree at most 7, all field elements can be represented as bytes.

This field and its construction are well understood; while MathComp did not include the construction of finite fields via quotients, we were able to define and prove general results about primitive polynomials and the finite field's construction without much issue. Then, we can prove correct the method the C code uses to populate the lookup tables used to compute in this field (Sect. 6.4).

One difficulty in using this field is the difference between the polynomials we used to define the field and the bytes that we would like to represent as field elements. To avoid manually converting everywhere, we defined another field structure directly on the byte type and used Coq's Canonical Structures.

#### **5 Verifying the Functional Model**

#### **5.1 Decoder Correctness**

To prove the RSE algorithm correct, we need to prove that the decoder actually reconstructs the original packets. That is, if the data and parity packets that were marked as "received" are correct and there are at most h missing packets, then running the decoder on the received packets should recover the original data. We state this in Coq below:

**Theorem** decoder correct: ∀ (h xh : nat) (Hh: xh ≤h) (data : 'M[F] (k, c)) (input: 'M[F] (k, c)) (parities: 'M[F] (h, c)) (missing packets : seq 'I k) (found parities : seq 'I h) (Hhh: h ≤max h) (x h : 'I h), (∗ Only the rows in [missing packets] are incorrect ∗) (∀ (x: 'I k) (y: 'I c), x \notin missing packets → data x y = input x y) → (∗ All found parity packets were produced by the encoder ∗) (∀ (x: 'I h) (y: 'I c), x \**in** found parities → parities x y = (encoder Hhh k leq n weights data) x y) → (∗ We have xh unique missing packets and found parities ∗) uniq missing packets → uniq found parities → size missing packets = xh → size found parities = xh → (∗ Then, the decoder recovers the original data ∗) decoder xh input parities missing packets found parities Hhh x h = data.

This theorem is expressed entirely in terms of MathComp matrices and operations; it does not rely on the C implementation at all. Its proof requires two main tasks: showing that W <sup>2</sup> is invertible and proving that the sequence of operations in the decoder is sufficient to recover the original data. The second task is fairly straightforward; we compare the matrices elementwise. Thus, the main challenge comes from proving the invertibility of the submatrix W 2.

**Proving the Invertibility of** *W-* **<sup>2</sup>.** Recall that W <sup>2</sup> is a dynamically chosen submatrix of W, the right submatrix of the row-reduced Vandermonde matrix V . Therefore, we want to prove the following theorem (any submx unitmx):

**Theorem 1.** *Let* V *be an* m × n *row-reduced Vandermonde matrix on distinct elements. Let* m ≤ n *and* z ≤ min(m, n − m)*. Let* Y *be the submatrix of* V *formed by taking* z *rows of* V *and* z *of the last* (n − m) *columns of* V *. Then* Y *is invertible.*

Formally proving this theorem in Coq is quite complicated, partly because Math-Comp does not include many of the definitions and results that we need. Namely, we need to define and prove properties about row operations and Vandermonde matrices, including the following well-known property (vandermonde unitmx):

**Theorem 2.** *Let* V *be an* n × n *Vandermonde matrix on distinct nonzero elements. Then* V *is invertible.*

The proof relies on the fact that a degree n polynomial with n + 1 zeroes is identically zero, a fact already included in MathComp. This marks the only direct use of polynomial properties (other than in the finite field construction); the rest of the results are purely based on linear algebra.

Note that the only property we required of the weight matrix W was that every z × z submatrix is invertible. Row-reduced Vandermonde matrices satisfy this property, but any other matrix that satisfies this property could be used, and the encoding-decoding scheme would still be correct.

#### **5.2 Gaussian Elimination**

Proving full Gaussian elimination (Algorithm 1) correct is fairly standard (though nontrivial to formalize completely in Coq), since the algorithm is very well-understood.

The real challenge is to determine the conditions under which RGE (Algorithm 2) will return the same result as Algorithm 1. It is easy to see that if the error case is never reached, then the two algorithms are equivalent. But it is not at all obvious how to avoid triggering the error. Invertibility is a necessary but quite insufficient condition; for instance, the restricted algorithm fails on diagonal and triangular matrices. Therefore, we had two tasks: determine the class of matrices for which RGE works correctly and prove that the matrices used in the RSE algorithm are in this class.

For the first task, we needed to determine when certain elements will be zero or nonzero at a given step in Gaussian elimination. This is difficult, since the elements are constantly changing; instead, we transformed the condition into a statement about the invertibility of certain submatrices, since Gaussian elimination preserves invertibility.

During the rth step of Gaussian elimination (assuming no error was reached), the r × r upper-left submatrix is a diagonal matrix with nonzero elements along the diagonal; all other elements in the first r columns are zero. With this, we defined the submatrix C*<sup>r</sup> <sup>k</sup>* (for k<r) as the submatrix of A consisting of the first r rows and the first r + 1 columns except column k. Then, for k<r, A*k,r* = 0 exactly when C*<sup>r</sup> <sup>k</sup>* is invertible (we prove this by showing that the rows of C*<sup>r</sup> k* are linearly independent). We can do something similar for k ≥ r; this time we consider R*<sup>r</sup> <sup>k</sup>*, defined to be the submatrix of A consisting of the first r+1 columns and rows {0, <sup>1</sup>,...r <sup>−</sup> <sup>1</sup>, k}. Similarly, <sup>R</sup>*<sup>r</sup> <sup>k</sup>* is invertible iff A*k,r* = 0. We will say that <sup>A</sup> is *strongly invertible* if, for all 0 <sup>≤</sup> r<m, <sup>C</sup>*<sup>r</sup> <sup>k</sup>* is invertible for all k<r and R*<sup>r</sup> <sup>k</sup>* is invertible for all k ≥ r. Finally, we prove that RGE is equivalent to full Gaussian elimination iff input A is strongly invertible.

Note that this condition requires a particular set of m<sup>2</sup> submatrices of the input m × n matrix to be invertible, quite a difficult condition to satisfy. However, in this application, Gaussian elimination is applied to only two kinds of matrices: the matrices W <sup>2</sup> in the decoder and a Vandermonde matrix on x*<sup>n</sup>max*−<sup>2</sup>,...,x<sup>2</sup>, x, 1 (where x is the primitive element of the field). The strong invertibility of each ultimately follows from properties of Vandermonde matrices: the result for the first matrix follows from Theorem 1, while the result for the second is harder to show, but ultimately follows from repeated applications of Theorem 2 and use of the fact that the field elements are consecutive powers of the primitive element. With this, we proved the previously unknown result that RGE suffices for this application and that the errors shown in Sect. 4.2 are never reached.

## **6 Verifying the Implementation**

The C code consists of five primary functions with the following signatures:

// Populate the field lookup tables **void** fec generate math tables(**void**) // (Restricted) Gaussian elimination on the i max × j max matrix p **int** fec matrix transform(**unsigned char** ∗p, **unsigned char** i max, **unsigned char** j max) // Generate weight matrix (row−reduced Vandermonde matrix) **void** fec generate weights(**void**) // Encode the data by appending h parity packets to the k data packets in pdata. // plen is an array of the lengths of the data packets. // pstat is a flag, all are initially FEC FLAG KNOWN. **int** fec blk encode(**int** k, **int** h, **int** c, **unsigned char** ∗∗pdata, **int** ∗plen, **char** ∗pstat) // Decode the packets in pdata. The ith flag in pstat is FEC FLAG WANTED if // the ith packet is missing, otherwise FEC FLAG KNOWN **int** fec blk decode (**int** k, **int** c, **unsigned char** ∗∗pdata, **int** ∗plen, **char** ∗pstat)

Each of these functions has a corresponding VST specification. We first describe key implementation differences and verification challenges, then discuss the specs for selected functions in Sect. 6.4 and Sect. 6.5.

#### **6.1 Implementation Differences from Algorithm**

Broadly, the C code implements the RSE algorithm from Sect. 2 with the parameters k*max* = 127 and h*max* = 128 (as well as a bound of 16000 on c, but this does not affect the correctness). However, neither this algorithm nor the functional model precisely align with the C implementation. Instead, the implementation makes a few changes, and we must prove that these changes do not modify the algorithm's behavior:


– The code takes as input a sequence of variable-length packets, and we want to recover the original data once the decoder has finished. The RSE algorithm only describes how to generate the recovered packets, but the implementation has to put each packet pointer in its correct position in the packet array and ensure that the length for each packet is correct. The functional model includes filling in missing packets, but it uses matrices of uniform length.

#### **6.2 Implementation-Specific Verification Challenges**

Aside from differences between the algorithm and implementation, the C code, first written 25 years ago and last modified over 15 years ago, does several things that make it poorly suited to verification:


#### **6.3 VST Specifications**

A C specification in VST looks like:

DECLARE f WITH v PRE [ *param typs* ] PROP(p1) PARAMS(*params*) GLOBALS(*globs*) SEP(s1) POST [ *ret ty* ] PROP(p2) RETURN(*ret*) SEP(s2)

where f is the function name, *param typs* are the C function parameter types, *ret ty* is the C return type, *params* are the (symbolic) values of the function parameters, *globs* are the global variables, and *ret* is the (symbolic) return value. The entire PRE block represents the precondition, which must hold before the function is run. The POST block is the postcondition, which is true after the function finishes. *p1* and *p2* are propositions in Coq's logic, while *s1* and *s2* are propositions in separation logic—they describe the contents of memory. Finally, the variables v in the WITH clause are logical variables, abstract mathematical values to which the precondition and postcondition can refer.

#### **6.4 Verifying fec generate math tables**

The first C function is fec generate math tables, the function that generates the power, logarithm, and inverse tables for the field elements. This function, like the others, is interesting because of how it modifies memory, not because of what it returns; thus the interesting part of the VST spec is the SEP clause. The precondition's SEP clause says that the global array fec 2 index (the power table) initially stores fec n zeroes. In the postcondition, this global array now stores the Coq list byte pows, which we define as the powers of field element x (the ith entry contains x*<sup>i</sup>* ). We have similar Coq lists and pre- and post-conditions for the log table and inverse table.

Proving that the field table generation is correct is largely straightforward, given the field definitions described in Sect. 4.3. However, there were two main complications. The first comes from the method of populating the tables: compute <sup>x</sup>*<sup>i</sup>* for all 0 <sup>≤</sup> i < 256 by repeatedly multiplying the result by <sup>x</sup> in each iteration (this can be implemented efficiently as a bitwise shift left and an xor). The correctness of this method relies on the fact that the modulus polynomial is primitive (i.e., the smallest <sup>n</sup> such that the modulus polynomial divides <sup>x</sup>*<sup>n</sup>* <sup>−</sup> <sup>1</sup> is 255), and is not trivial to show in Coq.

Separately, although in the functional model we prove results for arbitrary fields and irreducible polynomials, here we need to show that several specific polynomials are irreducible and primitive (several field sizes are allowed by the code, although only one is used). Both of these conditions require showing that a polynomial is not divisible by a set of polynomials, so the easiest way to show this is by direct computation along with a proof that this computation is sufficient. However, MathComp polynomials are opaque and not computable (dividing two MathComp polynomials results in a hanging computation), so we needed to define concrete, computable polynomials and operations and relate them to their MathComp equivalents. Then, we can prove that the particular polynomials that the C code uses satisfy all needed properties.

#### **6.5 Verifying fec blk decode**

The function spec for fec blk decode is quite long; it consists of many tedious preconditions to ensure that the input packets are stored correctly in memory, that the length and packet status arrays correspond to the actual packets in memory, and that the various integer parameters are within their correct bounds. The list of preconditions is long; however, these functions are called by client functions that do packet-handling and buffer management, and the verification of *those* functions will check that they do indeed set up their inputs correctly (see Sect. 8).

We focus on a key part of the spec: the precondition's SEP clause includes the predicate iter sepcon arrays packet ptrs packets, which states that the Coq list packets is stored in memory at the given pointers. In the postcondition's SEP clause, this becomes iter sepcon arrays packet ptrs (decoder list k c packets parities stats lengths parbound). In other words, after the function is run, the contents of the packet memory are represented by the *low-level functional model* of the decoder (the version that uses concrete types that VST can understand rather than opaque MathComp types).

Our decoder correct theorem (Sect. 5.1) states that the *high-level functional model* correctly reconstructs the missing packets that were originally given to the encoder. Lemma decoder list correct lowers that result to the low-level functional model, using some injectivity results between the two models.

Thus, a client of the code can compose the VST spec and the correctness theorem to prove that, after fec blk decode is run, as long as the received packets and parities were correct, the missing data is recovered and the original data is now stored in memory (see Sect. 8).

#### **6.6 Implementation Bug**

While verifying fec matrix transform, we discovered a bug in the following code:

```
q = (p + (i ∗ j max) + j max − 1);
m=q − j max;
for (n = q; n > m; n−−) {
        //loop body
}
```
Here, i ranges from 0 to i max, and p is a pointer to the input matrix. The problem is, when i = 0, q points to p+j max − 1 and thus m points to p − 1. By the C standard and the semantics of CompCert C, the comparison n > m is undefined behavior. In fact, in C11, even the line q − j max is undefined behavior [12, Section 6.5.6, #8].

This may seem harmless, but 21st-century C compilers optimize under the assumption that the program does not exhibit undefined behavior. A compiler can assume that m=q − j max cannot be reached when i=0, and it may mangle the loop body "knowing" that i= 0 . This has caused problems for systems code [26], and the solution is to avoid writing C programs with undefined behavior.

Fortunately, VST's machine-checked proof of soundness makes it impossible to prove a C program correct that contains undefined behavior (unless ruled out by a function precondition). The loop test n > m cannot be verified in VST, since undefined behavior cannot be ruled out.

Without formal methods, this type of bug is quite difficult to find: it depends on subtle C semantics, today's static analyzers won't catch it,<sup>2</sup> and testing cannot catch it until (in some future year) an optimizing C compiler gets more aggressive. VST provides blanket assurance against this entire class of errors.

Moreover, because VST uses separation logic, we specify exactly what effects the code is allowed to have. Thus, in principle, this kind of verification is 100% resistant to adversarial attacks that try and put exploits into code provided that those exploits can be defined as a functional property of the C code (such as which memory addresses it accesses, what system calls it makes, etc.). But our methods cannot defend against side-channel attacks.

#### **7 Related Work**

*Verification of Network Middleboxes.* Through several recent efforts, verification of network functions running in the dataplane has become increasingly feasible. Software dataplane verification [9] uses symbolic execution to prove certain lowlevel properties (such as memory safety) about programs written with Click, a popular framework for configuring routers and writing network functions. Gravel [31] uses symbolic execution and SMT solvers to verify many middlebox-specific properties of Click programs, including functional correctness. VigNAT [30] uses a mix of symbolic execution and proof checking to verify a Network Address Translation (NAT) implementation in C; this approach requires the use of a specialized data structure library and annotations on the C program but is quite automated overall. Vigor [29] builds on VigNAT to extend similar methods to more general network function verification. It uses a simpler but less expressive specification language, enabling fully automatic verification. Vigor and VigNAT use Verifast [13], a separation-logic-based tool for verifying C programs that is more automated than VST but is not connected to a proof assistant; this makes functional model proofs much more difficult.

These tools are considerably more automated than our work, but face significant restrictions on the type of code they can verify: none can verify code with

<sup>2</sup> "Conceptually, this undefined-behavior optimization bug is possible to trigger with STACK's approach [26]. But, as for the current implementation of STACK, the answer is likely no, because it depends on LLVM to do loop unrolling/inlining . . . and I doubt LLVM would do either . . . ." (Xi Wang, e-mail of May 23, 2022).

arbitrary unbounded loops, pointer arithmetic, or use of complex data structures. More importantly, none could handle the mathematical reasoning needed to prove the correctness of the functional model and ensure that the FEC correctly reconstructs packets.

*Verification of Error-Correcting Codes.* Since error-correcting codes are both ubiquitous and quite complex to implement correctly, there has been a long line of research in formalizing various codes. Most of these efforts take the form of either automated hardware verification of digital circuits or recent efforts to create formalized libraries of error-correcting codes. We believe that our work is the first to connect a high-level, mathematical specification with an efficient implementation.

Error-correcting codes are hard to verify with automated methods such as model checking and BDDs because of the large state space and the complexity of the algorithms. Some recent efforts [8] have used automated hardware verification tools to verify (non-Reed-Solomon) ECCs, but they can handle very few bit errors. BLUEVERI [16] is a tool for verifying hardware implementations of finite field operations and was applied to Reed-Solomon codes. It can handle more errors (up to almost a dozen bits), but requires extensive manual effort and knowledge of hardware implementation details.

In a separate vein, several recently-developed libraries of formalized coding theory are similar to the functional model in our work, but are not connected to an efficient implementation. Most notably, Affeldt, Garrigue, and Saikawa have developed a Coq library for error-correcting codes, including Hamming and acyclic LDPC [1], Reed-Solomon [2], and BCH [3] codes. This library is built atop MathComp, and includes many theoretical results about each of these codes as well as specific encoders and decoders. Ideally, we would have liked to use this library as part of our functional model, but the implementation we verified differs significantly from standard Reed-Solomon coding, which corrects errors rather than erasures. Their library's Euclidean-algorithm-based decoding is extremely different from the decoder in RSE.

In Lean, a coding theory library called Cotoleta was developed and used to prove results about Levenshtein distance [14] and Hamming(7,4) codes [11]. Separately, Hamming(7,4) and <sup>1</sup> <sup>2</sup> -rate convolutional codes were verified in the ACL2 theorem prover [20] with a particular focus on correcting memory errors; these codes were verified against a particular memory model. Both of these projects focused on verifying concrete-sized codes; thus they did not require the same level of abstraction or general mathematical reasoning as our work.

#### **8 Future Work**

In a real system, the encoder and decoder verified in this work are called by clients who handle receiving packets, assigning them to batches, and maintaining various data structures. We are currently working to verify a real-world version of such a system. This will permit a single, clean, end-to-end correctness result; right now, we have separate results for the decoder's correctness and the C program refinement which must be composed together. However, the specification of such a system introduces new challenges; it must reason about packet streams and network-specific features such as headers, timeouts, and packet reordering.

This C implementation of RSE has been useful in several projects at Bellcore/Telcordia/Peraton even though it cannot run at modern packet bit rates. We believe that a line-rate FPGA implementation of the finite-field matrixmultiply partial step is possible, and we are designing an API by which this could be controlled by a C program or a P4 program. Such an FPGA could be proved correct by a layered proof. The top layer would be our MathComp proof with no changes. The bottom layer could be proved using a Coq tool for hardware synthesis and functional-correctness verification, such as Kˆoika [7].

## **9 Conclusion**

We have presented an efficient, real-world C implementation of Reed-Solomon forward erasure correction that we formally verified using the Coq proof assistant and the Verified Software Toolchain. The code was verified with only minor changes; one macro was turned into a function for ease of verification and one bug that caused undefined behavior was fixed. While the code has been in use for over 25 years, the correctness of certain parts of the underlying algorithm, a modified form of Reed-Solomon erasure coding, were still ill-understood, including a very restricted form of Gaussian elimination. We were able to use Coq's Mathematical Components library to completely verify the correctness of this algorithm and VST to prove that the C code, with its various optimizations and modifications, correctly implements this algorithm. This demonstrates that tools like VST allow us to verify real-world, dusty-deck programs in C, even those whose correctness depends on a broad base of mathematics and those with numerous low-level optimizations. We believe this can be a viable approach to connect efficient low-level code with sophisticated high-level reasoning, enabling reliable software components for networks and other systems.

## **Appendix**

The appendix to this paper can be found in our git repo (see footnote 1) in doc/Appendix.pdf.

## **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **End-to-End Mechanized Proof of an eBPF Virtual Machine for Micro-controllers**

Shenghao Yuan1(B) , Fr´ed´eric Besson<sup>1</sup> , Jean-Pierre Talpin<sup>1</sup> , Samuel Hym<sup>2</sup>, Koen Zandberg<sup>4</sup>, and Emmanuel Baccelli3,4

> <sup>1</sup> Inria, Rennes, France {shenghao.yuan,frederic.besson, jean-pierre.talpin}@inria.fr <sup>2</sup> University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, 59000 Lille, France samuel.hym@univ-lille.fr <sup>3</sup> Freie Universit¨at Berlin, Berlin, Germany <sup>4</sup> Inria, Saclay, France {koen.zandberg,emmanuel.baccelli}@inria.fr

**Abstract.** RIOT is a micro-kernel dedicated to IoT applications that adopts eBPF (extended Berkeley Packet Filters) to implement so-called femto-containers. As micro-controllers rarely feature hardware memory protection, the isolation of eBPF virtual machines (VM) is critical to ensure system integrity against potentially malicious programs. This paper shows how to directly derive, within the Coq proof assistant, the verified C implementation of an eBPF virtual machine from a Gallina specification. Leveraging the formal semantics of the CompCert C compiler, we obtain an end-to-end theorem stating that the C code of our VM inherits the safety and security properties of the Gallina specification. Our refinement methodology ensures that the isolation property of the specification holds in the verified C implementation. Preliminary experiments demonstrate satisfying performance.

**Keywords:** Mechanized proof · Virtual machines · Fault isolation

#### **1 Introduction**

Hardware-enforced memory isolation (*e.g.,* Trustzone, Sanctum [6], Sancus [30]) is often not available on micro-controller units (MCU) which usually trade coarse-grain isolation for price and performance. To mitigate development variability and cost, common practices for MCU operating system design (RIOT [3], FreeRTOS, TinyOS, Fushia, and others [14]) advise to run all the device's code stack in a shared memory space, which can only be reasonably safe if that code can be trusted. While standard in safety-critical system design, such a trust requirement is oftentimes unsuitable for networked MCUs, where the extensibility of the OS kernel at runtime is an essential functionality. When system reconfiguration does not affect the entire network (via, *e.g.,* leader election), extensibility can easily be provided offline, by employing library OSs or unikernels [24], to reconfigure network endpoints independently (*e.g.,* cloud apps). Otherwise, the best solution is to load and execute system extensions (configurations, protocols, firewalls, etc.) as assembly-level Wasm [13] or Berkeley Packet Filters [25] scripts using an interpreter or a Just-In-Time (JIT) compiler on the target device.

*Femto-Containers.* RIOT adopts the extended Berkeley Packet Filters (eBPF) and tailors it to resource-constrained MCUs by implementing so-called femtocontainers: tiny virtual machine instances interpreting eBPF scripts. Compared to more expressive languages, like Wasm, experiments show that RIOT's eBPF implementation, rBPF, requires less memory [39]. The Linux kernel features an eBPF JIT compiler whose security depends on a sophisticated online verifier [29]. As an MCU architecture cannot host such a large verifier, executing JIT code would imply delegation of trust to a third-party, offline, verifier. The alternative is to rely on a defensive VM. Though a VM may be slower than a JIT, it can run untrusted, erroneous, adversary code in an open, and possibly hostile environment, and still isolate faults to protect its host's integrity.

*Approach and Goals.* This paper investigates an approach that trades high performance on low-power devices for defensive programming and low memory footprint. Our primary goal is to prevent faults that could compromise host devices and, by extension, force networked devices to reboot and resynchronize (*i.e.,* fault tolerance protocols). To maximize trust in the implementation of rBPF, our refinement methodology allows the verified extraction of C code directly from its mechanically proved definition in Gallina, the functional language embedded in the Coq proof assistant [4].

*Method.* To mechanically prove the correctness of an interpreter, a conventional approach consists in defining the reference semantics in a proof assistant and in showing that an executable optimized interpreter produces the same output. In this paper, our goal is to verify the interpreter of the virtual rBPF instruction set, implemented with the system programming language C. To this aim, we introduce a direct, end-to-end, validation workflow. The semantics of the source instruction set is directly defined by monadic functional terms in our proof assistant. We prove that this semantics enforces safety and security requirements regarding memory isolation and control-flow integrity. Then, C code is automatically derived from these monadic functional terms to implement the expected virtual machine. We prove that the extracted C code has the same stateful behavior as the monadic specification. Our method uses a monadic subset of Gallina of sufficient expressiveness to specify rBPF's semantics, supports the verified extraction of equivalent Clight [20] code, while provably implementing all required defensive runtime checks.

*Plan.* The rest of the paper is organized as follows. Section 2 states our contributions. Section 3 provides background on BPF and its variants, CompCert and the ∂x code extraction tool. Section 4 presents our workflow to formally refine monadic Gallina programs into C programs. Section 5 defines the proof model of our virtual machine: its semantics, consistency and isolation theorems. Section 6 refines the proof model of our femto-container into a synthesis model ready for code generation with CompCert. Section 7 proves the refinement between the synthesis and implementation models. Section 8 introduces our verified verifier which establishes the invariants needed by the VM. Section 9 case studies the performance of our generated VM implementation with respect to off-the-shelf RIOT femto-containers. Section 10 presents related works and Sect. 11 concludes.

#### **2 Contributions**

Implementing a fault-isolating virtual machine for MCUs faces two major challenges. One is to embed the VM inside the MCU's micro-kernel and, hence, to minimize its code size and execution environment. A second challenge is to minimize the verification gap between its proof model and the running code. We address these challenges and present the first end-to-end verification and synthesis of a full-scale, real-world, virtual machine for the BPF instruction set family: CertrBPF, an interpreter tailored to the hardware and resources constraints of MCU architectures running the RIOT operating system. CertrBPF employs a workflow of proof-oriented programming using the functional language Gallina embedded in the proof assistant Coq. The verified refinement and extraction of an executable C program is performed directly from its proof model. We report the successful integration of CertrBPF into the open source IoT operating system RIOT and the evaluation of its performance against micro-benchmarks.

*A Certified rBPF Interpreter.* CertrBPF is a verified model and implementation of rBPF in Coq. We formalize the syntax and semantics of all rBPF instructions, implement a formal model of its interpreter (femto-container), complete the proof of critical properties of our model, and extract and verify CompCert C code from this formalization. This method allows us to obtain a fully verified virtual machine. Not only is the Gallina specification of the VM proved kernel- and memory-isolated using the proof assistant, but the direct interpretation of its intended semantics as CompCert C code is, itself, verified correct. This yields a fully verified binary program of maximum security and minimal memory footprint and reduced the Trusted Computing Base (TCB): CertrBPF, a memory-efficient kernel-level virtual machine that isolates runtime software faults using defensive code and does not necessitate offline verification.

*End-to-End Proof Workflow.* An obvious choice is to use the existing Coq extraction mechanism to compile the Gallina model into OCaml. The downside of this approach is that Coq extraction has to be trusted. Moreover the OCaml runtime needs to be trimmed down to fit space requirements of our target architecture and also becomes part of the TCB. Our ambition is instead to minimize the verification gap and provide an end-to-end security proof linking our Gallina model to, bare-metal, extracted C code. Our intended TCB is hence restricted to the Coq type-checker, the C semantics of the CompCert compiler and a pretty-printer for the generated C Abstract Syntax Tree (AST).

To reach this goal, our starting point is a model of the rBPF semantics written in Gallina. We use this proof model to certify that all the memory accesses are valid and isolated to dedicated memory areas, thus ensuring isolation. From this proof model, we then derive a synthesis model of which we extract an executable version in Clight, that we finally prove to perform the same state transitions.

*Systems Integration and Micro-benchmarks.* We integrate CertrBPF as a drop-in replacement of the current, non-verified, rBPF interpreter in the RIOT operating system. We then comparatively evaluate the performance of CertrBPF integrated in RIOT, running on various 32-bit micro-controller architectures. Our benchmarks demonstrate that, in practice, CertrBPF not just gains security, but reduces memory footprint as well as execution time.

## **3 Background**

This section describes essential features of rBPF, of the CompCert compiler, and of the ∂x code generation tool, that are required by our refinement methodology.

*BPF, eBPF and rBPF.* Originally, the purpose of Berkeley Packet Filters [25] (BPF) was network packet filtering. The Linux community extended it to provide ways to run custom in-kernel VM code, hooked into various subsystems, for varieties of purposes beyond packet filtering [10]. eBPF was then ported to microcontrollers, yielding RIOT's specification: rBPF [38]. Just as eBPF, rBPF is designed as a 64-bit register-based VM, using fixed-size 64-bit instructions and a reduced instruction set architecture. rBPF uses a fixed-size stack (512 bytes) and defines no heap interaction, which limits the VM memory overhead in RAM. The rBPF specification, however, does not define special registers or interrupts for flow control, nor support virtual memory: the host device's memory is accessed directly and only guarded using permissions.

*The CompCert Verified Compiler.* CompCert [18] is a C compiler that is both programmed and proved correct using the Coq proof assistant. The compiler is structured into passes using several intermediate languages. Each intermediate language is equipped with a formal semantics and each pass is proved to preserve the observational behavior of programs.

*The Clight Intermediate Language.* Clight [20] is a pivotal language which condenses the essential features of C using a minimal syntax. The Verified Software Toolchain (VST) [2] verifies C programs at the Clight level that are obtained by the clightgen tool. Though we do not reuse the proof infrastructure of VST, we are reusing clightgen in order to get a Clight syntax from a C program.

*CompCert Values and Memory Model* [19,20]. The memory model and the representation of values are shared across all the intermediate languages of CompCert. The set of values val is defined as follows:

$$val \ni v ::= Vint(i) \mid Vlong(i) \mid Vptr(b, o) \mid Vundef \mid \dots$$

A value v ∈ *val* can be a 32-bit integer *Vint*(i); a 64-bit integer *Vlong*(i), a pointer *Vptr* (b, o) consisting of a block identifier b and an offset o, or the undefined value *Vundef* . The undefined value *Vundef* represents an unspecified value and is not, strictly speaking, an undefined behavior. Yet, as most of the C operators are strict in *Vundef* , and because branching over *Vundef* or de-referencing *Vundef* are undefined behaviors, our proofs will ensure the absence of *Vundef* . CompCert values also include floating-point numbers; they play no role in the current development. CompCert's memory consists of a collection of separate arrays. Each array has a fixed size determined at allocation time and is identified by an uninterpreted block b ∈ *block*. The memory provides an API for loading values from memory and storing values in memory. Operations are parameterised by a memory chunk k which specifies how many bytes should be written or read and how to interpret bytes as a value v ∈ *val*.

For instance, the memory chunk *Mint32* specifies a 32-bit value and *Mint64* a 64-bit value. The function *load* kmbo takes a memory chunk k, a memory m, a block b and an offset o. Upon success, it returns a value v obtained from the memory by reading bytes from the block b starting at index o. Similarly, the function *store* kmbov takes a memory chunk k, a memory m, a block b, an offset o and a value v. Upon success, it returns an updated memory m which is identical to m except that the block b contains the value v encoded into bytes according to the chunk k starting at offset o. The isolation properties offered by CompCert memory regions are worth mentioning: *load* and *store* operations fail (return None) for invalid offsets o and invalid permissions.

*The* ∂x *tool.* ∂x emerged from the toolchain used to design and verify the Pip proto-kernel [15]. Its aim was to allow writing most of Pip's source code in Gallina in a style as close to C as possible. ∂x extracts C code from a Gallina source program in the form of a CompCert C AST. The goal of ∂x is to provide C programmers with readily reviewable code and thus avoid misunderstanding between those working on C/assembly modules (that access hardware) and those working on Coq modules (the code and proofs). To achieve this, ∂x handles a C-like subset of Gallina. The functions that are to be converted to C rely on a monad to represent the side effects of the computation, such as modifications to the CPU state. Yet ∂x does not mandate a particular monad for code extraction.

∂x *'s Workflow.* ∂x proceeds in two steps. First, given a list of Gallina functions, or whole modules, it generates an intermediate representation (IR) for the subset of Gallina it can handle. The second step is to translate this IR into a CompCert C AST. Since Coq has no built-in reflection mechanism, the first step is written in Elpi [8], using the Coq-Elpi plugin [37]. That step can also process external functions (appearing as extern in the extracted C code) to support separate compilation with CompCert. In order to obtain an actual C file, ∂x also provides a small OCaml function that binds the extracted C AST to CompCert's C pretty-printer. Even though the ∂x language is a small subset of Gallina, it inherits much expressivity from the use of Coq types to manipulate values. For example, we can use bounded integers (*i.e.,* the dependent pair of an integer with the proof that it is within some given range), that can be faithfully and efficiently represented as a single int in C. To this end, ∂x expects a configuration mapping Coq types to C.

∂x *Memory Management.* A major design choice in the C-like subset of Gallina used by ∂x is memory management: its generated code executes without garbage collection. This affects the Coq types that can actually be used in ∂x: recursive inductive types, such as lists, cannot automatically be converted. However, this Gallina subset is particularly relevant to programs in which one wants to precisely control memory management and decide how to represent data structures in memory. This is typically the case of an operating system or, in our case, the rBPF virtual machine.

## **4 A Workflow for End-to-End Verification in Coq**

This section gives an overview of our methodology to derive a verified C implementation from a Gallina specification. In the following sections, the methodology will be instantiated to derive the C implementation of a fault-isolating rBPF virtual machine and its verifier. Our approach provides an end-to-end correctness proof, within the Coq proof assistant, that reduces the hurdle of reasoning directly over the C code.

As shown in Fig. 1, the original rBPF C implementation is first formalized by a proof model in Gallina, and the verification of expected properties (*e.g.,* safety) is performed within the Coq proof assistant. This specification is then refined into an optimized (and equivalent) synthesis model ready for C-code extraction.

**Fig. 1.** End-to-end verification and synthesis workflow

The refinement and optimization principle employed by our method consists of deriving a C-ready implementation, in Gallina, that is as close as possible to the expected target C code. This principle allows to i) prove optimizations correct, ii) improve the performance of the extracted code and, iii) facilitate review and validation of extracted code with the system designers. From the C-ready Gallina implementation, we leverage ∂x to automatically generate C code and verify it: i) the generated C code is first parsed as a CompCert Clight model by the clightgen tool of VST and ii) it is proved to refine the source Gallina model in Coq using translation validation. Because ∂x generates C code in a syntax-directed manner, a minimal *Clightlogic* is designed to facilitate the refinement proof. The rest of the section explains these different steps in details.

*Proof-Oriented Specification.* Our specification takes the form of an executable abstract machine in monadic form. It uses the standard option-state monad M.

M a state := state → **option**(a × state) returnM : a → M a state := λa.λst.**Some**(a, st) bindM : M a state → (a → M b state) → M b state := λA.f.λs.**match** A s **with** | **None** ⇒ **None** | **Some**(x, s ) ⇒ (f x) s

In the remainder, we write ∅ for **None** and x for **Some** x.

The monad threads the state along computations to model its in-place update. The safety property of the machine is implemented as an inline monitor: any violation leads to an unrecoverable error, *i.e.,* the unique error represented by ∅. One step of the machine has the following signature:

*step* : M r state

where r is the type of the result. The *step* function implements a defensive semantics, checking the absence of error, dynamically. For our rBPF interpreter (see Sect. 5), the absence of error ensures that the rBPF code only performs valid instructions. In particular, all memory accesses are restricted to a sandbox specified as a list of memory regions. Function *step* is part of the TCB and, therefore, a mis-specification could result, after refinement, in an invalid computation. The purpose of the error state is to specify state transitions that would escape the scope of the safety property and, therefore, shall never be reachable from a well-formed state st ∈ *wf* ⊆ P(state). We require well-formedness to be an inductive property of the *step* function.

**Theorem 1 (Well-formedness).** *The step function preserves well-formedness.*

$$\forall st, st', r.\ st \in wf \land step\ st = \lfloor (r, st') \rfloor \Rightarrow st' \in wf$$

We also require that well-formedness is a sufficient condition to prevent the absence of error and, therefore, the safety of computations.

**Theorem 2 (Safety).** *The step function is safe,* i.e., *a well-formed state never leads to an error.*

$$\forall st. \ st \in wf \Rightarrow step \ st \neq \emptyset$$

*C-Ready Implementation.* Our methodology consists in refining the step function into an interpreter step∂x complying with the requirements of ∂x. As ∂x performs syntax-directed code generation, the efficiency of the extracted code crucially depends on step∂x. In order to preserve the absence of errors, we need a simulation relation between the step and step∂x functions. A direct consequence of the simulation thoerem is that step∂x never raises an error.

**Theorem 3 (Simulation).** *Given simulation relations* Rs ⊆ state × state *and* Rr ⊆ r × r *, the function* step∂x *simulates the function* step*.*

$$\forall s\_1, s\_1', s\_2, r. (s\_1, s\_2) \in Rs \land step \ s\_1 = \lfloor r, s\_1' \rfloor \Rightarrow \exists s\_2', r'. \bigwedge \begin{cases} step\_{\partial x} \ s\_2 = \lfloor r', s\_2' \rfloor \\ (s\_1', s\_2') \in Rs \\ (r, r') \in Rs \end{cases}$$

*Translation Validation of C Code.* The next stage consists in refining the step∂x function into a Clight program by relying on ∂x to get a C program and on the clightgen tool to get a Clight step<sup>C</sup> program (see Sect. 6). As this pass is not trusted, we require the following translation validation theorem.

**Theorem 4 (Translation Validation).** *Given a simulation relation* Rs ⊆ state × val × mem *and a relation* Rr ⊆ res × val*, the Clight code* step<sup>C</sup> *refines the function* step∂x*:*

$$\forall r, s, s', v, k, m. (s, v, m) \in Rs \Rightarrow step\_{\partial x} \; s = \lfloor (r, s') \rfloor \Rightarrow \exists m', r'. Call state (step\_C, [v], k, m) \rightarrow^{\*t} ReturnState(r', call.cont(k), m') \land (s', v, m') \in Rs \land (r, r') \in Rr$$

Theorem 4 states that, if step∂x s runs without error and returns a result (r, s ), then, the Clight function step<sup>C</sup> successfully runs with argument v and, after a finite number of execution steps, returns a result r and a memory m that preserve the refinement relations. In our encoding, the unique argument v is a pointer to the memory allocated region refining the interpreter state and k represents the continuation of the computation. A corollary of Theorem 4 is that the Clight code step<sup>C</sup> is free of undefined behaviors. In particular, all memory accesses are valid. As the memory model does not allow to forge pointers, this yields a strong isolation property. In the remainder of this paper, for our rBPF virtual machine, we prove all the aforementioned properties within the Coq proof assistant.

#### **5 A Proof-Oriented Virtual Machine Model**

For our proof model, we define an explicit syntax for rBPF. We also define the state of the interpreter and semantic functions, in particular those implementing dynamic security checks. The rBPF instruction set, Fig. 2, features binary arithmetic and logic operations, negation, (un)conditional jumps relative to an offset, operations to load/store values from/to registers/memory, function calls, and termination. There are eleven 64-bit registers {R0,...,R10}; an immediate is 32-bit wide and an offset is 16-bit wide.


**Fig. 2.** Core syntax of rBPF instruction set

*Machine State.* A semantic state *st* is a tuple I, L, R, F, M, *MRs* consisting of a sequence of instructions I, the current location L, registers R, an interpreter flag F, a memory M and a specification of available memory regions *MRs*. The flag F characterizes the state of the rBPF interpreter. It may be i) a normal state, written Fn; ii) a final state, written Ft; iii) or an error state, written Fe. An error state f ∈ F<sup>e</sup> means that the defensive checks of the interpreter have detected that an invalid behavior is about to occur.

A memory region mr = start, size, p, ptr ∈ *MRs* associates a permission p ∈ {*Readable*,*Writable*} to the address range [start, start + size). We make the link between concrete physical addresses and the CompCert memory model using the pointer *ptr (= Vptr b 0)* where the block *b* is the abstract representation of the address start. We write I(L) for the instruction located at the program counter L. R[r] retrieves the value of the register r in the register map R. Functions alu and cmp reuse the CompCert's operators over the val type. The alu function returns ∅ if an error occurs, *e.g.,* division by zero. Functions load and store are those of CompCert's memory model (see Sect. 3).

$$\begin{array}{ll} \mathsf{alu} : op \to val \to val \to option \, val & \textbf{cmp} : comp \to val \to val \to book \\\mathsf{load} : chk \to mem \to block \to Z \to option \, val \\\ \mathsf{staree} : chk \to mem \to block \to Z \to val \to option \, mem \end{array}$$

*Dynamic Checks.* Function check alu dynamically checks the validity of an arithmetic to avoid *div-by-zero* and *undefined-shift* errors. For division instructions, check alu mandates the second argument to be non-zero. For arithmetic and logical shift instructions, the second argument has to be below n ∈ {32, 64} depending on whether the ALU instruction operates on 32 or 64 bit operands. For simplicity, the paper only considers 64-bit ALU instructions but CertrBPF also has the 32-bit variants.

check alu(op, v) def = ⎧ ⎨ ⎩ v = 0 *if* op ∈ {div, mod} 0 ≤ v<n *if* op ∈ {lsh, rsh, arsh} true otherwise

Function check mem returns a valid pointer (*Vptr* b ofs) if there exists a unique memory region mr in *MRs* such that i) the permission mr.perm is at least *Readable* for Load and *Writable* for Store, *i.e.,* mr.perm <sup>≥</sup> <sup>p</sup>; ii) the offset ofs is aligned, *i.e.,* ofs%Z(chk) = 0; iii) in bounds, *i.e.,* ofs ≤ max unsigned−Z(chk), iv) and the interval [ofs, hi ofs) is in the range of mr. Otherwise, check mem returns the null pointer *Vnullptr* . The function *Z(chk)* maps memory chunks *byte*, *halfword*, *word* and *double* to 1, 2, 4, and 8, respectively.

```
check mem(p, chk, addr, MRs) def
                              = if ∃! mr ∈ MRs, b.
  let ofs = addr − mr.start and hi ofs = ofs + Z(chk) in
    (mr.ptr == Vptr b 0) ∧ (mr.perm ≥ p) ∧ (ofs%Z(chk) == 0) ∧
      (ofs ≤ max signed − Z(chk)) ∧ (0 ≤ ofs ∧ hi ofs < mr.size))
    then Vptr b ofs else Vnullptr
```
*Semantics.* Functions interp and sem formalize the implementation of our proof model M<sup>p</sup> in the Coq proof assistant by defining a monadic interpreter of rBPF. The top-level recursion interp processes a (monotonically decreasing) fuel argument and a state s. The function sem processes individual instructions I(Lpc). *MRs* and I are read-only. During normal execution, the flag remains Fn. If the flag turns to F<sup>t</sup> or F<sup>e</sup> while processing an instruction, execution stops. For instance, if fuel reaches zero, the flag turns to Fe. We write s.F for the value of field F in record s and s{F = v} updates it to v.

```
interp = λfuel s. if fuel == 0 then 	((), s{F=Fe})
                                                    else
    match sem s with
    | 	((), t)
               => if t.F= Fn then 	((), t)

                   else interp (fuel-1) t{L = t.L+1}
    | ∅ => ∅
sem = λs. match s.I(s.L) with

                if f_ptr == Vnullptr then 	((), s{F = Fe})

                else 	((), s{R0 = exec_function f_ptr})


                      then 	((), s{L = s.L+ofs})
                                                 else 	((), s)


                    match alu(op, s.R[dst], s.R[src]) with
                    | 	v
                         => 	((), s{R[dst] = v})
                                                  | ∅ => ∅
                    else 	((), s{F = Fe})

  match check_mem(Readable, chk, s.R[reg]+ofs, s.MRs) with
  | Vptr b ofs => match load(chk, s.M, b, ofs) with
                  | 	v
                       => 	((), s{R[dst] = v})
                                                | ∅ => ∅
  | _ => 	((), s{F = Fe})

  match check_mem(Writable, chk, s.R[dst]+ofs, s.MRs) with
  | Vptr b ofs => match store(chk, s.M, b, ofs, S.R[src]) with
                  | 	N
                       => 	((), s{M = N})
                                           | ∅ => ∅
  | _ => 	((), s{F = Fe})

```
Result ∅ marks transitions to crash states that are proved unreachable given our carefully crafted definitions of the check alu and check mem functions. Note that the interpreter interp does not check the range of branching offsets (*i.e.,* 0 <=s.L< length(s.I)) and register-out-of-bounds. This properties are statically verified, once and for all, by the verifier of Sect. 8.

Exit terminates the program with flag Ft. The *Call* instruction selects (using *bpf get call*) the trusted system API service designated by an immediate number imm. It then calls the chosen service if available (*i.e.,* not a null pointer). Unconditional jump Ja increments the pc by ofs and a conditional Jump does so when cmp(c, src, dest) holds. For an arithmetic operation Alu op dst src, check alu first checks the validity of op with source src, evaluates op against destination dst using alu, stores the result v in register dst. For simplicity, we omit the case of immediate srcs. If the result is ∅, so becomes the monadic state (undefined behavior). Our definition of check alu, and well-formedness conditions (see Sect. 5.1) ensures that this will never happen and that, in case of error, the execution terminates with flag Fe. Similarly, the semantics of memory instructions (*Load-Store*) validates memory accesses using the check mem function. Its definition ensures the absence of undefined behaviors.

#### **5.1 Proof of Software-Fault Isolation**

Our proof model M<sup>p</sup> formalizes the semantics of rBPF. It is implemented in Coq using Gallina. Assessing its correctness consists of proving two essential properties: i) the well-formedness of the virtual machine's state, that is, its registers, memory and verifier invariants, and ii) software-fault isolation, that is, the isolation of all transitions to a crash state ∅ using runtime safety checks (*e.g.,* check mem), ergo the impossibility of a transition to an undefined behavior.

The register invariant states that all registers contain 64-bit integer values. This rules out 32-bit integers, *Vundef* but also pointers and floating-point numbers, for which the **alu** function may be undefined.

## **Definition 1 (register inv).** ∀r ∈ registers.∃l.R[r] = *Vlong* l

As expected, the memory consistency invariant is a bit more elaborate. It states that each CompCert memory region mr register 8-bit integer blocks b of memory m, designated by a pointer mr.ptr to the 32-bit physical mr.start address of b, the 32-bit mr.size of b and at least *Readable* permissions mr.perm across [0, size). Finally, every two regions point to disjoint physical address spaces in m (as per CompCert's memory regions for mr .ptr = mr.ptr).

**Definition 2 (memory inv).** ∀mr ∈ *MRs*, m. ∃b, start, size*. s.t.*

*mr*.*ptr* = *Vptr b 0* ∧ *Mem*.*valid block m b* ∧ *is byte block b m* ∧ *mr*.*start* = *Vint start* ∧ *mr*.*size* = *Vint size* ∧ *mr*.*perm* ≥ *Readable* ∧ *Mem*.*range perm m b 0* (*Int*.*unsigned size*) *Cur mr*.*perm* ∧ (∀*mr*- ∈ *MRs*, *mr*- = *mr* → *mr*- .*ptr* = *mr*.*ptr*)

Linux eBPF has a verifier to statically analyze eBPF programs and only accept those which are free of undefined behaviors. Our CertrBPF's verifier, introduced in Sect. 8, ensures the weaker invariant given by Definition 3. The invariant stipulates the minimal pre-condition so that the interpreter can safely run a sequence of instructions I. More precisely, the invariant states that each instruction I[i] references registers within the range [0, 10] and that the target of every jump instruction is within the program range *i.e.,* 0 ≤ i + ofs + 1 ≤ length(I) − 1.

**Definition 3 (verifier inv).** ∀i, I, ofs. 0 ≤ i ≤ length(I) − 1 → *0* ≤ *get dst*(*I* [*i*]) ≤ *10* ∧ *0* ≤ *get src*(*I* [*i*]) ≤ *10* ∧ ((*I* [*i*] = *Ja ofs* ∨ *I* [*i*] = *Jump ofs*) → *0* ≤ *i* + *ofs* + *1* ≤ *length*(*I* ) − *1*)

These three invariants implement well-formedness as proposed in Sect. 4. Therefore, the following Coq Theorem *sem preserve inv* proves Theorem 1 and states that well-formedness is preserved by the interp function. Similarly, Theorem *inv ensure no undef* proves Theorem 2. This proves that the dynamic checks of the model M<sup>p</sup> are sufficient to ensure the absence of error. In particular, all memory accesses are valid and performed within the dedicated memory regions. As a result, our model ensures software fault isolation. The corollary of Theorems *sem preserve inv* and *inv ensure no undef* is that our virtual machine, obtained by refinement of the proof model, will always isolate code from other memory regions of the operating system and never crash it.

```
Theorem sem_preserve_inv: ∀ (st st': state) (fuel: nat)
    (Hinv: register_inv st ∧ memory_inv st ∧ verifier_inv st)
    (Hsem: interp fuel st = 	(tt, st')
                                       ),
        register_inv st' ∧ memory_inv st' ∧ verifier_inv st'.
Theorem interp_no_undef: ∀ (st: state) (fuel: nat)
    (Hinv: register_inv st ∧ memory_inv st ∧ verifier_inv st),
        interp fuel st = ∅ .
```
#### **6 A Synthesis-Oriented eBPF Interpreter**

The coding style of the proof model M<sup>p</sup> is quite different from the original RIOT implementation in C and lacks optimizations used in the latter to improve runtime performance. The synthesis model M<sup>s</sup> firstly refines M<sup>p</sup> into an optimized, safe and behaviorally equivalent monadic model which is then automatically transformed into an effectful implementation model M<sup>c</sup> using ∂x.

*Synthesis Model* Ms*.* M<sup>s</sup> refines our proof model by following the principle "make M<sup>s</sup> as close as possible to the expected target C code". M<sup>s</sup> also refines Coq types because each Coq inductive type may correspond to several C types (*e.g.,* V int/V long to *signed* or *unsigned*, 32-bit or 64-bit). The case of V ptr is particularly delicate, as the target type contextually relies on bit-size and signedness. To sort this out, we rename Coq types to match the correct C type. For example, val<sup>64</sup> t, valu<sup>32</sup> <sup>t</sup>, vals<sup>32</sup> <sup>t</sup> are V al types mapped to unsigned long long, unsigned int and int, respectively.

*Equivalence.* Both M<sup>p</sup> and M<sup>s</sup> use the same monadic state st as in Sect. 5. Hence, the simulation relation R ⊆ st × st, required by Theorem 3, is equality. As a result, we prove the stronger result that both interp : nat → M unit, the M<sup>p</sup> interpreter, and interp dx : nat → M unit, the M<sup>s</sup> interpreter, denote the exact same function.

```
Theorem equivalence_relation: ∀ (st: state) (fuel: nat),
    interp fuel st = interp_dx fuel st.
```
∂x *configuration and Implementation model* Mc*.* To extract the implementation model, we supply ∂x with our monad M and a mapping relation from Gallina to C, Table 1.


**Table 1.** Mapping relation from Gallina to C

Inductive types map to C types, *e.g.,* reg to unsigned int (note that a manyto-one relation from Gallina to C is legal). Gallina constructs and constant functions map to C operators and constants, *e.g.,* '*Val*.*addl*' to '+', '*Int*.*repr* (−2)' and 'true' to '−2' and '1', etc. Gallina functions map to C functions. For any function operating the monadic state, the target C function has an additional argument st of type struct state∗ which corresponds to the implicit state of the monad. Gallina's *match-pattern* translates to C's *switch-case*, etc.

*Code Extraction with* ∂x*.* The extracted C implementation preserves the structure of the original Gallina code, and the extracted C functions directly operate on actual memory locations as CompCert memory operations map to C expressions with a dereference. Consider the example of the step mem st reg function.

```
Definition step_mem_st_reg (src: val64_t) (addr: valu32_t) (op: int8_t):
    M unit :=
  do opcode_st <- get_opcode_mem_st_reg op;
  match opcode_st with
  | op_BPF_STXW =>
    do addr_ptr <- check_mem Writable Mint32 addr;
      if eq_ptr_null addr_ptr then
        upd_flag BPF_ILLEGAL_MEM
      else (** i.e. Mem.storev Mint32 addr_ptr src *)
        do _ <- store_mem_reg Mint32 addr_ptr src; returnM tt
  ...
```
CompCert's Byte *int8 t* is mapped to *unsigned char*. Constructs *op BPF STXW*, *BPF ILLEGAL MEM* and *Writable* are respectively mapped to '99', '-2' and '2U'. The constant function eq ptr null is translated into an operation to check whether a pointer is null. The 'match opcode st with' is extracted to 'switch (opcode st) case'. Functions step mem st reg, check mem and store mem reg in C have an additional monadic argument st.

```
void step_mem_st_reg( struct bpf_state* st , unsigned long long
    src , unsigned int addr , unsigned char op){
  unsigned char opcode_st;
  unsigned char *addr_ptr;
  opcode_st = get_opcode_mem_st_reg(op);
  switch (opcode_st) {
    case 99:
      addr_ptr = check_mem(st , 2U, 4U, addr);
      if (addr_ptr == 0) {
        upd_flag(st , -2); return ;
      } else { // i.e. *(unsigned int *) addr_ptr = src
        store_mem_reg (st , 4U, addr_ptr , src); return ;
      }
    ...
```
## **7 Simulation Proof of the C rBPF Virtual Machine**

In this section, we explain how to establish Theorem 4 for the Clight code of our virtual machine, derived from ∂x, and compiled into a Clight AST in Coq using the clightgen tool.

*Simulation Relation.* A crucial ingredient of Theorem 4 is the simulation relation between the Gallina state monad and the Clight state which is essentially made of a CompCert memory. The Gallina state comprises a CompCert memory that models the various memory regions available to the rBPF program. This memory may also contain other blocks that are not modified by the virtual machine but represent other kernel data-structures. The simulation relation stipulates that such blocks also exist in the Clight memory and have the same content. The Clight memory contains additional blocks (*i.e.,* state block, ins block and

**Fig. 3.** Simulation relation R between st*rbpf* , left, and rBPFClight, right.

mrs block) to model the other fields of the Gallina state. The layout and content of those blocks are depicted in Fig. 3.

Solid arrows in Fig. 3 are simulation relations between state block and strbpf . Solid lines are the equalities between the rBPF memory m and blocks in rBPF-Clight memory. Dashed lines indicate relations of pointers to blocks in CompCert memory. The encoding exploits the fact that each field of the Gallina state has a known length. Thus, every field can be encoded as a continuous sub-block. As a result, the program counter is obtained from the first 4 bytes: loading a memory chunk of type *Mint32* at offset 0 retrieves the pc field of the Gallina state. The next 4 bytes encode the enumerated type flag. Here, each constructor of type flag is assigned an integer. The next 11 × 64 bits are used to encode the register bank of the Gallina state.

Rs(state, state block, m) def = ⎧ ⎪⎪⎪⎨ ⎪⎪⎪⎩ st*rbpf* .pc = load M int32 m*clight* state block 0 st*rbpf* .f lag = load M int32 m*clight* state block 4 st*rbpf* .R0 = load M int64 m*clight* state block 8 ...

The next elements of the Clight block represent the lists of instructions and of memory regions. In a functional language, lists are potentially of unbounded length and have a polymorphic type. Here, our lists always have fixed lengths and elements of fixed size. As a result, a list is directly encoded by a field specifying its length followed by a pointer to its memory block. The elements of the list are stored continuously in the pointed block.

*Systematic Proof of Simulation.* Since the ∂x tool is syntax-directed, there is a systematic correspondence between the source Gallina and the target C code. We exploit this property to design a minimal Clight logic geared toward our simulation proof. Our *Clightlogic* generalizes the translation validation theorem (Theorem 4) to accommodate Gallina functions and C functions with multiple arguments. In that case, we have a precondition which states that the Gallina and C arguments are linked pairwise by a refinement relation. Most of the arguments are numeric values and, in this case, the refinement relation states that the Gallina and C values are the same. The *Clightlogic* also provides a syntax-directed proof principle for each pair of Gallina/C syntactic construct. For instance, the *bindM* operator translates to a sequence in the C code. Also, the result of a Gallina function call is bound to a local variable in C. Moreover, the local variable v below stands for the monadic state in C and points to the state memory block.

$$\partial x (b \text{ind} M \, f \, (\lambda x. g)) = (vx = f\_C(v); g\_C(v, vx))$$

To exploit this pattern, our invariants take the form of an association list mapping each local variable to a set of C values that is obtained by partially evaluating a refinement relation with the Gallina value computed by the function (Fig. 3). To evaluate f, one needs to have a refinement relation Rs between the Gallina state st and the C value of v in memory m. Now, suppose that fst = r, st . Since f<sup>C</sup> is a correct refinement of f, relations Rs(st , v,m ) and Rr(r, x) hold for the value x of the local variable vx in the current environment. We conclude by mapping vx → Rr r and use this invariant to refine g by g<sup>C</sup> .

The translation validation theorem proves a forward simulation relation from Coq to Clight. A backward simulation relation can be constructed as Gallina programs are functions and Clight is *determinate*.

#### **8 CertrBPF Verifier**

Linux eBPF's compiler and runtime system do not enforce type or memory safety. Instead, safety is verified prior to execution using a static analyzer that checks programs validity. As both the size and complexity cannot fit the requirements of an MCU architecture, CertrBPF instead provides a simple (linear time) but formally verified verifier, CertrBPF-verifier, which ensures the invariant verif ier inv (Definition 3). Accordingly, it scans an input rBPF program (*i.e.,* a list of 64-bit bytecode instructions) and rejects it when: i) a source or destination register is greater than 10. ii) the offset of a jump instruction is out of the instruction sequence bounds. iii) or the last instruction is not the Exit instruction (opcode 0x95).

Static verification of these properties allows the interpreter to skip unnecessary dynamic checks. Our verifier adopts the same end-to-end verification method as the interpreter, Sect. 4. The virtual machine state in CertrBPFverifier is a strict subset of the interpreter's state: st<sup>v</sup> = I,M consists of a sequence of instructions I and a memory M.

```
Theorem verifier_well_formedness_and_safety :
  ∀ (st: verifier_state) (b: bool),
    verifier st = 	(b , st)
                            .
Theorem verifier_imply_inv :
  ∀ (st: verifier_state) (st': state)
    (Hinclude: st ⊂ st') (Hpre : verifier st = 	(true, st)
                                                            ),
      verifier_inv st'.
```
Theorem verif ier well formedness and safety proves both Theorem 1 and Theorem 2. The verifier has the following properties: i) no assumption (every state is well-formed); ii) never crashes (safety); iii) never modifies the VM state. In addition, the Coq theorem verif ier imply inv states that if the verif ier returns true, verif ier inv holds. Considering that the verifier's proof and synthesis models are exactly the same, the simulation relation R<sup>v</sup> ⊆ st<sup>v</sup> × st<sup>v</sup> required by Theorem 3 is equality. CertrBPF-verifier reuses the Clightlogic to prove the simulation proof of its C implementation.

#### **9 Evaluation: Case Study of RIOT's Femto-Containers**

We integrate CertrBPF as a drop-in replacement for the existing non-verified module optimized for size (vanilla-rBPF) in the IoT operating system RIOT to provide the expected femto-container functionalities [39].

*Implementation.* The proof model of the interpreter (Sect. 5) consists of 2.4k lines of Coq code and the corresponding isolation proof (Sect. 5.1) is more than 4.8k lines long. The synthesis model, Sect. 6, is approx. 3.2k lines long and the equivalence theorem is completed by 0.6k proof code. The final step (Sect. 7) includes 10.8k translation validation proofs between the Gallina specification and the extracted Clight model. As for the CertrBPF verifier (Sect. 8), the proof and synthesis models sport 1.4k lines of Coq code. The corresponding proofs are more than 0.5k long and the last simulation proof is about 8.3k long. In addition, the Clightlogic implementation has 4.4k lines of Coq code.

*Experimental Evaluation Setup.* Our experimental objects are the original nonverified rBPF interpreter (*i.e.,* vanilla-rBPF) and the automatically extracted and verified CertrBPF interpreter (without RIOT's API). We carry out our measurements on a selected set of popular, commercial, off-the-shelf low-power IoT hardware, representative of modern 32-bit micro-controller architectures and boards: i) Nordic nRF52840 (Arm Cortex-M); ii) Espressif WROOM-32 (Espressif ESP32); iii) Sipeed Longan Nano GD32VF103CBT6 (RISC-V). All code is compiled with GCC using size optimization enabled and the -foptimizesibling-calls GCC option to remove all tail-recursive calls and thus bound the stack size. This is critical to our isolation theorem as it relies on the implicit CompCert assumption that the stack cannot overflow. To avoid a possible mismatch between the CompCert semantics and the GCC semantics, we also pass the following options: i) -fwrapv, -fwrapv-pointer mean that both signed and pointer arithmetic wrap around according to the two's-complement encoding; ii) -fno-strict-aliasing means that there is no aliasing assumption.

*Results.* We first evaluate the memory footprint of the CertrBPF interpreter, compared to vanilla-rBPF. We measure i) *Flash size*: all read-only data, including the actual code; ii) *Stack*: the approximate ram used for stack space; iii) *Context*: the static RAM. In terms of Flash, our measurements show that CertrBPF actually reduces the footprint by 47% on RISC-V and by 35% on ESP32, and a 10% decrease on Cortex-M. In terms of stack requirements, CertrBPF reduces the footprint by 33% on Cortex-M, by 22% on RISC-V, and by 4% on ESP32. The context memory, however, increases from 92B to 144B on all platforms.

**Fig. 4.** Time per instructions on the Cortex-M4 platform

Next, we micro-benchmark the performance of core operations: single instructions from the arithmetic logic unit (ALU), for memory access (MEM) and branch instructions, with a mix of register and immediate value for the operands, Fig. 4. These results are averages over 1000 single identical instruction calls with a single return statement to make the application exit.

Finally, we benchmark the performance of actual IoT data processing, hosted in a femto-container with RIOT running on our selected hardware. In this use case, a sliding window average is performed within the femto-container, on available sensor data points. Figure 5 shows the performance we measured depending on the size of the window. We use this as blueprint for computation load scaling.

**Fig. 5.** Sliding window average on Cortex-M, ESP32, and RISC-V.

*Key Take-Away.* We observe that CertrBPF generally decreases the memory footprint. One reason is that calls to the RIOT API are currently not supported by CertrBPF. We observe, Fig. 4, that the execution slow-down is acute for Branch instructions, on Cortex-M. However, on all other platforms (RISC-V, ESP32 and Cortex-M), our micro-benchmarks show that most instructions enjoy speed-up with CertrBPF compared to vanilla-rBPF. This behavior is also visible in our sensor data processing benchmark, Fig. 5, where CertrBPF performs better than vanilla-rBPF on three platforms. All in all, CertrBPF gains both security and reduces memory footprint as well as execution time.

#### **10 Related Works**

*Methodologies for Systems and Compilers Verification.* The verification of compilers [18], static analyzers [16], and operating systems [12,17] have been the subjects of vast development and verification efforts due to the sheer code size of the artifacts at stake. These full-scale case studies gave rise to new strategies and methodologies to address the challenge of verifying large software. One such approach is Cogent [35] which aims at developing verified applications on top of the SeL4 [17] micro-kernel. Cogent [35] consists of a functional language with linear types to specify source programs and produces C code with Isabelle/HOL proof information. It provides a framework to prove that the extracted C code refines a high-level Isabelle/HOL functional correctness specification in the Isabelle/HOL proof assistant. Our method differs from co-specification in Cogent in that it is direct: it directly translates Coq specifications into C code and performs the end-to-end verification in Coq. CertiKOS [12] uses a multi-layered, refinementbased, and modular definition of a micro-kernel from its low-level memory model to its user-level interface and services. It is adopted in SeKVM [22], a layered Linux KVM hypervisor architecture for multiprocessor hardware. The CompCert project [18] adopted this "divide-and-conquer" strategy to decompose the verification of a full-scale ANSI C compiler into that of its successive transformations from source program to machine code, compositionally verifying each of the translation steps bisimilar. Its related static analyser, Verasco [16], employs static analysis of CompCert C code using a verified core abstract interpreter with composable abstract domains. Our problem statement is methodologically simpler: to build a safe and small VM that interprets rBPF virtual instructions on networked micro-controllers. We choose the radical approach of proof-oriented programming (`a la Low [34], Vale [5]) to prove an rBPF interpreter embedded in Coq correct and to directly extract verified code from its definition.

*Background on BPF and Its Verified Implementations.* Mogul et al. [26] introduce a stack-based virtual machine to interpret packet filters into the BSD kernel that BPF extended to 32-bit instructions. BPF gained adoption in the Linux community and became eBPF (extended BPF), a virtual 64-bit RISC-like architectures. To our knowledge, verification of BPF runtime systems has mainly focused on JIT translation for operation on micro-kernels. Myreen [28] verifies a JIT compiler targeting x86 for a stack language using the HOL4 proof assistant. The generated code only preserves the semantics of the source code but does not ensure any isolation property. Porncharoenwase et al. [33] use CompCert to extract an OCaml translator from BPF to assembly code, verified using the proof assistant Coq, using the OCaml runtime, an assembler, and a linker as TCB. Van Geffen et al. [11] present an optimized JIT compiler for Linux BPF with automated static analysis onboard, assuming offline verification using the Linux BPF verifier as TCB. For field deployment on networks of micro-controllers (IoT), all the above approaches would require a trusted, offline BPF verifier and, additionally, a secure upload protocol to sign verified scripts and perform authenticated uploads on target devices, which motivates our approach to use a fault-proof virtual machine instead.

*Background on Verified Virtual Machines.* Lochbihler [23] presents the verified implementation of a virtual machine modeling the semantics, memory model and byte-code semantics of Java, all by using the proof methodology of translation validation [18,32]. Desharnais and Brunthaler [7] propose the formal verification of an optimized and secure Javascript interpreter in Isabelle/HOL. Its proof methodology is based on concepts of bisimulation. The interpreter targets optimal security and run-time performance. To target MCU devices, our rBPF VM instead seeks optimal run-time memory footprint, to support the expected capability of dynamically running several isolated services on a small device with shared memory. Zhang et al. [40] present a different and ambitious workflow using the deductive programming environment Why3 [9] to specify a virtual machine of Etherium byte-code (EVM) and verify functional correctness of smart contracts against it. The EVM is extracted to OCaml binary code, yielding a TCB consisting of the OCaml runtime and the implementation of Eth's protocols.

*Background on Converting Gallina Programs into Executables.* Just as the prooforiented approach advocated by dependently-typed functional languages like F mentioned in Sect. 2, there are various alternatives to ∂x for extracting executables from Gallina programs. To begin with, Coq comes with a builtin extraction mechanism [21] that generates OCaml, Haskell or Scheme. This path has a rather large TCB (Coq extraction and a compiler). CertiCoq [1] is an ongoing project aiming at generating CompCert C code from Gallina using a specific IR and several passes. Once this effort is completed, it will allow one to rely on a small TCB. Œuf [27] is another tool to compile Gallina to C. It considers a carefully chosen subset of Gallina to tackle the tricky issue of verifying the reflection of Gallina into an AST. Both CertiCoq and Œuf, however, require a garbage collector and define how Coq inductives are represented at runtime. Codegen [36] converts Gallina to C with the goal of maximizing performance by, *e.g.,* allowing the user to control how Coq values are represented at runtime. Rupicola [31] considers an original and promising approach which regards a compiler as a partial decision procedure: it consists of a proof search procedure, which may fail, or else exhibit a target program in bedrock2 (a C-like low-level language AST embedded in Coq) with a proof of equivalence. It has, at present, only been tested for small algorithms. We chose to use ∂x for its simplicity and because it does not increase our TCB. It shares with Codegen the capability to configure the representation of values. Unlike Codegen, it produces C code that is structurally identical to source code. This direct and traceable translation simplifies the verification of generated code w.r.t. source programs, and facilitates source program optimisations.

#### **11 Conclusion and Future Works**

This paper uses a refinement methodology to directly derive a verified C implementation of rBPF, the implementation of BPF hosted by the RIOT operating system, from a Gallina specification in Coq. All the refinement steps are mechanically verified using the Coq proof assistant to minimize the TCB. We prove our rBPF virtual machine to isolate software faults and not to produce runtime errors. Performances are at par with the vanilla rBPF implementation in RIOT.

Our future works aim at instantiating our proof workflow to a (fault-isolating) JIT compiler, one challenge being that Linux's approach of using a verifier will not be feasible on resource-constrained devices, and another being that certain operations might only be expressible in assembly code. This calls for further studies on ways to substantially improve the efficiency of our VM.

**Acknowledgments.** The authors wish to thank the anonymous reviewers for their feedback and suggestions. This work is partly funded by Inria Challenge RIOT-fp, the ANR/BMBF project TinyPART, and the H2020 project Sparta.

**Artifacts.** The source code and proofs of our virtual machine, its generated code and benchmark data are available on https://gitlab.inria.fr/syuan/rbpf-dx/-/tree/CAV22- AE.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Hemiola: A DSL and Verification Tools to Guide Design and Proof of Hierarchical Cache-Coherence Protocols**

Joonwon Choi(B), Adam Chlipala, and Arvind

MIT CSAIL, Cambridge, USA joonwonc@alum.mit.edu, *{*adamc,arvind*}*@csail.mit.edu

**Abstract.** Cache-coherence protocols have been one of the greatest challenges in formal verification of hardware, due to their central complication of executing multiple memory-access transactions concurrently within a distributed message-passing system. In this paper, we introduce Hemiola, a framework embedded in Coq that guides the user to design protocols that never experience inconsistent interleavings while handling transactions concurrently. The framework provides a DSL, where any protocol designed in the DSL always satisfies the serializability property, allowing a user to verify the protocol assuming that transactions are executed one-at-a-time. Hemiola also provides a novel invariant proof method, for protocols designed in Hemiola, that only requires considering execution histories without interleaved memory accesses. We used Hemiola to design and prove hierarchical MSI and MESI protocols as case studies. We also demonstrated that the case-study protocols are hardware-synthesizable, by using a compilation/synthesis toolchain targeting FPGAs.

**Keywords:** formal verification · cache coherence · proof assistants

#### **1 Introduction**

Programming languages and compilers help engineers describe each system at the most expedient level of abstraction. The process of experimenting with new languages is most familiar from the software world, but hardware designers also benefit from it. Of course, Verilog and VHDL themselves are significant steps up from direct circuit descriptions. Some families of hardware languages go further, in roughly the sense that, say, Java goes further than C, providing abstractions that simplify reasoning about modular design. The rule-based hardware languages like Bluespec [23] allow hardware designers to *imagine* that system modules take turns executing local atomic state-change rules, with *no concurrency*. In reality, parallel execution is essential for performance, and compilers for these languages rely on static analysis to extract parallelism soundly.

Roughly speaking, a rule in Bluespec and its relatives must run within a single clock cycle. What happens when we want to simplify reasoning about c The Author(s) 2022

*longer-running processes*? A prime example is a cache-coherence protocol. A memory hierarchy is a distributed system, with many caches communicating through explicit message passing, requiring at least as many clock cycles as the longest dependency chain of message exchanges. The logic is notoriously difficult to get right. One reason is that many memory requests from processor cores may be handled simultaneously. One cache may be working on one request, while a neighboring cache is working on a different request. Might there be abstractions that remove this complication from the hardware designer's thought process, much as Bluespec allows the same designer to pretend that different hardware components do not execute state-change logic in parallel?

We answer affirmatively in presenting *Hemiola*, the first hardware-description language that presents cache-coherence transactions *as if they run atomically*, while realizing the usual parallel performance gains. We define a transaction as all the activity within the memory system in response to a single request from a processor core or other user of the memory. One request may trigger a flurry of activity in the protocol, but the designer may at least pretend that no other request is active in the same period.

The foundation of Hemiola is identifying *commonalities across practical cache-coherence protocols* and embodying them in a domain-specific language (DSL). We fix a notion of node hierarchy and message-passing channels, enumerating *rule templates* capturing relevant communication patterns. Protocols are then described in terms of single-cycle, per-cache rules, each instantiated from a template. Crucially, a locking discipline is built into the language and handled automatically by the templates.

In addition to the DSL, Hemiola provides formal tools significantly easing verification of all cache-coherence protocols designed in it. The DSL is embedded in the Coq proof assistant and has a fully machine-checked proof of soundness, formalized as *serializability*: any state invariant preserved with one-transactionat-a-time execution is also preserved in true parallel execution. The serializability property is once-and-for-all at the language level, freeing protocol designers from needing to reason about interleavings among transactions. In a sense, our work takes techniques that have been used for *per-protocol* verification and lifts them to apply at the level of a DSL, so that no verification effort need be expended on them per-protocol.

To sum up, the contribution of this paper consists of two parts<sup>1</sup>:


<sup>1</sup> Our framework and case studies are available as open source: https://github.com/ mit-plv/hemiola. Choi's dissertation [9] goes into additional detail.

**Fig. 1.** A simple MSI directory protocol and its rule-execution cases

mechanized proofs that share a large segment of reusable proofs across various cache-coherence protocols. We also demonstrate that the case-study protocols are hardware-synthesizable, by using a compilation/synthesis toolchain in Hemiola (Sect. 6).

#### **2 A Motivating Example**

Before introducing our proposed method to design and verify cache-coherence protocols, we provide a simple motivating example to explain the typical challenges and how we suggest to handle them. For simplicity, in this section, we will consider a protocol handling only *a single memory location*. We will see it is still nontrivial to design a correct protocol.

The overall goal of cache coherence is to preserve coherence among multiple candidate values in a memory subsystem. In other words, if the system is coherent, then it should behave like an atomic memory. Figure 1 shows caches and network channels for a directory-based MSI protocol. There are three caches (P, C1, and C2), and each of them has its own status (**M**odified, **S**hared, or **I**nvalid) and data (v). In this MSI protocol, a cache can read/write the data with the M status, only read with S, and cannot read/write with I. The parent P additionally has a data structure called a *directory* to track the statuses of the children. For example, a directory might be <sup>S</sup>1,2, meaning that both <sup>C</sup><sup>1</sup> and <sup>C</sup><sup>2</sup> have S status, in some logical snapshot of state.

Caches communicate through ordered channels, shown as (-) in the figure. Child caches (C<sup>1</sup> and C2) have channels to receive and respond to requests from processor cores. There are three types of channels between a parent and a child: one channel is for parent-to-child messages, and the other two channels are for child-to-parent requests and responses. It is natural to wonder why two separate child-to-parent channels are required; we will see the reason very soon.

Figure 1 also depicts some example state-transition cases depending on the cache statuses. In this setting, all the caches run concurrently by repeatedly executing *rules* that make atomic, local state transitions. A rule may take some messages from input channels, perform a state transition, and put messages in output channels. A rule may also have a precondition, blocking use of that rule when the precondition does not hold.

A rule execution <sup>1</sup> is a case where a child C<sup>1</sup> takes a request <sup>a</sup> rqWr from a processor to write data, but it does not have M status and thus further requests to the parent ( <sup>b</sup> rqM) to get the permission. At this moment, in many practical cache-coherence-protocol designs, C<sup>1</sup> changes its status to a *transient state* SM to record its current status (S) and the next expected status (M) and to make *any further processor requests stall*.

Due to the concurrent execution of caches, we might have another rule executed at the same time. <sup>2</sup> is executed concurrently with <sup>1</sup> , where C<sup>2</sup> also takes a processor request <sup>e</sup> rqWr and sends <sup>f</sup> rqM to the parent as well. Since <sup>1</sup> and <sup>2</sup> happened at the same time, the parent P needs to decide which request to deal with. Suppose that it decided to handle <sup>b</sup> rqM first.

<sup>3</sup> presents the next execution by P, taking the input message <sup>b</sup> rqM and making an invalidation request ( <sup>c</sup> rqI) to the other child C<sup>2</sup> to change its status to I. This request is required, since when a child has M, the others should not be able to read/write the data. The parent, at this moment, changes its directory status to a transient state to disallow any other requests from the children (e.g., <sup>f</sup> rqM), since otherwise it will handle two rqM messages simultaneously, which might lead to an incoherent state – two M statuses in the caches.

Lastly, <sup>4</sup> shows the case that C<sup>2</sup> handles the invalidation request ( <sup>c</sup> rqI). A number of corner cases should be handled carefully in this step:


A so-called three-channel system has been widely used and regarded as a good choice to make the design correct and live [33,34]. While there are other possible correct topology and network settings, the cases shown in Fig. 1 at least demonstrate that it is nontrivial to construct one of them. Note that the threechannel system is *logical* in the sense that the actual hardware implementation may use various hardware components that can simulate the requirements.

In terms of making a protocol design correct, transient states, topology, and network settings contribute to make *interleavings* correct. Considering the sequence of rule executions [ <sup>1</sup> ; <sup>3</sup> ; <sup>4</sup> ] (in red) as an execution flow – we will later call it a *transaction* – to handle a processor request <sup>a</sup> rqWr, we see that the other execution flow (in blue) could not happen after <sup>2</sup> , which is for another processor request <sup>e</sup> rqWr. As explained above case-by-case, proper transient states and network channels made <sup>f</sup> rqM stall. This mechanism to ensure safe interleavings is called *noninterference* [11,18], which ensures that no other transactions spuriously affect state transitions by an ongoing transaction.

**Hemiola in a Nutshell.** If transient states, proper topology, and network settings are essential for designing a correct protocol, can we craft a DSL where *only conformant protocols are expressible*?

That is exactly what we did with Hemiola. The Hemiola DSL helps designers design cache-coherence protocols in a safe way. Instead of requiring designers to use transient states coupled to a protocol, we discover *general* stall conditions that by themselves ensure noninterference and form those conditions as conceptual locks. The stall conditions are *extracted and abstracted* from the usual transient states, so they can apply to practical protocols.

For instance, a designer may write a rule for <sup>1</sup> without any DSL support like the left rule in the following code:

```
1 system memoryMSI {
2 cache C1 {
3 state status: MSI, value: valueT, in_transition: TrsMSI
4 ...
5 // Without any DSL support | // Using the Hemiola DSL
6 rule getMRqUpUp { | rule getMRqUpUp from template rquu {
7 msgIn = procToC1.deq(); | receive rqWr();
8 assert (msgIn.id == rqWr); | assert (status == S);
9 assert (!in_transition); | send rqM();
10 assert (status == S); | }
11 in_transition <= SM; |
12 c1ToPRq.enq({id: rqM, val: 0}); } |
13 } }
```
Note that a designer has to find proper input/output channels (procToC1 and c1ToPRq) and check/set a proper transient state (in\_transition) in order to define the rule.

On the other hand, the left rule can be written more easily by using the Hemiola DSL as the right rule. Instead of using explicit channels and transient states, the right rule just uses the rquu *rule template* (where rquu stands for request-up-up). The rule templates employ *proven-safe* network structures and automatically check/set/release associated locks, so users can design protocols without worrying about incorrect use of network channels, locks, etc.

#### **3 The Hemiola Domain-Specific Language**

As explained in Sect. 2, in designing a cache-coherence protocol, it is nontrivial to make concurrent execution of transactions correct. In this section, we introduce the Hemiola DSL to ease that burden. While conventional approaches deal with transient states directly to derive noninterference per-state, the Hemiola DSL limits protocols to satisfy *abstract conditions* that can guarantee noninterference by-construction. The conditions have already been mentioned in Sect. 2 – network topology and locking mechanisms extracted from transient states of practical cache-coherence protocols.

*Notations.* An overline (e.g., l) denotes a list. [] and (l + e) denote nil and single-element append, respectively. <sup>⊕</sup><sup>l</sup> flattens the list of lists <sup>l</sup> with repeated concatenation. (l<sup>1</sup> <sup>+</sup> <sup>l</sup>2), (l<sup>1</sup> <sup>−</sup> <sup>l</sup>2), and (l<sup>1</sup> # <sup>l</sup>2) denote append, subtraction, and disjointness of lists, respectively. We use the same operation (+) for the single-element and general append. Regarding a list of key-value pairs as a finite map, we override notations for lists. For example, (M + l) updates multiple key-value pairs in a finite map M. Moreover, we overload the same operation (M + (k, v)) for a single update for simplicity. (s.fd) is used as a shorter notation for (List.map (λs. s.fd) <sup>s</sup>). We use · to denote a struct and use a name (e.g., s.fd) to access a field value.

#### **3.1 Syntax**

The Hemiola DSL is similar to well-known rule-based hardware-description languages (HDLs) such as Bluespec [23], Kami [10], and Kˆoika [4]. A notable difference is that rule descriptions are restricted by predesigned *rule templates* to avoid spurious interleavings among transactions.

<sup>A</sup> *system* <sup>S</sup> ::= C,iin,irq,irs is the biggest unit of the language; it consists of caches (C) and channel indices for internal messages (iin) and external (processor) inputs(irq)/outputs(irs). A *cache* <sup>C</sup> ::= i, sinit, <sup>r</sup> consists of its index (unique within a system), an initial state (sinit), and rules (r). A *rule* (r) makes state transitions within the cache, and it is always defined by one of the rule templates provided by the language.

Each rule template must be instantiated with a rule index (should be unique within a cache), a precondition, and a transition function, where the types of the precondition and transition vary by template. A precondition of a rule template usually takes input messages and a (partial) current cache state and decides whether the rule can be executed or not. A transition function takes the same arguments in general but returns the next cache state and output messages. Neither state transition nor input-messages consumption happens if the precondition does not hold. We will introduce the detailed rule-template forms in the next section (Sect. 3.2).

<sup>A</sup> *message* <sup>m</sup> ::= ty, id, val is composed of a Boolean message type (request or response), a message ID (effectively from an enumeration of message kinds), and a value. We use *value* to refer to the set of legal contents of memory addresses. A pair im ::= (i, m) is used sometimes to represent a message m in a channel with an index i.

#### **3.2 Rule Templates**

The Hemiola DSL follows syntax and semantics of traditional rule-based HDLs, but the major difference is that Hemiola further restricts the way of describing rules, which itself guarantees noninterference among transactions.

**Topology and Network Requirements.** First of all, Hemiola requires that the caches in a given system form a tree topology. Most cache-coherent memory subsystems follow this topology, where leaf nodes correspond to L1 caches, and the root corresponds to the main memory. A child and its parent in the tree communicate using the three channels shown in Sect. 2.

**Fig. 2.** Locking mechanism in Hemiola

Note that the topology and network settings are required *logically*; the actual hardware implementation may use various hardware components (e.g., finitecapacity FIFOs or buses) that can simulate the requirements.

**Locking Mechanism.** We saw in Sect. 2 why transient states are required to ensure noninterference in cache-coherence protocols. Revisiting the issue described in Fig. 1, a child should be able to handle an invalidation request from the parent even if it is in a transient state (SM), and after handling the request it changes its transient state to IM.

Hemiola supports a locking mechanism reflecting this discovery; the locking is more general in that the framework looks at whether the message is from the parent or a child. This mechanism is still enough to describe practical cachecoherence protocols and sufficient to ensure noninterference.

In particular, Hemiola employs two kinds of locks: *uplocks and downlocks*. We say a cache is uplocked (or downlocked) when it holds an uplock (or downlock), respectively. Figure 2 depicts the locking mechanism in Hemiola. An uplock is set when a cache (P<sup>1</sup> in the figure) makes an upward request to its parent (P2); it is released when the cache gets a corresponding response from the parent. The cache cannot make any further upward requests while uplocked. On the contrary, a downlock is set when a cache (P<sup>2</sup> in the figure) makes a downward request(s) to some of its children; similarly it is released when the cache gets corresponding response(s) from the child requestee(s). The cache cannot make any further downward requests while downlocked.

Now every cache defined in Hemiola *does not need to set transient states* to consider all possible combinations among stable statuses. For instance, instead of setting a transient state SM, it is now desirable to maintain its status S and *set an uplock* to record it just made an upward request. We emphasize that the Hemiola locks *do not enforce more restrictions* on protocols than what is enforced by transient states; e.g., as an uplock makes certain messages like rqS and rqM stall, a transient state SM makes them stall as well.

Each cache defined by Hemiola has a semantic lock state holding a lock type (uplock or downlock) and related messages/indices. The user, however, does not need to deal with this lock state while using the DSL; locks are managed implicitly by Hemiola.

Note that the DSL supports design of single-cache-line protocols, and thus the uplock and downlock are assigned per-line. The single-line protocol is then

**Fig. 3.** Rule templates in Hemiola

naturally extended to all cache lines using a protocol compiler that will be introduced in Sect. 6. This approach is sound in terms of correctness, since a transaction does not affect coherence for the other lines.

**The Nine Rule Templates.** Hemiola provides a set of rule templates for describing protocols in a way that guarantees noninterference by-construction. Figure <sup>3</sup> presents the nine rule templates. Each diagram has the form {P}C[Q] and arrows (representing the directions of messages; e.g., a downward arrow indicates messages from a parent) with circles (◦ for inputs and • for outputs) and labels representing requests (rq(s)) and responses (rs(s)). It means that the rule template is for a cache <sup>C</sup>, requires input messages (◦) with the message types determined by the label, has a precondition P, performs a state transition <sup>Q</sup>, and generates output messages (•). The precondition and state transition are implicit in the sense that they are automatically checked and performed, respectively, whenever the rule is executed. Note that some rule templates may make local state transitions without any input/output messages (input/output messages marked with parentheses in Fig. 3).

UL, DL, !UL, and !DL in a precondition indicate that the cache should be uplocked, downlocked, uplock-free, and downlock-free, respectively. UL⇑, DL⇑, UL⇓, and DL⇓ in a state transition indicate setting an uplock, setting a downlock, releasing an uplock, and releasing a downlock, respectively. SLT annotates that the rule template forbids any state modification beside locking.

The rule templates are carefully designed to avoid any spurious interleavings among transactions. We see a number of cases that are worth analyzing:


$$\begin{array}{l} \text{SSIent:} \frac{\overbrace{\begin{subarray}{l} \overleftarrow{im} \ne \begin{subarray}{l} \overleftarrow{im} \ne \begin{subarray}{l} \overleftarrow{im}, \overleftarrow{i} \end{subarray} \overleftarrow{im}, \overleftarrow{i} \le \overleftarrow{S.\overline{i}\_{\text{rq}}}\\ \text{SOounts:} \frac{\overleftarrow{im} \ne \begin{subarray}{l} \overleftarrow{\overleftarrow{\bigleftarrow}}, M \end{subarray} \sqrt{\overleftarrow{\bigleftarrow}, M \end{subarray} \ldots \begin{subarray}{l} \overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow}, M \end{subarray}}{\left\langle \overleftarrow{\bigleftarrow}, M \right\rangle \xrightarrow{l\_{\text{lin}}(\overleftarrow{im})} \overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow} \rangle}\\ \text{SOounts:} \frac{\overleftarrow{\bigleftarrow}, M \right) \xrightarrow{l\_{\text{un}}(\overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow}, \overleftarrow{\bigleftarrow})} \left\langle \overleftarrow{\bigleftarrow}, M - \overleftarrow{\bigleftarrow} \right\rangle \end{subarray}}{\left\langle \overleftarrow{c\_{\text{u}}}, \overleftarrow{\bigleftarrow}\_{\text{in}}, \overleftarrow{\bigleftarrow}\_{\text{in}}, \overleftarrow{\bigleftarrow}\_{\text{in}} \right\rangle} \quad \left\langle \overleftarrow{\big-}, M - \overleftarrow{\big-}, \overleftarrow{\big-} \right\rangle}\\ \frac{S = \left\langle \overleftarrow{C}, \overleftarrow{i\_{\text{in}}}, \overleftarrow{i\_{\text{in}}}, \overleftarrow{i\_{\text{in}}} \right\rangle \cup S.\overline{i\_{\text{in}}}}{\left\langle \overleftarrow{m^{\text{unus}}}, \overleftarrow{\big-}, \overleftarrow{S.\overline{i\_{\text{in}}}} \right\rangle \rightarrow t.t. \left\langle \overleftarrow{c\_{\text{i}}}, \overleftarrow{\big-}$$

**Fig. 4.** Transition steps of the Hemiola DSL

a transaction needs to traverse all the caches in the system, e.g., invalidating all the other caches to obtain the M status. The forced order is important to avoid a deadlock.

#### **4 Verification in Hemiola**

We have introduced the Hemiola DSL in Sect. 3 and provided an intuition that rule templates ensure general noninterference, i.e., interleavings among any transactions are safe. That said, we have not yet showed how the rule templates guarantee such noninterference in a formal way. We also have not explained how noninterference eases the verification of cache-coherence protocols.

In this section, we provide the semantics of the Hemiola DSL and the formal meaning of general noninterference called *serializability*. We then introduce our novel approach to proving invariants called *predicate messages*, which eliminates the burden of considering interference while proving invariants.

#### **4.1 Semantics of the Hemiola DSL**

A system in Hemiola follows so-called "one-rule-at-a-time semantics" [4,5,10,34], i.e., any state transition by concurrent rule executions can be interpreted as a serial execution of rules. Thus, it is fair to consider that a state transition happens by executing a single rule.

*Transition Steps.* Figure 4 describes the complete semantics for transition steps of the Hemiola DSL. The semantics for a step is presented as a judgment s<sup>0</sup> l −→S s1, where S is the system to execute, s<sup>0</sup> is a prestate, s<sup>1</sup> is a poststate, and l is a label generated by the state transition. The state of a system (in domain S) is a pair c, M of cache states (c) and message states (M). Cache states are represented in a finite map from cache indices to cache states, and message states are represented in a finite map from channel indices to ordered queues of messages.

Rule [SSilent] represents the case where no state transition happens in the current step; an empty label (l-) is generated in this case. From now on, we assume that all the input/output messages used in the step definitions do not share the same channel, i.e., (List.NoDupim.i). [SIns] describes the case for external input messages coming to the system; an external-inputs label (lin(im)) is generated in this case. [SOuts] describes the opposite case, for output messages being released to the external world, generating an external-outputs label (lout(im)).

Lastly, [SInt] deals with a state transition by a rule (r) in a cache (C). It nondeterministically chooses a cache and a rule in the cache, checks that the precondition holds, and applies the transition to update the state of the system; an internal label (lint(C.i, r.i,imins,imouts)) is generated in this case, which records a cache index, a rule index, input messages, and output messages. Note that the semantics is based on ordered channels, so messages are *enqueued* and *dequeued* in each state-transition case.

The step semantics is naturally lifted to one for *multiple steps*, presented as a judgment s<sup>0</sup> l =⇒ S s1, where l is a sequence of labels generated by executions of the steps in order. We will sometimes call such a sequence of labels a *history*.

We say that a state s is *reachable* iff there is a history l such that Sinit l =⇒ S s holds, where Sinit is the initial state of the system S. We use a simpler notation <sup>S</sup> <sup>⇒</sup> <sup>s</sup> for reachable states. We also call such a history <sup>l</sup> *legal*, denoted as <sup>S</sup> <sup>l</sup> =⇒ •. We call <sup>I</sup> : <sup>S</sup> <sup>→</sup> <sup>P</sup><sup>2</sup> an *invariant* over a system <sup>S</sup> if <sup>I</sup> holds for all reachable states, i.e., <sup>∀</sup>s. (<sup>S</sup> <sup>⇒</sup> <sup>s</sup>) → I(s).

*Behaviors and Correctness.* A system <sup>S</sup> has a behavior <sup>l</sup> (denoted as <sup>S</sup> ⇓ <sup>l</sup>) iff <sup>S</sup>init l =⇒ S <sup>s</sup> holds, where · filters out silent (l-) and internal (lint) labels so only the external parts remain. We call such a sequence of labels a *trace*. Lastly, we say that a system I ("implementation") trace-refines another system <sup>S</sup> ("specification"), written as <sup>I</sup> <sup>S</sup>, iff every trace of <sup>I</sup> is also a trace of <sup>S</sup>:

$$I \sqsubseteq S \cong \forall \overline{t}. \; I \Downarrow \overline{t} \to S \Downarrow \overline{t}.$$

In order to prove trace refinement, we usually establish a *simulation relation* [6] between the implementation and the spec states and prove that the relation is preserved over steps, and it is crucial to state and prove proper invariants of the implementation for the simulation proof. Since the invariant proof is indeed the most significant part of the whole correctness proof, in this paper we would like to focus on how Hemiola helps a user state and prove invariants.

<sup>2</sup> P is Prop in Coq, which can reasonbly be interpreted as Boolean in this paper.

**Fig. 5.** An example of an atomic history

#### **4.2 Serializability in Hemiola**

Serializability [3,28] is a celebrated notion of concurrency correctness. While each transaction in a system affects multiple values, serializability guarantees that interleaved execution of such transactions is correct in that the effect (state change) is the same as if the transactions were executed serially, i.e., *atomically in some order with no interleaving*.

In order to define serializability formally, we first provide basic definitions of atomic histories and transactions. A history h is *atomic* iff it satisfies the predicate (iminit <sup>h</sup> ∼∼∼ imend) with *initial messages* iminit and *live messages* imend, constructed inductively by the following two cases:


Figure 5 presents an atomic history already shown in Fig. 1. h is generated by executions of three rules, <sup>r</sup><sup>1</sup> <sup>∈</sup> <sup>C</sup>1.r, <sup>r</sup><sup>2</sup> <sup>∈</sup> P.r, and <sup>r</sup><sup>3</sup> <sup>∈</sup> <sup>C</sup>2.r. Rule <sup>r</sup><sup>1</sup> takes an input message (1,rqWr) (from the channel with index 1) as an initial message of the history. Rule r<sup>2</sup> takes (3,rqM), the output message from r1. Finally, r<sup>3</sup> takes (8,rqI), the output message from r2. Summing up all the rule executions, by the definition of an atomic history we get the predicate lower-right in Fig. 5.

This example shows that an atomic history intuitively captures a *transaction flow* triggered by the initial messages. Note that an atomic history does not need to be completed, e.g., h in the example is incomplete in the sense that the live message (rsI) is not a response sent to an external channel.

We call an atomic history (iminit <sup>h</sup> ∼∼∼ imend) a *transaction* if its initial messages are external requests (iminit.i <sup>⊆</sup> S.irq); we denote it as <sup>S</sup> h.

With a clear notion of transactions, we can now easily define sequential histories and serializability. A history h is *sequential* iff the history is a concatenation of transactions:

**Fig. 6.** Interference breaks a predicate message

$$\mathsf{Sequential} \ S \ h \triangleq \exists \ \overline{t}. \ (\forall t \in \overline{t}. \ S \ \langle t \rangle \land h = \oplus \overline{t}.)$$

A legal history h is *serializable* in the system S iff there exists a sequential history that reaches the same state:

$$\text{\textbf{Senializablle} } S \text{ } h \triangleq \forall s. \, S\_{\text{init}} \, \mathop{\frac{h}{S}}\, s \to \exists \, h\_{\text{seq}}. \,\text{\textbf{Sequential} } S \text{ } h\_{\text{seq}} \land S\_{\text{init}} \, \mathop{\frac{h\_{\text{seq}}}{\rightleftharpoons}} s.$$

A system S is *serializable* iff every legal history is serializable:

Serializable <sup>S</sup> <sup>∀</sup>h. Serializable S h.

#### **4.3 Predicate Messages**

Now we discuss how to exploit our notion of serializability: how does it help prove global invariants of a system? In proving the correctness of a cache-coherence protocol, it is very common to state an invariant like "an important property holds *whenever the system includes a certain message in a certain channel*." We call such an invariant a *predicate message*, giving the intuition of messages that logically carry predicates that must be true so long as those messages remain in play. More formally, <sup>S</sup> im{P} <sup>∀</sup>s. (<sup>S</sup> <sup>⇒</sup> <sup>s</sup>) <sup>→</sup> im <sup>∈</sup> s.M <sup>→</sup> <sup>P</sup>(s), where s.M refers to the message state of the system. We will write just im{P} when the system <sup>S</sup> is clear from context, also often using a shorter version id{P} (considering only messages with a given ID) when it is not ambiguous.

Figure 6 presents an example of a predicate message. When a child C<sup>2</sup> is about to handle a response message rsM, which is a permission to change the cache status to M, we expect the parent and the other child C<sup>1</sup> to have I status (like {C1.st = <sup>I</sup> <sup>∧</sup>P.st = <sup>I</sup>} in the figure). However, between the sending of that message and receipt by C2, the predicate may be broken *by another transaction*; for instance, the predicate no longer holds if a state transition happens by <sup>r</sup><sup>1</sup> <sup>∈</sup> C1, which takes another (5,rsM) and updates the status of C<sup>1</sup> to M.

Investigating this corner case carefully, we find that actually no two different rsM messages can be in the system at the same time. It implies that now the predicate message for rsM should have a much-more-complicated form, which considers *all possible noninterference cases*. The complete desired predicate message for (8,rsM) will then look like:

$$\{\mathbf{8}, \mathbf{rsM}\} \begin{cases} C\_1 \text{s.t} = I \land P\_1 \text{s.t} = I \land // \text{The original predicate} \\ \text{// } \begin{cases} // \text{Noninterference with another transaction to get } M \text{ from } C\_1 \\ \text{(7, rs1)} \notin s.M \land \text{(5, rsM)} \notin s.M \land \\ \cdots // \text{More noninterference cases } \text{uses will be required} \end{cases} \end{cases}$$

It is indeed a burden to consider all possible interleavings per predicate message. We would not have faced such a complication if we could ensure that no other transactions interfere while handling a transaction. Serializability guarantees exactly that simplification, and Hemiola provides a way of designing and proving predicate messages in the simpler form, *not taking any interference into account*.

Our novel approach to employing predicate messages in atomic histories begins with formalizing the notion of atomic invariants. We say that I<sup>A</sup> : IM <sup>×</sup> <sup>S</sup> <sup>→</sup> <sup>P</sup> is an *atomic invariant* iff <sup>I</sup><sup>A</sup> (imo, s1) holds for any atomic history h with s<sup>0</sup> h =⇒ s<sup>1</sup> and im<sup>i</sup> h ∼∼∼imo.

S Figure 7 shows an example of predicate messages defined in an atomic history, formalized as an atomic invariant. An atomic invariant I<sup>A</sup> is a conjunction of clauses (im <sup>∈</sup> im<sup>o</sup> <sup>→</sup> <sup>P</sup>(s)), each claiming that the predicate <sup>P</sup> holds when im is in *the live messages* imo. We can prove that the atomic invariant <sup>I</sup><sup>A</sup> holds by induction on state-transition steps through the atomic history in the figure:


Note that the invariant proof was straightforward since no other state transitions interfere with an atomic history.

How do atomic invariants help prove conventional invariants? If the system S is serializable, by definition, for every reachable state there is a sequential history that reaches the same state. Since the sequential history is a concatenation of transactions, an invariant can be proven *by showing that any transaction preserves it*.

Since a transaction is an (external) atomic history, we can make use of corresponding atomic invariants. In other words, we can employ both conventional/atomic invariants (<sup>I</sup> and <sup>I</sup>A) to prove the ones for the next state (si+1):

$$\mathcal{T}\_A\left(\overline{m\_i}, s\_i\right) \land \mathcal{T}(s\_i) \to \left(\mathcal{T}\_A\left(\overline{m\_{i+1}}, s\_{i+1}\right) \land \mathcal{T}(s\_{i+1})\right).$$

For instance, in proving a cache-coherence protocol, we usually want to have an invariant claiming that at most one node of the system has M status at a time. The predicate messages defined in Fig. 7 will play a crucial role here, e.g., the one for (8,rsM) says that C<sup>1</sup> and P both have I status, which means that the state transition by (r<sup>2</sup> : <sup>C</sup>2.st <sup>←</sup> <sup>M</sup>) preserves the invariant. We will see more comprehensive uses of predicate messages in our case studies (Sect. 5).

#### **4.4 Serializability Guarantee by the Hemiola DSL**

The biggest contribution of the Hemiola framework includes the serializability proof. The highest-level theorem simply claims that use of good topology (OnTree S t) and the rule templates (GoodRules S t) guarantees serializability:

$$
\forall S, t. \text{ On } \mathsf{Tree} \ S \ t \land \mathsf{GoodRule} \ S \ t \to \mathsf{Serializable} \ S.
$$

In the proof we used a well-established technique called commuting reductions [15], showing that any interleaving transactions can be serialized by performing a finite number of reductions. Interested readers are referred to Choi's dissertation [9], which describes more details of the proof.

#### **5 Case Studies: Hierarchical MSI and MESI Protocols**

In this section we explain how we designed, specified, and formally proved the correctness of the following three hierarchical cache-coherence protocols: inclusive/noninclusive MSI protocols and a noninclusive MESI protocol. Each protocol is *parameterized by a tree* that decides the topology of the memory subsystem. In other words, whenever we instantiate the tree parameter, we get a cache-coherence design and its correctness proof *for free*.

The protocols are directory-based and support arbitrary evictions. The inclusive MSI protocol requires back-invalidation to maintain the cache-line inclusion [30]. The noninclusive protocols employ the noninclusive-cache inclusivedirectory (NCID) [38] structure to optimize cache space.

We will introduce common points among our case-study protocols. Particularly, we focus on *how predicate messages are used* (introduced in Sect. 4.3) to ease the invariant proofs required to prove protocol correctness. More details about the correctness proofs are provided in Choi's dissertation [9].

#### **5.1 Cache States**

A cache state consists of a status, a value, a directory, and a Boolean called an ownership bit. A status is either M, E, S, or I. The MESI protocol applies further optimizations to the MSI protocol: if a cache line has E status, then the line is exclusive to the cache but also clean.

A directory contains a status of its children called a directory status and a list of child-cache indices that have the directory status. An L1 cache does not have a directory since it has no children.

The *ownership bit* decides whether the cache is responsible for writing the value back to the parent when evicted. The ownership bit intuitively constrains which caches can have valid status; we will see how this intuition is formalized as an invariant in Sect. 5.3.

#### **5.2 Protocol Description with Rule Templates**

We present a number of rule descriptions, used in our case studies, that employ the rule templates provided in Hemiola. Each rule template is defined in Coq, taking in several parameters and generating a rule. We exploited Coq's notation mechanism to define each rule template compactly.

```
1 rule l1GetMRqUpUp from template rquu {
2 receive rqWr();
3 assert (status != M);
4 send rqM();
5 }
```
The above code presents an actual rule definition in an L1 cache, starting with an invocation of a particular rule template rquu, which takes an upward request (to the cache) and sends a further request to the parent. This rule receives a message with the ID rqWr from the processor core<sup>3</sup> to get a write permission. This rule template also requires to write down the precondition (assert) and the output message (send). In this example the cache simply forwards rqM to the parent. As explained in Sect. 3.2, the rquu rule template does not allow any state transition except locking – the template automatically sets an uplock.

```
1 rule liDownIRsUpDownM from template rsud {
2 receive downRsI();
3 hold {rsbTo, rqM()};
4 status <= I;
5 dir <= M [rsbTo];
6 owned <= false;
7 send rsM();
8 }
```
The above rule presents another case that sends the response to the child who requested rqM. Template rsud says that the rule takes responses from children and responds back to the original child requestor. The rule receives the response message with the ID downRsI. In order to execute this rule, the cache should hold

<sup>3</sup> It is a rule defined in an L1 cache, thus an upward request is from the core.

**Fig. 8.** Use of predicate messages in the case-study protocols

a downlock containing the index of the original requestor (rsbTo) and the request message with the ID rqM, acting like an assertion for the lock state.

As a state transition, this rule sets its status to I, sets the directory status to M by adding the requestor, and sets the ownership bit as false since the requestor will make the value dirty after it obtains M. It also sends a response (rsM) to the requestor. Lastly, the downlock is released *automatically and implicitly* by the rsud rule template.

```
1 rule liGetSImmME from template immd {
2 receive rqS() from cidx;
3 assert (status == E || status == M);
4 assert (dir.status == I);
5 status <= I;
6 dir <= E [cidx];
7 send rsE(value);
8 }
```
The above rule is for the MESI protocol, fired when an intermediate cache gets a request from a child to read the data, while the parent has status E or M. In this case, instead of responding with rsS, the cache sends rsE to provide <sup>E</sup>. Once the original requestor obtains E status, it can both read and write.

#### **5.3 Invariant Proof Using Predicate Messages**

Now we present how predicate messages (introduced in Sect. 4.3) are used to prove a nontrivial invariant required for all of our three case-study protocols.

Figure 8 shows a coordination between predicate messages and conventional invariants. Suppose that an L1 cache (shown as L<sup>1</sup> in gray in the figure) requested to the parent to get the M status. When it finally handles the response rsM, it should know all the other caches (except itself) have been invalidated to prove the desired invariant about M (denoted as <sup>L</sup>1.st <sup>=</sup> <sup>M</sup> <sup>→</sup> Invalid (λc. c <sup>=</sup> <sup>L</sup>1)). This proof case can be supported using the predicate message for rsM, stating Invalid (tr−<sup>1</sup> (C)) (the caches outside of the subtree rooted to C are invalid) when the message goes to C. Since L<sup>1</sup> is a leaf node in the tree, it is trivial to prove Invalid (tr−<sup>1</sup> (L1)) <sup>→</sup> Invalid (λc. c <sup>=</sup> <sup>L</sup>1), so we see an example of a predicate message helping prove a conventional invariant.

Figure 8 also shows another coordination to prove a predicate message. When a child C<sup>i</sup> sends the invalidation response rsI, it should know that all the caches inside the subtree of C<sup>i</sup> have been invalidated (denoted as Invalid (tr (Ci))). When the parent P subsequently handles the responses, it responds with rsM to the original requestor (C<sup>0</sup> in the figure), requiring to prove Invalid (tr−<sup>1</sup> (C0)), the predicate message for rsM.

While <sup>P</sup> also changes its status to I in this state transition, how do we infer that the caches outside P have already been invalidated, which is required to prove the predicate over rsM? In this case, we should know that 1) P has the ownership bit true (from a simple cache-level invariant of P) and 2) the caches outside of a cache with ownership bit set should have I status (denoted as P.owned <sup>=</sup> → Invalid (tr−<sup>1</sup> (P))) as an invariant. Combining all the predicates and the state transition by P, we can prove the next predicate message for rsM to the original requestor C0.

#### **6 Compilation and Synthesis to Hardware**

So far we have dealt with cache-coherence protocols for a single line. In order to build a hardware-synthesizable multiline implementation, we developed a compiler that takes a single-line Hemiola protocol and generates a multiline implementation described in Kami [10].

Kami is a hardware formal-verification framework, where its own HDL and proof tools are defined in Coq, allowing users to design, specify, verify, and synthesize their hardware components. Since Kami already has a hardware-synthesis toolchain, we can just compile a Hemiola program to Kami and use the toolchain to run it on FPGAs.

#### **6.1 Compilation of Hemiola Protocols**

The compiler uses prebuilt hardware components described in Kami. One of them is NCID [38], whose interfaces include asynchronous read and write of the line status and value. Another prebuilt component holds a *finite* number of miss-status holding registers (MSHRs), whose abstract interface includes registering, updating, and releasing MSHRs with respect to their types (uplock or downlock) and locking addresses. The compiler also takes a cache configuration as an argument to set the capacity of a cache, the number of MSHRs, etc.

One of the biggest differences between a source Hemiola protocol and the target Kami implementation is that the target accesses multiple lines *asynchronously*. In the source protocol, a single-line value is read (or written) *immediately*, whereas in the target the value is accessed first by making a read (or write) request to a cache and next by handling the response. In order to optimize such line accesses, the compiler uses a prebuilt pipeline to deal with multiple line accesses in parallel.

#### **6.2 Synthesis of Hemiola Protocols**

Once we have obtained a multiline cache-coherence protocol implementation from the compiler, we can use Kami's synthesis toolchain to transliterate it to a Bluespec [23] implementation and synthesize it to load on an FPGA.

Before synthesis, we first evaluated two Hemiola protocols, Hemiola<sup>2</sup> and Hemiola3, instantiated from our hierarchical noninclusive MESI protocol described in Sect. 5, using the Bluespec simulator. Hemiola<sup>3</sup> is a 3-level protocol, consisting of four 32 KB 4-way set-associative L1 caches, two 128 KB 8-way L2 caches, and a 512 KB 16-way last-level cache. Hemiola<sup>2</sup> is 2-level, consisting of four L1 caches and the last-level cache. Each line holds 32 bytes in all the protocols. We compared the performance with an existing Bluespec implementation, RiscyOO [37], featuring a 2-level inclusive MESI protocol with self-invalidation [30]. We set the cache sizes of RiscyOO the same as for Hemiola2.

Figure 9 shows the performance result. We measured performance by counting the number of transactions performed in 5 <sup>×</sup> <sup>10</sup><sup>5</sup> simulation cycles, with various workloads that make random requests but mimic some amount of temporal/spatial locality of memory accesses. Though one should not draw too many conclusions from the precise measurements, the result shows that the Hemiola protocols are competitive with a practical implementation coded by hand.

Next we synthesized the Hemiola protocols, also shown in Fig. 9. We used Xilinx's Virtex-7 VC707 FPGA [1] for synthesis. Each protocol uses a minimal clock length that can safely cover its critical path. Both Hemiola<sup>3</sup> and Hemiola<sup>2</sup> stayed within the FPGA's budget of lookup tables (LUTs) and flip-flops (FFs). We performed tandem verification covering over 10<sup>9</sup> memory requests for each protocol on the FPGA, by connecting it to a tester module that generates a random workload and a reference memory to check its safety and liveness.


**Fig. 9.** Evaluation and synthesis of Hemiola protocols

Optimization and verification of the cache-controller design are nontrivial; the pipeline requires correct stall logic, which is as sophisticated as the logic in pipelined processors. While the verification of the pipeline is one of our futurework directions, we see it as orthogonal to the verification of cache-coherence protocols, our focus with Hemiola.

#### **7 Related Work**

*Model Checking.* Model checking has long been widely used to verify cachecoherence protocols. Various model checkers like Murphi [12], SMV [20], and TLA+ [13,14] have been used.

In order to overcome the usual state-space-explosion problem, model checkers have developed noninterference lemmas to deal with the state-space explosion by interleavings [11,18]. In order to obtain effective lemmas, a number of approaches used descriptions in terms of transactions (called "message flows") [24,31,32]. Instead of looking at each transaction, Hemiola provides serializability that guarantees noninterference among any transactions defined on top of the framework.

In order to verify cache-coherence protocols with arbitrary numbers of cores (but no hierarchy), parameterization has been used in designing and modelchecking the protocols [2,35,36]. Since Hemiola is built on Coq, we can take full advantage of parameterization, and indeed the framework supports verification of cache-coherence protocols with an arbitrary tree shape as a parameter.

In order to increase scalability further, recent approaches used modularity in protocol design and successfully verified hierarchical cache-coherence protocols [7,8,16,17]. The enforced modularity, however, made it hard to design and verify noninclusive protocols. [7,8] tried to solve this problem using assumeguarantee reasoning and history variables, while still maintaining the concept of compositional verification, but faced state-space explosion again, and thus they just verified a two-level MSI protocol with three L2 caches. [16,17] have developed the Neo theory as a safe way to compose "subtrees" of caches to have a hierarchical protocol. They argued it is possible to verify noninclusive protocols in the Neo framework when a directory is still inclusive but did not provide the actual design and proof. We provided the proofs of hierarchical noninclusive cache-coherence protocols in Hemiola, without any such restrictions.

Another notable success of cache-coherence verification employed program synthesis to generate a protocol for a given atomic specification [25,26]. The ProtoGen/HieraGen synthesizer can generate various hierarchical protocols including 3-hop protocols and even unconventional protocols like TSO-CC but does not support noninclusive protocols as well. Furthermore, they used Murphi to verify synthesized protocols, but in ProtoGen [26] they only succeeded up to three caches without exhausting memory, and in HieraGen [25] they succeeded only with the root, two cache-H, and two cache-L nodes. Since Hemiola supports noninclusive protocols but not 3-hop ones, we see protocol-design-space coverage between Hemiola and ProtoGen/HieraGen as incomparable. That said, in terms of verification, Hemiola provides a much higher level of formal assurance by allowing verification of protocols with arbitrary tree topologies.

*Theorem Proving.* Theorem proving also has been used to verify cache-coherence protocols. A number of works proved correctness of specific protocols [22,29]. A recent success was a proof of a hierarchical MSI protocol with an arbitrary tree topology using Coq [34], but it was not structured to promote streamlined reuse of results for other protocols. It also included rather complex and ad-hoc invariants that needed to characterize transient states.

Another notable project designed a modular-specification approach for cache coherence, verifying each cache against the spec while generating/proving invariants automatically, using the Ivy verification tool [19,21,27]. While in Hemiola a user should state and prove invariants manually, the framework provides serializability as a large essential invariant that can be reused by various protocols, and then invariants become easier to prove on top of it.

## **8 Conclusion**

We have developed a framework called Hemiola for simplified design and formal proof of cache-coherence protocols. The template-based DSL ensures that the only protocols that can be expressed are those that admit a form of permemory-access serializability. On top of the framework, we proved the correctness of hierarchical MSI and MESI protocols as case studies, demonstrating that Hemiola indeed eases proof burden. We also built a protocol compiler and demonstrated these protocol implementations running on FPGAs.

**Acknowledgements.** We thank our reviewers for their feedback and detailed comments. This work was supported by the Defense Advanced Research Projects Agency (DARPA) under Grant No. HR001118C0018. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA or the U.S. Government.

## **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Machine Learning**

## **Specification-Guided Learning of Nash Equilibria with High Social Welfare**

Kishor Jothimurugan(B) , Suguman Bansal, Osbert Bastani, and Rajeev Alur

> University of Pennsylvania, Philadelphia, USA kishor@seas.upenn.edu

**Abstract.** Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement learning framework for training joint policies that form a Nash equilibrium. In our approach, rather than providing lowlevel reward functions, the user provides high-level specifications that encode the objective of each agent. Then, guided by the structure of the specifications, our algorithm searches over policies to identify one that provably forms an --Nash equilibrium (with high probability). Importantly, it prioritizes policies in a way that maximizes social welfare across all agents. Our empirical evaluation demonstrates that our algorithm computes equilibrium policies with high social welfare, whereas state-ofthe-art baselines either fail to compute Nash equilibria or compute ones with comparatively lower social welfare.

#### **1 Introduction**

Reinforcement learning (RL) is an effective strategy for automatically synthesizing controllers for challenging control problems. As a consequence, there has been interest in applying RL to multi-agent systems. For example, RL has been used to coordinate agents in cooperative systems to accomplish a shared goal [22]. Our focus is on non-cooperative systems, where the agents are trying to achieve their own goals [17]; for such systems, the goal is typically to learn a policy for each agent such that the joint strategy forms a Nash equilibrium.

A key challenge facing existing approaches is how tasks are specified. First, they typically require that the task for each agent is specified as a reward function. However, reward functions tend to be very low-level, making them difficult to manually design; furthermore, they often obfuscate high-level structure in the problem known to make RL more efficient in the single-agent [14] and cooperative [22] settings. Second, they typically focus on computing an arbitrary Nash equilibrium. However, in many settings, the user is a social planner trying to optimize the overall social welfare of the system, and most existing approaches are not designed to optimize social welfare.

We propose a novel multi-agent RL framework for learning policies from high-level specifications (one specification per agent) such that the resulting

The extended version of this paper can be found at [3].

c The Author(s) 2022

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 343–363, 2022. https://doi.org/10.1007/978-3-031-13188-2\_17

**Fig. 1.** Intersection Example

joint policy (i) has high social welfare, and (ii) is an --Nash equilibrium (for a given -). We formulate this problem as a constrained optimization problem where the goal is to maximize social welfare under the constraint that the joint policy is an --Nash equilibrium.

Our algorithm for solving this optimization problem uses an enumerative search strategy. First, it enumerates candidate policies in decreasing order of social welfare. To ensure a tractable search space, it restricts to policies that conform to the structure of the user-provided specification. Then, for each candidate policy, it uses an explore-then-exploit self-play RL algorithm [4] to compute *punishment strategies* that are triggered when some agent deviates from the original joint policy. It also computes the maximum benefit each agent derives from deviating, which can be used to determine whether the joint policy augmented with punishment strategies forms an --Nash equilibrium; if so, it returns the joint policy.

Intuitively, the enumerative search tries to optimize social welfare, whereas the self-play RL algorithm checks whether the --Nash equilibrium constraint holds. Since this RL algorithm comes with PAC (Probably Approximately Correct) guarantees, our algorithm is guaranteed to return an --Nash equilibrium with high probability. In summary, our contributions are as follows.


*Motivating Example.* Consider the road intersection scenario in Fig. 1. There are four cars; three are traveling east to west and one is traveling north to south. At any stage, each car can either move forward one step or stay in place. Suppose each car's specification is as follows:


We also require that the cars do not crash into one another.

Clearly, not all agents can achieve their goals. The next highest social welfare is for three agents to achieve their goals. In particular, one possibility is that all cars except the black car achieve their goals. However, the corresponding joint policy requires that the black car does not move, which is not a Nash equilibrium—there is always a gap between the blue car and the other two cars behind, so the black car can deviate by inserting itself into the gap to achieve its own goal. Our algorithm uses self-play RL to optimize the policy for the black car, and finds that the other agents cannot prevent the black car from improving its outcome in this way. Thus, it correctly rejects this joint policy. Eventually, our algorithm computes a Nash equilibrium in which the black and blue cars achieve their goals.

#### **1.1 Related Work**

*Multi-agent RL.* There has been work on learning Nash equilibria in the multiagent RL setting [1,12,13,21,23,24]; however, these approaches focus on learning an arbitrary equilibrium and do not optimize social welfare. There has also been work on studying weaker notions of equilibria in this context [9,27], as well as work on learning Nash equilibria in two agent zero-sum games [4,20,26].

*RL from High-Level Specifications.* There has been recent work on using specifications based on temporal logic for specifying RL tasks in the single agent setting; a comprehensive survey may be found in [2]. There has also been recent work on using temporal logic specifications for multi-agent RL [10,22], but these approaches focus on cooperative scenarios in which there is a common objective that all agents are trying to achieve.

*Equilibrium in Markov Games.* There has been work on computing Nash equilibrium in Markov games [17,25], including work on computing --Nash equilibria from logical specifications [6,7], as well as recent work focusing on computing welfare-optimizing Nash equilibria from temporal specifications [18,19]; however, all these works focus on the planning setting where the transition probabilities are known. Checking for existence of Nash equilibrium, even in deterministic games, has been shown to be NP-complete for reachability objectives [5].

*Social Welfare.* There has been work on computing welfare maximizing Nash equilibria for bimatrix games, which are two-player one-step Markov games with known transitions [8,11]; in contrast, we study this problem in the context of general Markov games.

#### **2 Preliminaries**

#### **2.1 Markov Game**

We consider an n-agent Markov game M = (S, A, P, H, s0) with a finite set of states S, actions A = A<sup>1</sup> ×···× A<sup>n</sup> where A<sup>i</sup> is a finite set of actions available to agent i, transition probabilities P(s | s, a) for s, s ∈ S and a ∈ A, finite horizon H, and initial state s<sup>0</sup> [20]. A *trajectory* ζ ∈ Z = (S×A)<sup>∗</sup> × S is a finite sequence ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→ <sup>s</sup><sup>1</sup> <sup>a</sup><sup>1</sup> −→··· <sup>a</sup>t−<sup>1</sup> −−−→ s<sup>t</sup> where s<sup>k</sup> ∈ S, a<sup>k</sup> ∈ A; we use |ζ| = t to denote the length of the trajectory ζ and a<sup>i</sup> <sup>k</sup> ∈ A<sup>i</sup> to denote the action of agent i in ak.

For any i ∈ [n], let D(Ai) denote the set of distributions over Ai—i.e., D(Ai) = {Δ : A<sup>i</sup> → [0, 1] | - <sup>a</sup>i∈A<sup>i</sup> <sup>Δ</sup>(ai)=1}. A *policy* for agent <sup>i</sup> is a function π<sup>i</sup> : Z→D(Ai) mapping trajectories to distributions over actions. A policy π<sup>i</sup> is *deterministic* if for every ζ ∈ Z, there is an action a<sup>i</sup> ∈ A<sup>i</sup> such that πi(ζ)(ai) = 1; in this case, we also use πi(ζ) to denote the action ai. A *joint policy* π : Z→D(A) maps finite trajectories to distributions over joint actions. We use (π1,...,πn) to denote the joint policy in which agent i chooses its action in accordance to πi. We denote by D<sup>π</sup> the distribution over H-length trajectories in M induced by π.

We consider the reinforcement learning setting in which we do not know the probabilities P but instead only have access to a simulator of M. Typically, we can only sample trajectories of M starting at s0. Some parts of our algorithm are based on an assumption which allows us to obtain sample trajectories starting at any state that has been observed before. For example, if taking action a<sup>0</sup> in s<sup>0</sup> leads to a state s1, we assume we can obtain future samples starting at s1.

**Assumption 1.** *We can obtain samples from* P(· | s, a) *for any previously observed state* s *and any action* a*.*

#### **2.2 Specification Language**

We consider the specification language Spectrl to express agent specifications. We choose Spectrl since there is existing work on leveraging the structure of Spectrl specifications for single-agent RL [16]. However, we believe our algorithm can be adapted to other specification languages as well.

Formally, a Spectrl specification is defined over a set of *atomic predicates* P0, where every p ∈ P<sup>0</sup> is associated with a function <sup>p</sup> : S → <sup>B</sup> <sup>=</sup> {true, false}; we say a state <sup>s</sup> *satisfies* <sup>p</sup> (denoted <sup>s</sup> <sup>|</sup><sup>=</sup> <sup>p</sup>) if and only if <sup>p</sup>(s) = true. The set of *predicates* <sup>P</sup> consists of conjunctions and disjunctions of atomic predicates. The syntax of a predicate b ∈ P is given by the grammar b ::= p | (b<sup>1</sup> ∧ b2) | (b<sup>1</sup> ∨ b2), where p ∈ P0. Similar to atomic predicates, each predicate b ∈ P corresponds to a function <sup>b</sup> : S → <sup>B</sup> defined naturally over Boolean logic. Finally, the syntax of Spectrl is given by<sup>1</sup>

<sup>φ</sup> ::= achieve <sup>b</sup> <sup>|</sup> <sup>φ</sup><sup>1</sup> ensuring <sup>b</sup> <sup>|</sup> <sup>φ</sup>1; <sup>φ</sup><sup>2</sup> <sup>|</sup> <sup>φ</sup><sup>1</sup> or <sup>φ</sup>2,

where b ∈ P. Each specification φ corresponds to a function <sup>φ</sup> : Z → <sup>B</sup>, and we say ζ ∈ Z satisfies φ (denoted ζ |= φ) if and only if <sup>φ</sup>(ζ) = true. Letting <sup>ζ</sup> be a finite trajectory of length t, this function is defined by

<sup>ζ</sup> <sup>|</sup><sup>=</sup> achieve <sup>b</sup> if <sup>∃</sup> <sup>i</sup> <sup>≤</sup> t, s<sup>i</sup> <sup>|</sup><sup>=</sup> <sup>b</sup> <sup>ζ</sup> <sup>|</sup><sup>=</sup> <sup>φ</sup> ensuring <sup>b</sup> if <sup>ζ</sup> <sup>|</sup><sup>=</sup> <sup>φ</sup> and <sup>∀</sup> <sup>i</sup> <sup>≤</sup> t, s<sup>i</sup> <sup>|</sup><sup>=</sup> <sup>b</sup> ζ |= φ1; φ<sup>2</sup> if ∃ i < t, ζ0:<sup>i</sup> |= φ<sup>1</sup> and ζi+1:<sup>t</sup> |= φ<sup>2</sup> <sup>ζ</sup> <sup>|</sup><sup>=</sup> <sup>φ</sup><sup>1</sup> or <sup>φ</sup><sup>2</sup> if <sup>ζ</sup> <sup>|</sup><sup>=</sup> <sup>φ</sup><sup>1</sup> or <sup>ζ</sup> <sup>|</sup><sup>=</sup> <sup>φ</sup>2.

Intuitively, the first clause means that the trajectory should eventually reach a state that satisfies the predicate b. The second clause says that the trajectory should satisfy specification φ while always staying in states that satisfy b. The third clause says that the trajectory should sequentially satisfy φ<sup>1</sup> followed by φ2. The fourth clause means that the trajectory should satisfy either φ<sup>1</sup> or φ2.

#### **2.3 Abstract Graphs**

Spectrl specifications can be represented by *abstract graphs* which are DAGlike structures in which each vertex represents a set of states (called subgoal regions) and each edge represents a set of concrete trajectories that can be used to transition from the source vertex to the target vertex without violating safety constraints.

**Definition 1.** An *abstract graph* G = (U, E, u0, F, β, Zsafe) is a directed acyclic graph (DAG) with vertices U, (directed) edges E ⊆ U ×U, initial vertex u<sup>0</sup> ∈ U, final vertices <sup>F</sup> <sup>⊆</sup> <sup>U</sup>, subgoal region map <sup>β</sup> : <sup>U</sup> <sup>→</sup> <sup>2</sup><sup>S</sup> such that for each <sup>u</sup> <sup>∈</sup> <sup>U</sup>, <sup>β</sup>(u) is a subgoal region,<sup>2</sup> and *safe trajectories* <sup>Z</sup>safe <sup>=</sup> <sup>e</sup>∈<sup>E</sup> <sup>Z</sup><sup>e</sup> safe ∪ <sup>f</sup>∈<sup>F</sup> <sup>Z</sup><sup>f</sup> safe, where <sup>Z</sup><sup>e</sup> safe ⊆ Z denotes the safe trajectories for edge <sup>e</sup> <sup>∈</sup> <sup>E</sup> and <sup>Z</sup><sup>f</sup> safe ⊆ Z denotes the safe trajectories for final vertex f ∈ F.

Intuitively, (U, E) is a standard DAG, and u<sup>0</sup> and F define a graph reachability problem for (U, E). Furthermore, β and Zsafe connect (U, E) back to the original MDP M; in particular, for an edge e = u → u , <sup>Z</sup><sup>e</sup> safe is the set of safe trajectories in M that can be used to transition from β(u) to β(u ).

**Definition 2.** A trajectory ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→ <sup>s</sup><sup>1</sup> <sup>a</sup><sup>1</sup> −→ ··· <sup>a</sup>t−<sup>1</sup> −−−→ s<sup>t</sup> in M satisfies the abstract graph G (denoted ζ |= G) if there is a sequence of indices 0 = k<sup>0</sup> ≤ k<sup>1</sup> < ··· < k ≤ t and a path ρ = u<sup>0</sup> → u<sup>1</sup> →···→ u in G such that

<sup>1</sup> Here, achieve and ensuring correspond to the "eventually" and "always" operators in temporal logic.

<sup>2</sup> We do not require that the subgoal regions partition the state space or that they be non-overlapping.

– u ∈ F, – for all z ∈ {0,..., }, we have sk<sup>z</sup> ∈ β(uz), – for all z< , letting <sup>e</sup><sup>z</sup> <sup>=</sup> <sup>u</sup><sup>z</sup> <sup>→</sup> <sup>u</sup>z+1, we have <sup>ζ</sup>kz:kz+1 ∈ Ze<sup>z</sup> safe, and – ζk-:<sup>t</sup> ∈ Zu- safe.

The first two conditions state that the trajectory should visit a sequence of subgoal regions corresponding to a path from the initial vertex to some final vertex, and the last two conditions state that the trajectory should be composed of subtrajectories that are safe according to Zsafe.

Prior work shows that for every Spectrl specification φ, we can construct an abstract graph G<sup>φ</sup> such that for every trajectory ζ ∈ Z, ζ |= φ if and only if ζ |= G<sup>φ</sup> [16]. Finally, the number of states in the abstract graph is linear in the size of the specification.

#### **2.4 Nash Equilibrium and Social Welfare**

Given a Markov game <sup>M</sup> with unknown transitions and Spectrl specifications φ1,...,φ<sup>n</sup> for the n agents respectively, the score of agent i from a joint policy π is given by

$$J\_i(\pi) = \Pr\_{\zeta \sim \mathcal{D}\_\pi}[\zeta \mid = \phi\_i].$$

Our goal is to compute a *high-value* --Nash equilibrium in M w.r.t these scores. Given a joint policy π = (π1,...,πn) and an alternate policy π <sup>i</sup> for agent i, let (π−<sup>i</sup>, π i) denote the joint policy (π1,...,π <sup>i</sup>,...,πn). Then, a joint policy π is an -*-Nash equilibrium* if for all agents i and all alternate policies π i, Ji(π) ≥ Ji((π−<sup>i</sup>, π i))−-. Our goal is to compute a joint policy π that maximizes the social welfare given by

$$\mathtt{w1fare}(\pi) = \frac{1}{n} \sum\_{i=1}^{n} J\_i(\pi)$$

subject to the constraint that π is an --Nash equilibrium.

#### **3 Overview**

Our framework for computing a high-welfare --Nash equilibrium consists of two phases. The first phase is a *prioritized enumeration* procedure that learns deterministic joint policies in the environment and ranks them in decreasing order of social welfare. The second phase is a *verification phase* that checks whether a given joint policy can be extended to an --Nash equilibrium by adding punishment strategies. A policy is returned if it passes the verification check in the second phase. Algorithm 1 summarizes our framework.

For the enumeration phase, it is impractical to enumerate all joint policies even for small environments, since the total number of deterministic joint policies is <sup>Ω</sup>(|A||S|H−<sup>1</sup> ), which is Ω(2<sup>n</sup>|S|H−<sup>1</sup> ) if each agent has atleast two actions. Thus,

#### **Algorithm 1** HighNashSearch

**Inputs:** Markov game (with unknown transition probabilities) M with n-agents, agent specifications φ1,...,φn, Nash factor -, precision δ, failure probability p. **Outputs:** --NE, if found.


in the prioritized enumeration phase, we apply a specification-guided heuristic to reduce the number of joint policies considered. The resulting search space is independent of |S| and H, depending only on the specifications {φi}<sup>i</sup>∈[n]. Since the transition probabilities are unknown, these joint policies are trained using an efficient compositional RL approach.

Since the joint policies are trained cooperatively, they are typically not --Nash equilibria. Hence, in the verification phase, we use a probably approximately correct (PAC) procedure (Algorithm 2) to determine whether a given joint policy can be modified by adding *punishment strategies* to form an --Nash equilibrium. Our approach is to reduce this problem to solving two-agent zero-sum games. The key insight is that for a given joint policy to be an --Nash equilibrium, unilateral deviations by any agent must be successfully punished by the coalition of all other agents. In such a *punishment game*, the deviating agent attempts to maximize its score while the coalition of other agents attempts to minimize its score, leading to a competitive min-max game between the agent and the coalition. If the deviating agent can improve its score by a margin ≥ -, then the joint policy cannot be extended to an --Nash equilibrium. Alternatively, if no agent can increase its score by a margin ≥ -, then the joint policy (augmented with punishment strategies) is an --Nash equilibrium. Thus, checking if a joint policy can be converted to an --Nash equilibrium reduces to solving a twoagent zero-sum game for each agent. Each punishment game is solved using a self-play RL algorithm for learning policies in min-max games with unknown transitions [4], after converting specification-based scores to reward-based scores. While the initial joint policy is deterministic, the punishment strategies can be probabilistic.

Overall, we provide the guarantee that with high probability, if our algorithm returns a joint policy, it will be an --Nash equilibrium.

#### **4 Prioritized Enumeration**

We summarize our specification-guided compositional RL algorithm for learning a finite number of deterministic joint policies in an unknown environment under Assumption 1. These policies are then ranked in decreasing order of their (estimated) social welfare.

**Fig. 2.** Abstract Graph of black car.

**Fig. 3.** Abstract Graph of blue car. **Fig. 4.** Product Abstract Graph of black and blue cars. <sup>Z</sup><sup>v</sup><sup>1</sup> and <sup>Z</sup><sup>v</sup><sup>2</sup> refer to safe trajectories after the black and blue cars have reached their final states, respectively.

Our learning algorithm harnesses the structure of specifications, exposed by their abstract graphs, to curb the number of joint policies to learn. For every set of *active agents* B ⊆ [n], we construct a product abstract graph, from the abstract graphs of all active agents' specifications. A property of this product is that if a trajectory ζ in M corresponds to a path in the product that ends in a final state then ζ satisfies the specification of *all* active agents. Then, our procedure learns one joint policy for every path in the product graph that reaches a final state. Intuitively, policies learned using the product graph corresponding to a set of active agents B aim to maximize satisfaction probabilities of all agents in B. By learning joint policies for every set of active agents, we are able to learn policies under which some agents may not satisfy their specifications. This enables learning joint policies in non-cooperative settings. Note that the number of paths (and hence the number of policies considered) is independent of |S| and H, and depends only on the number of agents and their specifications.

One caveat is that the number of paths may be exponential in the number of states in the product graph. It would be impractical to na¨ıvely learn a joint policy for every path. Instead, we design an efficient compositional RL algorithm that learns a joint policy for each edge in the product graph; these edge policies are then composed together to obtain joint policies for paths in the product graph.

#### **4.1 Product Abstract Graph**

Let φ1,...,φ<sup>n</sup> be the specifications for the n-agents, respectively, and let <sup>G</sup><sup>i</sup> = (Ui, Ei, u<sup>i</sup> <sup>0</sup>, Fi, βi, Zsafe,i) be the abstract graph of specification φ<sup>i</sup> in the environment M. We construct a product abstract graph for every set of active agents in [n]. The product graph for a set of active agents B ⊆ [n] is used to learn joint policies which satisfy the specification of all agents in B with high probability.

**Definition 3.** *Given a set of agents* B = {i1,...,im} ⊆ [n]*, the product graph* G<sup>B</sup> = (U, E, u0, F , β, Zsafe) *is the asynchronous product of* G<sup>i</sup> *for all* i ∈ B*, with*


We denote the i-th component of a product vertex u ∈ U by u<sup>i</sup> for agent i ∈ B. Similarly, the i-th component in an edge e = u → v is denoted by e<sup>i</sup> = u<sup>i</sup> → v<sup>i</sup> for i ∈ B; note that e<sup>i</sup> can be a self loop which is not an edge in Gi. For an edge e ∈ E, we denote the set of agents i ∈ B for which e<sup>i</sup> ∈ Ei, and not a self loop, by progress(e).

Abstract graphs of the black car and the blue car from the motivating example are shown in Figs. 2 and 3 respectively. The vertex v<sup>1</sup> denotes the subgoal region βblack(v1) consisting of states in which the black car has crossed the intersection but the orange and green cars have not. The subgoal region βblue(v2) is the set of states in which the blue car has crossed the intersection but the black car has not. Z<sup>1</sup> denotes trajectories in which the black car does not collide and Z<sup>2</sup> denotes trajectories in which the blue car does not collide and stays a car length ahead of the orange and green cars. The product abstract graph for the set of active agents B = {black, blue} is shown in Fig 4. The safe trajectories on the edges reflect the notion of *achieving* a product edge which we discuss below.

A trajectory ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→ <sup>s</sup><sup>1</sup> <sup>a</sup><sup>1</sup> −→ ··· <sup>a</sup>t−<sup>1</sup> −−−→ s<sup>t</sup> *achieves* an edge e = u → v in G<sup>B</sup> if all progressing agents i ∈ progress(e) reach their target subgoal region βi(vi) along the trajectory and the trajectory is safe for all agents in B. For a progressing agent i ∈ progress(e), the initial segment of the rollout until the agent reaches its subgoal region should be safe with respect to the edge ei. After that, the rollout should be safe with respect to every future possibility for the agent. This is required to ensure continuity of the rollout into adjacent edges in the product graph GB. For the same reason, we require that the entire rollout is safe with respect to all future possibilities for non-progressing agents. Note that we are not concerned with non-active agents in [n]\B. In order to formally define this notion, we need to setup some notation.

For a predicate b ∈ P, let the set of safe trajectories w.r.t. b be given by Z<sup>b</sup> = {ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→ <sup>s</sup><sup>1</sup> <sup>a</sup><sup>1</sup> −→ ··· <sup>a</sup>t−<sup>1</sup> −−−→ s<sup>t</sup> ∈Z|∀ 0 ≤ k ≤ t, s<sup>k</sup> |= b}. It is known that safe trajectories along an edge in an abstract graph constructed from a Spectrl specification is either of the form <sup>Z</sup><sup>b</sup> or <sup>Z</sup><sup>b</sup><sup>1</sup> ◦ Z<sup>b</sup><sup>2</sup> , where b, b1, b<sup>2</sup> ∈ P and ◦ denotes concatenation [16]. In addition, for every final vertex <sup>f</sup>, <sup>Z</sup><sup>f</sup> safe is of the form Z<sup>b</sup> for some b ∈ P. We define First as follows:

$$\mathsf{First}(\mathcal{Z}') = \begin{cases} \mathcal{Z}\_b, & \text{if } \mathcal{Z}' = \mathcal{Z}\_b \\ \mathcal{Z}\_{b\_1}, & \text{if } \mathcal{Z}' = \mathcal{Z}\_{b\_1} \circ \mathcal{Z}\_{b\_2} \end{cases}$$

We are now ready to define the notion of satisfiability of a product edge.

**Definition 4.** A rollout ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→ <sup>s</sup><sup>1</sup> <sup>a</sup><sup>1</sup> −→ ··· <sup>a</sup>t−<sup>1</sup> −−−→ s<sup>k</sup> *achieves an edge* e = u → v in G<sup>B</sup> (denoted ζ |=<sup>B</sup> e) if


We can now define what it means for a trajectory to achieve a path in the product graph GB.

**Definition 5.** Given B ⊆ [n], a rollout ζ = s<sup>0</sup> → ··· → s<sup>t</sup> *achieves* a path ρ = u<sup>0</sup> → ··· → u in G<sup>B</sup> (denoted ζ |=<sup>B</sup> ρ) if there exists indices 0 = k<sup>0</sup> ≤ k<sup>1</sup> ≤ ··· ≤ k ≤ t such that (i) u ∈ F, (ii) ζ<sup>k</sup>z:kz+1 achieves u<sup>z</sup> → uz+1 for all 0 ≤ z< , and (iii) ζ<sup>k</sup>-:<sup>t</sup> ∈ Z<sup>u</sup>-,i safe,i for all i ∈ B.

**Theorem 2.** *Let* ρ = u<sup>0</sup> → u<sup>1</sup> → ··· → u *be a path in the product abstract graph* G<sup>B</sup> *for* B ⊆ [n]*. Suppose trajectory* ζ |=<sup>B</sup> ρ*. Then* ζ |= φ<sup>i</sup> *for all* i ∈ B*.*

That is, joint policies that maximize the probability of achieving paths in the product abstract graph G<sup>B</sup> have high social welfare w.r.t. the active agents B.

#### **4.2 Compositional RL Algorithm**

Our compositional RL algorithm learns joint policies corresponding to paths in product abstract graphs. For every B ⊆ [n], it learns a joint policy π<sup>e</sup> for each edge in the product abstract graph GB, which is the (deterministic) policy that maximizes the probability of achieving e from a given initial state distribution. We assume all agents are acting cooperatively; thus, we treat the agents as one and use single-agent RL to learn each edge policy. We will check whether any deviation to this co-operative behaviour by any agent can be punished by the coalition of other agents in the verification phase. The reward function is designed to capture the reachability objective of progressing agents and the safety objective of all active agents.

The edges are learned in topological order, allowing us to learn an induced state distribution for each product vertex u prior to learning any edge policies from u; this distribution is used as the initial state distribution when learning outgoing edge policies from u. In more detail, the distribution for the initial vertex of G<sup>B</sup> is taken to be the initial state distribution of the environment; for every other product vertex, the distribution is the average over distributions

**Fig. 5.** π<sup>i</sup> augmented with punishment strategies.

induced by executing edge policies for all incoming edges. This is possible because the product graph is a DAG.

Given edge policies Π along with a path ρ = u<sup>0</sup> → u<sup>1</sup> → ···→ u = u ∈ F in GB, we define a *path policy* π<sup>ρ</sup> to navigate from u<sup>0</sup> to u. In particular, π<sup>ρ</sup> executes πe[z], where e[z] = u<sup>z</sup> → uz+1 (starting from z = 0) until the resulting trajectory achieves e[z], after which it increments z ← z + 1 (unless z = ). That is, π<sup>ρ</sup> is designed to achieve the sequence of edges in ρ. Note that π<sup>ρ</sup> is a finite-state deterministic joint policy in which vertices on the path correspond to the memory states that keep track of the index of the current policy. This way, we obtain finite-state joint policies by learning edge policies only.

This process is repeated for all sets of active agents B ⊆ [n]. These finitestate joint policies are then ranked by estimating their social welfare on several simulations.

#### **5 Nash Equilibria Verification**

The prioritized enumearation phase produces a list of path policies which are ranked by the total sum of scores. Each path policy is deterministic and also finite state. Since the joint policies are trained cooperatively, they are typically not --Nash equilibria. Thus, our verification algorithm not only tries to prove that a given joint policy is a --Nash equilibrium, but also tries to modify it so it satisfies this property. In particular, our verification algorithm attempts to modify a given joint policy by adding *punishment strategies* so that the resulting policy is an --Nash equilibrium.

Concretely, it takes as input a finite-state deterministic joint policy π = (M, α, σ, m0) where M is a finite set of *memory states*, α : S×A× M → M is the memory update function, σ : S × M → A maps states to (joint) actions and m<sup>0</sup> is the initial policy state. The *extended memory update function* αˆ : Z → M is given by ˆα(-) = m<sup>0</sup> and ˆα(ζstat) = α(st, at, αˆ(ζ)). Then, π is given by π(ζst) = σ(st, αˆ(ζ)). The policy π<sup>i</sup> of agent i simply chooses the i th component of π(ζ) for any history ζ.

The verification algorithm learns one punishment strategy τij : Z→D(Ai) for each pair (i, j) of agents. As outlined in Fig. 5, the modified policy for agent i uses π<sup>i</sup> if every agent j has taken actions according to π<sup>j</sup> in the past. In case some agent j has taken an action that does not match the output of πj- , then agent i uses the punishment strategy τij , where j is the agent that deviated the earliest (ties broken arbitrarily). The goal of verification is to check if there is a set of punishment strategies {τij | i = j} such that after modifying each agent's policy to use them, the resulting joint policy is an --Nash equilibrium.

#### **5.1 Problem Formulation**

We denote the set of all punishment strategies of agent i by τ<sup>i</sup> = {τij | j = i}. We define the composition of π<sup>i</sup> and τ<sup>i</sup> to be the policy ˜π<sup>i</sup> = π<sup>i</sup> τ<sup>i</sup> such that for any trajectory ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→··· <sup>a</sup>t−<sup>1</sup> −−−→ st, we have


Given a finite-state deterministic joint policy π, the verification problem is to check if there exists a set of punishment strategies τ = <sup>i</sup> τ<sup>i</sup> such that the joint policy ˜π = π τ = (π<sup>1</sup> τ1,...,π<sup>n</sup> τn) is an --Nash equilibrium. In other words, the problem is to check if there exists a policy ˜π<sup>i</sup> for each agent i such that (i) ˜π<sup>i</sup> follows π<sup>i</sup> as long as no other agent j deviates from π<sup>j</sup> and (ii) the joint policy ˜π = (˜π1,..., π˜n) is an --Nash equilibrium.

#### **5.2 High-Level Procedure**

Our approach is to compute the best set of punishment strategies τ <sup>∗</sup> w.r.t. π and check if π τ <sup>∗</sup> is an --Nash equilibrium. The best punishment strategy against agent j is the one that minimizes its incentive to deviate. To be precise, we define the best response of j with respect to a joint policy π = (π 1,...,π <sup>n</sup>) to be br<sup>j</sup> (π ) ∈ arg max<sup>π</sup>-- <sup>j</sup> J<sup>j</sup> (π <sup>−</sup><sup>j</sup> , π <sup>j</sup> ). Then, the best set of punishment strategies τ <sup>∗</sup> w.r.t. π is one that minimizes the value of br<sup>j</sup> (π <sup>τ</sup> ) for all <sup>j</sup> <sup>∈</sup> [n]. To be precise, define τ [j] = {τij | i = j} to be the set of punishment strategies *against* agent j. Then, we want to compute τ <sup>∗</sup> such that for all j,

$$\forall \tau^\* \in \arg\min\_{\tau} J\_j((\pi \bowtie \tau)\_{-j}, \text{br}\_j(\pi \bowtie \tau)). \tag{1}$$

We observe that for any two sets of punishment strategies τ , τ with τ [j] = τ [j] and any policy π <sup>j</sup> , we have <sup>J</sup><sup>j</sup> ((<sup>π</sup> <sup>τ</sup> )−<sup>j</sup> , π <sup>j</sup> ) = <sup>J</sup><sup>j</sup> ((<sup>π</sup> τ )−<sup>j</sup> , π <sup>j</sup> ). This is because, for any τ , punishment strategies in τ\τ [j] do not affect the behaviour of the joint policy ((π <sup>τ</sup> )−<sup>j</sup> , π <sup>j</sup> ), since no agent other than agent j will deviate from π. Hence, br<sup>j</sup> (π τ ) as well as J<sup>j</sup> ((π <sup>τ</sup> )−<sup>j</sup> , br<sup>j</sup> (<sup>π</sup> τ )) are independent of τ \ τ [j]; therefore, we can separately compute τ <sup>∗</sup>[j] (satisfying Eq. 1) for each j and take τ <sup>∗</sup> = <sup>j</sup> τ <sup>∗</sup>[j]. The following theorem follows from the definition of τ <sup>∗</sup>.

**Theorem 3.** *Given a finite-state deterministic joint policy* π = (π1,...,πn)*, if there is a set of punishment strategies* τ *such that* π τ *is an* -*-Nash equilibrium, then* π τ <sup>∗</sup> *is an* -*-Nash equilibrium, where* τ <sup>∗</sup> *is the set of best punishment strategies w.r.t.* π*. Furthermore,* π τ <sup>∗</sup> *is an* -*-Nash equilibrium iff for all* j*,*

$$J\_j((
\pi \bowtie \tau^\*)\_{-j}, \text{br}\_j(\pi \bowtie \tau^\*)) - \epsilon \le J\_j(\pi \bowtie \tau^\*) = J\_j(\pi).$$

Thus, to solve the verification problem, it suffices to compute (or estimate), for all j, the optimal deviation scores

$$\mathbf{dev}\_j^{\pi} = \min\_{\tau \left[ j \right]} \max\_{\pi\_j'} J\_j((\pi \bowtie \tau)\_{-j}, \pi\_j'). \tag{2}$$

#### **5.3 Reduction to Min-Max Games**

Next, we describe how to reduce the computation of optimal deviation scores to a standard self-play RL setting. We first translate the problem from the specification setting to a reward-based setting using *reward machines*.

*Reward Machines.* A *reward machine (RM)* [14] is a tuple R = (Q, δu, δr, q0) where Q is a finite set of states, δ<sup>u</sup> : S×A× Q → Q is the state transition function, δ<sup>r</sup> : S × Q → [−1, 1] is the reward function and q<sup>0</sup> is the initial RM state. Given a trajectory ζ = s<sup>0</sup> <sup>a</sup><sup>0</sup> −→ ... <sup>a</sup>t−<sup>1</sup> −−−→ st, the reward assigned by R to ζ is R(ζ) = t−1 <sup>k</sup>=0 δr(sk, qk), where qk+1 = δu(sk, ak, qk) for all k. For any Spectrl specification φ, we can construct an RM such that the reward assigned to a trajectory ζ indicates whether ζ satisfies φ.

**Theorem 4.** *Given any* Spectrl *specification* <sup>φ</sup>*, we can construct an RM* <sup>R</sup><sup>φ</sup> *such that for any trajectory* ζ *of length* t + 1*,* Rφ(ζ) = **1**(ζ0:<sup>t</sup> |= φ)*.*

For an agent <sup>j</sup>, let <sup>R</sup><sup>j</sup> denote <sup>R</sup><sup>φ</sup><sup>j</sup> = (Q<sup>j</sup> , δ<sup>j</sup> u, δ<sup>j</sup> r, q<sup>j</sup> <sup>0</sup>). Letting <sup>D</sup>˜<sup>π</sup> be the distribution over length <sup>H</sup>+1 trajectories induced by using <sup>π</sup>, we have <sup>E</sup><sup>ζ</sup>∼D˜<sup>π</sup> [R<sup>j</sup> (ζ)] = J<sup>j</sup> (π). The deviation values defined in Eq. 2 are now min-max values of expected reward, except that it is not in a standard min-max setting since the policy of every non-deviating agent <sup>i</sup> <sup>=</sup> <sup>j</sup> is constrained to be of the form <sup>π</sup><sup>i</sup> τi. This issue can be handled by considering a product of M with the reward machine R<sup>j</sup> and the finite-state joint policy π. The following theorem follows naturally.

**Theorem 5.** *Given a finite-state deterministic joint policy* π = (M, α, σ, m0)*, for any agent* j*, we can construct a simulator for an augmented two-player zerosum Markov game* <sup>M</sup><sup>π</sup> <sup>j</sup> *(with rewards) which has the following properties.*


#### **Algorithm 2** VerifyNash

Inputs: Finite-state deterministic joint policy π, specifications φ<sup>j</sup> for all j, Nash factor -, precision δ, failure probability p.

Outputs: True or False along with a set of punishment strategies <sup>τ</sup> .

```
1: existsNE ← True
2: τ ← ∅
3: M ←˜ BFS-Estimate(M, δ, p) // Only run if M has not been estimated before.
4: for agent j ∈ {1,...,n} do
5: Rj ← ConstructRM(φj )
6: M˜ j ← ConstructGame(M˜ , j, Rj , π)
7: dev˜ j ← minπ¯2 maxπ¯1 J¯M˜ j (¯π1, π¯2)
8: ¯π∗
       2 ← arg minπ¯2 maxπ¯1 J¯M˜ j (¯π1, π¯2)
9: existsNE ← existsNE ∧ (dev˜ j ≤ Jj (π) + -
                                               − δ)
10: τ ← τ ∪ PunStrat(¯π∗
                           2 )
11: return existsNE, τ
```
*– The min-max value of the two player game corresponds to the deviation cost of* j*, i.e.,*

$$\mathsf{d}\mathsf{e}\mathsf{v}\_{j}^{\pi} = \min\_{\bar{\pi}\_{2}} \max\_{\bar{\pi}\_{1}} J\_{j}^{\pi}(\bar{\pi}\_{1}, \bar{\pi}\_{2}),$$

*where* J¯<sup>π</sup> <sup>j</sup> (¯π1, <sup>π</sup>¯2) = <sup>E</sup> -H <sup>k</sup>=0 R<sup>j</sup> (¯sk, ak) | π¯1, π¯<sup>2</sup> *is the expected sum of rewards w.r.t. the distribution over* (H + 1)*-length trajectories generated by using the joint policy* (¯π1, <sup>π</sup>¯2) *in* <sup>M</sup><sup>π</sup> j *.*

*– Given any policy* <sup>π</sup>¯<sup>2</sup> *for player 2 in* <sup>M</sup><sup>π</sup> <sup>j</sup> *, we can construct a set of punishment strategies* <sup>τ</sup> [j] = PunStrat(¯π2) *against agent* <sup>j</sup> *in* <sup>M</sup> *such that*

$$\max\_{\bar{\pi}\_1} \bar{J}\_j^{\pi}(\bar{\pi}\_1, \bar{\pi}\_2) = \max\_{\pi\_j'} J\_j((\pi \bowtie \tau[j])\_{-j}, \pi\_j').$$

*Given an estimate* <sup>M</sup>˜ *of* <sup>M</sup>*, we can also construct an estimate* <sup>M</sup>˜ <sup>π</sup> <sup>j</sup> *of* <sup>M</sup><sup>π</sup> j *.*

We omit the superscript <sup>π</sup> from <sup>M</sup><sup>π</sup> <sup>j</sup> when there is no ambiguity. We denote by ConstructGame(M˜ , j, <sup>R</sup><sup>j</sup> , π) the product construction procedure that constructs and returns <sup>M</sup>˜ <sup>π</sup> j .

#### **5.4 Solving Min-Max Games**

The min-max game M<sup>j</sup> can be solved using self-play RL algorithms. Many of these algorithms provide probabilistic approximation guarantees for computing the min-max value of the game. We use a model-based algorithm, similar to the one proposed in [4], that first estimates the model M<sup>j</sup> and then solves the game in the estimated model.

One approach is to use existing algorithms for reward-free exploration to estimate the model [15], but this approach requires estimating each M<sup>j</sup> separately. Under Assumption 1, we provide a simpler and more sample-efficient algorithm, called BFS-Estimate, for estimating <sup>M</sup>. BFS-Estimate performs a search over the transition graph of M by exploring previously seen states in a breadth first manner. When exploring a state s, multiple samples are collected by taking all possible actions in s several times and the corresponding transition probabilities are estimated. After obtaining an estimate of M, we can directly construct an estimate of <sup>M</sup><sup>π</sup> <sup>j</sup> for any π and j when required. Letting |Q| = max<sup>j</sup> |Q<sup>j</sup> | and |M| denote the size of the largest finite-state policy output by our enumeration algorithm, we get the following guarantee.

**Theorem 6.** *For any* δ > <sup>0</sup> *and* <sup>p</sup> <sup>∈</sup> (0, 1]*,* BFS-Estimate(M, δ, p) *computes an estimate* <sup>M</sup>˜ *of* <sup>M</sup> *using* <sup>O</sup> |S|3|M<sup>|</sup> <sup>2</sup>|Q<sup>|</sup> <sup>4</sup>|A|H<sup>4</sup> <sup>δ</sup><sup>2</sup> log |S||A| p *sample steps such that with probability at least* 1 − p*, for any finite-state deterministic joint policy* π *and any agent* j*,*

$$\left| \min\_{\bar{\pi}\_2} \max\_{\bar{\pi}\_1} \bar{J}^{\tilde{\mathcal{M}}\_j^{\pi}}(\bar{\pi}\_1, \bar{\pi}\_2) - \mathsf{dec}\_j^{\pi} \right| \le \delta, \eta$$

*where* J¯M˜ <sup>π</sup> <sup>j</sup> (¯π1, π¯2) *is the expected reward over length* H+1 *trajectories generated by* (¯π1, <sup>π</sup>¯2) *in* <sup>M</sup>˜ <sup>π</sup> <sup>j</sup> *. Furthermore, letting* π¯<sup>∗</sup> <sup>2</sup> <sup>∈</sup> arg minπ¯<sup>2</sup> maxπ¯<sup>1</sup> <sup>J</sup>¯M˜ <sup>j</sup> (¯π1, <sup>π</sup>¯2) *and* τ [j] = PunStrat(¯π<sup>∗</sup> <sup>2</sup>)*, we have*

$$\left| \max\_{\bar{\pi}\_1} J^{\tilde{\mathcal{M}}\_j^{\pi}}(\bar{\pi}\_1, \bar{\pi}\_2^\*) - \max\_{\pi\_j'} J\_j((\pi \bowtie \tau[j])\_{-j}, \pi\_j') \right| \leq \delta. \tag{3}$$

The min-max value of <sup>M</sup>˜ <sup>π</sup> <sup>j</sup> as well as ¯π<sup>∗</sup> <sup>2</sup> can be computed using value iteration. Our full verification algorithm is summarized in Algorithm 2. It checks if dev˜ <sup>j</sup> <sup>≤</sup> J<sup>j</sup> (π) + - <sup>−</sup> <sup>δ</sup> for all <sup>j</sup>, and returns True if so and False otherwise. It also simultaneously computes the punishment strategies τ using the optimal policies for player 2 in the punishment games. Note that BFS-Estimate is called only once (i.e., the first time VerifyNash is called) and the obtained estimate <sup>M</sup>˜ is stored and used for verification of every candidate policy π. The following soundness guarantee follows from Theorem 6.

**Corollary 1 (Soundness).** *For any* p ∈ (0, 1]*,* ε > 0 *and* δ ∈ (0, ε)*, with probability at least* <sup>1</sup> <sup>−</sup> <sup>p</sup>*, if* HighNashSearch *returns a joint policy* <sup>π</sup>˜ *then* <sup>π</sup>˜ *is an* -*-Nash equilibrium.*

#### **6 Complexity**

In this section, we analyze the time and sample complexity of our algorithm in terms of the number of agents n, size of the specification |φ| = max<sup>i</sup>∈[n] |φi|, number of states in the environment |S|, number of joint actions |A|, time horizon H, precision δ and the failure probability p.

*Sample Complexity.* It is known [16] that the number of edges in the abstract graph G<sup>i</sup> corresponding to specification φ<sup>i</sup> is O(|φi| <sup>2</sup>). Hence for any set of active agents B, the number of edges in the product abstract graph G<sup>B</sup> is O(|φ| 2|B| ). Hence total number of edge policies learned by our compositional RL algorithm is - <sup>B</sup>⊆[n] <sup>O</sup>((|φ<sup>|</sup> <sup>2</sup>)|B<sup>|</sup> ) = O((|φ| <sup>2</sup>+1)n). We learn each edge using a fixed number of sample steps C, which is a hyperparameter.

The number of samples used in the verification phase is the same as the number used by BFS-Estimate. The maximum size of a candidate policy output by the enumeration algorithm |M| is at most the length of the longest path in a product abstract graph. Since the maximum path length in a single abstract graph G<sup>i</sup> is bounded by |φi| and at least one agent must progress along every edge in a product graph, the maximum length of a path in any product graph is at most n|φ|. Also, the number of states in the reward machine R<sup>j</sup> corresponding to <sup>|</sup>φ<sup>j</sup> <sup>|</sup> is <sup>O</sup>(2|φ<sup>j</sup> <sup>|</sup> ). Hence, from Theorem 6 we get that the total number of sample steps used by our algorithm is O (|φ| <sup>2</sup> + 1)<sup>n</sup><sup>C</sup> <sup>+</sup> <sup>2</sup>4|φ<sup>|</sup> |S|3n2|φ<sup>|</sup> <sup>2</sup>|A|H<sup>4</sup> <sup>δ</sup> log  |S||A| p .

*Time Complexity.* As with sample complexity, the time required to learn all edge policies is O((|φ| <sup>2</sup> + 1)<sup>n</sup>(<sup>C</sup> <sup>+</sup> |A|)) where the term |A| is added to account for the time taken to select an action from A during exploration (we use Qlearning with ε-greedy exploration for learning edge policies). Similarly, time taken for constructing the reward machines and running BFS-Estimate is <sup>O</sup>( <sup>2</sup>4|φ<sup>|</sup> |S|3n2|φ<sup>|</sup> <sup>2</sup>|A|H<sup>4</sup> <sup>δ</sup> log  |S||A| ).

p The total number of path policies considered for a given set of active agents B is bounded by the number of paths in the product abstract graph G<sup>B</sup> that terminate in a final product state. First, let us consider paths in which exactly one agent progresses in each edge. The number of such paths is bounded by (|B||φ|)|B||φ<sup>|</sup> since the length of such paths is bounded by <sup>|</sup>B||φ<sup>|</sup> and there are at most |B||φ| choices at each step—i.e., progressing agent j and next vertex of the abstract graph G<sup>φ</sup><sup>j</sup> . Now, any path in G<sup>B</sup> can be constructed by merging adjacent edges along such a path (in which at most one agent progresses at any step). The number of ways to merge edges along such a path is bounded by the number of groupings of edges along the path into at most |B||φ| groups which is bounded by (|B||φ|)|B||φ<sup>|</sup> . Therefore, the total number of paths in G<sup>B</sup> is at most 2<sup>2</sup>|B||φ<sup>|</sup> log(n|φ|) - . Finally, the total number of path policies considered is at most <sup>B</sup>⊆[n] <sup>2</sup><sup>2</sup>|B||φ<sup>|</sup> log(n|φ|) <sup>≤</sup> ((n|φ|)<sup>2</sup>|φ<sup>|</sup> + 1)<sup>n</sup> <sup>=</sup> <sup>O</sup>(2<sup>2</sup>n|φ<sup>|</sup> log(2n|φ|)).

Now, for each path policy <sup>π</sup>, the verification algorithm solves <sup>M</sup>˜<sup>π</sup> <sup>j</sup> using value iteration which takes <sup>O</sup>(|S||A| ˜ Hf(|A|)) = <sup>O</sup>(2|φ<sup>|</sup> n|φ||S||A|Hf(|A|)) time, where f(|A|) is the time required to solve a linear program of size |A|. Also accounting for the time taken to sort the path policies, we arrive at a time complexity bound of 2<sup>O</sup>(n|φ<sup>|</sup> log(n|φ|))poly(|S|, |A|, H, <sup>1</sup> <sup>p</sup> , <sup>1</sup> δ ).

It is worth noting that the procedure halts as soon as our verification procedure successfully verifies a policy; this leads to early termination for cases where there is a high value --Nash equilibrium (among the policies considered). Furthermore, our verification algorithm runs in polynomial time and therefore one could potentially improve the overall time complexity by reducing the search space in the prioritized enumeration phase—e.g., by using domain specific insights.

#### **7 Experiments**

We evaluate our algorithm on finite state environments and a variety of specifications, aiming to answer the following:


We compare our approach to two baselines described below, using two metrics: (i) the social welfare welfare(π) of the learned joint policy <sup>π</sup>, and (ii) an estimate of the minimum value of for which π forms an --Nash equilibrium:

$$\epsilon\_{\min}(\pi) = \max \{ J\_i(\pi\_{-i}, \text{br}\_i(\pi)) - J\_i(\pi) \mid i \in [n] \}.$$

Here, min(π) is computed using single agent RL (specifically, Q-learning) to compute bri(π) for each agent i.

*Environments and Specifications.* We show results on the *Intersection environment* illustrated in Fig. 1, which consists of k-cars (agents) at a 2-way intersection of which k<sup>1</sup> and k<sup>2</sup> cars are placed along the N-S and E-W axes, respectively. The state consists of the location of all cars where the location of a single car is a non-negative integer. 1 corresponds to the intersection, 0 corresponds to the location one step towards the south or west of the intersection (depending on the car) and locations greater than 1 are to the east or north of the intersection. Each agent has two actions. STAY stays at the current position. MOVE decreases the position value by 1 with probability 0.95 and stays with probability 0.05. We consider specifications similar to the ones in the motivating example.

*Baselines.* We compare our NE computation method (HighNashSearch) to two approaches for learning in non-cooperative games. The first, maqrm, is an adaption of the reward machine based learning algorithm proposed in [22]. maqrm was originally proposed for cooperative multi-agent RL where there is a single specification for all the agents. It proceeds by first decomposing the specification into individual ones for all the agents and then runs a Q-learning-style algorithm (qrm) in parallel for all the agents. We use the second part of their algorithm directly since we are given a separate specification for each agent. The second baseline, nvi, is a model-based approach that first estimates transition probabilities, and then computes a Nash equilibrium in the estimated game using value iteration for stochastic games [17]. To promote high social welfare, we select the highest value Nash solution for the matrix game at each stage of value iteration. Note that this greedy strategy may not maximize social welfare. Both maqrm and nvi learn from rewards as opposed to specification; thus, we supply rewards in the form of reward machines constructed from the specifications. nvi is guaranteed to return an --Nash equilibrium with high probability, but maqrm is not guaranteed to do so.


**Table 1.** Results for all specifications in Intersection Environment. Total of 10 runs per benchmark. Timeout = 24 h.

*Results.* Our results are summarized in Table 1. For each specification, we ran all algorithms 10 times with a timeout of 24 h. Along with the average social welfare and min, we also report the average number of sample steps taken in the environment as well as the number of runs that terminated before timeout. For a fair comparison, all approaches were given a similar number of samples from the environment.

*Nash Equilibrium.* Our approach learns policies that have low values of min, indicating that it can be used to learn --Nash equilibria for small values of -. nvi also has similar values of -, which is expected since nvi provides guarantees similar to our approach w.r.t. Nash equilibria computation. On the other hand, maqrm learns policies with large values of min, implying that it fails to converge to a Nash equilibrium in most cases.

*Social Welfare.* Our experiments show that our approach consistently learns policies with high social welfare compared to the baselines. For instance, φ<sup>3</sup> corresponds to the specifications in the motivating example for which our approach learns a joint policy that causes both blue and black cars to achieve their goals. Although nvi succeeds in learning policies with high social welfare for some specifications (φ<sup>1</sup>, φ<sup>3</sup>, φ<sup>4</sup>), it fails to do so for others (φ<sup>2</sup>, φ<sup>5</sup>). Additional experiments (see extended version [3]) indicate that nvi achieves similar social welfare as our approach for specifications in which all agents can successfully achieve their goals (cooperative scenarios). However, in many other scenarios in which only some of the agents can fulfill their objectives, our approach achieves higher social welfare.

#### **8 Conclusions**

We have proposed a framework for maximizing social welfare under the constraint that the joint policy should form an --Nash equilibrium. Our approach involves learning and enumerating a small set of finite-state deterministic policies in decreasing order of social welfare and then using a self-play RL algorithm to check if they can be extended with punishment strategies to form an --Nash equilibrium. Our experiments demonstrate that our approach is effective in learning Nash equilibria with high social welfare.

One limitation of our approach is that our algorithm does not have any guarantee regarding optimality with respect to social welfare. The policies considered by our algorithm are chosen heuristically based on the specifications, which may lead to scenarios where we miss high welfare solutions. For example, φ<sup>2</sup> corresponds to specifications in the motivating example except that the blue car is not required to stay a car length ahead of the other two cars. In this scenario, it is possible for three cars to achieve their goals in an equilibrium solution if the blue car helps the cars behind by staying in the middle of the intersection until they catch up. Such a joint policy is not among the set of policies considered; therefore, our approach learns a solution in which only two cars achieve their goals. We believe that such limitations can be overcome in future work by modifying the various components within our enumerate-and-verify framework.

**Acknowledgements.** We thank the anonymous reviewers for their helpful comments. This work is supported in part by NSF grant 2030859 to the CRA for the CIFellows Project, ONR award N00014-20-1-2115, DARPA Assured Autonomy award, NSF award CCF 1723567 and ARO award W911NF-20-1-0080.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Synthesizing Fair Decision Trees via Iterative Constraint Solving**

Jingbo Wang(B), Yannan Li, and Chao Wang

University of Southern California, Los Angeles, CA 90089, USA jingbow@usc.edu

**Abstract.** Decision trees are increasingly used to make socially sensitive decisions, where they are expected to be both accurate and fair, but it remains a challenging task to optimize the learning algorithm for fairness in a predictable and explainable fashion. To overcome the challenge, we propose an iterative framework for choosing decision attributes, or *features*, at each level by formulating *feature selection* as a series of mixed integer optimization problems. Both fairness and accuracy requirements are encoded as numerical constraints and solved by an off-the-shelf constraint solver. As a result, the trade-off between fairness and accuracy is quantifiable. At a high level, our method can be viewed as a generalization of the entropy-based greedy search techniques such as CART and C4.5, and existing fair learning techniques such as IGCS and MIP. Our experimental evaluation on six datasets, for which *demographic parity* is used as the fairness metric, shows that the method is significantly more effective in reducing bias than other methods while maintaining accuracy. Furthermore, compared to non-iterative constraint solving, our iterative approach is at least 10 times faster.

#### **1 Introduction**

Decision trees are one of the most widely used machine learning models in statistical analysis, data mining and decision making. Compared to other predictive models such as deep neural networks, decision trees have the advantage of being easily understandable by humans, which makes them a favorite building block in systems that require interpretability [34]. However, when they are used to make socially sensitive decisions in business, finance and law enforcement, decision trees may introduce bias against certain groups [16]. In this context, a widely used group fairness metric is *demographic parity* [11,38], also known as the *80% rule* [8]. Bias against demographic groups, in general, comes from two sources. First, historical data used to learn models may be biased. Second, learning algorithms may be biased even if they operate on unbiased data.

State-of-the-art decision tree learning algorithms such as CART and C4.5 [10, 29], which are the ones used by popular machine learning toolkits, rely on a

c The Author(s) 2022

This work was partially funded by the U.S. National Science Foundation grants CNS-1813117 and CNS-1702814.

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 364–385, 2022. https://doi.org/10.1007/978-3-031-13188-2\_18

**Fig. 1.** SFTree – our symbolic method for synthesizing a fair decision tree.

greedy search technique that is optimized solely for high learning speed and classification accuracy. Since they do not consider fairness as an optimization requirement at all, they often produce decision trees that are severely biased. To mitigate the bias, modifications have been proposed to make the greedy search discrimination-aware [24] (e.g., IGCS). Unfortunately, these modifications are not always effective as shown by our own experimental evaluation in Sect. 5 and, more importantly, the impact of *ad hoc* modifications is often unpredictable and difficult to explain.

Meanwhile, there is a line of work in operational research that formulates decision tree learning as a mixed-integer optimization (MIO) problem [7,35]. Given a finite set F of decision attributes, or features, and a maximum tree depth K, the set of all possible decision trees is captured symbolically as a set of numerical constraints, which is then fed to a solver to compute the *globallyoptimal* decision tree. While optimality was defined initially to minimize the tree size and accuracy loss [7,35], later on, fairness was added as a goal of the optimization [1,5]. However, the approach remains largely theoretical due to its limited scalability: since the entire decision tree must be encoded as a monolithic MIO problem, only small training datasets (with sample sizes in the 1000s) and small decision trees (with depths up to 4 or 5) can be handled [2,7].

To overcome the limitations of the existing approaches, we propose an *iterative constraint solving* technique for synthesizing decision trees in a practically efficient fashion while simultaneously optimizing for fairness and accuracy. Instead of encoding the decision tree as a monolithic MIO formula, we break it down to a series of small steps to avoid the scalability bottleneck. Specifically, starting from the root node, we use constraint solving to conduct a depthbounded look-ahead search at each level of the decision tree, to compute the best feature. Within the look-ahead search, we encode both fairness and accuracy requirements explicitly as numerical constraints, to make the fairness-accuracy trade-off not only predictable but also easy to explain.

The overall flow of our method, SFTree, is shown in Fig. 1. Given a set of training examples (E), a set of features (F), and a sensitive feature (f<sup>s</sup> P F) as input, SFTree returns the synthesized decision tree (T ) as output. Internally, SFTree encodes the hierarchical structure of a partial decision tree symbolically starting from the current node and its training set E, covering a fixed number of tree levels. Then, it uses an MIO solver to compute the optimal feature, <sup>f</sup> <sup>∗</sup>, that minimizes the bias against the protected group, the classification error, and the tree size. Assuming that f <sup>∗</sup> P {0, 1} is a Boolean predicate, the training set is partitioned into subsets E<sup>f</sup> <sup>∗</sup> and E<sup>f</sup> <sup>∗</sup> , one for each child node. Our method iteratively partitions the child nodes until the training subset becomes empty, or all examples in E belong to the same class, or all features in F have been used.

To demonstrate its effectiveness, we have implemented SFTree and evaluated it on six supervised learning datasets, consisting of three small datasets and three large ones. Since the small datasets can be handled even by the monolithic MIO approach (named MIP [1]) to obtain globally-optimal and fair solutions, we used them to evaluate the quality of decision trees learned by our method. The large datasets, which are out of the reach of MIP, were used to evaluate scalability. For comparison, we also evaluated CART [27], a mainstream decision tree learning algorithm, and IGCS [24], a discrimination-aware learning algorithm.

The experimental results show that, among all methods (CART, IGCS, MIP, and SFTree), SFTree produces the best overall solution in terms of fairness and accuracy. In contrast, CART produces unfair decision trees in most cases and, while IGCS does well on the small datasets, it produces mostly unfair decision trees for the large datasets. Neither CART nor IGCS is effective in satisfying the well-known *80% Rule* [8] for demographic parity [11,38]. In contrast, SFTree satisfies the *80% Rule* in all cases. In terms of scalability, MIP fails to handle any of the large datasets, while SFTree handles all of them. In fact, among all four methods, SFTree is the only one that produces fair and accurate decision trees for datasets with ą40, 000 training samples.

To sum up, this paper makes the following contributions:

	- By formulating feature selection as a series of mixed integer optimization subproblems, we make the constraints efficiently solvable.
	- By encoding fairness and accuracy explicitly as symbolic constraints, we make the trade-off quantifiable and easy to explain.

The remainder of this paper is organized as follows. In Sect. 2, we review the basics of decision tree learning and group fairness. In Sect. 3, we present our method. In Sect. 4, we present generalization and performance enhancement techniques. In Sect. 5, we present our experimental results. After reviewing the related work in Sect. 6, we give our conclusions in Sect. 7.

#### **2 Background**

## **2.1 Training Dataset** *E*

The training dataset is a finite set of examples, E " {(xi, yi)}, where i P N is the index, input x<sup>i</sup> " f1,...,fk is a vector of features, and output y<sup>i</sup> is a class

**Fig. 2.** An example training dataset *E* (left) and the related decision tree *T* (right).

label. Let F be the set of all features. For ease of comprehension, let us assume for now that all input features and the output class label are Boolean. In this case, every input <sup>x</sup><sup>i</sup> <sup>P</sup> {0, <sup>1</sup>}<sup>k</sup> is a <sup>k</sup>-bit vector in the feature space, the output y<sup>i</sup> P {0, 1} is a bit, and a decision tree trained using E is a k-input Boolean function. To make the presentation clear, we may also use y<sup>i</sup> P {´, `} instead of y<sup>i</sup> P {0, 1} as the output, where ´ means "no" and ` means "yes".

Figure 2 shows a training set E, where each row in the table represents an example. The input features are a job candidate's *gender* (0 = Female, 1 = Male), *college rank* (0 = Low, 1 = High), *experience* (0 = No, 1 = Yes), and *interview score* (0 = Not-Good, 1 = Good), while the output shows whether the job is offered (0 = No, and 1 = Yes). At the root of the decision tree, for instance, the input goes to the left branch when (f<sup>4</sup> " 0) and to the right branch when (f<sup>4</sup> " 1). The example illustrates three important notions associated with the training set: (1) partition of E (2) entropy, and (3) conditional entropy.

*Partition.* Given a set E and a feature f<sup>j</sup> , we can partition E into subsets E<sup>f</sup>*j*"<sup>0</sup> and E<sup>f</sup>*j*"<sup>1</sup>, or E<sup>f</sup>*<sup>j</sup>* and E<sup>f</sup>*<sup>j</sup>* , respectively, in shorthand notation. Here, E<sup>f</sup>*<sup>j</sup>* " {(xi, yi) P E | f<sup>j</sup> (xi) " 0} consists of examples whose f<sup>j</sup> is 0, and E<sup>f</sup>*<sup>j</sup>* " {(xi, yi) P E | f<sup>j</sup> (xi) " 1} consists of examples whose f<sup>j</sup> is 1. By definition, we have E<sup>f</sup>*<sup>j</sup>* Ď E and E<sup>f</sup>*<sup>j</sup>* Ď E, E<sup>f</sup>*<sup>j</sup>* X E<sup>f</sup>*<sup>j</sup>* " H and E<sup>f</sup>*<sup>j</sup>* Y E<sup>f</sup>*<sup>j</sup>* " E.

For our example in Fig. 2, partitioning the dataset by *gender* (f1) results in subsets E<sup>f</sup>1"<sup>F</sup> " E<sup>f</sup><sup>1</sup> " {(x1, y1)(x4, y4)(x7, y7)} and E<sup>f</sup>1"<sup>M</sup> " E<sup>f</sup><sup>1</sup> " {(x2, y2)(x3, y3)(x5, y5)(x6, y6)}.

*Entropy.* The diversity (or purity) of a set E may be measured by Shannon entropy. Let |E`| be the number of examples in E with positive output label, and |E´| be the number of examples with negative output label. The percentage of positive examples is |E`|{|E|, and the percentage of negative examples is |E´|{|E|. Thus, the entropy is <sup>H</sup>(E) " ´<sup>|</sup>E`| <sup>|</sup>E<sup>|</sup> log(<sup>|</sup>E`| <sup>|</sup>E<sup>|</sup> ) ´ <sup>|</sup>E´| <sup>|</sup>E<sup>|</sup> log(<sup>|</sup>E´| <sup>|</sup>E<sup>|</sup> ).

For our example in Fig. 2, since |E´| " 3 and |E`| " 4, the entropy is H(E) " ´3 <sup>7</sup> log( <sup>3</sup> <sup>7</sup> ) ´ <sup>4</sup> <sup>7</sup> log( <sup>4</sup> <sup>7</sup> ) « 0.985.

*Conditional Entropy.* Given a partition of the set E by the feature f<sup>j</sup> , the entropy of each subset, E<sup>f</sup>*<sup>j</sup>* or E<sup>f</sup>*<sup>j</sup>* , is defined similarly. For our example, since E<sup>f</sup><sup>1</sup> has 2/3 negative examples and 1/3 positive examples, the entropy is H(E<sup>f</sup><sup>1</sup> ) " ´<sup>2</sup> <sup>3</sup> log( <sup>2</sup> <sup>3</sup> ) ´ <sup>1</sup> <sup>3</sup> log( <sup>1</sup> <sup>3</sup> ) " 0.918. Similarly, since Ef<sup>1</sup> has 1/4 negative examples and 3/4 positive examples, the entropy is H(Ef<sup>1</sup> ) " ´1 <sup>4</sup> log( <sup>1</sup> <sup>4</sup> ) ´ <sup>3</sup> <sup>4</sup> log( <sup>3</sup> <sup>4</sup> ) " 0.811.

The conditional entropy of E, with respect to f<sup>j</sup> , is defined as follows:

$$H(\mathcal{E} \mid f\_j) = \frac{|\mathcal{E}\_{\neg f\_j}|}{|\mathcal{E}|} H(\mathcal{E}\_{\neg f\_j}) + \frac{|\mathcal{E}\_{f\_j}|}{|\mathcal{E}|} H(\mathcal{E}\_{f\_j})$$

For our running example, since there are 3 female and 4 male candidates, we have |E<sup>f</sup><sup>1</sup> |{|E| " 3{7 and |E<sup>f</sup><sup>1</sup> |{|E| " 4{7. Thus, the conditional entropy is <sup>H</sup>(<sup>E</sup> <sup>|</sup> <sup>f</sup>1) " <sup>3</sup> 7H(E<sup>f</sup><sup>1</sup> ) ` <sup>4</sup> <sup>7</sup>H(E<sup>f</sup><sup>1</sup> ) « 0.857.

The difference between H(E) and H(E | f<sup>j</sup> ) is called the *information gain*, a metric for evaluating how effective f<sup>i</sup> is in separating positive examples from negative examples in E. For our example, since H(E) « 0.985 and H(E | f1) « 0.857, the information gain (of partitioning E) by *gender (* f1*)* is 0.985´0.857 " 0.128. In contrast, the information gain by *interview (* f4*)* is 0.985 ´ 0.516 " 0.469. Thus, f<sup>4</sup> is more effective as a decision attribute.

*Real-Valued Features.* It is important to note that, while the above examples use Boolean features, our method is more general in that it allows all features have real values, i.e., <sup>x</sup><sup>i</sup> <sup>P</sup> [0, 1]<sup>k</sup> instead of <sup>x</sup><sup>i</sup> <sup>P</sup> {0, <sup>1</sup>}<sup>k</sup>. We accomplish this by applying one-hot encoding to any categorical feature and normalizing any real-valued feature to the [0, 1] domain. Thus, the branch predicates become (f<sup>j</sup> ă bv) and (f<sup>j</sup> bv), instead of (f<sup>j</sup> " 0) and (f<sup>j</sup> " 1), where b<sup>v</sup> P (0, 1] is a threshold computed by our method. For example, if f<sup>j</sup> is the (normalized) salary and b<sup>v</sup> " 0.5, the branch predicates are (f<sup>j</sup> ă 0.5) and (f<sup>j</sup> -0.5).

#### **2.2 Decision Tree Learning**

A decision tree T is a binary tree consisting of a set of nodes and a set of edges. Let the set of nodes be V Y L, where V is the subset of branch nodes (including the root) and L is the subset of leaf nodes. Let E be the set of edges between these nodes. A path in T is a sequence of nodes and edges, denoted v0, e1, v<sup>1</sup> ...vn, en, ln, where v<sup>0</sup> is the root, l<sup>n</sup> is a leaf node, v<sup>1</sup> ...v<sup>n</sup> are the internal nodes, and e1,...,e<sup>n</sup> are the edges.

Each edge has a branch condition. The edge is activated only if the condition holds for a given input x. In Fig. 2, for example, the left-most path of the decision tree has the condition f4(x) " 0 and output *offer* " 0, while the right-most path has the condition (f4(x) " 1) ^ (f1(x) " M) and output *offer* " 1.

Given a training set E " {(xi, yi)}, where x<sup>i</sup> is an input and y<sup>i</sup> is the known output, mainstream algorithms aim to learn a decision tree T that minimizes the classification error. They also aim to minimize the tree size which, in general, allows T to generalize well on the test examples.

**The Baseline Algorithm.** Algorithm 1 shows the top-level procedure of these mainstream algorithms. It takes the training set E and the feature set F as input, and returns a decision tree (T ) as output. These mainstream algorithms use a **Algorithm 1.** The baseline decision tree learning procedure <sup>T</sup> <sup>=</sup> DTL(E, <sup>F</sup>).

1: **Input:** training set *E* " *{*(x1, y1),..., (x*n*, y*n*)*}* and feature set *F* " *{*f1, f2,...,f*k}* 2: **Output:** decision tree *T*

3: **if** all examples in *<sup>E</sup>* have the same label <sup>l</sup>=Label(*E*)

4: **return** *T* " LeafNode(l)

5: **else if** *F* " H and the most common label of *E* is l <sup>∗</sup> <sup>=</sup> MostCommonLabel(*E*) 6: **return** *T* " LeafNode(l ∗)

7: **else if** *E* " H and in *E*.parent, we have l <sup>∗</sup> <sup>=</sup> MostCommonLabel(*E*.parent)

8: **return** *T* " LeafNode(l ∗)

9: **else**

10: *<sup>T</sup>* " BranchNode(<sup>f</sup> <sup>∗</sup>), where <sup>f</sup> <sup>∗</sup> " FindNextFeature(*E*, *<sup>F</sup>*)

11: **foreach** value i P *{*0, 1*}* of the chosen feature f <sup>∗</sup>

12: *T<sup>i</sup>* " DTL(*E<sup>f</sup>*∗"*<sup>i</sup>*, *F\{*f <sup>∗</sup>*}*)

13: Add an edge from *T* to *T<sup>i</sup>* with label (f <sup>∗</sup>(x) " i)

14: **return** *T*

**Algorithm 2.** Subroutine FindNextFeature(E, <sup>F</sup>) used in CART.


greedy method to recursively select decision attributes from F and use them to partition the training set E. At each step, it selects the best feature f <sup>∗</sup> using the subroutine FindNextFeature.

In CART, for example, FindNextFeature is entropy-based, to maximize the information gain of partitioning E by f as shown in Algorithm 2. While this is fast and often leads to high classification accuracy, it does not consider fairness and thus often produces biased decision trees. In this work, we use *iterative constraint solving* to overcome the limitation.

After f <sup>∗</sup> is computed by FindNextFeature, Algorithm 1 uses it to partition the training set <sup>E</sup>, and recursively process the two subsets: DTL(E<sup>f</sup> <sup>∗</sup>"<sup>0</sup>, F \{<sup>f</sup> <sup>∗</sup>}) and DTL(E<sup>f</sup> <sup>∗</sup>"<sup>1</sup>, F\{<sup>f</sup> <sup>∗</sup>}). The recursion ends when


#### **2.3 Fairness Metric**

Given a training set E and a sensitive feature f<sup>s</sup> P F, e.g., race or gender, the goal is to construct a decision tree T that maximizes classification accuracy while minimizing bias. The metric concerned in this work, *demographic parity* [11,38], comes from the legal guideline in the United States for avoiding employment discrimination. Known as the *80% rule* [8], it says the percentage at which



candidates from one protected group are offered jobs should be at least 80% of the percentage at which candidates from another group are offered jobs.

This is formalized using the fairness index, Fs(T , E), defined as follows:

$$F\_{f\_s}(T, \mathcal{E}) = \frac{\Pr[\mathcal{T}(x) = + \mid f\_s(x) = 0]}{\Pr[\mathcal{T}(x) = + \mid f\_s(x) = 1]} \tag{1}$$

where P r[T (x) "`| fs(x) " 0], or P r` <sup>f</sup>*<sup>s</sup>* in short, is the probability of positive examples under the condition fs(x) " 0, and P r[T (x) "`| fs(x) " 1], or P r` f s in short, is the probability of positive examples under the condition fs(x) " 1. Thus, we have P r` <sup>f</sup>*<sup>s</sup>* " <sup>|</sup>{<sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"<sup>0</sup> ^ <sup>T</sup> (x)"`}<sup>|</sup> <sup>|</sup>{<sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"0}<sup>|</sup> and P r` <sup>f</sup>*<sup>s</sup>* " |{x P E | f*s*(x)"1 ^ T (x)"`}| <sup>|</sup>{<sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"1}<sup>|</sup> .

Demographic parity means 0.8 Fs(T , E) (1{0.8) " 1.25. For the example in Fig. 2, since F<sup>f</sup><sup>1</sup> (T , E) " 0.44 for *gender (* f1*)*, the tree fails to satisfy the *80% rule* due to bias against female. The bias is explicit in that f<sup>1</sup> is actually used in the edge labels of the right most two paths of the decision tree. However, even if f<sup>1</sup> is not used in T explicitly, T may still be biased against female, for example, if other non-sensitive features (or their combinations) are statistically correlated to f<sup>1</sup> and, as a result, introduce bias against female. This is the reason why mitigating bias during decision tree learning is a challenging task.

#### **3 Our Method**

To minimize the bias and, at the same time, maximize the classification accuracy, we proposed to follow the top-level procedure in Algorithm 1, but formulate *feature selection* as a series of mixed-integer optimization (MIO) subproblems.

As shown in Algorithm 3, each of our MIO subproblems consists of an objective function O and a constraint Φ, and the solution is an assignment of the numerical variables (shared by O and Φ) that minimizes O while satisfying Φ. In the remainder of this section, we present our symbolic encoding of the objective function, O, and the constraint, Φ, respectively.

#### **3.1 The Objective Function** *O*

We define the function as O :" Oaccu ` αOtree ´ βOfair, consisting of components for accuracy loss (Oaccu), tree size (Otree), and fairness score (Ofair), respectively. The constants, α and β, are used to make trade-offs. In our implementation, α is fixed to 1{(2K`<sup>1</sup>´2) while β is the optimal value in [0, 1] selected using n-fold cross-validation.

Specifically, we test the values 0.02, 0.04, 0.06, ... to 1.00 and, for each fold of the dataset, we compute the objective function and choose β with the minimal objective value. In general, a bigger β means more fairness. Our experiments show that, as β gets larger, Ofair remains constant initially and then starts increasing while Oaccu remains constant, and then Oaccu starts increasing.

Since the decision tree structure is not known *a priori*, we encode a complete binary tree while allowing all branch and leaf nodes to be activated or de-activated. Recall that L is the subset of leaf nodes, V is the subset of branch nodes, l P L denotes a leaf node, and v P V denotes a branch node.

**Tree Size (**Otree :" - <sup>v</sup>P<sup>V</sup> <sup>p</sup>v**).** We assign a variable <sup>p</sup><sup>v</sup> to each branch node v P V, to indicate if a feature is used to split v. Thus, p<sup>v</sup> " 1 means v is split, while p<sup>v</sup> " 0 means v is not split. To get a valid decision tree, p<sup>v</sup> must be constrained also by formula Φ (Sect. 3.2) . Assuming the number of p<sup>v</sup> variables is |V|, the tree size is the number of p<sup>v</sup> variables with value 1.

**Accuracy Loss (**Oaccu :" <sup>1</sup> |L| - <sup>l</sup>P<sup>L</sup> <sup>L</sup>l**).** We assign a variable <sup>L</sup><sup>l</sup> to each leaf node l P L to represent the misclassification error at l. Since we start with a complete tree, each leaf node corresponds to a distinct path. The actual value of L<sup>l</sup> is defined by formula Φ (Sect. 3.3). Assuming the number of L<sup>l</sup> variables is |L|, the accuracy loss is measured by averaging the L<sup>l</sup> values.

**Fairness Score (**Ofair :" F**).** We assign a variable F to represent the overall fairness score of the decision tree. The value of F is defined by formula Φ (Sect. 3.4) according to the definition of demographic parity.

Next, we present our encoding of formula Φ :" Φtree ^ Φaccu ^ Φfair, where Φtree encodes the hierarchical structure of the tree, Φaccu encodes the accuracy requirement, and Φfair encodes the fairness requirement. They share variables with Otree, Oaccu and Ofair in the objective function, such as pv, Ll, and F. Note that, since the constraint will be solved by an off-the-shelf MIO solver, <sup>Φ</sup> must be encoded as a conjunction of equality/inequality constraints. If *logical-or* operators are needed, they must be converted to equality/inequality operators.

#### **3.2 Encoding of the Decision Tree (***Φtree* **)**

Given a node, which may be the root of the decision tree under construction, or any of its branch nodes, we consider a depth-K *complete binary tree* rooted at that node. Since it is a complete binary tree, there are precisely T<sup>K</sup> " 2<sup>K</sup>`<sup>1</sup> ´ 1 nodes with indices 1 ...T<sup>K</sup> and, for any node n, the left and right child nodes have indices 2n and 2n ` 1, respectively. Furthermore, the set of leaf nodes is <sup>L</sup> " {2<sup>K</sup>, <sup>2</sup><sup>K</sup> ` <sup>1</sup> ... <sup>2</sup><sup>K</sup>`<sup>1</sup> ´ <sup>1</sup>}, where <sup>|</sup>L| " <sup>2</sup><sup>K</sup>, and the set of branch nodes is <sup>V</sup> " {1, <sup>2</sup> ... <sup>2</sup><sup>K</sup> ´ <sup>1</sup>}, where <sup>|</sup>V| " <sup>2</sup><sup>K</sup> ´ 1.

**Fig. 3.** Example of a complete binary tree, where *V* " *{*1, 2, 3*}* are branch nodes, *L* " *{*4, 5, 6, 7*}* are leaf nodes, and the decision thresholds b1, b<sup>2</sup> and b<sup>3</sup> belong to [0, 1]. (Color figure online)


Figure 3 shows a depth-2 binary tree whose branch nodes are colored in *teal* and leaf nodes are colored in *red*. The thresholds b1, b<sup>2</sup> and b<sup>3</sup> may be either 0 or a value in (0, 1]: only when they are non-zero, the corresponding nodes are split by features.

For instance, when b<sup>2</sup> is set to 1, if edge condition (w<sup>T</sup> <sup>2</sup> x ă 1) holds, input x goes to the left child, and if (w<sup>T</sup> <sup>2</sup> x - 1) holds, x goes to the right child. When b<sup>2</sup> is set to 0, however, since edge condition (w<sup>T</sup> <sup>2</sup> x ă 0) is always false and (w<sup>T</sup> <sup>2</sup> x - 0) is always true, input x always goes to the right child. In other words, b<sup>2</sup> " 0 disallows splitting at node v " 2.

**Symbolic Variables.** To model how a feature splits the training set, we define some symbolic variables first.


condition (w<sup>T</sup> <sup>v</sup> x ă 0) is unsatisfiable. Otherwise, x goes to the left child when (w<sup>T</sup> <sup>v</sup> x ă bv), and to the right child when (w<sup>T</sup> <sup>v</sup> x bv).


**Formula** Φtree**.** We define the formula as Φtree :" Πsplit ^ Πedge ^ Πleaf ^ Πbranch where Πsplit encodes how features are used to split branch nodes, Πedge encodes the constraints on edges, Πleaf encodes the constraints on leaf nodes, and Πbranch encodes the constraints on branch nodes.

**Subformula** Πsplit**.** We construct Πsplit by constraining pv, wvj , and bv:


Thus, we have Πsplit :" <sup>v</sup>P<sup>V</sup> ( - <sup>j</sup>P{1,....k} <sup>w</sup>vj " <sup>p</sup>v) ^ (0 <sup>b</sup><sup>v</sup> <sup>p</sup>v).

**Subformula** Πedge**.** We construct Πedge by constraining the p<sup>v</sup> variables: If node v P V stops splitting, its child nodes also stop splitting. That is, when p<sup>v</sup> " 0, both p2<sup>v</sup> and p2v`<sup>1</sup> must also be 0.

Thus, we have Πedge " <sup>v</sup>P<sup>V</sup> (p<sup>v</sup> p2<sup>v</sup>) ^ (p<sup>v</sup> p2v`<sup>1</sup>).

**Subformula** Πleaf **.** We construct Πleaf by constraining variables zit and It:


Thus, we have Πleaf :" <sup>i</sup>P{1,...,n} ( - <sup>l</sup>P<sup>L</sup> <sup>z</sup>il " 1) ^ <sup>l</sup>PL(zil <sup>I</sup>l).

**Subformula** Πbranch**.** We construct Πbranch by constraining wvj , bv, and zit:


$$\begin{array}{l} -\ II\_{br2} := \bigwedge\_{v \in \{2^d, \ldots, 2^{d+1}-1\}} \left( z\_{iv} = z\_{i(2v)} + z\_{i(2v+1)} \right) \\ -\ III\_{br3} := \bigwedge\_{v \in \{2^d, \ldots, 2^{d+1}-1\}} \left( \sum\_{j \in \{1, \ldots, k\}} w\_{vj} x\_{ij} - \gamma\_L (1 - z\_{iL}) < b\_v \right) \\ -\ III\_{br4} := \bigwedge\_{v \in \{2^d, \ldots, 2^{d+1}-1\}} \left( \sum\_{j \in \{1, \ldots, k\}} w\_{vj} x\_{ij} + (1 - z\_{iR}) \geqslant b\_v \right) \end{array}$$

Thus, we have Πbranch :" iP{1,...,n} <sup>d</sup>P{1,...,K´1}(Πbr<sup>1</sup> ^ <sup>Π</sup>br<sup>2</sup> ^ <sup>Π</sup>br<sup>3</sup> ^ <sup>Π</sup>br4).

**Explanation of** Πbr<sup>3</sup> **and** Πbr4**.** What we would like to encode in Πbr<sup>3</sup> is the fact that branch condition (<sup>w</sup>vjxij <sup>ă</sup> <sup>b</sup>v) may be either TRUE (x<sup>i</sup> goes to the left child <sup>L</sup> when <sup>z</sup>iL " 1 and <sup>b</sup><sup>v</sup> <sup>P</sup> (0, 1]) or FALSE (x<sup>i</sup> goes to the right child R when ziL " 0 and b<sup>v</sup> P (0, 1], or when b<sup>v</sup> " 0). However, since off-the-shelf MIO solvers do not support *logical-or* operators, we have to encode these different scenarios in a single inequality constraint. This is accomplished by adding a slack value, ´γL(1 ´ ziL), to the branch condition. Similarly, in Πbr4, we add a slack value, (1 ´ ziR), to the branch condition (wvjxij bv).

#### **3.3 Encoding of the Accuracy Requirement (***Φaccu* **)**

To minimize the accuracy loss defined in <sup>O</sup>accu :" <sup>1</sup> |L| - <sup>l</sup>P<sup>L</sup> <sup>L</sup><sup>l</sup> (Sect. 3.1), we need to constrain the L<sup>l</sup> variables in Φaccu such that L<sup>l</sup> models the misclassification error at the leaf node l P L. In the depth-K complete binary tree, there are <sup>|</sup>L| " <sup>2</sup><sup>K</sup> leaf nodes. For each leaf node <sup>l</sup>, variable <sup>L</sup><sup>l</sup> represents the number of misclassified examples (xi, yi) P E: it is misclassified if the given output y<sup>i</sup> does not match the predicted output T (xi).

The formula Φaccu :" Φ<sup>p</sup> ^ Φ<sup>N</sup> ^ Φ<sup>θ</sup> ^ Φloss consists of four subformulas.

**Subformula** Φ<sup>P</sup> **.** For each (xi, yi) P E, where i P {1,...,n} and n " |E|, and for each output value m P {0, 1}, we use pim to model if (y<sup>i</sup> " m). The value of pim, which is either 0 or 1, is constim :" (y<sup>i</sup> " m)?0:1.

Thus, we have <sup>Φ</sup><sup>p</sup> :" <sup>n</sup> i"1 1 <sup>m</sup>"<sup>0</sup>(pim " constim).

**Subformula** Φ<sup>N</sup> **.** We use variable N<sup>l</sup> to represent the number of examples associated with leaf node l, and Nlm to represent those with output value m.

Thus, we have Φ<sup>N</sup> :" <sup>l</sup>PL(N<sup>l</sup> " n <sup>i</sup>"<sup>1</sup> <sup>z</sup>il) ^ (Nlm " <sup>1</sup> 2 n <sup>i</sup>"<sup>1</sup> <sup>z</sup>il(1 ` <sup>p</sup>im)).

**Subformula** Φθ**.** According to Lines 5–8 of Algorithm 1, each leaf node has an output class label θ<sup>l</sup> " **argmax**<sup>m</sup>P{0,1} Nlm. Since **argmax** cannot be directly encoded, we define a matrix of θlm variables in {0, 1}, where θlm " 1 means the output label of node l is m. By definition, only one θlm variable can be 1.

Thus, we have Φ<sup>θ</sup> :" <sup>l</sup>P<sup>L</sup> ( - <sup>m</sup>P{0,1} <sup>θ</sup>lm " 1).

**Subformula** Φloss**.** Assuming that m is the output label predicted by the leaf node l. The misclassification error, Ll, is equal to the number of examples associated with l, denoted Nl, minus the number of examples that have the most common label m, denoted **max**<sup>m</sup>P{0,1}Nlm.

To avoid **max/min** in L<sup>l</sup> " N<sup>l</sup> ´ **max**<sup>m</sup>P{0,1}Nlm " **min**<sup>m</sup>P{0,1}(N<sup>l</sup> ´ Nlm), we use θlm variables and constant n " |E| to rewrite the constraint as :

$$(L\_l \gg 0) \land \bigwedge\_{m \in \{0, 1\}} (L\_l \gg N\_l - N\_{lm} - n(1 - \theta\_{lm})) \land (L\_l \lessapprox N\_l - N\_{lm} + n\theta\_{lm})$$

Thus, we have Φloss :" <sup>l</sup>PL((L<sup>l</sup> - 0)^ <sup>m</sup>P{0,1}(L<sup>l</sup> - Nl´Nlm´n(1´θlm))^ (L<sup>l</sup> N<sup>l</sup> ´ Nlm ` nθlm)).

#### **3.4 Encoding of the Fairness Requirement**

Formula Φfair :" ΦF*s*^ΦFM has two subformulas. Here, ΦF*<sup>s</sup>* encodes the fairness index and ΦFM encodes the constraints on variables used in ΦF*<sup>s</sup>* .

According to Eq. 1 (Sect. 2.3), the fairness index is defined as F<sup>s</sup> " (P r` <sup>f</sup>*<sup>s</sup>* {P r` <sup>f</sup>*<sup>s</sup>* ), where f<sup>s</sup> is a sensitive feature such that fs(x), for any input x P E, may be 0 or 1 (e.g., female and male) while T (x) " ` means the output generated by T is positive (e.g., a job is offered). According to the *"80% rule"*, demographic parity is achieved if F<sup>s</sup> is above 80%. In this work, our goal is to find a solution that (1) satisfies (F<sup>s</sup> ą 0.8) and, at the same time (2) maximizes the value of Fs.

However, the definition of F<sup>s</sup> shown in Eq. 1 has division operators, which are not supported by off-the-shelf MIO solvers. Furthermore, the divisor part of the equation varies even for a fixed set E of examples, which makes the encoding a challenging task. To overcome the challenge, we refine the definition of as follows:

$$\frac{Pr\_{f\_s=0}^+}{Pr\_{f\_s=1}^+} = \frac{|\{x \in \mathcal{E} \mid f\_s(x) = 0, T(x) = +\}| \;/\; |\{x \in \mathcal{E} \mid f\_s(x) = 0\}|}{|\{x \in \mathcal{E} \mid f\_s(x) = 1, T(x) = +\}| \;/\; |\{x \in \mathcal{E} \mid f\_s(x) = 1\}|} = \frac{S\_0^+ / S\_0}{S\_1^+ / S\_1} \tag{2}$$

For each of the four components, we create a symbolic variable. Variable S<sup>0</sup> represents the number of examples whose sensitive feature has the value 0 (e.g., *female*) for the *gender (* f1*)* feature. Variable S` <sup>0</sup> represents the number of examples in S<sup>0</sup> that have the positive output (e.g., a job is offered). Variable S<sup>1</sup> represents the number of examples whose sensitive feature has the value 1 (e.g., *male*) for the *gender (* f1*)* feature. Variable S` <sup>1</sup> represents the number of examples in S<sup>1</sup> that have the positive output.

**Subformula** <sup>Φ</sup><sup>F</sup>*<sup>s</sup>* **.** We use <sup>Φ</sup><sup>F</sup>*<sup>s</sup>* to enforce the *80% rule*: <sup>F</sup><sup>s</sup> " <sup>S</sup>` <sup>0</sup> {S<sup>0</sup> S` <sup>1</sup> {S<sup>1</sup> - 0.8. Assuming S<sup>0</sup> ą 0, S` <sup>0</sup> ą 0, S<sup>1</sup> ą 0, and S` <sup>1</sup> ą 0, we encode the rule as follows:

$$\Phi\_{F\_\*} := (S\_0^+ \times S\_1 - 0.8 \times S\_0 \times S\_1^+ \gg 0),$$

There are two advantages of this encoding. First, the resulting constraint can be solved by off-the-shelf MIO solvers, whereas a direct encoding of Eq. <sup>2</sup> cannot. Second, the value of (S` <sup>0</sup> ˆS1´0.8ˆS0ˆS` <sup>1</sup> ) increases as F<sup>s</sup> increases; therefore, it can be used as part of the objective function, Ofair, to maximize Fs.

**Subformula** ΦFM**.** We use ΦFM to constrain the variables S0, S` <sup>0</sup> , S1, and S` 1 . Toward this end, we need to define the following variables:


$$(\theta\_{lm} = 1 \,\,\wedge \,\, m = 1 \,\,\wedge \,\, z\_{il} = 1 \,\,\wedge \,\, S\_{0i} = 1)$$

In the condition above, (θlm " 1) means the output label produced by the leaf node l is m, and (m " 1) means m is the positive output ("`").

– S<sup>1</sup><sup>i</sup> and S<sup>1</sup> ` il*:* We define variables S<sup>1</sup><sup>i</sup> and S<sup>1</sup> ` il similar to S<sup>0</sup><sup>i</sup> and S<sup>0</sup> ` il.

Thus, we have  $\Phi\_{FM} \coloneqq (S\_0 = \sum\_{i \in \{1, \ldots, n\}} S\_{0i}) \wedge (S\_0^+ = \sum\_{i \in \{1, \ldots, n\}} \sum\_{l \in \mathcal{L}} S\_{0il}^+) \wedge (S\_1 = \sum\_{i \in \{1, \ldots, n\}} S\_{1i}^+).$  $(S\_1 = \sum\_{i \in \{1, \ldots, n\}} S\_{1i})$  $\wedge$ 

**Putting It All Together.** Recall that, in Sect. 3.3, we have constrained the accuracy loss, Ll, in the objective function Oaccu, and defined the objective function Otree in Sect. 3.1, which is used to minimize the tree size and thus reduce over-fitting. As for the objective function Ofair (Sect. 3.1), we define the fairness score as follows: F " (S` <sup>0</sup> ˆ S<sup>1</sup> ´ 0.8 ˆ S<sup>0</sup> ˆ S` <sup>1</sup> ).

Thus, we have the entire MIO problem as follows:

$$\begin{aligned} \text{minimize} & \qquad \frac{1}{|\mathcal{L}|} \sum\_{l \in \mathcal{L}} L\_l + \alpha \sum\_{v \in \mathcal{V}} p\_v - \beta F \\ \text{subject to} & \qquad \Phi\_{accu}(L\_l) \wedge \Phi\_{tree}(p\_v) \wedge \Phi\_{fail}(F) \end{aligned} \tag{3}$$

#### **4 Generalization and Performance Enhancement**

In this section, we first explain how our method relates to various existing algorithms (Sect. 4.1). Next, we present techniques for speeding up constraint solving while maintaining the quality of the solution (Sect. 4.2). Finally, we show that, beyond *demographic parity*, our method can encode other group fairness metrics, such as *equal opportunity* and *equal odds* (Sect. 4.3).

#### **4.1 Relating to Existing Algorithms**

Recall that our method performs feature selection by symbolically encoding a depth-K binary tree, to perform a bounded look-ahead search of the optimal feature using the MIO solver. For ease of presentation, let us call the selected feature *depth-*K *optimal*, where K P {1,..., `8}.

**Depth-1 Optimal.** When K " 1, the tree consists of the root node only and, as a result, look-ahead search is disabled. In this case, our method is the same as a purely greedy search method. Depending on whether fairness is encoded, there are two cases.


Our experimental evaluation (in Sect. 5) shows that neither CART nor IGCS is effective in improving fairness, especially for larger datasets, primarily due to their inability to look beyond the current node.

**Depth-**8 **Optimal.** When K is set to a sufficiently-large number, our method is able to find the globally optimal feature for not only the root node, but also other nodes in the decision tree. Thus, it would compute the entire decision tree in one shot.


Our experimental evaluation (in Sect. 5) shows that the computational overhead of the monolithic MIO approach or MIP is too high to be practically useful. We discuss how to set the value of K in our method in the next subsection.

#### **4.2 Performance Enhancement**

We propose two techniques for speeding up our method by (1) choosing the K value adaptively and (2) sampling the training examples in E.

**Choosing the** K **Value Adaptively.** There is a trade-off between looking further ahead and reducing the constraint solving time. Given n " |E| training examples, and 2<sup>K</sup> leaf nodes in a depth-K binary tree, the number of decision variables (such as S0il) would be (n ˆ 2<sup>K</sup>). Since mixed-integer optimization is NP-hard, the complexity of constraint solving is O(2<sup>n</sup>ˆ2*<sup>K</sup>* ). Empirically, we have found that Gurobi, a state-of-the-art solver, may take 1–2 h to solve a problem for n " 1000 training examples and tree depth K " 7—this is consistent with prior experimental results, e.g., Bertsimas and Dunn [7]. Unfortunately, supervised learning datasets in practice often bring as many as 50,000 training examples to the root node of a decision tree, although the number decreases gradually and may reach 0 for some leaf nodes. Therefore, setting K to 7, or any predetermined value, would not work well in practice.

Instead, we propose to set the K value adaptively. Given a time-out limit (T/O) for learning a decision tree, we start with a relatively small <sup>K</sup> value, say K " 2, to synthesize a decision tree. Then, we increase the K value to synthesize a better decision tree. We keep increasing the K value as long as the time limit is not yet reached, and the quality of the decision tree is improved. We measure the quality of the tree using the value of the objective function, O, which consists of the tree size, the accuracy loss, and the fairness score.

**Sampling the Training Examples.** We propose to reduce the size of the constraints in Φ by sampling the training examples in E, before using them to construct the formula Φ. Our experience shows that sampling can reduce the value of <sup>n</sup> significantly and, at the same time, maintaining the quality of the MIO solution. For the adult dataset, which has 48, 842 training examples, even with a small K value, the symbolic constraints would take more than 1 h to solve.


1: Let f*<sup>s</sup>* be the sensitive feature 2: **if** <sup>|</sup>*E*<sup>|</sup> <sup>8000</sup> **then** (O, Φ) = DtlEncoding(*E*, *<sup>F</sup>*, f*s*) 3: **else** (O, Φ) = DtlEncoding(*E*|*sampled*, *<sup>F</sup>*, f*s*)

4: **return** f <sup>∗</sup> = MioSolver(O, Φ)

Empirically, we have observed that the feature computed by depth-K lookahead using 8,000 randomly-chosen examples is almost as good as the feature computed using all examples. Based on this observation, we set the threshold (n 8000), i.e., at most 8,000 examples from E are used in the symbolic constraints in Algorithm 4, where <sup>Φ</sup> <sup>=</sup> DtlEncoding(E, <sup>F</sup>, fs) is invoked if |E| 8000. Otherwise, E is replaced by the randomly-sampled subset E |sampled.

Our sampling method is not directly applicable to the original MIP approach because, if sampled data are used as input, the MIP solving procedure would permanently discard the rest of the data, which would significantly degrade its accuracy. In contrast, sampling in our method only causes the rest of the data to be ignored temporarily (for this particular node) but, for the child nodes in the subtree, the entire data will still be used in the subsequent computation.

#### **4.3 Encoding Other Group Fairness Metrics**

Beyond *demographic parity*, there are two popular metrics for group fairness, of which one is *equal opportunity* and the other is *equalized odds*.

**Equal Opportunity.** In addition to the sensitive feature fs, there is a decisioncritical feature fc. Let P ` <sup>f</sup>*s*"0,f*c*"<sup>1</sup> " <sup>|</sup><sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"0, f*c*(x)"1, <sup>T</sup> (x)"`| <sup>|</sup><sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"0, f*c*(x)"1<sup>|</sup> " <sup>S</sup>` 0 <sup>S</sup><sup>0</sup> and P ` <sup>f</sup>*s*"1,f*c*"<sup>1</sup> " <sup>|</sup><sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"1, f*c*(x)"1, <sup>T</sup> (x)"`| <sup>|</sup><sup>x</sup> <sup>P</sup> <sup>E</sup> <sup>|</sup> <sup>f</sup>*s*(x)"1, f*c*(x)"1<sup>|</sup> " <sup>S</sup>` 1 <sup>S</sup><sup>1</sup> . A decision tree T satisfies *equal opportunity* if the following condition holds (for a small ).

$$P\_{f\_s=1, f\_c=1}^+ - P\_{f\_s=0, f\_c=1}^+ \lesssim \epsilon \tag{4}$$

In our method, Eq. 4 may be encoded as Φeq :" S` <sup>1</sup> S<sup>0</sup> ´ S` <sup>0</sup> S<sup>1</sup> ´ S0S<sup>1</sup> 0, to replace Φ<sup>F</sup>*<sup>s</sup>* in the fairness requirement Φfair :" Φ<sup>F</sup>*<sup>s</sup>* ^ ΦFM. The definitions of variables S0, S` <sup>0</sup> , S<sup>1</sup> and S` <sup>1</sup> are analogous to that in Sect. 3.4. Similarly, we can define fairness decision variables S<sup>0</sup>i, S<sup>0</sup>il, S<sup>1</sup>i, and S<sup>1</sup>il. For example, the value of S<sup>0</sup><sup>i</sup> is set to 1 if fs(xi) " 0 ^ fc(xi) " 1 and is set to 0 otherwise.

**Equalized Odds.** To satisfy *equalized odds*, we must satisfy Eq. 4, as well as the condition below:

$$P\_{f\_s=1, f\_c=0}^+ - P\_{f\_s=0, f\_c=0}^+ \leqslant \epsilon. \tag{5}$$

Since Eq. 5 can be encoded similarly to Eq. 4, the details are omitted for brevity.


**Table 1.** Comparing our method with existing methods on small benchmarks.

#### **5 Experiments**

We have implemented our method, SFTree, using Python, Julia 1.5.1 [15], and Gurobi 9.03 [21], where Julia is used to encode the MIO constraints and Gurobi is used to solve the constraints. We compared SFTree with three state-of-the-art techniques: CART, which is a mainstream algorithm for decision tree learning, IGCS, which is a discrimination-aware learning algorithm, and MIP, which is a monolithic MIO approach to learning fair tress. We conducted all experiments with Catalina running on a macOS with 2.4 GHz 8-Core CPU and 64G RAM.

**Benchmarks.** Our evaluation uses six popular benchmarks from the fairness literature. They are divided to three small datasets and three large datasets. Since the small datasets can be handled by the less-scalable but more-accurate MIP to obtain globally optimal solutions, they are useful in evaluating the quality of our method. The large datasets, in contrast, are out of the reach of MIP and thus useful in evaluating the scalability of our method.



**Table 2.** Comparing our method with existing methods on large benchmarks.

During learning, we apply the standard 5-fold cross validation expect for Compas, to which we apply 4-fold cross validation to be consistent with prior work.

**Results on the Small Benchmarks.** We compare the quality of the decision trees learned by our method and three existing methods on the small benchmarks. The results are shown in Table 1, where Column 1 shows name of the dataset, Columns 2–3 shows the result of our method in terms of accuracy and fairness, computed by cross-validation, Columns 4–5 show the result of CART, Columns 6–7 show the result of IGCS, and Columns 8–9 show the result of MIP. Since the datasets are small, MIP is able to compute the best solutions: without violating the *80% Rule*, it maximizes accuracy.

The result shows that, overall, CART has the best accuracy but the worst fairness score. IGCS improves over CART, but still violates the *80% Rule* in 5 out of the 15 cases. In contrast, SFTree satisfies the fairness requirement in all 15 cases and, at the same time, achieves high accuracy. Furthermore, it runs more than 10 times faster than MIP.

**Results on the Large Benchmarks.** We use these benchmarks to evaluate both the quality and the scalability of our method. Table 2 shows the result of the quality comparison, which has the same format as Table 1. CART has the highest accuracy but fails to satisfy the fairness requirement in all 14 cases. Although IGCS is somewhat effective for the small benchmarks in Table 1, here, it fails to satisfy the fairness requirement in 12 of the 14 cases. In contrast, our method is the only one that satisfies the fairness requirement in all cases and, at the same time, has accuracy comparable to CART and IGCS.

Table <sup>3</sup> shows the execution time comparison. MIP times out in all 14 cases (T/O = 3h), while our method finishes each within 1 h. Thus, our method runs more than 10 times faster than MIP. Although CART and IGCS are faster, they are equivalent to depth-1 look-ahead search in our method and, due to the limited ability to look ahead, they almost never satisfy the fairness requirement.


**Table 3.** Comparing the run time of methods on large benchmarks (T/O = 3h).

**Fig. 4.** How accuracy and fairness of the learned decision tree change with the K value for the *Student* dataset. For each K " 1,..., 7, we plot the fairness and accuracy scores.

*Evaluating the Impact of the* K*-value.* We have also evaluated how the K value affects the quality of the learned decision tree using the Student Fold1 benchmark. Since the benchmark is small enough, we set K to fixed values 1,..., 7 instead of letting it adapt, so we can assess the impact. Figure 4 shows the result, where the x-axis is accuracy and the y-axis is the fairness score. Thus, the closer a dot is to the right-top corner, the higher the overall quality is. The result shows that the quality of our solution increases dramatically as the K value increases from 1 to 7, due to the increasingly deeper look-ahead search.

**Summary of Additional Results.** While we have also evaluated the scalability of our method with respect to the dataset size, we omit the results for brevity and instead provide a summary. What we have found is that, as the dataset gets larger, the execution time of our method increases modestly at first, and then stops increasing after a threshold is reached. This is due to the use of performance enhancement techniques presented in Sect. 4. Thus, our method does not have scalability issues. In fact, among all four methods, SFTree is the only one that consistently produces fair and accurate decision trees for datasets with ą40,000 training samples.

#### **6 Related Work**

At a high level, our method can be viewed as an *in-processing* approach to mitigating bias in machine learning models. Broadly speaking, there are three approaches: *pre-processing* [17,25,31], *in-processing* [11,19,24,30,33] and *postprocessing* [18,22], depending on whether the focus is on de-biasing the training data, the learning algorithm, or the classification output.

Since the *pre-processing* approach focuses on de-biasing the training data [17, 25,31], it is applicable to any machine learning model; however, it cannot remove bias introduced by the learning algorithms, which is problematic because, even if the training data is not biased, learning algorithms may introduce new bias. While the *post-processing* approach can remove such bias by modifying the predicted output [18,22], the result is often hard to predict and difficult to explain. In contrast, our method does not have these limitations.

Compared to other *in-processing* techniques for fair learning decision trees, including IGCS [24] and similar greedy search methods [11,19,30,33], our method has the advantage of being more systematic and quantifiable. This is because we encode both accuracy and fairness requirements explicitly as numerical constraints. Thus, it would be easy to explain, at every step, why a feature is chosen over another feature, and quantify how much more effective it is in minimizing bias and accuracy loss at the same time. Compared to the monolithic constraint solving approach, including MIP [1] and similar methods [5,35], our method has the advantage of being significantly more scalable.

Our method differs from the recent work of Torfah et al. [32] in that their method uses a small training set sampled from a known distribution and thus does not need techniques such as incremental solving. Furthermore, their method assumes the decision predicates are given, but in our method, the predicates are synthesized from real-valued features. Finally, our fairness constraint is also different from the explainability constraint.

Besides synthesis, there are techniques for improving fairness by repairing an existing machine learning model [4,9,20,26], and techniques for verifying that an existing machine learning model is indeed fair, e.g., by using probabilistic analysis methods [3,6,28]. While these techniques are related, they differ from our method in that they cannot synthesize new decision trees from training data while ensuring the decision trees are fair by construction.

#### **7 Conclusion**

We have presented a method for synthesizing a fair and accurate decision tree, by formulating feature section as a series of mixed-integer optimization problems and solve them using an off-the-shelf constraint solver. The method is flexible in expressing group fairness metrics including *demographic parity*, *equal opportunity*, and *equal odds*. On popular datasets, it is able to learn decision trees that satisfy the fairness requirement and, at the same time, achieve a high classification accuracy.

## **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## SMT-Based Translation Validation for Machine Learning Compiler

Seongwon Bang<sup>1</sup>, Seunghyeon Nam<sup>1</sup>, Inwhan Chun<sup>1</sup>, Ho Young Jhoo<sup>1</sup>, and Juneyoung Lee2(B)

<sup>1</sup> Seoul National University, Seoul, Korea {seongwon.bang,seunghyeon.nam,inwhan.chun, hoyoung.jhoo}@sf.snu.ac.kr <sup>2</sup> CryptoLab, Seoul, Korea aqjune@cryptolab.co.kr

Abstract. Machine learning compilers are large software containing complex transformations for deep learning models, and any buggy transformation may cause a crash or silently bring a regression to the prediction accuracy and performance. This paper proposes an SMT-based translation validation framework for Multi-Level IR (MLIR), a compiler framework used by many deep learning compilers. It proposes an SMT encoding tailored for translation validation that is an over-approximation of the FP arithmetic and reduction operations. It performs abstraction refinement if validation fails. We also propose a new approach for encoding arithmetic properties of reductions in SMT. We found mismatches between the specification and implementation of MLIR, and validated high-level transformations for SqueezeNet, MobileNet, and text\_classification with proper splitting.

### 1 Introduction

Machine learning compilers play a crucial role in the deep learning ecosystem. Their primary goal is to lower high-level tensor operations into fast machine instructions. To boost the speed of training and inference, they utilize several optimizations. Tensors' layouts may be changed for spatial locality, and loops lowered from tensor operations may be fused and offloaded into GPUs if beneficial. Any bug in the optimizations may cause a crash or silently bring a regression to the prediction accuracy and performance.

However, verifying machine learning compilers is a challenging goal. Opensource compilers like XLA, Glow, and MLIR are being updated daily. As their intermediate representations (IRs) are for internal uses, they are sometimes underspecified, making their formalization hard. Furthermore, programmers want to boost the performance at the expense of precision by allowing unsafe arithmetic properties such as associativity of addition.

c The Author(s) 2022 S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 386–407, 2022. https://doi.org/10.1007/978-3-031-13188-2\_19

Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-3-031-13188-2\_19.

Recently, SMT-based automatic translation validation has gained attention [33] because it fits well with fast-moving industrial compilers. Translation validation is an approach to checking whether a specific compilation is correct by inspecting the source (input) and target (output) programs. To cover a variety of compiler optimizations, it uses an SMT solver which is an automatic theorem prover for first-order logic. Using an SMT solver allows us to quickly explore possible semantics for IRs by implementing them and validating compilations of various programs.

A key challenge is how to make SMT solvers prove the verification condition in a reasonable time. To use an SMT solver, the given problem must not be too complex. Bit-vector and uninterpreted function (UF) theories are well-supported by the majority of solvers, whereas floating-point numbers are not [14]. This implies that finding an efficient encoding for tensors and their operations is important for practical validation of machine learning compilers.

In this paper, we propose an SMT-based translation validation framework for Multi-Level IR (MLIR). MLIR is a compiler framework for facilitating the modular development of domain-specific compilers by sharing IRs and relevant transformations. MLIR is primarily used by TensorFlow, TFLite, and IREE. More deep learning frameworks like PyTorch are adding supports for MLIR.

Our goal is to validate high-level, target-independent intraprocedural transformations in MLIR. These include lowering high-level tensor operations to loops, bufferizing tensors, simplifying tensor/buffer operations, and simple loop optimizations. Our tool does not receive hints about the ongoing transformation from the compiler.

The list of contributions of our paper is as follows:


#### 2 Multi-level Intermediate Representation (MLIR)

The MLIR project is an open-source compiler infrastructure that facilitates the modular development of domain-specific compilers by sharing reusable parts. The reusable parts are dialects and relevant compiler transformations. A dialect is a subset of a compiler's intermediate representation language. An intermediate representation (IR) program in MLIR is expressed using one or more dialects. They are ultimately lowered into the input languages of low-level code generation frameworks such as LLVM IR or SPIR-V through first-class dialects. We will introduce several core dialects in MLIR, which are also our targets for validation. Dialects for Tensors. The tensor and tosa (Tensor Operator Set Architecture) dialects define the tensor type and operations. Pre-trained machine learning models can be lowered to them via importers.

A tensor type consists of an element type and dimensions. The tensor dimensions can be dynamic, which are retrievable in runtime. Its elements can be accessed through, e.g., tensor.extract with valid indices. Tensor registers do not alias each other and are in the static single assignment (SSA) form. tosa provides a set of operations commonly employed by deep neural networks, such as a convolution or pooling.

Fig. 1. Dialects for tensors and buffers in MLIR.

The @calc function in Fig. 1(a) takes two tensor arguments, performs convolution (tosa.conv2d), and returns the result. The input bias and output tensor are stored at tensor-typed virtual registers %bias and %res. Note that different dialects – tensor, tosa, and arith (dialect for simple arithmetic operations) – can exist in one IR program.

MemRef Dialect. The memref dialect has a type for memory references (which is also called memref) and relevant operations. The memref type is similar to a pointer type in C but has richer information than that. It has a layout map that maps multidimensional, logical indices into a one-dimensional, physical address<sup>1</sup>. It is used to create a view of a specific memory region in the form of a tensor. It supports arbitrary access patterns such as strided accesses or a transposed view. MLIR transformations assume that the layout map is injective.

Figure 1(b) shows two memref arguments with different layout maps. %arg0 has a default row-major layout map. On the other hand, %arg1 has a columnmajor layout map meaning that %arg1[i][j] is located at offset (*<sup>i</sup>* + *<sup>j</sup>* <sup>×</sup> 3) from the reference point. The values multiplied by offsets d0, d1 (1 and 3 in #col\_major) are called strides.

<sup>1</sup> Its domain can also be multidimensional in general, but we do not support the case.

Linalg Dialect. The linalg dialect contains various loop-like operations on tensors and buffers. linalg operations are more primitive than tosa's and can be performed on buffers.

In linalg, one can represent a generic loop in a structured form using the linalg.generic operation. Each loop explicitly takes input tensors or buffers as its operands. The loop's indexing maps describe which elements are chosen at each iteration. The elements chosen from the inputs at an iteration are represented as input arguments of the loop body region.

The loop body yields a value at each iteration, and the results constitute the output elements. A loop that takes an output buffer writes the resulting elements to the buffer. A loop that takes an output tensor stores the resulting tensor in a new tensor register, which can later be used as another input tensor.

Figure 1(c) shows how to represent %C+%A×%B for three matrices %A, %B, and %C in linalg.generic. %C and the resulting tensor (%output)'s shapes must be the same. The linalg.generic is a triple nested loop that has three induction variables d0, d1, d2. The indexing\_maps describe which elements of the tensors are retrieved in each iteration. The retrieved elements are assigned into block arguments %a, %b, and %c of the loop body. The loop body performs integral multiplication (arith.muli) followed by addition (arith.addi) and yields it to the next iteration which again becomes %c. iterator\_type shows that the third (innermost) loop is a reduction loop because it is doing summation, whereas the two outer loops can be parallelized.

linalg is the source and target dialect of several key transformations. First, tosa's operations can be lowered into the combination of linalg's operations on tensors. Second, bufferization on linalg's operations changes their tensor operands into buffers. Third, the linalg.generic loops can be optimized into fused linalg.generic loops or simpler operations. Fourth, conversions from linalg to lower level dialects yield for loops (affine, scf) or control-flow graphs (standard).

Transformations in MLIR. MLIR provides transformations that (1) convert the input programs written in high-level dialects into the low-level ones, or (2) optimize the input program into more efficient form. Except for those that intentionally change the input program's behavior, transformations must preserve the behavior.

#### 3 Overview

In this section, we introduce mlir-tv, a translation validation framework for MLIR. Like other frameworks [22,33,36,41,42], mlir-tv takes two programs written in the IR and checks whether the transformation is correct. Since mlir-tv targets intraprocedural transformations, functions in the two programs with the same signature are checked pairwisely. mlir-tv relies on an SMT solver to automatically prove that the transformation is correct or find a counterexample if incorrect.

mlir-tv symbolically encodes each MLIR instruction in a function and emits its final state in a logical formula. After encoding the final states of the source and target functions *f*src and *f*tgt, mlir-tv checks a refinement predicate using an SMT solver. The predicate states that for any input state *I* consisting of an initial memory and argument values, *<sup>f</sup>*src(*I*) *<sup>f</sup>*tgt(*I*) must hold where - is a refinement relation between two final states (Sect. 6.3). If the SMT solver finds an input that breaks the refinement, mlir-tv concludes that the compiler transformation is incorrect. If the SMT solver proves that such input does not exist, the transformation is correct.

#### 3.1 Abstraction for Floating-Point Arithmetic

For practical validation of tensor transformations, it is crucial to efficiently represent floating-point (FP) arithmetic in SMT. SMT-LIB 2 formally supports IEEE-754 [10] under the name of the FPA theory [37]. SMT solvers supporting the FPA theory typically simulate the hardware implementation of the FP arithmetic by representing their bits as boolean variables and converting FP operations into boolean expressions (called *bit-blasting*) [13]. Then, the formula can be efficiently solved using their highly optimized SAT solvers.

However, there are two challenges in using the FPA theory to prove transformations on tensors. First, encoding FP arithmetic in SMT is expensive because solvers internally yield large expressions. Also, a significant portion of tensor transformations does not require such precise encoding. For example, bufferization is agnostic to the representation of the underlying values because its goal is *moving* the virtual registers to memory buffers correctly. Second, machine learning compilers want to support transformations that are incorrect under IEEE-754 for performance. We cannot simply rely on FPA in this case because it will invalidate the transformations.

To address these concerns, mlir-tv abstractly encodes the FP operations (Sect. 4). We find an abstract domain for FP numbers that is *specific* to the transformation to validate. It uses over-approximations meaning that a successful validation implies the correctness of the transformation. If it is not validated, mlir-tv refines the abstraction and try validation again (Sect. 6.3).

#### 3.2 The Formal Semantics of Dialects

Since there is no official formal semantics for MLIR dialects yet, we read the textual specification of MLIR dialects and represented them in the encoding function. The function returns the final state in SMT expressions. Therefore, it implicitly defines the big-step formal semantics of the dialects in MLIR. Also, the function contains encoding rules for each instruction, which implicitly represent its small-step semantics.

Note that we are not proposing new formal semantics for unsafe FP arithmetic. We assume that there exists a valid FP semantics that satisfies certain arithmetic properties. The concrete semantics of FP operations is hidden under the uninterpreted functions used for the abstract encoding. The semantics of unsafe FP arithmetic is often explained using nondeterministic execution [11] and encoding it in SMT requires universal quantification which is expensive.

#### 4 Encoding Floating-Point Numbers and Tensors

To overcome the challenges described in Sect. 3.1, we devise an abstract encoding of FP arithmetic tailored for translation validation. In this abstract encoding, an FP number is represented as a bit-vector that is typically smaller than its original bit width. The operations on FP numbers are represented as UFs satisfying arithmetic properties like commutativity. Our encoding does not miss bugs because it is an over-approximation of the FP arithmetic. On the other hand, validation failure does not always mean that the transformation is wrong.

#### 4.1 Abstract Domain of Floating-Point Numbers

We begin with defining an abstract domain for FP numbers that is specific to the transformation to validate. We count the number of distinct FP numbers that are required to express at least one counterexample if the transformation is incorrect. As a result, if it is possible to prove that no counterexample is found in this abstract domain, no concrete counterexample can exist.

Consider a transformation that swaps the two operands of FP addition. An invocation of the source function (top) can observe at most three distinct FP numbers because it has three FP registers %a, %b, and %c\_src. Similarly, the target function (below) can observe at most three different numbers. The number of distinct FP numbers required to validate the transformation is not greater than 4=3+3 <sup>−</sup> 2 since two of those are shared as arguments.

After counting the number, we abstractly represent the values of FP registers and con-

```
// The source function
func @f(%a: f32, %b: f32) {
  %c_src = addf %a, %b: f32
  return %c_src: f32
}
// The target function
func @f(%a: f32, %b: f32) {
  %c_tgt = addf %b, %a: f32
  return %c_tgt: f32
}
```
stants using bit-vectors. For the above example, 2 bits are enough in theory because 4 <sup>≤</sup> 2<sup>2</sup>. We will use notation [[%a]] to represent the abstract bit-vector value of %a. In SMT, two bit-vector variables are declared for %a and %b because they can be any value, and %c\_src and %c\_tgt are defined as expressions with respect to the variables.

Defining Operations. To abstractly define addf, we declare a UF for addition. If the arithmetic properties of addition are ignored, [[addf(%a, %b)]] may be defined as addfSMT([[%a]]*,* [[%b]]) where the definition of UF addfSMT is arbitrarily determined by the SMT solver. Since the solver's goal is to find a counterexample, it will try to find a definition of addfSMT that breaks the transformation. If the solver couldn't find one, the transformation is correct under any definition of FP addition.

Note that validating the above example requires encoding commutativity 'addfSMT([[%a]]*,* [[%b]]) = addfSMT([[%b]]*,* [[%a]])'. Instead of using an expensive universal quantification, we encode addition as 'addf SMT(*x, y*) & addf SMT(*y, x*)' where & is the bitwise and operation and addf SMT is another UF. Without loss of generality, it encodes all possible commutative functions<sup>2</sup>.

To encode the result of operations on <sup>±</sup>0*,* <sup>±</sup>1*,* <sup>±</sup>fMAX (finite max)*,* ±∞ and NaN, we use the ite (if-then-else) expression in SMT. For example, to encode 'NaN + *<sup>y</sup>* = NaN', the expression is wrapped with an ite that checks if one of the inputs is NaN. Combined with the commutativity encoding, the expression for *<sup>x</sup>* + *<sup>y</sup>* becomes as follows. '*<sup>x</sup>* is NaN' is the SMT formula checking *<sup>x</sup>* is NaN by inspecting *x*'s abstract representation which will be described later.

ite(*<sup>x</sup>* is NaN <sup>∨</sup> *<sup>y</sup>* is NaN*,* NaN*,* addf SMT(*x, y*) & addf SMT(*y, x*))

Using UFs and ites, we abstractly encode +, <sup>−</sup>, <sup>×</sup>, */* and *<sup>x</sup><sup>y</sup>*. Subtraction is defined as an addition of the negated second operand. Division is not equivalent to multiplication of the inversed operand due to the existence of subnormal values. Therefore, it is encoded using a separate UF.

Comparisons, |*x*| and −*x* are precisely encoded because our bit-vector representation natively supports them. Their representation will be described below.

Bit-vector Structure. A bit-vector for FP consists of a sign bit (SB) at its most significant bit and magnitude bits (MB) at the entire lesser significant bits. They represent the sign and the order of absolute value of the original number, respectively. Therefore, comparing the magnitudes of two finite FP numbers is equivalent to simply comparing their MBs. If MB[1 *...* <sup>|</sup>MB| − 1] are all set to 1, the original value is ∞ (MB[0] = 0) or NaN value (1). Unlike IEEE-754 [10] which have multiple NaN values per sign, we have one representation per sign<sup>3</sup>.

The bit-vector representation of an FP constant number is a concatenation of the sign bit and magnitude bits which is a bit-vector variable in SMT. The bitvector variables are given preconditions so that a constant with a larger absolute value is guaranteed to have larger MB.

Supporting Floating Point Casts. To support FP casts, MB is further split into three parts: limit bits (LB), truncated bits (TB), and precision bits (PB) in descending significance order. These parts determine the result of casting the value into a smaller FP type. LB represents the overflow condition. If LB is 0, a cast to the smaller size yields a finite value. If not, it yields ±∞. TB represents the magnitude floored to the target type. Its bit width is equivalent to the bit width of MB of the smaller type. PB represents the offset from the floored value. If PB is 0, the value is truncated to the exact value without loss of precision. Otherwise, the value must be rounded, and the direction is determined by a UF

<sup>2</sup> We describe its formal proof in our online supplementary material [5].

<sup>3</sup> We chose this policy because respecting the bits invalidates several transformations in LLVM and the behavior of processors canonicalizing NaN values [33].

returning boolean. Extension is done by copying MB to TB and filling LB and PB with 0<sup>4</sup>.

#### 4.2 Encoding Tensors

In SMT, a tensor is represented as an array expression from the address spacesized bit-vector to the element type. A multidimensional tensor is encoded as a one-dimensional array in row-major order. The dimension sizes of dynamically shaped tensor arguments are encoded as bit-vector variables. The number of elements of a tensor cannot exceed the size of the address space.

For each tensor argument in MLIR function, a new SMT array variable is assigned because its value can be fully arbitrary. The results of tensor operations are encoded as lambda expressions in SMT which is described in Sect. 5.1.

Uninitialized Tensors. A tensor may contain uninitialized elements. In SMT, a tensor carries another boolean array that indicates uninitialized elements.

We define accessing uninitialized elements as an undefined behavior (UB) for the following reason. During bufferization, linalg.init\_tensor operation that returns an uninitialized tensor is lowered into memref.alloc. The memref.alloc operation is then converted into a malloc call in LLVM IR, reading uninitialized bytes of which and using them may raise UB.

Tensor arguments in MLIR are assumed to be fully initialized. linalg's init\_tensor is the only operation that creates an uninitialized tensor. Operations like tensor.insert can create a partially initialized tensor.

#### 4.3 Calculating the Bit Width

The bit width of the abstract representation of FP numbers is decided by the number of float registers and constants. Since all FP registers can store distinct FP numbers, the number of different FP numbers that may appear during the source and target program execution is bounded by the number of FP registers and distinct constants.

However, an operation that does not return an FP number can internally observe an unseen number. For example, suppose is\_int(x) that returns true if float <sup>x</sup> is an integral value. Given an UF floorSMT(*x*) that returns an abstract float with its decimal truncated, this operation can be encoded as '<sup>x</sup> == floorSMT(*x*)', which hides an unseen number in floorSMT(*x*).

Therefore, we count the number of UFs applied to abstract FP numbers while encoding the source and target instructions. The size of the BV field is log2*N* where *N* is the number of applied UFs added by the number of FP arguments as well as distinct constants of the source and target functions. From the above example, is\_int(x) must increment *N* even if it returns boolean because floorSMT(*x*) can return an unseen FP value.

<sup>4</sup> We describe the full encoding of constants of different types and other details in our online supplementary material [5].

Fig. 2. Reducing elementwise tensor operations into scalar operations.

Considering Tensors and Memory. In general, a tensor with *M* elements must increase *N* by *M* because it can have *M* different floats. To reduce the bound, we again rely on the fact that finding only *one* counter-example is enough. If that counter-example is a tensor, one mismatched element is sufficient.

If all tensor operations in functions are elementwise, we can simply ignore tensors' dimensions and count them as FP numbers when evaluating *N*. Consider the example in Fig. 2(a). To validate that transforming the upper f to the lower f is correct, we must check whether %z\_src[i] and %z\_tgt[i] are equal for any i. Therefore, we can rewrite the functions into the form in Fig. 2(b) without affecting the correctness of the transformation. Note that the return types of two functions are changed from tensor to float. Since tosa.add is an instruction that performs addf elementwisely, choosing i from tosa.add only requires i'th elements from its input tensors. Therefore, the functions can again be rewritten as in Fig. 2(c). Since only the i'th elements of tensors %x and %y are used, the functions can again be rewritten to take %x\_i and %y\_i as function arguments instead, which is not depicted in the figure. Therefore, validating the initial pair is equivalent to validating two functions taking and adding two FP numbers.

Given a memref value, one can only access in-bounds locations. Thus, its size is added into *N*. If all tensor operations are elementwise, it is counted as one.

#### 5 Supporting Tensor Operations and Loops

In this section, we introduce the SMT encoding of tensor operations and loops.

#### 5.1 Encoding Tensor Operations

The result of a tensor operation is encoded as a lambda expression in SMT. For example, a negation of tensor <sup>t</sup> is encoded as 'lambda *i,* negate(select(t*, i*))' where *<sup>i</sup>* is a 32 bit-vector variable, 'select(t*, i*)' selects the *<sup>i</sup>*-th element from the SMT array of <sup>t</sup>, and 'negate(*bv*)' is an alias for an SMT expression extracting the sign bit of *bv* and concatenating its negation with its BV bits. Note that it does not check whether *i* is within the bound of the tensor. It is because the values at out-of-bounds indices cannot affect the program's behavior.

For operations returning a multidimensional tensor, the lambda chooses and returns the element in row-major order. For example, transpose of t whose size is *<sup>N</sup>* <sup>×</sup> *<sup>N</sup>* is encoded as 'lambda *i,*select(t*, i*%*<sup>N</sup>* <sup>×</sup> *<sup>N</sup>* + *i/N*)'.

Encoding Reduction Operations. In general, reduction operations like summation of an array cannot be precisely encoded in SMT-LIB 2. To support them, we abstractly encode the reduction operations using UFs. For example, we declare sum which is a UF taking an array and returning a float number. Since this is an over-approximation, the validation may fail. In this case, we perform abstraction refinement, which will be described in Sect. 6.3.

The out-of-bounds elements of an array are wiped out before applying to UF because they must not affect the result. This is done by wrapping the input array with lambda and select. The select returns the value that do not affect the result of the reduction (e.g., <sup>−</sup>0*.*0 for a summation) if the index is out of bounds.

Tensor Operations and Undefined Behavior. The documentation was not clear about the behavior of a program violating the assumptions that tensor operations expect at runtime. The violations include out-of-bounds access, size mismatch of the dynamic-shaped tensors, and reading an uninitialized element. If it is defined as having well-defined side effects such as calling exit, dead tensor operations cannot be freely removed and lowering to LLVM IR whose behavior may be undefined cannot be explained. Therefore, we define them as UB.

#### 5.2 Encoding Loops

In MLIR, linalg loops are typically generated from high-level tensor operations. Compared to loops in general programs, they are simple and syntactically provide rich information. The loop consists of instructions without side-effect (modulo UB), and linalg loops explicitly state input/output tensors' index mappings as well as parallelizable induction variables. Therefore, we can construct the output tensor or buffer without synthesizing loop invariants.

```
#id = affine_map<(d0, d1) -> (d0, d1)>
#transposed = affine_map<(d0, d1) -> (d1, d0)>
// %C = %A + %B^T, %C's shape = %out's shape
%C = linalg.generic {indexing_maps = [#id, #transposed, #id],
                     iterator_types = ["parallel", "parallel"]}
    ins(%A, %B : tensor<?x?xf32>) outs(%out : tensor<?x?xf32>) {
^bb0(%a: f32, %b: f32, %unused: f32):
  %c = arith.addf %a, %b: f32
  linalg.yield %c : f32
} -> tensor<?x?xf32>
```
Consider the above loop that adds tensors %A and %B*<sup>T</sup>* . Indexing maps (#id, #transposed, #id) are mappings from two induction variables (hence a doubly nested loop) to the indices of input (%A, %B) and output (%out) tensors. The loop body shows that the initial value of %out is not used. Since iterations over each dimension have no dependency because they are parallel (iterator\_types), we can conclude that %out[i][j] = %A[i][j] + %B[j][i].

In this section, we propose an encoding of loops in linalg using the lambda theory and a universal quantification. Encoding a loop in linalg starts with finding loop bounds. Loop bounds are determined by matching the ranges of the indexing maps with the tensor (buffer) sizes. Then, the loop body which yields the element of the resulting tensor is encoded. If the output type is tensor, the resulting tensor is encoded in lambda in row-major order. If the output type is buffer, the memory locations are accordingly updated.

For the above example, the yielded result at each iteration is described as a lambda expression with two parameters: 'lambda (*d*0*, d*<sup>1</sup>)*,* add(%A[*d*0*, d*<sup>1</sup>]*,* %B[*d*1*, d*<sup>0</sup>])'. Then, the output tensor %C is encoded as a lambda with a single parameter *<sup>i</sup>*. It selects (*i /N, i* % *<sup>N</sup>*) from the first lambda where *<sup>N</sup>* is %out's width.

Determining Loop Bounds. If the sizes of %A and %B are larger than that of %out, should the linalg.generic raise UB or add parts of the inputs?

To find its valid semantics, the first transformation to consider is linalg's conversion from linalg.generic to a canonical for loop in another dialect. The conversion generates a for loop with the upper bounds of induction variables explicitly given. The conversion sequentially visits the indexing maps, and finds the first dimension that exactly matches. Exact matching means that the range of the indexing map must be identity, not e.g., d0 + 1. If such dimension cannot be found, the linalg.generic is considered syntactically invalid.

The second transformation is the canonicalization of linalg.generic. If a linalg.generic loop iterates over the input tensors and simply returns the elements, its output is replaced with the input tensors regardless of the input/output tensors' shapes. However, if we determine the loop bounds only by the shape of the first matched tensor, this transformation cannot be justified when input tensors have different sizes.

Therefore, we encode the loop bounds of linalg.generic as follows. First, we find loop bounds according to the algorithm of the first transformation (generic to for). For the above example, the upper bounds of d0 and d1 are the dimension's sizes of %A because the first indexing map is for %A. Second, all input tensors' shapes must match the determined loop bounds, otherwise UB. In the case of the above example, %A, %B and %out's shapes must be equal.

Encoding Loops on Buffers. If inputs/outputs are buffers, tensors are loaded from the inputs, the loop is performed on the tensors, and the resulting tensor is stored into the output buffer. The input and output buffers of linalg.generic must be disjoint (Sect. 6.2). If the output buffer's layout map is identity, the output memory block is updated using lambda. If not, a fresh SMT array for the updated block is created, and the equalities between old/new elements of the block and the output tensor are encoded using forall quantifications.

Encoding Reduction Loops. Induction variables which have "parallel" in the iterator\_types attribute must appear as the parameters of the SMT lambda expression. Other variables, however, must be accordingly encoded. To encode reduction loops, we syntactically match the operand of the last yield and use the corresponding UF for the reduction (Sect. 5.1). This worked well in practice because the reduction loops in MLIR had common patterns.

#### 5.3 Supporting Arithmetic Properties of Reductions

Floating-point addition and multiplication are not associative, but programmers sometimes want to boost performance at the expense of precision by allowing compiler optimizations that rely on the property. To encode the property, the definition of addition and multiplication must be different from IEEE-754 because using it causes inconsistency in the underlying logic.

Then, what is the semantics of *<sup>x</sup>* + *<sup>y</sup>* + *<sup>z</sup>*? One possible solution is that its evaluation nondeterministically yields either (*x*+*y*)+*<sup>z</sup>* or *<sup>x</sup>*+(*y*+*z*) [11]. However, encoding the semantics in SMT requires introducing quantified variables.

Therefore, as described in Sect. 5.1, we start from abstractly encoding reduction operations in UFs. For example, UF sum takes an array [*x, y, z*] and returns its summation. A question is how to encode their arithmetic properties like sum([sum([*x, y*])*, z*]) = sum([*x,* sum([*y, z*])]). We introduce a new technique that works when the length of the input array is constant. This technique is not specific to a summation but can be applied to any reduction.

Encoding Commutativity. The first arithmetic property to consider is commutativity: 'sum([...*, x,* ...*, y,* ...]) = sum([...*, y,* ...*, x,* ...])'.

A straightforward solution is to use the multiset theory. Two sums are considered equal if the multisets converted from input arrays are equal. For the solvers that do not support the multiset theory, a multiset can be simulated using an array taking an element and returning its count. However, this multiset-based approach does not scale well (Sect. 7.3). We conjecture that existing algorithms in the solvers are not good at checking the equality of two multisets (cvc5)/counter arrays (Z3).

We suggest a hash-based approach for encoding the multiset equality. Our approach begins with defining a hash function *F* on an array. If two arrays are equal, their hash values must be equal. The inverse holds when the range of *F* is sufficiently large. It only uses the theory of UF and BV, which are cheap.

To define *F*, we define another hash function *f* on floating-point numbers. *<sup>F</sup>*(*A*) is defined as a summation of hash values of its elements - *<sup>x</sup>*∈*<sup>A</sup> <sup>f</sup>*(*x*). By the arithmetic property of bit-vector addition, *<sup>F</sup>*(*A*) = *<sup>F</sup>*(*A* ) if *<sup>A</sup>* is a permutation of *<sup>A</sup>*. The inverse direction also holds. We prove that if *<sup>F</sup>*(*A*) = *<sup>F</sup>*(*A* ) for any *f*, *A* is a permutation of *A*.

Theorem 1. *Given A and A that are arrays of type T, if* ∀*f .* - *<sup>x</sup>*∈*Af*(*x*) = *x*∈*A<sup>f</sup>*(*x*) *where <sup>f</sup>* <sup>∈</sup> *<sup>T</sup>* <sup>→</sup>*BV* (log<sup>2</sup> *max*(|*A*|*,* <sup>|</sup>*A* <sup>|</sup>))*, <sup>A</sup> is a permutation of <sup>A</sup>. Proof.* Let's assume that *count*(*S, x*) is the number of *<sup>x</sup>* in multiset *<sup>S</sup>*. For example, *count*({1*,* 1*,* 3}*,* 1) is 2. We first prove the following lemma.

Lemma 1. *Given two multisets S and S , <sup>S</sup>* = *<sup>S</sup> holds if*

$$\forall g, \left(\sum\_{x \in S} count(S, x) \times g(x)\right) = \left(\sum\_{x \in S'} count(S', x) \times g(x)\right)$$

*where <sup>g</sup>* <sup>∈</sup> *<sup>T</sup>* <sup>→</sup> *BV* (log<sup>2</sup> *max*(|*S*|*,* <sup>|</sup>*S* <sup>|</sup>))*.*

*Proof.* Assume that *<sup>g</sup><sup>k</sup>*(*x*) is a function that returns 1 if *<sup>x</sup>* = *<sup>k</sup>* and 0 otherwise. By picking each element of *<sup>S</sup>* as *<sup>k</sup>* and *<sup>g</sup>* = *<sup>g</sup>k*, *<sup>S</sup>* <sup>=</sup> *<sup>S</sup>* holds. -

Assume that *S* is a multiset from array *A* and *S* from *A* . From the assumption ∀*f .* - *<sup>x</sup>*∈*<sup>A</sup> <sup>f</sup>*(*x*) = - *x*∈*A <sup>f</sup>*(*x*), we can derive ∀*g,* - *<sup>x</sup>*∈*<sup>S</sup> count*(*S, x*) <sup>×</sup> *<sup>g</sup>*(*x*) = - *x*∈*S count*(*S , x*) <sup>×</sup> *<sup>g</sup>*(*x*) . Then, we can apply the lemma. By the conclusion of the lemma, the two multisets are equal, hence *A* is a permutation of *A* . -

For each pair of two sum function calls appearing in the source and target, their equality is encoded as a constraint. Since *<sup>P</sup>* =<sup>⇒</sup> *<sup>Q</sup>* iff <sup>¬</sup>*<sup>Q</sup>* =⇒ ¬*P*, the universal quantification in the Theorem 1 can be converted into an existential form 'sum(*A*) = sum(*A* ) =⇒ ∃*f .* - *<sup>x</sup>*∈*<sup>A</sup> <sup>f</sup>*(*x*) <sup>=</sup> - *x*∈*A <sup>f</sup>*(*x*)'. Since <sup>∃</sup>*<sup>f</sup>* can be moved out, the precondition is quantifier-free.

Encoding Flattening of a Nested Reduction. By expanding the hash function based approach, we can encode the equality between nested reductions. Consider this equality: 'sum([sum(*A*)*,* sum(*B*)]) = sum(*<sup>A</sup>* ++ *<sup>B</sup>*)'.

Since the array [sum(*A*)*,* sum(*B*)] is not a permutation of *<sup>A</sup>* ++ *<sup>B</sup>*, the previous encoding does guarantee that the two summations are equivalent. To support this case, given a hash function *<sup>F</sup>* and summation sum(*A*), we add a precondition *<sup>F</sup>*(sum(*A*)) = - *<sup>x</sup>*∈*<sup>A</sup> <sup>F</sup>*(*x*). That is, the hash value of sum(*A*) is equivalent to the summation of hash values of *x* ∈ *A*.

Note that the hash function is individually defined per a pair of summations in the programs. This causes additional preconditions for each hash pair to relate inner and outer summation. We reduce the number of preconditions by unifying hash functions into one<sup>5</sup>.

#### 6 Encoding Memory and Refinement

MLIR has several dialects providing memory operations, such as memref, affine, and bufferization. We propose a memory model for these dialects. Also, we illustrate our SMT encoding for the model.

<sup>5</sup> Due to the limited space, we prove that the unified hash function's range must not be smaller than [0*, p*<sup>2</sup>*n*) where *p* is the number of summation pairs and *n* is the maximum size of an array in our online supplementary material [5].

#### 6.1 Memory Model

Memory Block. A memory is made up of smaller memory blocks in our memory model. A memory block is a unit of a memory allocation, and is either created by memref.alloc, memref.alloca, clone-like operations of bufferization, or defining a global variable. memref.alloca allocates a block at stack whereas memref.alloc has no such constraint. memref.dealloc frees the block.

A memory block is uniquely identified with a block id. Its properties consist of the number of elements, block type, writability, liveness, and the list of elements with the list of booleans indicating whether each element is initialized. The block type is a boolean value which shows whether it is created by memref.alloc. Allocating instruction creates a new memory block which is initially alive, writable, and fully uninitialized. The clone-like operation marks the source block with permanent read-only. The behavior of accessing a dead block is undefined, and also accessing an uninitialized element is undefined behavior. This decision is described in Sect. 4.2 as well.

Memory Reference. The memref type is a reference to a specific memory area. It consists of the pointing block's id, block offset, layout map, dimension sizes of the pointing area, and a flag indicating whether it is a view reference. A block offset may be non-zero because memref allows creating an aliased reference via memref.view, which may not point to the head of the block. memref may point to an out-of-bounds area of the block, and accessing that area is UB.

Loading a tensor from memref is well-defined if (1) the referenced area is within the bounds of the memory block, (2) the block is alive (i.e. not deallocated yet), and (3) the visited offsets are fully initialized. Writing a tensor is welldefined if the area is in-bounds and the block is alive and writable.

#### 6.2 Encoding the Memory Model

The properties of memory blocks are encoded as SMT variables size, writable, liveness, block\_type, elements, initialized. By default, all properties are defined as SMT variables because we cannot make any assumption on how and when a block is created in general. If the block's definition is visible (e.g., it is a global variable), they are initialized with literals in SMT. elements and initialized are encoded as SMT arrays from the offset to the value and boolean.

The number of blocks necessary to validate the transformation is determined via static analysis, which is described in [30]. The number is bounded because we do not support loops containing allocating operations. This works in practice because allocations are usually located outside of the loops. After the analysis, each block and property declares one SMT variable. The blocks for global variables and allocating operations are assigned constant block ids.

Local and Non-local Blocks. We adopt the notion of local and non-local blocks from [30]. Local blocks are created by the allocating instructions that belong to the validated function, whereas non-local blocks are not. Only the non-local blocks are checked at the refinement of final states. We do not consider escaped local blocks because (1) memref cannot have memref as its element type, and (2) we do not support call instructions.

Encoding Memory Access. The SMT encodings of memory load/store operations follow the encodings described in [30]. The result of loading a value from memref %m is encoded as ite(%m*.*bid = 0*,* arr<sup>0</sup>[%m*.*ofs]*,* ite(%m*.*bid = 1*,...*)) where arr<sup>0</sup> has the elements of memory block 0. Storing a value to memref updates the elements of possibly aliased blocks with ites.

Encoding disjointnesses of two memref accesses – which is required by several buffer operations – is hard in general because a memref can point to noncontiguous locations in arbitrary patterns. Therefore, we support encoding a disjointness of memrefs with trivial, row-major layout maps only, raising an error otherwise.

#### 6.3 Compiler Correctness and Abstraction Refinement

Finally, we compare the final states of the source and target functions. A final state is defined as (*ub, m, v*) where *ub* is UB, *<sup>m</sup>* is the memory, and *<sup>v</sup>* is the return value. A final state refines another, or (*ub, m, v*) - (*ub , m , v* ), if (1) *ub* is true, or (2) *ub* = *ub* <sup>∧</sup> *<sup>v</sup>* <sup>=</sup> *<sup>v</sup>*<sup>6</sup> <sup>∧</sup> *<sup>m</sup>* refines *<sup>m</sup>* . A memory *m* refines *m* if for non-local blocks (*b, b* ) with same id in the source and target, if (1) reading *<sup>b</sup>* at offset *o* is successful, so does the access to *o* at *b* , and (2) if *b* is writable, so does *b* . For any input state *I* consisting of an initial memory and argument values, *<sup>f</sup>*src(*I*) *<sup>f</sup>*tgt(*I*) must hold where *<sup>f</sup>*(*I*) denotes the final state of function *<sup>f</sup>*. In SMT, the formula is inverted to remove the outermost quantification.

Abstraction Refinement. To make validations cheap on average, we progressively refine the abstraction scheme that describes the abstraction level of encodings. Abstraction refinement happens when a validation fails or timeouts.

In the first round, the integer and FP dot operations are encoded using independent UFs which are not related to a summation. Also, FP numbers of different types are independently encoded and casts are defined as full UFs. If validation fails, the dot operations are encoded as a composition of a summation and multiplications, and the encoding for casts described in Sect. 4.1 is used. If this also fails, summations of arrays having small constant numbers of elements are unrolled into a sequence of additions and validated again. This validates, for example, folding sum([1*.*0*,* 2*.*0]) into 1*.*0+2*.*0.

<sup>6</sup> For floats, NaNs of different signs are considered equivalent. For memrefs, we do not support references to local blocks because it needs universal quantifications [30]. Validating functions returning such values may result in false alarms.

Our abstraction cannot validate the constant folding optimization in general. To address this, mlir-tv provides a command-line option for using IEEE-754. It disables the unsafe properties on reductions because they are not compatible.

#### 7 Implementation and Evaluation

mlir-tv consists of 8,900 lines of C++ codes. It supports 25 tosa ops, 11 memref ops, 13 linalg ops, 10 tensor ops, 29 arith ops, 3 bufferization ops, and 8 other ops. mlir-tv uses Z3 4.8.13 and cvc5 0.0.3 as a solver, with 30 s timeout. The experiments in this section are performed using Z3 because Z3 showed better performance than cvc5 in mlir-tv's sanity tests. We used the Apple M1 CPU and 16 GB RAM with a fixed version of MLIR (b5a0f0f, 26/Dec).

We wrote 57 function pairs to check that it validates correct transformations and finds counterexamples for wrong pairs. From these tests, we observed that using the abstract encoding was 13.6x faster on average than the concrete IEEE-754 encoding. Shrinking the bit width of abstract FP (Sect. 4.3) was important because it brought 2.2x speedup compared to simply using 32 bits.

#### 7.1 Validating MLIR Unit Tests

We validated the unit tests in the official MLIR project using mlir-tv. The unit tests (1) apply specific transformations to small, pre-defined MLIR programs, and (2) check whether the output programs syntactically match the test patterns. Using mlir-tv, we validated that the outputs of the transformations preserved the semantics of the inputs as well. We bounded the size of dynamic-shaped tensors to 100 to avoid timeouts. Bugs in tests with such tensors may have been missed.

Among the MLIR's unit tests, which consist of 2,467 function pairs in total, mlir-tv validated 433 tests, raised timeout for 8 tests, and failed for 8 tests. Validating the tests did not require encoding the unsafe arithmetic properties for reductions, but we are aware of uncovered transformations that require them.

We could find several issues in the semantics of MLIR dialects.

Signed Zero, NaN and *−∞.* The tosa.conv2d and tosa.depthwise\_conv2d operations are lowered to linalg.pad\_tensor with the input tensors padded with +0*.*0. Also, we found that MLIR was folding *<sup>x</sup>*+(+0*.*0) into *<sup>x</sup>*. However, this is incorrect since (1) *<sup>x</sup>*+(+0*.*0) = *<sup>x</sup>* if *<sup>x</sup>* = <sup>−</sup>0*.*0 and (2) the tosa specification [6] states that an FP type must support signed zero. This problem was also found from tosa.fully\_connected and tosa.reduce\_sum operations whose lowered loops fill the initial tensors with +0*.*0. We reported these issues to the LLVM community. After the report, *<sup>x</sup>* + (+0*.*0) <sup>→</sup> *<sup>x</sup>* folding has been fixed [1]. We also found that lowering tosa.clamp and tosa.max\_pool2d does not preserve their outputs if the inputs contain a NaN and −∞ value.

Fig. 3. (a) A graph showing the effectiveness of our encoding of unsafe arithmetic properties of reduction operations. The numbers below the labels indicate the sizes of input tensors. The Y-axis shows the running time of mlir-tv. Timeout is 30,000 ms. (b) Descriptions of the test cases. *X* is a 1D or 2D tensor, Σ is a summation, rev is a reverse, *·* is a dot product, and ++ is a concatenation.

memref Operations and Read-Only Blocks. We couldn't find a good semantics for linalg.fill with a memref reference to a read-only memory block given as its operand. If linalg.fill with a read-only memref raises UB, it cannot explain the linalg-bufferize transformation because it creates linalg.fill with its memref operand pointing to a read-only block. If it is well-defined, it cannot explain the linalg-generalize-named-ops transformation because this converts linalg.fill into a loop storing a value to the pointer.

We found that buffer-deallocation transformation was introducing UB. It inserts memref.dealloc to free the unused result of memref.clone. But, mutating the result of clone is UB according to the specification.

Also, it was not clearly stated in the document when memref.clone makes the referenced location read-only. We discussed this issue in the online LLVM Discussion Forums, and the document was fixed to clearly state that it is immediately after the operation that the block becomes read-only [2].

#### 7.2 Validating Compilation of Deep Learning Models

We compiled the TFLite models of text\_classification\_v2, SqueezeNet [25] and MobileNet [24], taken from the official TensorFlow website [7,8], by running them through tosa-to-linalg, tosa-to-standard, canonicalize, fuse-elementwise-ops, tensor-constant-bufferize, linalg-bufferize, tensor-bufferize transformations. To address validation failures of tosato-linalg due to the problem in tosa's <sup>±</sup>0*.*0 handling, we tweaked mlir-tv so that it recognizes +0*.*0 used by certain operations as <sup>−</sup>0*.*0 instead.

To validate them in a reasonable time, we split the source and target programs into smaller functions. Since the networks did not have complex control flows other than loops, splitting was not very hard. The split functions contain an average 9.5 to 11.6 instructions. All transformations were validated correctly in text\_classification\_v2, but the last two transformations were failed in the other models since they have unsupported operations.

#### 7.3 Performance Evaluation of Hash-Based Encoding

We compared the performance of our hash-based encoding to the multisetbased encoding. In the latter encoding, two reductions are assumed to be equal if the multisets converted from the input arrays are equal. For cvc5, we used its native multiset theory. For Z3, we simulated multisets by defining an array that counts the numbers of elements. We set QF\_AUFBV logic to Z3 by default and used ALL logic only when the solver failed. For cvc5, we used HO\_AUFBV and HO\_ALL

Fig. 4. Running times of dot-RV by tensor size. Timeout is 30,000 ms.

logic respectively. We ran tests 10 times and calculated their average execution times. The timeout was set to 30 s.

Our hash-based encoding was faster than multiset-based encoding in overall cases (Figs. 3 and 4). cvc5's multiset theory performed better than the Z3's array encoding, but was still slower than the hash-based encoding. The hash-based encoding showed consistent running time regardless of tensor size.

#### 8 Related Work

Verifying Programs with Floating-Points. Strategies for verifying programs using FP arithmetic (FPA) vary with their goals and background theories. Several works using abstract interpretation [29], SMT solvers [23,40] or computer algebra systems [31] target checking round-off errors of FP operations automatically. Axiomatizing and verifying FPA in theorem provers [12] enable us to make analysis sound and complete, but they require significant efforts.

To realize bit-precise FP reasoning in SMT, one can use a bit-vector representation of FP numbers (bit-blasting). Since bit-blasting can generate large and complex formulae, researchers have tried to find better FP abstraction. [15] presents an abstraction technique using either large or reduced precision of FPA. UppSAT [44] proposes an abstraction framework including fixed-point and real arithmetic. SymFPU [13] gives an effective yet correct bit-vector encoding of FPA considering various types and special cases of IEEE 754.

Verifying Programs Using Arrays. Several works have proposed their approaches to embedding the theory of arrays [34] into SMT solvers. [21,35] consider array read and write terms as uninterpreted functions, and regard the theory of array as axioms. FreqHorn [20] and SPACER [26] utilize constrained Horn clauses (CHC) engines. [27] analyzes array programs with broader theories by translating the axioms of the theory of array into the CHC format.

Yet another approach uses mathematical induction-based techniques to reason about array-manipulating programs with loops. [16] verify the validity of a given parameterized Hoare triple where the length of array N is used as a parameter of the pre- and post-condition.

Machine Learning Compilers. Optimizing deep learning specific workloads has been a major working field for both hardware vendors [3,4] and software developers. [9,19] These frameworks translate neural-net representations in several frameworks into high-level computation graphs. Then, they optimize the graphs via well-known optimizations such as operator fusion or data layout transformation. Recent works allow optimization of dynamic workloads [38,45] and supports optimizations for heterogeneous systems [43].

[39] surveyed the bugs in DL compilers. They reported that the high-level IR transformations are the most buggy ones and stated that finding wrong code generation is challenging and should receive more attention.

Compiler Verification. [11] relaxes FPA semantics since a compiler can ignore strict IEEE-754 behavior like fast-math optimizations in LLVM. They propose Icing which is a language allowing IEEE 754-unsafe FPA optimizations, and CakeML [28] which is a verified compiler with the optimizations. [32] proposes a verified tensor optimizer whose optimizations can be explored via Coq's tactics.

As for translation validation (TV), [18] proposes a practical TV framework for Halide which is a language for processing arrays. To support fast-math optimizations, it mainly uses Z3's type for real numbers. For general-purpose compilers, many different tools have been developed [36]. Alive2 [33], LLVM-MD [42] and Peggy [41] validate the transformations in LLVM using various techniques. The SMT memory model for Alive2 [30] uses a technique that is similar to our approach in order to bound the number of memory blocks. Some TV tools [17,22] split the original programs and validate the smaller pairs.

#### 9 Conclusion

We propose mlir-tv, an SMT-based translation validation framework for MLIR. It abstractly encodes the FP arithmetic and reduction operations in SMT. Since the abstraction is an over-approximation, mlir-tv does not miss bugs unless a flag for bounding the size of dynamically shaped tensors is given. If validation fails, mlir-tv tries again with refined abstractions. We also propose a hash-based approach for encoding arithmetic properties of reductions, which outperformed a multiset-based one. mlir-tv found several mismatches between the specification and implementation of MLIR from the unit tests. Finally, mlir-tv validated high-level transformations for three pretrained DL models.

Acknowledgements. The authors thank Chung-Kil Hur, Nuno P. Lopes and anonymous reviewers for their feedbacks on previous versions of this paper. We thank Jiun Kim for helping our experiments by writing useful scripts. We also thank Hong-Seok Kim, Jieung Kim, Hyunchul Park, Inho Seo and engineers from Google for their precious comments. Seunghyeon Nam, Inwhan Chun and Ho Young Jhoo were supported by the Basic Science Research Program through the National Research Foundation of Korea under Project Number 2020R1A2C2011947.

#### References


Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Verifying Fairness in Quantum Machine Learning**

Ji Guan1(B) , Wang Fang1,2, and Mingsheng Ying1,3

<sup>1</sup> State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China {guanj,fangw,yingms}@ios.ac.cn <sup>2</sup> University of Chinese Academy of Sciences, Beijing 100049, China <sup>3</sup> Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China

**Abstract.** Due to the beyond-classical capability of quantum computing, quantum machine learning is applied independently or embedded in classical models for decision making, especially in the field of finance. Fairness and other ethical issues are often one of the main concerns in decision making. In this work, we define a formal framework for the fairness verification and analysis of quantum machine learning decision models, where we adopt one of the most popular notions of fairness in the literature based on the intuition—any two similar individuals must be treated similarly and are thus unbiased. We show that quantum noise can improve fairness and develop an algorithm to check whether a (noisy) quantum machine learning model is fair. In particular, this algorithm can find bias kernels of quantum data (encoding individuals) during checking. These bias kernels generate infinitely many bias pairs for investigating the unfairness of the model. Our algorithm is designed based on a highly efficient data structure—Tensor Networks—and implemented on Google's TensorFlow Quantum. The utility and effectiveness of our algorithm are confirmed by the experimental results, including income prediction and credit scoring on real-world data, for a class of random (noisy) quantum decision models with 27 qubits (2<sup>27</sup>-dimensional state space) tripling (2<sup>18</sup> times more than) that of the state-of-the-art algorithms for verifying quantum machine learning models.

**Keywords:** Quantum Machine Learning · Fairness Verification · Quantum Noise · Quantum Decision Model

#### **1 Introduction**

**Quantum Machine Learning**: Google's quantum supremacy (or advantage) experiment demonstrated that a quantum computer *Sycamore* with 53 noisy superconducting qubits can do a specific calculation, namely sampling, in 200 s that would take (arguably) 10,000 years on the largest classical computer using existing Algorithms [1]. More recently, a quantum computer *Jiuzhang* with 76 noisy photonic qubits was used to perform a type of Boson sampling in 20 s that would require 600 million years for a classical computer [2]. These experiments mark the beginning of the Noisy Intermediate-Scale Quantum (NISQ) computing era, where quantum computers with tens-to-hundreds of qubits become a reality, but quantum noise still cannot be avoided.

Quantum machine learning is believed to be a far frontrunner in setting a path for practical beyond-classical applications of NISQ quantum devices. This stimulates the fast development of various quantum machine learning (see [3] for a review). Stepping into industries, Google recently built up a framework *TensorFlow Quantum* for the design and training of quantum machine learning within its well-known classical machine learning platform—*TensorFlow* [4].

Classical machine learning has led to automated decision models assuming a significant role in making real-world decisions, especially in finance [5]. Such (financial) decision tasks are known to face the curse of dimensionality as there are too many features available to model customers/users. Principal component analysis (PCA) is one of the most popular methods for dimensionality reduction. It was recently shown that quantum PCA Algorithm [6] can run exponentially faster on a quantum processor. At the same time, the training process of quantum machine learning could be sped up exponentially (compared with classical training) by using quantum PCA to implement iterative gradient descent methods for network training [7]. It is worth noting that this quantum approach is generic in the sense that it can be applied to various types of neural networks, including shallow, convolutional, and recurrent networks, and thus can mitigate the high complexity issue of classical training. Because of these reasons, quantum machine learning has been introduced to be applied independently or embedded in classical decision-making models, e.g. fraud detection (in transaction monitoring) [8,9], credit assessments (risk scoring for customers) [10,11], and recommendation systems for content dissemination [12] (see reviews [13,14] for more information). Similar to the classical counterparts, the quantum models are trained on individuals' information, e.g. saving, employment, salary (encoded as quantum data).

**Fairness in Machine Learning**: It is well-known that classical decision models are prone to discriminating against users/consumers on the basis of characteristics such as race and gender [15], and have even led to legal mandates of ensuring *fairness*. To develop fair models, various attempts have been made to precisely define and quantify fairness. They broadly fall into two categories: *group* and *individual* fairness. Group fairness aims to achieve through statistical parity the same outcomes across different protected groups (e.g. gender or race) [16,17], whereas individual fairness advocates treating similar individuals similarly (receiving the similar outcomes) [18] (see [19,20] for various definitions of fairness and discussions about their relationship). The computer science community has endeavoured to check and avoid bias in classical decision models in the sense of different types of fairness (e.g. [18,19,21]). In particular, several verifiers for formal analysis and fairness verification have been designed and implemented, including FairSquare [22], VeriFair [23] and Justicia [24].

Inevitably, the same issue of fairness arises in the quantum models too. Furthermore, as quantum machine learning is principled by quantum mechanics, which is usually hard to explain to the end-users, it is even more important to verify fairness when a decision is made by a quantum machine learning algorithm. However, to the best of our knowledge, the verification problem of fairness in quantum algorithms has not yet been touched.

**Contributions of this Paper**: In this work, we define a formal framework so that the fairness of quantum machine learning decision models can be verified and analyzed in a principled way. Our *design decision* is as follows: we focus on individual fairness—*treating similar individuals similarly* [18]. The trace distance—one of the most widely used quantities in quantum information [25, Section 9.2]—is chosen as the metric for measuring the similarity of quantum data (individuals) in defining fairness. Our main technical contributions include:


#### **1.1 Related Works and Challenges**

To put our work in an appropriate context, let us further discuss some related works and the challenges we face in this paper.

**Classical Versus Quantum Models:** In order to identify and mitigate the bias of classical machine learning decision models, an algorithm for maximizing utility with fairness guarantee was proposed [18]. Then the strategy of searching input data with linear and integral constraints is employed in a verifier for proving individual fairness of a given decision model [21]. The verifier is sound but not complete in general. But in the case of linear models, it is exact (both sound and complete) if the worst-case exponential time is allowed. However, although quantum decision models are always linear, the above technique cannot be directly generalized from the classical case to the quantum case. The main obstacle here is that the corresponding constraints in the quantum models are nonlinear, and thus searching the data set in a linear domain is ineffective in the quantum case. In this paper, we surmount this obstacle by reducing the quantum fairness verification problem to determining the distinguishability of a quantum measurement, which is independent of input data. Then we resolve the latter by eigenvalue analysis with polynomial time in the dimension of input quantum data. As a result, our algorithm is exact (sound and complete) and efficient.

**Fairness Versus Robustness:** As in the classical case, the individual fairness considered in this paper can be thought of as a kind of global robustness [21]. This will be formally discussed in Sect. 3. In the last few years, quite a few papers have been devoted to (adversarial) robustness verification of quantum machine learning (e.g. [26–28]), where a verifier is given a nominal input quantum datum and it checks robustness in a neighborhood of that particular input datum. However, the techniques developed in these works cannot be directly generalized to solve our problem of fairness verification, because we are required to check a global property. Instead, we transfer the impact of the evolution of the quantum machine learning model on input quantum data to quantum measurements.

**Efficiency:** As the dimension of input data increases exponentially with the number of qubits, efficiency is always a key issue in the verification of quantum machine learning models. The state-of-the-art algorithms for robustness verification mentioned above can only cope with quantum machine learning models with 9 qubits<sup>1</sup>. In this paper, we boost the scale up to 27 qubits on a small server, which represents a big step toward the demand in practical applications of NISQ devices (≥50 qubits). The speedup originates from not only the high efficiency of our algorithm but also the based data structure we adopted—*Tensor Network* [29]—which can exploit the locality and regularity of the underlying circuits of quantum decision models and thus further optimize the algorithm.

#### **2 Quantum Decision Models**

For convenience of the reader, in this section, we review the setup of quantum (machine learning) decision models in their most basic form.

**Classical Models:** In the classical world, a *classification decision model* is a mapping <sup>f</sup><sup>c</sup> : C→O, where <sup>C</sup> is a set of data to be classified, and <sup>O</sup> is a set of outcomes corresponding to the classes we are interested in; for example

<sup>1</sup> The experiments of [26] were performed on a personal computer and the size is at most 8 qubits. We have estimated and tested the same experiments on the server we used in this paper and only 9 qubits can be handled.

<sup>O</sup> <sup>=</sup> {0, <sup>1</sup>} in the simplest non-trivial (binary) case. Such a model <sup>f</sup><sup>c</sup> can be generalized to be a randomized mapping <sup>f</sup><sup>r</sup> : C→D(O), where <sup>D</sup>(O) denotes the set of probability distributions over <sup>O</sup>. <sup>f</sup><sup>r</sup> is known as a *regression decision model* to predict distributions and naturally describes a randomized classification procedure: to classify <sup>x</sup> ∈ C, choose an outcome <sup>o</sup> ∈ O according to the distribution fr(x). For example, o is chosen as the outcome corresponding to the maximum probability of fr(x). Therefore, the basic form of a classical decision model is a randomized mapping f = f<sup>r</sup> (f = f<sup>c</sup> when f is degenerated to be a deterministic mapping).

**Fig. 1.** Noisy Quantum (Machine Learning) Decision Model

**Quantum Models:** Due to the statistical nature of quantum mechanics, a quantum decision model is inherently a randomized mapping A : D(H) → D(O). Here D(H) is the set of *quantum states* (data) and to be specific later. Inspired by the classical models, A is not predefined but initialized as A<sup>θ</sup> by a parameterized quantum circuit <sup>E</sup><sup>θ</sup> (see Fig. 1) with a set of free parameters <sup>θ</sup> <sup>=</sup> {θj}<sup>L</sup> <sup>j</sup>=1. Following the training strategy of classical machine learning, A<sup>θ</sup> is trained on a set of input quantum states (training dataset) by tuning θ subject to some loss function <sup>L</sup>(θ).

In the following, we explain the noisy quantum decision model from the left side to the right one of Fig. 1. For the details of the training process, we refer to a comprehensive review paper [30].

**Input State** *ρ* **:** The input state of the model is a quantum *mixed state* ρ, which is mathematically modelled by a positive semi-definite complex matrix, written as <sup>ρ</sup> <sup>≥</sup> 0, with unit trace<sup>2</sup>. <sup>ρ</sup> admits a decomposition form<sup>3</sup>: <sup>ρ</sup> <sup>=</sup> - <sup>k</sup> <sup>p</sup>kψ<sup>k</sup>

<sup>2</sup> ρ has unit trace if tr(ρ) = 1, where trace tr(ρ) of ρ is defined as the summation of diagonal elements of ρ.

<sup>3</sup> This kind of decomposition is generally infinitely many, and one instance is eigendecomposition, i.e., p<sup>k</sup> and |ψkare eigenvalues and eigenvectors of ρ, respectively.

where {pk} is a probability distribution and each <sup>ψ</sup><sup>k</sup> is a rank-one positive semidefinite matrix, i.e., <sup>ψ</sup><sup>k</sup> <sup>=</sup> <sup>|</sup>ψkψk|. Here, <sup>|</sup>ψk is a unit vector and ψk<sup>|</sup> is the entry-wise conjugate transpose of <sup>|</sup>ψk, i.e., ψk<sup>|</sup> <sup>=</sup> <sup>|</sup>ψk†. Physically, <sup>|</sup>ψk represents a *pure state*, and <sup>ρ</sup> represents an ensemble {(pk, <sup>|</sup>ψk)}k, often called a mixed state, meaning that <sup>ρ</sup> is at <sup>|</sup>ψk with probability <sup>p</sup>k. In particular, if <sup>ρ</sup> <sup>=</sup> <sup>ψ</sup> for some pure state <sup>|</sup>ψ, then the ensemble is deterministic; that is, it is degenerated to a singleton {(1, ψ)}. In general, the statistical feature of <sup>ρ</sup> may result from quantum noise, which is unavoidable in the current NISQ era, from the surrounding environment.

*Example 1 (Qubits – Quantum Bits).* A pure state of a single qubit q is described by a 2-dimensional unit vector and in the Dirac notation it can be written as:

$$|\psi\rangle = \begin{pmatrix} a \\ b \end{pmatrix} = a|0\rangle + b|1\rangle \text{ for } |0\rangle = \begin{pmatrix} 1 \\ 0 \end{pmatrix}, |1\rangle = \begin{pmatrix} 0 \\ 1 \end{pmatrix} \text{ and } |a|^2 + |b|^2 = 1,$$

and ensembles {( <sup>1</sup> , <sup>|</sup>0),( <sup>1</sup> , <sup>|</sup>+)} and {( <sup>1</sup> , <sup>|</sup>1),( <sup>5</sup> , <sup>|</sup>φ)} of <sup>q</sup> are represented by the same 2-by-2 mixed state

$$\rho = \frac{1}{4} \begin{pmatrix} 3 \ 1 \\ 1 \ 1 \end{pmatrix} = \frac{1}{2} |0\rangle\langle 0| + \frac{1}{2} |+\rangle\langle +| = \frac{1}{6} |1\rangle\langle 1| + \frac{5}{6} |\phi\rangle\langle \phi|,$$

where |+ = <sup>√</sup> 1 <sup>2</sup> (|0 <sup>+</sup> <sup>|</sup>1) and <sup>|</sup>φ <sup>=</sup> <sup>√</sup> 1 <sup>10</sup> (3|0 + |1).

For a system of multiple qubits q1, ..., qn, the state space is a 2<sup>n</sup>-dimensional Hilbert (linear) space, denoted by H. As a result, pure and mixed states on H are 2<sup>n</sup>-dimensional unit vectors and 2<sup>n</sup> <sup>×</sup> <sup>2</sup><sup>n</sup> positive semi-definite matrices with unit trace, respectively. It is worth noting that *the dimension* 2<sup>n</sup> *of the state space* <sup>H</sup> *of quantum states is exponentially increasing with the number* <sup>n</sup> *of qubits*. Thus, describing a quantum system with a large number of qubits and verifying its properties on a classical computer is challenging. For our purpose of verifying fairness in quantum machine learning, we adopt a compact data structure—*Tensor Networks*—to mitigate this issue (see this in Sect. 6).

**Parameterized Quantum Circuit** *E<sup>θ</sup>* **:** Several different types of parameterized quantum circuits have been proposed; e.g. quantum neural networks (QNNs) [31] and quantum convolutional neural networks (QCNNs) [32]. Basically, E<sup>θ</sup> consists of a sequence of quantum operations: E<sup>θ</sup> = Ed,θ<sup>d</sup> ◦ ··· ◦ <sup>E</sup>1,θ<sup>1</sup> . For each input quantum state <sup>ρ</sup>, the output of the circuit is <sup>E</sup>θ(ρ) = <sup>E</sup>d,θ<sup>d</sup> (... <sup>E</sup>2,θ<sup>2</sup> (E1,θ<sup>1</sup> (ρ))). In the current NISQ era, each component <sup>E</sup>i,θ<sup>i</sup> is:


Note that in constructing a quantum machine learning model, only quantum gate Ui,θ<sup>i</sup> is parameterized, and noises E<sup>i</sup> are not because they come from the outside environment.

It should be pointed out that, in a practical model, as shown in Fig. 1, each quantum operation E = Ei,θ<sup>i</sup> non-trivially applies on one or two qubits. For example, if <sup>E</sup> only works on the first qubit, then <sup>E</sup> <sup>=</sup> <sup>E</sup><sup>1</sup> <sup>⊗</sup> id<sup>2</sup> <sup>⊗</sup> ... <sup>⊗</sup> id<sup>n</sup> and <sup>E</sup>(ρ1⊗ρ2⊗...⊗ρn) = <sup>E</sup>1(ρ1)⊗ρ2⊗...⊗ρn, where <sup>ρ</sup><sup>i</sup> is the mixed state applied on qubit <sup>q</sup><sup>i</sup> and tensor product <sup>ρ</sup><sup>1</sup> <sup>⊗</sup>ρ<sup>2</sup> <sup>⊗</sup>...⊗ρ<sup>n</sup> is the joint state of multiple qubits q1,...,qn. This locality feature will be exploited by Tensor Networks to optimize our verification algorithm for fairness in the Evaluation Section—Sect. 6.

*Example 2.* Consider the 1-qubit noise model: <sup>E</sup><sup>U</sup> (ρ) = (1 <sup>−</sup> <sup>p</sup>)<sup>ρ</sup> <sup>+</sup> pU ρU† where <sup>0</sup> <sup>≤</sup> <sup>p</sup> <sup>≤</sup> 1 is a probability and <sup>U</sup> is a unitary matrix. It includes the following typical noises depending on the choice of U: U = X for bit flip, U = Z for phase flip and U = Y = ıXZ for bit-phase flip [25, Section 8.3], where I, X, Y, Z are *the Pauli matrices*:

$$X = \begin{pmatrix} 0 \ 1 \\ 1 \ 0 \end{pmatrix}, \ Y = \begin{pmatrix} 0 \ -i \\ i \ 0 \ \end{pmatrix}, \ Z = \begin{pmatrix} 1 & 0 \\ 0 & -1 \\ \cdot & \cdot \end{pmatrix}, \ I = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ \cdot & \cdot \end{pmatrix},$$

where ı denotes imaginary unit. The depolarizing noise combines the above three kinds of noise: <sup>E</sup>D(ρ) = (1 <sup>−</sup> <sup>p</sup>)<sup>ρ</sup> <sup>+</sup> <sup>p</sup> <sup>I</sup> <sup>2</sup> = (1 <sup>−</sup> <sup>3</sup><sup>p</sup> <sup>4</sup> )<sup>ρ</sup> <sup>+</sup> <sup>p</sup> <sup>4</sup> (XρX <sup>+</sup> Y ρY <sup>+</sup> ZρZ).

**Measurement** *{Mi}<sup>i</sup>∈O* **:** At the end of parameterized quantum circuit Eθ, we cannot directly read out the output <sup>E</sup>θ(ρ). The only way allowed by quantum mechanics to extract classical information from <sup>E</sup>θ(ρ) is through a quantum measurement, which is mathematically modeled by a set {Mi}<sup>i</sup>∈O of matrices with O being the set of possible outcomes and - <sup>i</sup>∈O <sup>M</sup>† <sup>i</sup> <sup>M</sup><sup>i</sup> <sup>=</sup> <sup>I</sup>. This observing process is probabilistic: for the measurement on state <sup>E</sup>θ(ρ), an outcome <sup>i</sup> ∈ O is obtained with probability <sup>p</sup><sup>i</sup> = tr(MiEθ(ρ)M† <sup>i</sup> )<sup>4</sup>. Therefore, the output of quantum machine learning model <sup>A</sup><sup>θ</sup> upon an input <sup>ρ</sup> is a probability distribution <sup>A</sup>θ(ρ) = {p<sup>i</sup> : <sup>p</sup><sup>i</sup> = tr(MiEθ(ρ)M† <sup>i</sup> )}, as depicted at the rightmost of Fig. 1.

In this paper, we focus on the well-trained quantum machine learning models (i.e., <sup>θ</sup> has been tuned), so we ignore the <sup>θ</sup> in <sup>E</sup><sup>θ</sup> and <sup>A</sup>θ. Now, we can formally specify quantum decision model A as follows:

**Definition 1.** *A quantum decision model* <sup>A</sup> = (E, {Mi}<sup>i</sup>∈O) *is a randomized mapping:*

$$\mathcal{A}: \mathcal{D}(\mathcal{H}) \to \mathcal{D}(\mathcal{O}) \qquad \mathcal{A}(\rho) = \{ \text{tr}(M\_i \mathcal{E}(\rho) M\_i^\dagger) \}\_{i \in \mathcal{O}} \quad \forall \rho \in \mathcal{D}(\mathcal{H}),$$

*where* <sup>E</sup> *is a super-operator on Hilbert space* <sup>H</sup>*, and* {Mi}<sup>i</sup>∈O *is a quantum measurement on* H *with* O *being the set of measurement outcomes (classical information) we are interested in.*

<sup>4</sup> After measuring <sup>E</sup>θ(ρ) with outcome <sup>i</sup> ∈ O, the state <sup>E</sup>θ(ρ) will be collapsed (changed) to ρ- <sup>i</sup> = MiEθ(ρ)M† <sup>i</sup> /pi. As we can see, the post-measurement state ρ- i is dependent on the measurement outcome i. This special property is vitally different from the classical computation.

Like their classical counterparts, quantum decision models are usually classified into two categories: *regression* and *classification* models. Regression models generally predict a value/quantity, whereas classification models predict a label/class. More specifically, a regression model A<sup>R</sup> uses the output of A directly as the predicted value of the regression variable <sup>ρ</sup> ∈ D(H). That is <sup>A</sup>R(ρ) = <sup>A</sup>(ρ) for all <sup>ρ</sup> ∈ D(H). In the classical world, regression models have been successfully applied to many real-world applications, such as stock market prediction and object detection. Quantum regression models were recently used to predict molecular atomization energies [33] and the demonstration of IBM's programming platform—Qiskit [34, Variational Quantum Regression]. On the other hand, classification model A<sup>C</sup> further uses the measurement outcome probability distribution <sup>A</sup>(ρ) to sign a class label on the input state <sup>ρ</sup>. The most common way is as follows:

$$\mathcal{A}\_{\mathcal{C}} : \mathcal{D}(\mathcal{H}) \to \mathcal{O} \qquad \mathcal{A}\_{\mathcal{C}}(\rho) = \arg\max\_{i} \mathcal{A}(\rho)\_{i} \quad \forall \rho \in \mathcal{D}(\mathcal{H}), i \in \mathcal{O},$$

where <sup>A</sup>(ρ)<sup>i</sup> denotes the <sup>i</sup>-th element of distribution <sup>A</sup>(ρ). Classical classification models have broad applications in our daily life, such as face recognition and medical image classification. Quantum classification models have been used to implement quantum phase recognition [32] and cluster excitation detection [4] from real-world physical problems, and fraud detection [8] in finance.

As we saw above, although classical and quantum decision models <sup>f</sup> and <sup>A</sup> are both randomized mappings, the input data to them and their procedure of processing the data are fundamentally different. These differences make that the techniques for verifying classical models cannot be directly applied to quantum models and we have to develop new techniques for the latter.

#### **3 Defining Fairness**

As discussed in the Introduction, an important issue in classical machine learning is: how fair is the decision made by machines. The same issue exists for quantum machine learning. Intuitively, the fairness of quantum decision model A is to treat all input states equally, i.e., there is not a pair of two closed input states that has a large difference between their corresponding outcomes. Formally,

**Definition 2 (Bias Pair).** *Suppose we are given a quantum decision model* <sup>A</sup> = (E, {Mi}<sup>i</sup>∈O)*, two distance metrics* <sup>D</sup>(·, ·) *and* <sup>d</sup>(·, ·) *on* <sup>D</sup>(H) *and* <sup>D</sup>(O)*, respectively, and two small enough threshold values* <sup>1</sup> <sup>≥</sup> ε, δ > <sup>0</sup>*. Then* (ρ, σ) *is said to be an* (ε, δ)*-bias pair if the following is true*

$$[D(\rho, \sigma) \le \varepsilon] \land [d(\mathcal{A}(\rho), \mathcal{A}(\sigma)) > \delta]. \tag{1}$$

The first condition in (1) indicates that the distance between input states ρ and σ is within ε, and the second condition shows the difference between outcomes <sup>A</sup>(ρ) and <sup>A</sup>(σ) is beyond <sup>δ</sup>. Sometimes, without any ambiguity, (ρ, σ) is called a bias pair if ε and δ are preset.

**Definition 3 (Fair Model).** *Let* <sup>A</sup> = (E, {Mi}i∈O) *be a decision model. Then* <sup>A</sup> *is* (ε, δ)*-fair if there is no any* (ε, δ)*-bias pair.*

The intuition behind this notion of fairness is that small or non-significant perturbation of a sample <sup>ρ</sup> to <sup>σ</sup> (i.e. <sup>D</sup>(ρ, σ) <sup>≤</sup> <sup>ε</sup>) must not be treated "differently" by a fair model. The choice of input distance function <sup>D</sup>(·, ·) identifies the perturbations to be considered non-significantly, while the choice of the output distance function <sup>d</sup>(·, ·) limits the changes allowed to the perturbed outputs in the model.

**Fairness Implying Robustness**: As the same in the classical situation [21], robustness of quantum machine learning is a special case of fairness defined above. Formally, robustness is defined on a specific state ρ: given a quantum model <sup>A</sup> = (E, {Mi}<sup>i</sup>∈O), <sup>ρ</sup> is (ε, δ)-robust if for all <sup>σ</sup> ∈ D(H), <sup>D</sup>(ρ, σ) <sup>≤</sup> <sup>ε</sup> implies <sup>d</sup>(A(ρ), <sup>A</sup>(σ)) <sup>≤</sup> <sup>δ</sup>. In contrast, fairness is established on all quantum states: <sup>A</sup> is (ε, δ)-fair if and only if <sup>ρ</sup> is (ε, δ)-robust for all states <sup>ρ</sup> ∈ D(H). So, *fairness implies robustness and can be thought of as global robustness.*

**Choice of Distances**: The reader should have noticed that the above definition of fairness for quantum decision models is similar to that for classical decision models. But an intrinsic distinctness between them comes from the choice of distances <sup>D</sup>(·, ·) and <sup>d</sup>(·, ·). In the classical case, the distances define the similarity between individuals and their appropriate choices have been intensively discussed [18]. One of the most used distances is total variation distance, measuring the closeness of individuals encoded by probability distributions. In this paper, we use it as <sup>d</sup>(·, ·) for measurement outcome distributions in Definition <sup>1</sup> and choose <sup>D</sup>(·, ·) to be the trace distance. Trace distance is essentially a generalization of total variation distance, and has been widely used by the quantum computation and quantum information community to define the closeness of quantum states [25, Section 9.2]. Formally, for two quantum states ρ, σ ∈ D(H),

$$D(\rho, \sigma) = \frac{1}{2} \text{tr}(|\rho - \sigma|),$$

where <sup>|</sup><sup>ρ</sup> <sup>−</sup> <sup>σ</sup><sup>|</sup> <sup>=</sup> <sup>Δ</sup><sup>+</sup> <sup>+</sup> <sup>Δ</sup><sup>−</sup> if <sup>ρ</sup> <sup>−</sup> <sup>σ</sup> <sup>=</sup> <sup>Δ</sup><sup>+</sup> <sup>−</sup> <sup>Δ</sup><sup>−</sup> with tr(Δ+Δ−) = 0 and <sup>Δ</sup><sup>±</sup> being positive semi-definite matrix. On the other hand, for two probability distributions <sup>p</sup> <sup>=</sup> {pi}<sup>i</sup>∈O, <sup>q</sup> <sup>=</sup> {qi}<sup>i</sup>∈O over <sup>O</sup>, <sup>d</sup>(p, q) = <sup>1</sup> 2 - <sup>i</sup> <sup>|</sup>p<sup>i</sup> <sup>−</sup> <sup>q</sup>i|. In particular, for the measurement outcome distributions, we have:

$$d(\mathcal{A}(\rho), \mathcal{A}(\sigma)) = \frac{1}{2} \sum\_{i} |\text{tr}(M\_i^\dagger M\_i \mathcal{E}(\rho - \sigma))|.$$

If <sup>ρ</sup> and <sup>σ</sup> are both diagonal matrices, i.e., <sup>ρ</sup> = diag(p1, ··· , p|O|) and <sup>σ</sup> <sup>=</sup> diag(q1, ··· , q|O|), then <sup>D</sup>(ρ, σ) = <sup>d</sup>(p, q).

#### **4 Characterizing Fairness**

In this section, we give a characterization of fairness in terms of the Lipschitz constant and clarify its relationship with quantum noises.

#### **4.1 Fairness and Lipschitz Constant**

The Lipschitz constant has been widely used in classical machine learning for applications ranging from robustness and fairness certification of classifiers to stability analysis of closed-loop systems with reinforcement learning controllers (e.g. [35,36]). In this subsection, we show that there also exists a close connection between the Lipschitz constant and fairness in the quantum setting. Let us start from an observation:

**Lemma 1.** *Let* <sup>A</sup> = (E, {Mi}<sup>i</sup>∈O) *be a quantum decision model. Then*

$$d(\mathcal{A}(\rho), \mathcal{A}(\sigma)) \le D(\rho, \sigma). \tag{2}$$

*Proof.* See Appendix A in [37] for the proof.

The above lemma indicates that quantum decision model A is automatically (ε, δ)-fair whenever <sup>ε</sup> <sup>=</sup> <sup>δ</sup>. Furthermore, we see that <sup>A</sup> is unconditionaly *Lipschitz continuous*: there exists a constant K > 0 (<sup>K</sup> <sup>≤</sup> 1 by Lemma 1) such that for all ρ, σ ∈ D(H),

$$d(\mathcal{A}(\rho), \mathcal{A}(\sigma)) \le KD(\rho, \sigma). \tag{3}$$

As usual, <sup>K</sup> is called a *Lipschitz constant* of <sup>A</sup>. Furthermore, the smallest <sup>K</sup>, denoted by <sup>K</sup>∗, is called the (best) Lipschitz constant of <sup>A</sup>.

In the context of quantum machine learning, the following theorem shows that <sup>K</sup><sup>∗</sup> actually measures the fairness of decision model <sup>A</sup>, i.e., the best (maximum) ratio of δ and ε in a fair model, and the states ψ, φ achieving K<sup>∗</sup> can be used to find bias pairs in fairness verification.

**Theorem 1.** *1. Given a quantum decision model* <sup>A</sup> = (E, {Mi}<sup>i</sup>∈O) *and* <sup>1</sup> <sup>≥</sup> ε, δ > <sup>0</sup>*,* <sup>A</sup> *is* (ε, δ)*-fair if and only if* <sup>δ</sup> <sup>≥</sup> <sup>K</sup>∗ε*.*

*2. If* <sup>A</sup> *is not* (ε, δ)*-fair, then* (ψ, φ) *achieving* <sup>K</sup><sup>∗</sup> *is a bias kernel; that is, for any quantum state* <sup>σ</sup> ∈ D(H)*,* (ρψ, ρφ) *is a bias pair where*

$$
\rho\_{\psi} = \varepsilon \psi + (1 - \varepsilon)\sigma \qquad \rho\_{\phi} = \varepsilon \phi + (1 - \varepsilon)\sigma. \tag{4}
$$

*Proof (Outline).* The "if" direction of the first claim is derived by the definitions of (ε, δ)-fairness and K<sup>∗</sup> together with (3). The "only if" direction of the first claim and the second claim are both based on the existence of pure states <sup>|</sup>ψ and <sup>|</sup>φ achieving <sup>K</sup>∗: <sup>d</sup>(A(ψ), <sup>A</sup>(φ)) = <sup>K</sup>∗D(ψ, φ). The detailed proof is presented in Appendix B in [37].

#### **4.2 Fairness and Noises**

In this subsection, we turn to consider the relation between fairness and noise. Let us first examine a simple example. Assume a noiseless quantum decision model <sup>A</sup> = (U, {Mi}<sup>i</sup>∈O) where <sup>U</sup> is a unitary operator, i.e., <sup>U</sup> <sup>=</sup> {U} for some unitary matrix U. The 1-qubit depolarizing noise in Example 2 can be generalized to a large-size system with the following form:

$$
\mathcal{E}(\rho) = (1 - p)\rho + p\frac{I}{N} \quad \forall \rho \in \mathcal{D}(\mathcal{H}),
$$

where 0 <sup>≤</sup> <sup>p</sup> <sup>≤</sup> 1 and <sup>N</sup> is the dimension of the state space <sup>H</sup> of the system. By introducing it into <sup>A</sup>, we obtain a noisy model <sup>A</sup><sup>E</sup> = (E◦U, {Mi}i∈O). Let K<sup>∗</sup> and K<sup>∗</sup> <sup>E</sup> be the Lipschitz constants of <sup>A</sup> and <sup>A</sup><sup>E</sup> , respectively. A calculation (with the help of Theorem 3 below) yields:

$$K\_{\mathcal{E}}^{\*} = (1 - p)K^{\*}.\tag{5}$$

Theorem 1 indicates that the less the Lipschitz constant is, the fairer the quantum machine learning model will be. So, depolarizing noise improves fairness by the order of (1 <sup>−</sup> <sup>p</sup>). By the way, it was shown in [38] that depolarizing noise can improve the robustness of quantum machine learning. This result can be strengthened by using (5) to quantitatively characterize the robustness improvement.

The observation in the above example can actually be generalized to the following:

**Theorem 2.** *Let* <sup>A</sup> = (U, {Mi}<sup>i</sup>∈O) *be a quantum decision model. Then for any quantum noise represented by a super-operator* <sup>E</sup>*, we have* <sup>K</sup><sup>∗</sup> <sup>E</sup> <sup>≤</sup> <sup>K</sup>∗*, where* <sup>K</sup><sup>∗</sup> *and* K<sup>∗</sup> <sup>E</sup> *are the Lipschitz constants of* <sup>A</sup> *and* <sup>A</sup><sup>E</sup> = (E◦U, {Mi}<sup>i</sup>∈O)*.*

*Proof (Outline).* The proof of this theorem mainly depends on the observation that the range of <sup>A</sup><sup>E</sup> is a subset of the range of <sup>A</sup>, i.e. {E ◦ U(ρ) : <sup>ρ</sup> ∈ D(H)} ⊆ {U(ρ) : <sup>ρ</sup> ∈ D(H)} <sup>=</sup> <sup>D</sup>(H). Subsequently, by Definition <sup>2</sup> of fairness, the output distributions of A<sup>E</sup> are contained in that of A. A restatement of this theorem in terms of quantum states (measurements) distinguishability and its full proof are presented in Appendix C in [37].

*Remark 1.* The above theorem indicates that adding noises at the end of noiseless computation can always improve fairness. Indeed, this is also true when the noises appear in the middle (after any gate in the circuit).

## **5 Fairness Verification**

In this section, we develop an algorithm for the fairness verification of quantum decision models based on the theoretical results obtained in the last section. Formally, the major problem concerned in this paper is the following:

*Problem 1 (Fairness Verification Problem).* Given a quantum decision model <sup>A</sup> and 1 <sup>≥</sup> ε, δ > 0, check whether or not <sup>A</sup> is (ε, δ)-fair. If not then (at least) one bias pair (ρ, σ) is provided.

#### **5.1 Computing the Lipschitz Constant**

First of all, we note that essentially, Theorem 1 gives a verification condition for fairness in terms of the Lipschitz constant K∗. Therefore, computing K<sup>∗</sup> is crucial for fairness verification. However, this problem is much more difficult than that in the classical counterpart as discussed in Subsect. 1.1. The following theorem provides a method to compute the Lipschitz constant K<sup>∗</sup> by evaluating the eigenvalues of certain matrices.

**Theorem 3.** *1. Given a quantum decision model* <sup>A</sup> = (E, {Mi}i∈O)*. The Lipschitz constant* K<sup>∗</sup> *is:*

$$K^\* = \max\_{A \subseteq \mathcal{O}} [\lambda\_{\max}(M\_A) - \lambda\_{\min}(M\_A)] \text{ with } M\_A = \sum\_{i \in A} \mathcal{E}^\dagger(M\_i^\dagger M\_i),$$

*where* <sup>E</sup>† *is the conjugate map*<sup>5</sup> *of* <sup>E</sup>*, and* <sup>λ</sup>max(MA) *and* <sup>λ</sup>min(MA) *are the maximum and minimum eigenvalues of positive semi-definite matrix* MA*, respectively.*

*2. Furthermore, let* <sup>A</sup><sup>∗</sup> ⊆ O *be an optimal solution of reaching the Lipschitz constant, i.e.,*

$$A^\* = \arg\max\_{A \subseteq \mathcal{O}} [\lambda\_{\max}(M\_A) - \lambda\_{\min}(M\_A)],$$

*and* <sup>|</sup>ψ *and* <sup>|</sup>φ *be two normalized eigenvectors corresponding to the maximum and minimum eigenvalues of* M<sup>A</sup><sup>∗</sup> *, respectively. Then we have*

$$d(\mathcal{A}(\psi), \mathcal{A}(\phi)) = K^\* D(\psi, \phi) = K^\*,$$

*where* <sup>ψ</sup> <sup>=</sup> <sup>|</sup>ψψ<sup>|</sup> *and* <sup>φ</sup> <sup>=</sup> <sup>|</sup>φφ|*.*

*Proof (Outline).* This theorem can be proved by reducing the problem of calculating the Lipschitz constant to determining the distinguishability of a quantum measurement. Then we claim that the distinguishability is the maximum difference between the eigenvalues of the matrices generated by the measurement. The details are quite involved, and we postpose them into Appendix C in [37].

Based on the above theorem, we are able to develop Algorithm 1 for computing the Lipschitz constant K∗. The correctness and complexity are provided in the next subsection.

#### **5.2 Fairness Verification Algorithm**

Now we are ready to present our main algorithm—Algorithm 2—for verifying fairness of quantum decision models.

To see the correctness of Algorithm 2, let us first note that the second part of Theorem <sup>3</sup> shows that <sup>K</sup><sup>∗</sup> can be achieved by <sup>d</sup>(A(ψ), <sup>A</sup>(φ)) for two mutually orthogonal quantum (pure) states ψ and φ. On the other hand, the second part of Theorem 1 asserts that such states ψ and φ form a bias kernel. Moreover, since state <sup>σ</sup> ∈ D(H) in (4) is arbitrary and <sup>D</sup>(H) is an infinite set, infinitely many bias pairs can be generated from this kernel.

To analyze the complexities of Algorithm 2 and its subroutine—Algorithm 1, we first see by Theorem 1 that for evaluating the (ε, δ)-fairness of quantum decision model <sup>A</sup>, the Lipschitz constant <sup>K</sup><sup>∗</sup> is sufficient and necessary. Thus the first step (Line 1) of Algorithm 2 is to call Algorithm 1 to compute K<sup>∗</sup> by the mean of Theorem 3. The complexity of Algorithm 1 mainly attributes to computing W<sup>i</sup> = - <sup>j</sup>∈J <sup>E</sup>† jM† <sup>i</sup> <sup>M</sup>iE<sup>j</sup> for each <sup>i</sup> ∈ O, and for each <sup>A</sup> ⊆ O, - <sup>i</sup>∈<sup>A</sup> <sup>W</sup><sup>i</sup>

 $\mathcal{S}^5 \mathcal{E}^\dagger(\rho) = \sum\_{j \in \mathcal{J}} E\_j^\dagger \rho E\_j$  if  $\mathcal{E}$  admits Kraus matrix form  $\mathcal{E}(\rho) = \sum\_{j \in \mathcal{J}} E\_j \rho E\_j^\dagger$ .

#### **Algorithm 1.** Lipschitz(A)

**Input:** A quantum decision model A = (E = {Ej}<sup>j</sup>∈J , {Mi}<sup>i</sup>∈O) on a Hilbert space H with dimension N.

**Output:** The Lipschitz constant K<sup>∗</sup> and (ψ, φ) as in Theorem 3.

1: **for each** i ∈ O **do** 2: W<sup>i</sup> = E†(M† <sup>i</sup> Mi) = - <sup>j</sup>∈J <sup>E</sup>† jM† <sup>i</sup> MiE<sup>j</sup> 3: **end for** 4: K<sup>∗</sup> = 0, A<sup>∗</sup> = ∅ be an empty set and M<sup>A</sup><sup>∗</sup> = **0**, zero matrix. 5: **for each** A ⊆ O **do** 6: M<sup>A</sup> = - <sup>i</sup>∈<sup>A</sup> <sup>W</sup><sup>i</sup> and <sup>K</sup><sup>A</sup> <sup>=</sup> <sup>λ</sup>max(MA) <sup>−</sup> <sup>λ</sup>min(MA) 7: **if** K<sup>A</sup> > K<sup>∗</sup> **then** 8: K<sup>∗</sup> = KA, A<sup>∗</sup> = A and M<sup>A</sup><sup>∗</sup> = M<sup>A</sup> 9: **end if** 10: **end for** 11: |ψ and |φare obtained two normalized eigenvectors corresponding to the maxi-


## **Algorithm 2.** FairVeriQ(A, ε, δ)

**Input:** A quantum decision model A = (E = {Ej}<sup>j</sup>∈J , {Mi}<sup>i</sup>∈O) on a Hilbert space H with dimension N, and real numbers 1 ≥ ε, δ > 0.

**Output: true** indicates A is (ε, δ)-fair or **false** with a bias kernel pair (ψ, φ) indicates A is not (ε, δ)-fair.

1: (K∗, (ψ, φ))=Lipschitz(A) *// Call Algorithm 1*

2: **if** δ ≥ K∗ε **then**

3: **return true**

4: **else**

5: **return false** and (ψ, φ)

6: **end if**

and its maximum and minimum eigenvalues (and the corresponding eigenvectors for A = A<sup>∗</sup> at the end). The former calculation needs O(N<sup>5</sup>) as the multiplication of <sup>N</sup> <sup>×</sup> <sup>N</sup> matrices needs <sup>O</sup>(N<sup>3</sup>) operations, and the number |J | of the Kraus operators {Ej}<sup>j</sup>∈J of <sup>E</sup> can be at most <sup>N</sup><sup>2</sup> [39, Chapter 2.2]; the complexity of the latter one is <sup>O</sup>(2|O||O|N<sup>2</sup>) since the number of subsets of <sup>O</sup> is 2|O|, <sup>|</sup>A| ≤ |O| for any <sup>A</sup> ⊆ O and computing maximum and minimum eigenvalues with corresponding eigenvectors of <sup>N</sup> <sup>×</sup> <sup>N</sup> matrix costs <sup>O</sup>(N<sup>2</sup>). Therefore, the total complexity of Algorithm <sup>1</sup> is <sup>O</sup>(N<sup>5</sup> + 2|O||O|N<sup>2</sup>). After that, in Lines 2-6, we simply compare δ and K∗ε to answer the fairness verification problem. So, Algorithm 2 shares the same complexity with Algorithm 1.

**Theorem 4.** *The worst case complexities of Algorithms 1 and 2 are both* <sup>O</sup>(N<sup>5</sup> + 2|O||O|N<sup>2</sup>)*, where* <sup>N</sup> *is the dimension of input Hilbert state space* <sup>H</sup> *and* |O| *is the number of the measurement outcome set* O*.*

Like their classical counterparts, quantum machine learning models usually downscale large-dimension input data to small-size outputs. This means that the number |O| of the measurement outcome set O is far smaller than the dimension <sup>N</sup> of input Hilbert state space <sup>H</sup>. It is even a constant 2 in most real-world tasks for binary decisions/classifications, such as income prediction and credit scoring (see the examples in Sect. 6), and in this case, the complexities of Algorithms 1 and 2 are both O(N<sup>5</sup>). However, the dimension N is exponential in the number n of the input qubits, i.e., N = 2n. Thus the complexity turns out to be O(2<sup>5</sup>n). In verification of classical models, this *state-space explosion problem* [40] can be mitigated by using some custom-made data structures to capture the features of the underlying data, e.g. Binary Decision Diagrams (BDDs) [41]. In the quantum case, we cross this difficulty by employing a quantum data structure—*Tensor Networks (TNs)*, originating from quantum many-body physics—to exploit the locality and regularity of the circuits representing quantum machine learning models. As a result, quantum models with up to n = 27 qubits can be handled by our verification algorithm.

#### **6 Evaluation**

In this section, we evaluate the efficiency of our verification algorithm (Algorithm 1) on noisy quantum decision models. The algorithm is implemented on *TensorFlow Quantum* [4]—a platform of Google for designing and training quantum machine learning algorithms. Then we test it by verifying the fairness of two groups of examples:


All source codes can be found at: https://github.com/Veri-Q/Fairness. All our experiments are carried out on a server with Intel Xeon Platinum 8153 @ 2.00 GHz × 256 Processors, 2048 GB Memory and no dedicated GPU. The machine runs Centos 7.7.1908 and each experiment is run with at most 80 processors. We use the NumPy and Google TensorNetwork [44] Python packages to compute Lipschitz constants and bias kernels for small-scale models and medium-scale models, respectively. These two packages have their own advantages in different sizes.

#### **6.1 A Practical Application in Finance**


These datasets contain some categorical features, which are transformed into different integer numbers for further operations. Then we have <sup>n</sup> ∈ {8, <sup>9</sup>} numbering features in total and use the following data-encoding feature map:

$$x = (x\_1, x\_2, \dots, x\_n) \mapsto |\psi(x)\rangle = \bigotimes\_{j=1}^n X^{x\_j} |0\rangle.$$

for Pauli matrix X defined in Example 2 to encode an n-dimensional feature vector *x* (each dimension is normalized by its maximum value) to an n-qubits quantum state <sup>ψ</sup>(*x*) = <sup>|</sup>ψ(*x*)ψ(*x*)|.

**Models:** For the quantum decision model, we choose the basic rotation and entangling building blocks [47] to construct parameterized quantum circuits (see Fig. 2). In the rotation block, without any ambiguity, we directly use X and <sup>Z</sup> to represent parameterized <sup>X</sup>-rotation <sup>e</sup>−<sup>ı</sup> <sup>θ</sup><sup>1</sup> <sup>2</sup> <sup>X</sup> and parameterized Z-rotation e−<sup>ı</sup> <sup>θ</sup><sup>2</sup> <sup>2</sup> <sup>Z</sup> on one qubit, respectively. It is worth noting that the parameterized (Z-X-Z)-rotation induces universal gates on each qubit [25, Theorem 4.1], and thus the expressiveness of the models on one qubit is ensured. In the entangling block, XX stands for the parameterized (<sup>X</sup> <sup>⊗</sup> <sup>X</sup>)-rotation <sup>e</sup>−<sup>ı</sup> <sup>θ</sup><sup>3</sup> <sup>2</sup> <sup>X</sup>⊗<sup>X</sup> on two qubits. The entangling block can create entanglement between each qubit. Here entanglement is a unique feature of quantum models to express the interactions of qubits. The model is constructed by alternately using these two blocks with a quantum measurement M at the end of the model.

Since TensorFlow Quantum is inefficient in training noisy models, we only use 3 rotation blocks and 2 entangling blocks in the training models. In addition, to

**Fig. 2.** Parameterized Quantum Circuits for Quantum Finance Decision Models.


**Table 1.** Experimental results of Lipschitz constant K<sup>∗</sup> of the trained models.

simulate noisy models, we put different quantum noises introduced in Example 2 on each qubit, including bit flip, phase flip, depolarizing, and the mixtures of them, behind the first rotation block. Note that the number of qubits for the models is the same as the number of features of datasets due to the above choice of the data-encoding feature map. The final measurement <sup>M</sup> <sup>=</sup> {M<sup>0</sup> <sup>=</sup> <sup>I</sup> ⊗ |00|, M<sup>1</sup> <sup>=</sup> <sup>I</sup> ⊗ |11|} is a local measurement performed on the last qubit. With the binary classification task, the loss function we choose is binary crossentropy: <sup>−</sup> <sup>1</sup> N -N <sup>j</sup>=1 <sup>c</sup><sup>j</sup> · log ¯c<sup>j</sup> + (1 <sup>−</sup> <sup>c</sup><sup>j</sup> ) log(1 <sup>−</sup> <sup>c</sup>¯<sup>j</sup> )), where <sup>N</sup> is the size of the batch fixed in the training process, c<sup>j</sup> is the true label and ¯c<sup>j</sup> is the outcome of the measurement. All models are well trained and achieve around 70% train and test accuracy (see Column "Accuracy" in Table 1), matching that of the previously used classical and quantum finance decision models (e.g. [10,21]).

**Evaluation Details and Results:** The results of evaluating Algorithm 1 on the models trained from different datasets and different quantum noises are presented in Table 1. For different datasets, we train noise-free models to serve as the baseline for training and test accuracy (see Row "None"). Furthermore, different types of noise are added with different levels of probabilities. We list the Lipschitz constant <sup>K</sup><sup>∗</sup> and the running time of Algorithm <sup>1</sup> aided by NumPy for each column. It can be seen that the higher level of noise's probability, the smaller value of constant K∗. Therefore, the claim of quantum noise improving fairness in Sect. 4.2 is confirmed by the numerical results. This is also observed in Table 2 later.

#### **6.2 Scalability in the NISQ Era**

**Models:** To reflect an actual application in the NISQ era, we choose not to randomly generate a parameterized quantum circuit model. Instead, we expanded the existing example of Quantum Convolutional Neural Network (QCNN) [32] in the QCNN tutorial<sup>6</sup> of TensorFlow Quantum from 8 qubits (see Fig. 3) to 27 qubits. In the experiment, we use the QCNN model with one convolution layer and one pooling layer. The noise is applied between convolution and pooling layers on each qubit. The final measurement is <sup>M</sup> <sup>=</sup> {M<sup>0</sup> <sup>=</sup> <sup>I</sup> ⊗ |00|, M<sup>1</sup> <sup>=</sup> <sup>I</sup> ⊗ |11|} performed on the last qubit with a gate <sup>U</sup> appended before. Since training a noisy model of this size is currently intractable on TensorFlow Quantum, the parameters in the model are all randomly sampled.

<sup>6</sup> https://tensorflow.google.cn/quantum/tutorials/qcnn.

**Fig. 3.** The QCNN model in the tutorial of TensorFlow Quantum. Each C<sup>i</sup> in the convolution layer is a parameterized 2-qubit gate to find a new state between adjacent qubits. Each P<sup>i</sup> in the pooling layer is also a parameterized 2-qubit gate with another form that attempts to extract the information of two qubits into a single qubit.

**Evaluation Details and Results:** We choose the models with 25 and 27 qubits to run experiments. Since the parameters are randomly sampled, for each noise with different levels of probability, we generate the model and evaluate the Lipschitz constant <sup>K</sup><sup>∗</sup> for 3 times. However, because a 2<sup>25</sup> <sup>×</sup> <sup>2</sup><sup>25</sup> or 2<sup>27</sup> <sup>×</sup> <sup>2</sup><sup>27</sup> complex matrix consumes a huge amount of memory, it is not feasible to directly use Algorithm 1 as the previous experiment, where we represent the M<sup>A</sup> in Algorithm <sup>1</sup> as a matrix and use the package NumPy to evaluate eigenvalue. We instead use a tensor network [48] to represent the M<sup>A</sup> and the subroutine of evaluating eigenvalue in Algorithm 1 is implemented with the basic power method for eigenvalue problem [49] by using TensorNetwork package. Although there are some packages for sparse matrix in Python that can collaborate with TensorNetwork, their implementation for computing eigenvalues still consumes a huge amount of memory. The evaluation results on QCNN models with randomly sampled parameters and different quantum noises are listed in Table 2. These results prove that our fairness verification algorithm is efficient and can handle 27-qubit quantum decision models on a small server. For further exploring the scalability of our verification algorithm, we also test on 29-qubit QCNN models; Please see Appendix D in [37] for the results.

Last but not least, it is worth noting that in all experiments, we also obtain bias kernels by Algorithm 1 at the running time presented in Tables 1 and 2, but as they are large-size (up to 2<sup>27</sup>-dimensional) vectors, we do not show them.


**Table 2.** Experimental results of Lipschitz constant K<sup>∗</sup> of QCNN models.

#### **7 Conclusion**

In this work, we initiate the studies on algorithmic verification of fairness of quantum machine learning decision models. In particular, we showed that this verification problem can be reduced to computing the Lipschitz constant of the decision models, and then resolved the latter by introducing and estimating single measurement distinguishability. Based on these theoretical results, we developed an algorithm that can verify the (ε, δ)-fairness of quantum decision models and provides useful bias kernels for explaining the unfairness of the models.

An interesting topic for future research is how to improve the results presented in this paper for training quantum decision models with fairness guarantee. On the other hand, further investigations are required to better understand the bias kernels detected by our verification algorithm, especially through more experiments on real-world applications.

**Acknowledgments.** Ji Guan would like to thank Jiayi Chen for her linguistic assistance during the preparation of this paper. This work was partly supported by the National Key R&D Program of China (Grant No: 2018YFA0306701), the National Natural Science Foundation of China (Grant No: 61832015).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## MoGym: Using Formal Models for Training and Verifying Decision-making Agents

Timo P. Gros<sup>1</sup> , Holger Hermanns1,2 , Jörg Hoffmann<sup>1</sup> , Michaela Klauck1(B) , Maximilian A. Köhl1(B) , and Verena Wolf<sup>1</sup>

<sup>1</sup> Saarland University, Saarland Informatics Campus, Saarbrücken, Germany <sup>2</sup> Institute of Intelligent Software, Guangzhou, China {timopgros,hermanns,hoffmann,klauck,koehl,wolf}@cs.uni-saarland.de

Abstract. MoGym, is an integrated toolbox enabling the training and verification of machine-learned decision-making agents based on formal models, for the purpose of sound use in the real world. Given a formal representation of a decision-making problem in the JANI format and a reachavoid objective, MoGym (a) enables training a decision-making agent with respect to that objective directly on the model using reinforcement learning (RL) techniques, and (b) it supports rigorous assessment of the quality of the induced decision-making agent by means of deep statistical model checking (DSMC). MoGym implements the standard interface for training environments established by OpenAI Gym, thereby connecting to the vast body of existing work in the RL community. In return, it makes accessible the large set of existing JANI model checking benchmarks to machine learning research. It thereby contributes an efficient feedback mechanism for improving in particular reinforcement learning algorithms. The connective part is implemented on top of Momba. For the DSMC quality assurance of the learned decision-making agents, a variant of the statistical model checker modes of the Modest Toolset is leveraged, which has been extended by two new resolution strategies for non-determinism when encountered during statistical evaluation.

Keywords: Formal Methods · Statistical Model Checking · Reinforcement Learning

#### 1 Introduction

Making optimal decisions in an uncertain environment is the crux of many practical problems. Reinforcement Learning (RL) is a popular method to compute near-optimal policies for sequential decision-making problems [60]. In the last years, RL algorithms that approximate optimal decision policies by training deep

Authors are listed alphabetically. This work was partially supported by the German Research Foundation (DFG) under grant No. 389792660 as part of TRR 248, by the European Regional Development Fund (ERDF), and by the Key-Area Research and Development Program Grant 2018B010107004 of Guangdong Province.

neural networks have exhibited unprecedented performance in various tasks [47]. However, the expressivity of these models makes them difficult to interpret or to be checked for consistency for some desired properties. This is an impediment to the use of such representations in safety-critical applications [61]. In addition, the environment of the decision-making agent executing the policy during training is typically specified implicitly in the form of simulation code. In the academic context, for instance the Arcade Learning Environment is widely used, which provides game simulators for different ATARI 2006 benchmarks [6].

If one strives for a principled understanding of the power of RL algorithms or of the properties of a specific learned agent in the (possibly uncertain) environment, a formal, mathematically precise and unambiguous description of the *training environment* appears central. The formal methods community has developed appropriate language concepts for the description of such environment models. Their advantage lies in their succinctness and modularity as well as their underlying mathematically rigorous formal semantics based on stochastic process models such as Markov Decision Processes (MDPs) [53], the main semantic object of probabilistic model checking [40]. A widespread format to describe MDP models of environments is the JANI format [14], providing a modular, automata-like syntax, supported by several model checkers, like Storm, the Modest Toolset, ePMC [29,30,33], and via a translation also by PRISM [41].

This paper presents MoGym, a toolbox that bridges the gap between formal methods and RL by enabling (a) formally specified training environments to be used with machine-learned decision-making agents, and (b) the rigorous assessment of the quality of learned agents. For (a), it implements and extends the OpenAI Gym API [11], which is the widely used standard interface for deep reinforcement learning [16,26,35,50,55]. MoGym is based on Momba [39], a Python toolbox for dealing with quantitative models from construction to analysis centered around JANI. MoGym can process JANI models for the description of a training environment and, based on the induced formal MDP semantics, makes it possible to train agents using popular RL algorithms.

For (b), the environment format itself is accessible to state-of-the-art model checkers. This enables probabilistic model checking of a specific agent acting in the environment specified by the model. This can be crucial to determining if further training improves the agent's quality and, whenever synthesis of the optimal agent is feasible, it allows a comparison of the agent's behavior to the optimal one. As such, the environment provides a stable and fully controllable training and checking context to assert the safety risk induced by an agent during and after training. More concrete, MoGym leverages deep statistical model checking (DSMC) [20,21]. As shown in these works on DSMC, the quality assessment of an agent during training is not trivial and can especially not always be derived from the observed training returns. Hence, analyzing the quality of the decision-making agents after training clearly is of interest [20,21], especially for badly interpretable agent structures such as neural networks (NN). In DSMC this is done by using the decision-making agent as an oracle resolving the non-determinism in the MDP specifying the environment. When resolving

Fig. 1. The architecture of MoGym and its components

the non-determinism, a Markov chain results on which the probability of satisfying a given reach-avoid objective can be calculated. A prominent technique for doing so with very low memory requirements is statistical model checking (SMC) [5,7,32,34,44,64,67]. The satisfaction probability for the reach-avoid objective calculated using statistics based on a set of simulation runs of the resulting Markov chain, can serve as an indicator of the quality of the decision-making agent for solving the reach-avoid task it was originally trained on.

MoGym comprises the following components:


Figure 1 shows how the different parts of MoGym are interconnected. First, a decision-making agent can be trained on a formal model and a reach-avoid property, defined in a JANI model, against the OpenAI Gym API by using Momba Gym with different reinforcement learning techniques, which can be implemented and defined by the user. Afterwards, the trained agent can be verified w.r.t. reach-avoid objectives by invoking the DSMC API, which makes use of the DSMC extension of the statistical model checker modes. Alternatively, the training step can be skipped, or can be done in any other way, and an arbitrary external oracle can be checked.

We are not aware of any other work that enables a direct connection of formal verification models and reinforcement learning that directly allows the analysis of different RL agents for a variety of verification benchmarks.

*Outline of the paper.* In Sect. 2 we describe the Momba Gym Python API and explain how MoGym is used to train agents on existing JANI MDPs. Sect. 3 presents the DSMC API of Momba, and discusses its use to assess the quality of decision-making agents or arbitrary oracles via DSMC, together with the new DSMC functionality of modes. In Sect. 4 we provide empirical insight into the full functionality of MoGym. Sect. 5 concludes the paper.

A preview of the Jupyter Notebook demonstrating the code we used to execute the experiments shown in the paper can be found online. It will later be part of the full artifact for the tool paper.

#### 2 Formal Models as Training Environments

At the heart of MoGym is an implementation of the OpenAI Gym API in *Momba Gym*, which now enables the usage of JANI models as training environments. OpenAI Gym [11] constitutes *the* standard API for interfacing environments with different reinforcement learning algorithms enabling their comparison and fostering development of new techniques. It is widely used by both, algorithms that interact with the interface [16,26,35,50,55], as well as various benchmarks that implement (and sometimes extend) the interface [3,15,18,62,63,66]. With Momba Gym, MoGym provides an extension of this API for general JANI MDP models equipped with reach-avoid properties. JANI is a JSON-based format for exchanging formal models between tools [14]. It is the standard format in the quantitative verification community and directly supported by state-of-theart tools, like Storm [33], the Modest Toolset [30], and ePMC [29]. Translations from and to other languages such as the PRISM language [41,42], Modest [28] and even the planning language PPDDL [36,37] exist.

A JANI model is a network of interacting automata with variables. Each automaton consists of a set of *locations* and a set of probabilistic *edges* from a *source* location to possibly multiple *destination* locations. Edges can be labeled with *edge labels* and annotated, depending on the destination, with assignments to variables. The *transitions* of the network are then obtained by synchronizing the automata, i. e., in every transition, potentially multiple automata participate with one edge, respectively. For our purposes, we assume that a decision-making agent controls a single automaton in the network, i. e., resolves the non-determinism of this automaton. Fig. 2 exemplifies the construction of an automata network from two automata: a controlled automaton (a) and a non-controlled automaton (b). Depending on which of the edges of automaton (b) is taken, the probability of ending up in state b is either 1.0 (action α) or 0.2 (action β). The final composition (c) is then the product of both automata synchronizing over the shared edge labels α and β. Controlling automaton (a) here

Fig. 2. Networks of interacting automata.

implies selecting which of the transitions in the final compositional does happen. By choosing the edge labeled with α, the transition α, α in the composition is selected and analogously for β. The choice of α in the controlled automaton obviously is the one maximizing the probability of reaching the green state y, b in the composition (c). In fact, the state is reached with certainty. Technically, this approach would extend to a multi-agent setting where different agents resolve the non-determinism in different parts of the model. We plan to provide a multi-agent setting in future work and assume here that all non-determinism not resolved by the controlled automaton is resolved uniformly.<sup>1</sup>

For training an agent in an environment, the OpenAI Gym API requires the definition of an *action space* and an *observation space*. In response to receiving observations from the observation space, the trained agent makes a decision from the action space. To enable the usage of general JANI MDP models as environments, an action space and observation space have to be extracted from the model. Depending on the model, there are multiple ways to do so. Momba Gym implements different strategies for this extraction. For the action space, edges of the controlled automaton can be selected by index or by label. For the observation space, (i) only global variables, (ii) global variables and local variables of the controlled automaton, or (iii) all variables can be declared as observable.<sup>2</sup> Other strategies can easily be added to Momba Gym.

Whenever the agent makes a decision in response to an observation, the decision is mapped to an edge of the controlled automaton and then to a transition of the network. If present, other non-deterministic influences are resolved uniformly at random, as mentioned above. In this case, the user receives a warning message so that this is taken into account when inspecting the results. After taking the respective transition, the environment continues the trace through the model until a state is reached where the agent can make a decision again.

Momba Gym supports reach-avoid properties of the form φ **U** ψ where φ and ψ are propositional logic formulas over the model's states. φ **U** ψ encodes the property that a state satisfying ψ is reached eventually and that φ holds on all states prior to reaching ψ. In a *bad state*, which should be avoided, ψ is not

<sup>1</sup> That is, each of the remaining non-deterministic options is considered equiprobable. MoGym can easily be extended with other mechanisms to resolve non-determinism.

<sup>2</sup> For more details about those strategies see https://momba.dev/gym/.

satisfied and (i) there are no remaining transitions or (ii) φ is violated. In a *goal state* ψ is satisfied. To apply RL techniques, Momba Gym supports providing a *reward structure* specifying the reward for reaching a goal, the (usually negative) reward for reaching a bad state, the reward for taking a decision neither leading to a goal nor to a bad state (usually zero), and the reward for taking a nonapplicable decision. Using the Momba Gym API integrated in Momba, one can create a training environment from an arbitrary JANI MDP model as follows:

```
from momba import jani, gym
model = jani.load_model(JANI_SOURCE)
# ...
env = gym.create_generic_env(model, automaton)
```
In this command, automaton is the automaton the agent controls. The function create\_generic\_env takes additional optional parameters specifying the strategy for the extraction of the action and observation space (i. e., by index or by label, see above) as well as the reward structure (by defining the four reward values indicated above) and parameters of the JANI model. The resulting env implements the OpenAI Gym API such that it can be directly used to train an agent for the given property using arbitrary RL algorithms based on the OpenAI Gym API. Thereby, Momba Gym makes JANI MDP models accessible to the RL community to train and evaluate their algorithms on. The implementation of the Momba Gym environment uses the explicit state space exploration engine of Momba which is written in Rust. It is sufficiently performant such that it can be used to train different agents using state-of-the-art RL algorithms.

Momba Gym extends the OpenAI Gym API with the ability to *fork* the environment and query the applicable actions. The former is useful for algorithms based on *Monte-Carlo Tree Search (MTS)* [12], known to act favorably on prominent benchmarks, like Atari Games [24]. Further, MTS forms the basis of DeepMind's famous algorithms around AlphaGo and AlphaZero [57].

In addition to the general Momba Gym API, we provide exemplary code to train an agent for an arbitrary formal model. While we ourselves implemented *deep Q-learning* [47], MoGym is open to any (deep) reinforcement learning algorithm. Using our implementation of deep Q-learning, enables training of a decision-making agent for an arbitrary JANI MDP model.<sup>3</sup> We note however that deep RL is known to be hyperparameter sensitive [45], so intensive tweaking of hyperparameters might be needed for the learning to work. In this regard our deep Q-learning implementation is no exception.

#### 3 Verifying Agents Using Statistical Model Checking

If given a formal model and a decision-making agent trained on it, MoGym supports verification by deep statistical model checking. To this end, the DSMC API of MoGym implements two functions, one for verifying arbitrary agents in the form of Python functions and one for verifying PyTorch neural networks.

<sup>3</sup> Details will be included in the artifact of the paper.

Both functions rely on our DSMC extension of the statistical model checker modes [13] of the Modest Toolset [30], which accepts both forms of decision entities, and returns the reach-avoid probability calculated by the model checker.

Statistical model checking is based on Monte-Carlo simulation [56,65]. Using statistics, a probability estimate is derived from a set of simulation runs, regarding the satisfaction of a reach-avoid property, the error of which is bounded by a confidence interval. This is determined by the probability of the error in the computation being larger than is smaller than δ: P(*error* > ) < δ. For SMC to be applicable, the non-determinism of the model needs to be resolved [8,13]. In our DSMC setting this is done by the agent and otherwise resolved uniformly, i. e., equiprobable across all options (see Sect. 2). The computed reach-avoid probability can serve as an indicator of the overall quality of the decisions made by the agent [20]. The DSMC implementation in modes provides the same functionality regarding the observation space (global and/or local variables) and action space (select by index or label) as the Momba Gym training infrastructure described in Sect. 2.

As mentioned above, modes can deal with two variants of decision-making agents. An arbitrary Python function mapping observations to decisions can be checked with the DSMC API of MoGym by executing:

#### gym.checker.check\_oracle(oracle, model, automaton)

Here, oracle is the Python function implementing the decision-making agent. Notably, this is not limited to trained agents in any way. Any arbitrary Python function with an appropriate signature can be used. The other parameters are analogous to create\_generic\_env. In particular, check\_oracle also allows optionally specifying a strategy for extracting the action and observation spaces (see above).

While check\_oracle involves executing Python code, a more efficient approach is available when the decision-making agent is a PyTorch neural network. In this case, the network can directly be verified with check\_nn:

#### gym.checker.check\_nn(nn, model, automaton)

To this end, we assume that the network is a sequence of layers. The function check\_nn extracts these layers from the provided neural network nn and exports them in a JSON-based format. The neural network is then loaded by modes and used for model checking without calling back into the Python runtime. With the help of TorchSharp [25] (a .NET library providing access to the library that powers PyTorch) our extension of modes supports networks with arbitrary dimensions and activation functions.

Alternatively to the DSMC API provided by MoGym, it is also possible to invoke modes on the command line to check a NN or to connect it to an arbitrary decision-making agent via a socket connection. The agent could be any program taking the information of the observation space as input and sending an action decision back.

#### 4 Experimental Insights

With MoGym it is now possible to train agents and assess their quality for arbitrary JANI MDP models by evaluating them using the DSMC extension of the statistical model checker modes. In the following, we demonstrate all parts of the workflow when MoGym is used from training to evaluation. For our case studies, the training was performed by using a well-established standard RL algorithm, the deep Q-learning algorithm [47].

*Benchmarks.* Working with MoGym starts with devising a formal model to train a decision-making agent on. For example, the *Quantitative Verification Benchmark Set* (QVBS) [31] contains JANI models originally collected for competitions among quantitative verification tools. With the help of MoGym they are now accessible for use in the learning community. For our case studies, we selected three MDP benchmarks from the QVBS: *cdrive.2*, *elevators* and *firewire\_dl*. With respect to the observation spaces, we use the Momba Gym API default setting, in which only global variables are observable.

In *cdrive.2* a car drives in a city modeled using locations connected by roads with traffic lights. The car should reach a destination without an accident [10]. In the *elevators* case, a certain number of elevators is available to transport coins to a predefined level. An elevator can fall down on a lower level [10,38]. The *firewire\_dl* benchmark models the leader election protocol in the Tree Identify Protocol of the IEEE 1394 High Performance Serial Bus [43,59].

Another popular benchmark is Racetrack, which has been adopted for decision making under uncertainty in many works [2,4,9,19,46,51,52]. In Racetrack, a vehicle needs to be driven on a discretized grid track towards a goal as fast as possible without crashing. A preview of the Jupyter Notebook showing the code we used for the experiments, which will later be part of the tool paper's artifact, is available online.

*Training.* We trained agents for all of the considered benchmarks by using the calls to the Momba Gym API as introduced in Sect. 2, which can be inspected in Sect. 2.1 and 2.2 of the Jupyter notebook.

Fig. 3 (a) and (b) shows the training progress of *cdrive.2* and *Racetrack*, respectively, depicted in blue. The training for *cdrive.2* took around 1 *min*, and for *Racetrack* about 22 *min*, on a standard laptop. In contrast to these two benchmarks, learning for *elevators* and *firewire\_dl* failed. During training, the agent was able to reach the goal, but the NN was not able to generalize.

*Verification.* For *cdrive.2* and *Racetrack*, the training return increases over the number of training episodes and is quite stable at the end. The training return is commonly regarded as an estimator of the training progress [47,48]. Here it appears to indicate that the quality of the trained neural networks does neither increase nor decrease from a certain episode on.

However, we now can use DSMC to check the actual quality of the trained agents, i. e., we can determine how high the probability is that they indeed

Fig. 3. Blue: Training curve showing sliding mean of the training return, i. e., the accumulated discounted reward over the last 500 training episodes, on the left y-axis. Note the different scale for (a) and (b). Red: Goal reachability probability on the right y-axis. Both are plotted over the number of training episodes on the x-axis. (a) Shows results for *cdrive.2* and (b) for *Racetrack*.

reach the goal in their respective environments defined by the MDP model. We do so by making use of the DSMC API of MoGym, introduced in Sect. 3 using modes as backend. We check the goal reachability probability of the NN policies extracted every 1000 training episodes as shown in Sect. 2.1 and 2.2 of the Jupyter notebook.

As depicted by Fig. 3, the return during training is not as expressive as expected. While the training return is relatively consistent for both *cdrive.2* and *Racetrack*, the goal reachability probability (depicted in the red points) over training is not. In contrast, it both increases and decreases over the training episodes. So, the training return alone turns out not to be a good indicator for deciding which of the extracted policies actually is the best one. For *cdrive.2* (Fig. 3 (a)), this can be considered as fine tuning, as most of the policies perform near-optimal. In contrast, for *Racetrack* (Fig. 3 (b)), we observe a huge difference between the policies, including near-optimal policies as well as policies with a goal reachability probability of only about 20%. These deeper insights regarding the neural networks' quality are only possible by using DSMC.

Having selected the best policy for each benchmark, the analysis yields a goal reachability probability of 86.57% for *cdrive.2*, where a policy acting optimally would reach the goal with a probability of 86.45%. <sup>4</sup> The optimal value has been calculated with the exhaustive probabilistic model checking engine mcsta [27] of the Modest Toolset. The goal reachability probability of the best NN policy

<sup>4</sup> Note that the goal reachability probability of the NN policies is estimated by statistical model checking. Thus, even though it might seem surprising at first sight, it is of course possible that the analysis of our policy yields a slightly higher goal reachability probability than optimally possible as long as this is within the given confidence interval. We use P(*error* > ) < δ, where = 0.01 and δ = 0.05, i. e., a confidence of 95%.

of the trained agent for *Racetrack* is 97.30% where the optimal policy reaches the goal with a probability of 99.99%.

#### 5 Conclusion and Future Work

We presented MoGym, an integrated toolbox to train, analyze and verify decisionmaking agents on formal models. These formal models are made available through Momba Gym, which implements and extends the well-established OpenAI Gym API for arbitrary reinforcement learning techniques. Using these techniques to obtain NNs or, alternatively, some general decision-making agents, they can then be rigorously verified with DSMC using the new extension of modes. The approach is open to all JANI MDPs and modes can in principle handle arbitrary fully connected and even convolutional networks.

On the basis of the QVBS and *Racetrack*, we showed how the toolchain of MoGym works. As presented, our formal-model-based approach enables deeper insights for specified properties than non-formal, implicitly defined simulationbased environments.

In the future, we want to address the problem which caused the training for *elevators* and *firewire\_dl* to fail. Given the successes of deep RL across many diverse environments [1,23,47,49,54,57,58], one is tempted to expect it to work well on the considered environments [22,31], too. Still, deep reinforcement learning is known to perform badly in domains with large action spaces [17], and we suspect this to be the root of the problem we observe. The action structures arising in networks of automata are of a specific kind. Rooted in process algebra, their main role is to enable and orchestrate synchronization across automata, and this is indeed the case for the JANI models *elevators* and *firewire\_dl*. A more meaningful construction of an action space of compositional models suitable for learning appears needed.

Furthermore, the extension of our tool to other model types and an extension to control all of the modeled automata, making the learning task a multi-agent one, would clearly be of interest. Apart from that, we plan to build upon MoGym to develop DSMC techniques further. With DSMC Evaluation Stages [21] it has already been shown that DSMC can be applied during deep RL to determine state space regions with weak performance to concentrate on them during the learning process. With the help of MoGym this technique can now be done much more integrated and there is room for further implementations into this direction in our tool chain.

#### References


Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Synthesis and Concurrency**

## **Synthesis and Analysis of Petri Nets from Causal Specifications**

Mateus de Oliveira Oliveira(B)

University of Bergen, Bergen, Norway mateus.oliveira@uib.no

**Abstract.** Petri nets are one of the most prominent system-level formalisms for the specification of causality in concurrent, distributed, or multi-agent systems. This formalism is abstract enough to be analyzed using theoretical tools, and at the same time, concrete enough to eliminate ambiguities that would arise at implementation level. One interesting feature of Petri nets is that they can be studied from the point of view of true concurrency, where causal scenarios are specified using partial orders, instead of approaches based on interleaving.

On the other hand, message sequence chart (MSC) languages, are a standard formalism for the specification of causality from a purely behavioral perspective. In other words, this formalism specifies a set of causal scenarios between actions of a system, without providing any implementation-level details about the system.

In this work, we establish several new connections between MSC languages and Petri nets, and show that several computational problems involving these formalisms are decidable. Our results fill some gaps in the literature that had been open for several years. To obtain our results we develop new techniques in the realm of slice automata theory, a framework introduced one decade ago in the study of the partial order behavior of bounded Petri nets. These techniques can also be applied to establish connections between Petri nets and other well studied behavioral formalisms, such as the notion of Mazurkiewicz trace languages.

**Keywords:** MSC Languages · Mazurkiewicz Traces · Petri Nets

#### **1 Introduction**

Petri nets are one of the most prominent system-level formalisms for the specification of causality in concurrent, distributed or multi-agent systems. This formalism is abstract enough to be analyzed using theoretical tools, and at the same time, concrete enough to eliminate ambiguities that would arise at implementation level. One interesting feature of Petri nets is that they can be studied from the point of view of true concurrency, where causal scenarios are specified using partial orders, instead of approaches based on interleaving [18,36]. On the other hand, message sequence chart (MSC) languages [16,19], are a standard formalism for the specification of causality from a purely behavioral perspective. In other words, this formalism specifies a set of causal scenarios between actions of a system, without providing any implementation-level details about the system.

In this work, we show that given an MSC automaton M specifying a set of partial orders L*po*(M), and a b-bounded Petri net N with causal behavior P*cau* (N), it is decidable whether L*po*(M)∩P*cau* (N) = ∅, and whether L*po*(M) ⊆ <sup>P</sup>*cau* (N) (Theorem 8). Additionally, for any given <sup>b</sup> <sup>∈</sup> <sup>N</sup>+, one can synthesize a b-bounded Petri net N that best captures the behavior specified by M (Theorem 9). More specifically, L*po*(M) ⊆ P*cau* (N), and there is no other b-bounded Petri net N such that L*po*(M) ⊆ P*cau*(N ) - P*cau* (N). Finally, if the MSC automaton M is *locally synchronized*, a well studied property in the context of MSC language theory [1,19,31], then one can also test whether P*cau* (N) ⊆ L*po*(M) (Theorem 8).

The feasibility of all computational problems described above have been open even for 1-bounded Petri nets, despite the fact that both Petri nets and MSC languages have been defined several decades ago. The key of our results is a new connection between MSC automata and slice automata, a formalism introduced in [33] in the study of the partial order behavior of bounded Petri nets. More specifically, we show that for each MSC automaton M, one can construct a slice automaton A such that L*po*(M) = L*po*(A) (Theorem 7). A crucial feature of this construction is that it preserves good decidability properties. More precisely, if the input MSC automaton M is locally synchronized, then the obtained slice automaton A satisfies a property called saturation, which is crucial for the analysis of the causal behavior of Petri nets against safety specifications. To establish the connection mentioned above, we develop new slice-theoretic machinery of independent interest. In particular, we introduce the notions of slice-traces, and the notion of a locally synchronized slice automaton. In Sect. 8, we show that this new framework can also be used to establish connections between slice automata (and therefore, Petri nets), and the formalism of Mazurkiewicz trace languages [8,12,20,24,28,28], which is another well-studied formalism for the specification of sets of partial orders. In this case, it also holds that our reductions preserve good decidability properties, in the sense that finite automata accepting traceclosed languages are mapped to saturated slice automata.

**Related Work.** During the last four decades many partial order formalisms have been introduced and several connections have been established between these formalisms [9,13,17,21,25,29,36]. In particular, the expressiveness of finite message-passing automata with a priori unbounded FIFO channels was studied in [5], where it was shown that these automata capture exactly the class of MSC languages that are definable in existential monadic second-order logic interpreted over MSCs. Asynchronous cellular automata for traces were originally introduced by Zielonka [37]. A notion of asynchronous cellular automaton for pomsets without auto-concurrency was devised in [10]. Existentially bounded communicating automata have been considered in [14] where an equivalence was established between communication automata, globally cooperative compositional message sequence graphs and monadic second-order logic. Several connections between communicating automata with bounded channels and Mazurkiewicz traces have been considered in [15]. Generalizations of Mazurkiewicz traces have been considered in [22], and some extensions of message sequence graphs that are suitable for model checking under MSO specifications have been considered in [27]. Series parallel languages have been considered in [26]. It is important to note that the class of partial orders that can be accepted by slice automata are incomparable with the class of series parallel partial orders. On the one hand, series parallel partial orders are not necessarily k-bounded in the sense considered in this work. On the other hand, it is easy to construct k-bounded partial orders that are not series parallel. In particular, for k ≥ 4, slice automata are able to define k-bounded partial orders whose underlying undirected graph have the complete graph K<sup>4</sup> as a minor, whereas it is known that no such partial order can be series parallel. It is also worth noting that none of the formalisms described in this paragraph are able to represent the causal behavior of arbitrary bounded Petri nets. Generalizations of finite automata accepting infinite words have been considered in several contexts. For instance, regular sets of infinite message sequence charts [23]. Automata over message sequence charts capable of accepting infinite MSCs were studied in [4]. We note that we do not consider automata capable of accepting infinite partial orders in this work.

#### **2 The Causal Semantics of Petri Nets**

In this section, we briefly define the classic notion of Petri-nets and describe their partial-order semantics. Within this semantics, partial orders are used to represent the causality between events in concurrent runs of a Petri net.

A Petri net is a tuple N = (P, T, W, m0) where P is a set of *places*, T is a set of *transitions* such that <sup>P</sup> <sup>∩</sup> <sup>T</sup> <sup>=</sup> <sup>∅</sup>, <sup>W</sup> : (<sup>P</sup> <sup>×</sup> <sup>T</sup>) <sup>∪</sup> (<sup>T</sup> <sup>×</sup> <sup>P</sup>) <sup>→</sup> <sup>N</sup> is a function that assigns a weight W(x, y) to each element (x, y) ∈ (P × T) ∪ (T × P), and <sup>m</sup><sup>0</sup> : <sup>P</sup> <sup>→</sup> <sup>N</sup> is a function that assigns a non-negative integer <sup>m</sup>0(p) to each place p ∈ P.

A marking for <sup>N</sup> is any function of the form <sup>m</sup> : <sup>P</sup> <sup>→</sup> <sup>N</sup>. Intuitively, a marking m assigns a number of tokens to each place of N. The marking m<sup>0</sup> is called the *initial marking* of N. If m is a marking and t is a transition in T, then we say that t is enabled at m if m(p)−W(p, t) ≥ 0 for every place p ∈ P. If this is the case, the firing of t yields the marking m which is obtained from m by setting m (p) = m(p) − W(p, t) + W(t, p) for every place p ∈ P. A firing sequence for N is a mixed sequence of markings and transitions s ≡ m<sup>0</sup> <sup>t</sup><sup>1</sup> −→ <sup>m</sup><sup>1</sup> <sup>t</sup><sup>2</sup> −→ ... <sup>t</sup>*<sup>n</sup>* −→ <sup>m</sup><sup>n</sup> such that for each i ∈ {1, ..., n}, t<sup>i</sup> is enabled at m<sup>i</sup>−<sup>1</sup>, and m<sup>i</sup> is obtained from m<sup>i</sup>−<sup>1</sup> by the firing of ti. We say that such a firing sequence is b-bounded if for each i ∈ {0, ..., n} and each p ∈ P, mi(p) ≤ b. We say that N is b-bounded if each of its firing sequences is b-bounded.

The partial order semantics of Petri nets is defined using the notion of Petrinet processes introduced by Goltz and Reisig in [18]. The information about the causality between events is extracted from objects called Petri net *processes*, which encode the production and consumption of tokens along a concurrent run of the Petri net in question. The definition of processes, in turn, is based on the notion of *occurrence net*.

An occurrence net is a DAG <sup>O</sup> = (<sup>B</sup> <sup>∪</sup>˙ V,F) where the vertex set <sup>B</sup> <sup>∪</sup>˙ <sup>V</sup> is partitioned into a set B, whose elements are called conditions, and a set V , whose elements are called events. The edge set F ⊆ (B × V ) ∪ (V × B) is restricted in such a way that for every condition b ∈ B, |{(b, v) | v ∈ V }| ≤ 1 and |{(v, b) | v ∈ V }| ≤ 1. In other words, conditions in an occurrence net are unbranched. For each condition b ∈ B, we let InDegree(b) denote the number of edges having b as target. A process of a Petri net N is an occurrence net whose conditions are labeled with places of N, and events are labeled with transitions of N. Processes are intuitively used to describe the token game in a concurrent execution of the net.

**Definition 1 (Process** [18]**).** *A process of a Petri net* N = (P, T, W, m0) *is a labeled DAG* <sup>π</sup> = (B∪˙ V, F, ρ) *where* (B∪˙ V,F) *is an occurrence net and* ρ : (B ∪ V ) → (P ∪ T) *is a labeling function satisfying the following properties.*


Item 1 says that the conditions of a process are labeled with places, while the events are labeled with transitions. Item 2 says that the minimal vertices of the process, are conditions. Intuitively, each of these conditions represent a token in the initial marking of N. Thus for each place p of N the process has m0(p) minimal conditions labeled with the place p. Item 3, determines that the token game of a process corresponds to the token game defined by the firing of transitions in the Petri net N. Thus if a transition t consumes W(p, t) tokens from place p and produces W(t, p) tokens at place p, then each event labeled with t must have W(p, t) in-neighbours that are conditions labeled with p, and W(t, p) out-neighbours that are conditions labeled with p.

**Fig. 1.** A 2-bounded Petri net N. A process π of N. The partial order <sup>π</sup> derived from π. The extension ˆ<sup>π</sup> of π.

Let R ⊆ X × X be a binary relation over a set X. We denote by **tc**(R) the transitive closure of R. If π = (B ∪ V, F, ρ) is a process then the *causal order* of π is the partial order <sup>π</sup> = (V, **tc**(F)|<sup>V</sup> <sup>×</sup><sup>V</sup> , ρ|<sup>V</sup> ) which is obtained by taking the transitive closure of F and subsequently by restricting **tc**(F) to pairs of events of V . In other words the causal order of a process π is the partial order induced by π on its events.

If = (V, <, l) is a partial order, then we let <sup>∗</sup> = (V , < , l ) be the *extended version* of , where V = V ∪ {vι, vε}, < =< ∪({vι}×V )∪(V × {vε})∪ {(vι, vε)}, l |<sup>V</sup> = l, l (vι) = ι and l (vε) = ε. In other words, is obtained from by the addition of an element v<sup>ι</sup> that is smaller than all other elements, and an element v<sup>ε</sup> that is greater than all other elements. The addition of these minimal and maximal elements to a partial order are made to avoid the consideration of special cases in some of our future lemmas. All of our results work if ignore this step, but at the expense of more repetitive proofs that deal with corner cases. We denote by P*cau* (N) the set of all extended versions of partial orders derived from processes of <sup>N</sup>: <sup>P</sup>*cau* (N) = {ˆπ|<sup>π</sup> is a process of <sup>N</sup>}. We say that <sup>P</sup>*cau* (N) is the causal language of N. We observe that several processes of N may correspond to the same partial order in P*cau* (N).

Recall that the Hasse diagram of a partial order = (V, <, l) is the DAG H = (V,E) with the smallest number of edges with the property that <= **tc**(E). It is a well known result in partial order theory that this DAG is unique. We say that is a <sup>k</sup>-partial-order, for some <sup>k</sup> <sup>∈</sup> <sup>N</sup>, if there exist <sup>k</sup> paths <sup>p</sup>1,..., <sup>p</sup><sup>k</sup> in <sup>H</sup> that cover all vertices and edges of H. In other words, V = - <sup>i</sup> V<sup>i</sup> and E = - <sup>i</sup> E<sup>i</sup> where for each i ∈ {1,...,k}, V<sup>i</sup> and E<sup>i</sup> are the vertex-set and edge-set of the path p<sup>i</sup> respectively. We note that the paths in the cover are not necessarily vertex-disjoint nor edge-disjoint.

For each <sup>k</sup> <sup>∈</sup> <sup>N</sup>, let <sup>P</sup>*cau* (N, k) denote the set of <sup>k</sup>-partial-orders which are causal-orders of N. It can be shown that if N is b-bounded, then every causalorder of N = (P, T, W, m0) is a (b·|P|)-partial-order. In other words, each causalorder of N can be covered by at most b · |P| paths. This implies that P*cau*(N) = P*cau* (N, b · |P|).

#### **3 Message Sequence Chart Languages**

Message Sequence Charts (MSCs) are a suitable formalism for the representation of the exchange of messages between processes of a concurrent systems. In particular, during the last two decades, MSCs have been used to specify runs of telecommunication protocols. Intuitively, an MSC can be formalized as a partial-order that represents the causality between messages exchanged in a given concurrent run. Infinite families of MSCs, and therefore infinite families of partial-orders, can be specified using equivalent formalisms such as *message sequence graphs*, *hierarchical (or high-level) message sequence charts (HMSCs)* [2,30,32], or *message sequence chart automata* which will be defined below.

We formalize MSCs according to the terminology used in [30]. Let J be a finite set of processes, also called instances. For each instance i ∈ J , we associate a finite set of actions Σ<sup>i</sup> = Σint <sup>i</sup> <sup>∪</sup> <sup>Σ</sup>! <sup>i</sup> <sup>∪</sup> <sup>Σ</sup>? <sup>i</sup> . This set is partitioned into a set Σint i of *internal actions*, a set Σ! <sup>i</sup> = {i!j : j ∈ J \{i}} of *send actions*, and a set Σ? <sup>i</sup> = {i?j : j ∈ J \{i}} of *receive actions*. We shall assume that for each two distinct instances i, j ∈ J , Σ<sup>i</sup> ∩ Σ<sup>j</sup> = ∅. The set of actions associated with J is defined as Σ<sup>J</sup> = - <sup>i</sup>∈J <sup>Σ</sup>i. Given an action <sup>a</sup> <sup>∈</sup> <sup>Σ</sup><sup>J</sup> , *Ins*(a) denotes the unique instance i such that a ∈ Σi. For each Σ<sup>J</sup> -labeled partial-order = (V, <, l) and each vertex v ∈ V , we let *Ins*(v) = *Ins*(l(v)) be the instance of J where the action l(v) occurs. For each i, j ∈ J with i = j, and each subset X ⊆ V we let #i!<sup>j</sup> (X) = |{<sup>v</sup> <sup>∈</sup> <sup>X</sup> <sup>|</sup> <sup>l</sup>(v) = <sup>i</sup>!j}| be the number of messages sent from <sup>i</sup> to <sup>j</sup>, and by #i?<sup>j</sup> (X) = {<sup>v</sup> <sup>∈</sup> <sup>X</sup> <sup>|</sup> <sup>l</sup>(v) = <sup>i</sup>?j} be the number of messages received by <sup>i</sup> which were sent by j. We write v ≤ v as a shortcut for v<v ∨v = v . For each v ∈ V we let ↓ v = {v | v ≤ v} be the set of all nodes of which are smaller or equal to v. We write v ≺ v to indicate that v<v and for every u ∈ V , v<u ≤ v ⇒ u = v . In other words, v ≺ v if v is an out-neighbour of v in the Hasse diagram of .

**Definition 2 (Message Sequence Chart (MSC)).** *Let* J *be a set of processes. A message sequence chart over* J *is a* Σ<sup>J</sup> *-labeled partial-order* M = (V, <, l) *satisfying the following properties.*


Intuitively, Condition 1 states that actions occurring on the same process are linearly ordered. Condition 2 states that for each two distinct processes i, j, the number of messages send from i to j is equal to the number of messages received by <sup>j</sup> coming from <sup>i</sup>. Condition <sup>3</sup> states that for each <sup>n</sup> <sup>∈</sup> <sup>N</sup>, the <sup>n</sup>-th message sent from i to j is received when the n-th action j?i occurs, i.e., the channels in which these messages are transmitted are assumed to be FIFO. Finally, Condition 4 establishes a causal dependence between send and receive actions from distinct processes.

Let M = (V, <, l) and M = (V , < , l ) be MSCs over J . The composition of M with M is the MSC M ◦M = (V , <, l) where V = V ∪V , l = l∪l , and < is the transitive closure of the relation < ∪ < ∪ {(v, v ) ∈ V ×V |Ins(v) = Ins(v )}.

To define infinite families of partial-orders, we use the notion of *message sequence chart automata* (MSC Automata). Let <sup>M</sup><sup>J</sup> be the set of all finite MSCs over <sup>J</sup> . Here, the set <sup>M</sup><sup>J</sup> may be regarded as an (infinite) alphabet of MSCs.

**Definition 3.** *Let* J *be a set of processes. A message sequence chart automaton (MSC automaton) over* J *is a finite automaton* M = (Q, R, Q0, F) *where* Q *is finite a set of states,* Q<sup>0</sup> ⊆ Q *is a set of initial states,* F *is a set of final states and* <sup>R</sup> <sup>⊆</sup> <sup>Q</sup> <sup>×</sup> <sup>M</sup><sup>J</sup> <sup>×</sup> <sup>Q</sup>*.*

We say that a sequence M1M2...M<sup>n</sup> of MSCs is accepted by M if there is a sequence q<sup>0</sup> <sup>M</sup><sup>1</sup> −−→ <sup>q</sup><sup>1</sup> <sup>M</sup><sup>2</sup> −−→ ... <sup>M</sup>*<sup>n</sup>* −−→ <sup>q</sup><sup>n</sup> where <sup>q</sup><sup>0</sup> <sup>∈</sup> <sup>Q</sup>0, <sup>q</sup><sup>n</sup> <sup>∈</sup> <sup>F</sup> and (q<sup>i</sup>−<sup>1</sup>, Mi, qi) <sup>∈</sup> <sup>R</sup> for each i ∈ {1, ..., n}. An MSC automaton generates two languages. At the syntactic level, L(M) is the set of all sequences M1M2...M<sup>n</sup> of MSCs accepted by M. At the semantic level,

$$\mathcal{L}\_{po}(\mathcal{M}) = \{M\_1 \circ \dots \circ M\_n \mid n \in \mathbb{N}, M\_1 \dots M\_n \in \mathcal{L}(\mathcal{M})\}$$

is the set of all MSCs obtained by composing each sequence of MSCs in L(M). We note that an MSC language can be represented by an MSC automaton if and only if it can be represented by the more traditionally used message sequence graphs [2,30,32]. Nevertheless, we choose to work with MSC automata due to the fact that the proof of our results will be shorter.

If M is an MSC, then the communication graph of M, denoted by G(M), has the processes of M as vertices, and has one edge e with source in a process p and target in a process q if and only if p sends some message to q in M. We say that M is locally-synchronized if the graph G(M) has a unique non-trivial<sup>1</sup> strongly connected component, and every vertex that is not in such component is isolated. We say that an MSC automaton M is locally-synchronized if for each loop q<sup>1</sup> <sup>M</sup><sup>1</sup> −−→ <sup>q</sup><sup>2</sup> <sup>M</sup><sup>2</sup> −−→ ...q<sup>n</sup> <sup>M</sup>*<sup>n</sup>* −−→ <sup>q</sup><sup>1</sup> in <sup>M</sup>, the MSC <sup>M</sup><sup>1</sup> ◦ <sup>M</sup><sup>2</sup> ◦ ... ◦ <sup>M</sup><sup>n</sup> is locally-synchronized.

The partial-order language accepted by an MSC automaton is linearizationregular [19] if the set of linearizations of partial-orders in L*po*(M) can be recognized by a finite automaton over the alphabet Σ<sup>J</sup> . In other words, L*po*(M) is linearization-regular if the following set of strings over Σ<sup>J</sup> is regular in the usual sense of finite automata theory.

$$\operatorname{lin}(\mathcal{M}) = \bigcup\_{\ell \in \mathcal{L}\_{\text{po}}(\mathcal{M})} \operatorname{lin}(\ell). \tag{1}$$

It can be shown that an MSC language generated by an MSC automaton M is linearization-regular if and only if M is locally synchronized.

**Theorem 1 (**[2,31]**).** *Let* M *be an MSC automaton. Then* L*po*(M) *is linearization-regular if and only if* M *is locally-synchronized.*

#### **4 Slice Automata**

In this section we define *slices* and *slice automata*. Slice automata will be used to provide a static representation of infinite families of DAGs and infinite families of partial-orders. We note that slices can be related to several formalisms such as, multi-pointed graphs, [11], co-span decompositions [7] and graph transformations [3,6,11,35].

In what follows, T denotes a finite set of labels. A slice **S** = (V, E, l, s, t, [I, C, O]) is a (<sup>T</sup> <sup>∪</sup> <sup>N</sup>)-labeled DAG where the vertex set <sup>V</sup> <sup>=</sup> <sup>I</sup> <sup>∪</sup>˙ <sup>C</sup> <sup>∪</sup>˙ <sup>O</sup> is partitioned into an in-frontier I, a center C and an out-frontier O. The function <sup>l</sup> : <sup>V</sup> <sup>→</sup> <sup>T</sup> <sup>∪</sup> <sup>N</sup> labels the center vertices in <sup>C</sup> with elements of <sup>T</sup>, and the in- and out-frontier vertices with positive integers in such a way thatl(I) = {1, ..., |I|} and

<sup>1</sup> A strongly connected component is trivial if it has a unique vertex.

l(O) = {1, ..., |O|}. We require that each frontier-vertex v in I ∪ O is the endpoint of exactly one edge e ∈ E and that the edges are directed from the in-frontier to the out-frontier. More precisely, for each edge e ∈ E, we assume that s(e) ∈ I ∪ C and that t(e) ∈ C ∪ O. We may also speak of a slice **S** with frontiers (I,O) to indicate that the in-frontier of **S** is I and that the out-frontier of **S** is O.

**Fig. 2.** i) A slice and its pictorial representation. ii) Composition of slices.

A slice **S**<sup>1</sup> = (V1, E1, l1, s1, t1) with frontiers (I1, O1) can be glued to a slice **S**<sup>2</sup> = (V2, E2, l2, s2, t2) with frontiers (I2, O2) provided |O1| = |I2|. In this case the glueing gives rise to the slice **S**1◦**S**<sup>2</sup> = (V3, E3, l3, s3, t3) with frontiers (I1, O2) which is obtained by taking the disjoint union of **S**<sup>1</sup> and **S**2, and by fusing, for each i ∈ {1, ..., |O1|}, the unique edge e<sup>1</sup> ∈ E<sup>1</sup> for which l1(t1(e1)) = i with the unique edge e<sup>2</sup> ∈ E<sup>2</sup> for which l2(s2(e2)) = i. Formally, the fusion of e<sup>1</sup> with e<sup>2</sup> is performed by creating a new edge e<sup>12</sup> with source s3(e12) = s1(e1) and target t3(e12) = t2(e2), and by deleting e<sup>1</sup> and e2. Thus in the glueing process the vertices in the glued frontiers disappear.

A *unit slice* is a slice with exactly one vertex in its center. A slice is *initial* if it has empty in-frontier and *final* if it has empty out-frontier. The width of a slice **S** with frontiers (I,O) is defined as w(**S**) = max{|I|, |O|}. A *slice alphabet* is any finite set of slices. In particular, for each finite set of symbols <sup>T</sup> and each <sup>k</sup> <sup>∈</sup> <sup>N</sup> we let −→*Σ*(k, T) be the set of all unit slices **S** of width at most k whose center vertex is labeled with an element from T. Observe that the alphabet −→*Σ*(k, T) is finite and has asymptotically <sup>|</sup>T| · <sup>2</sup><sup>O</sup>(<sup>k</sup> log <sup>k</sup>) slices. A sequence **<sup>U</sup>** <sup>=</sup> **<sup>S</sup>**1**S**2...**S**<sup>n</sup> of unit slices is called a unit decomposition if **S**<sup>i</sup> can be glued to **S**i+1 for each <sup>i</sup> ∈ {1, ..., n−1}. In this case, we let ◦ **U**= **S**<sup>1</sup> ◦**S**<sup>2</sup> ◦ ... ◦**S**<sup>n</sup> be the DAG associated with **U**, which is obtained by glueing each two consecutive slices in **U**. The width of **U**, denoted by w(**U**), is defined as the maximum width of a slice occurring in **U**. We let −→*Σ*(k, T)<sup>∗</sup> be the set of all sequences of slices over −→*Σ*(k, T), and −→*Σ*(k, T)be the set of all unit decompositions over −→*Σ*(k, T).

**Definition 4 (Slice Automaton).** *Let* T *be a finite set of symbols and let* <sup>k</sup> <sup>∈</sup> <sup>N</sup>*. A slice automaton over* −→*Σ*(k, T) *is a finite automaton* <sup>A</sup> = (Q, <sup>R</sup>, q0, F) *where* Q *is a set of states,* q<sup>0</sup> ∈ Q *is an initial state,* F ⊆ Q *is a set of final states, and* <sup>R</sup> <sup>⊆</sup> <sup>Q</sup> <sup>×</sup> −→*Σ*(k, T) <sup>×</sup> <sup>Q</sup> *is a transition relation such that for every* q, q , q <sup>∈</sup> <sup>Q</sup> *and every* **<sup>S</sup>** <sup>∈</sup> −→*Σ*(k, T)*:*


**Fig. 3.** <sup>i</sup>) A slice automaton <sup>A</sup>. ii) A unit decomposition **<sup>U</sup>** accepted by <sup>A</sup>. iii) The DAG ◦ **U** obtained by glueing each two consecutive slices in **U**.

**Languages of a Slice Automaton.** A slice automaton A can be used to represent three types of languages. At a syntactic level, we have the slice language L(A) which consists of the set of all unit decompositions accepted by A.

$$\mathcal{L}(\mathcal{A}) = \{ \mathbf{S}\_1 \mathbf{S}\_2 ... \mathbf{S}\_n \mid \mathbf{S}\_1 \mathbf{S}\_2 ... \mathbf{S}\_n \text{ is accepted by } \mathcal{A} \}\tag{2}$$

At a semantic level, we have the graph language LG(A) which consists of all DAGs represented by unit decompositions in L(A), and the partial-order language L*po*(A), which consists of all partial-orders which arise as the transitive closure (**tc**) of DAGs in LG(A). Formally, the graph language, and the partialorder languages accepted by A are defined as follows.

$$\mathcal{L}\_{\mathcal{G}}(\mathcal{A}) = \{ \mathbf{\tilde{U}} \: \mid \: \mathbf{U} \in \mathcal{L}(\mathcal{A}) \} \qquad \qquad \mathcal{L}\_{po}(\mathcal{A}) = \{ \mathbf{tc}(\mathbf{\tilde{U}}) \: \mid \: \mathbf{\tilde{U}} \in \mathcal{L}\_{\mathcal{G}}(\mathcal{A}) \}. \tag{3}$$

Let H be a DAG whose vertices are labeled with elements from a finite set T. Then we let *ud*(H, −→*Σ*(k, T)) denote the set of all unit decompositions **U** in −→*Σ*(k, T) for which ◦ **<sup>U</sup>**<sup>=</sup> <sup>H</sup>. We say that a slice automaton <sup>A</sup> over −→*Σ*(k, T) is *saturated* if for every DAG <sup>H</sup> ∈ LG(A) we have that *ud*(H, −→*Σ*(k, T)) ⊆ L(A).

The transitive reduction of a DAG H = (V,E,l) is the (unique) minimal subgraph **tr**(H) of H with the same transitive closure as H. Note that **tc**(**tr**(H)) = **tc**(H). We say that a DAG H is transitively reduced if H = **tr**(H). Alternatively, we call a transitively reduced DAG a Hasse diagram. We say that a slice automaton A is transitively reduced if every DAG in LG(A) is transitively reduced. Theorem 2 states that any slice automaton A can be converted into a transitively reduced slice automaton **tr**(A) representing the same partial-order language in such a way that the saturation property is preserved.

**Theorem 2 (**[34]**).** *Let* <sup>A</sup> *be a slice automaton over* −→*Σ*(k, T)*. Then one can construct in time* <sup>2</sup><sup>O</sup>(<sup>k</sup> log <sup>k</sup>) ·|A| *a transitively reduced slice automaton* **tr**(A) *such that* L*po*(**tr**(A)) = L*po*(A)*. Additionally, if* A *is saturated, then so is* **tr**(A)*.*

Transitively reduced saturated slice automata are important for our setting because they can be used to canonically represent infinite families of partial-orders, and because they enjoy several nice decidability/closure properties. For instance, inclusion and emptiness of intersection of partial-order languages represented by such slice automata are decidable.

**Lemma 1 (Properties of Saturated Slice Automata).** *Let* A *and* A *be transitively-reduced slice automata over* −→*Σ*(k, T)*. Assume that* <sup>A</sup> *is saturated.*


Additionally, the partial order behavior of bounded Petri nets can be represented using transitively-reduced, saturated slice automata.

**Theorem 3 (**[33]**).** *Let* N = (P, T, W, m0) *be a* b*-bounded Petri net. Then for each* <sup>k</sup> <sup>∈</sup> <sup>N</sup> *one can construct in time* <sup>2</sup><sup>O</sup>(|<sup>P</sup> |·k·log <sup>b</sup>·k) · |T<sup>|</sup> <sup>|</sup><sup>P</sup> <sup>|</sup> *a transitively-reduced, saturated slice automaton* <sup>A</sup>(N, k) *over* −→*Σ*(k, T) *such that* L*po*(A(N, k)) = P*cau* (N, k)*.*

We note that every partial-order in the causal language of a b-bounded Petri net is a k-partial-order for some k ≤ b · |P|. Therefore, if we set A(N) = A(N, b · |P|) then L*po*(A(N)) = P*cau* (N). Finally, synthesis of Petri nets from (any) slice automata is decidable.

**Theorem 4 (Synthesis** [33]**).** *Let* <sup>A</sup> *be a slice automaton over* −→*Σ*(k, T)*. For each* <sup>b</sup> <sup>∈</sup> <sup>N</sup> *one can construct a* <sup>b</sup>*-bounded Petri net* <sup>N</sup> *satisfying the following properties.*


#### **5 Weak Saturation**

In this section, we introduce the notion of weak-saturation, a relaxation of the notion of saturation that is more suitable for applications involving other partial order formalisms. The main result of this section states that weak-saturated slice automata can be effectively transformed into saturated slice automata.

Let H = (V, E, l, s, t) be a T-labeled DAG and ω = (v1, ..., vn) be a topological ordering of the vertices of H. In other words, ω is a sequence of vertices from H such that for each i, j with i<j, there is no edge e ∈ E with source s(e) = v<sup>j</sup> and target <sup>t</sup>(e) = <sup>v</sup>i. We say that a unit decomposition **<sup>U</sup>** <sup>=</sup> **<sup>S</sup>**1**S**2...**S**<sup>n</sup> over −→*Σ*(k, T) is compatible with <sup>ω</sup> if ◦ **U** = H and for each i, v<sup>i</sup> is the center vertex of **S**i. Note that given a graph H and a topological ordering ω, there may be several unit decompositions of H compatible with ω. We denote by *ud*(H, ω, −→*Σ*(k, T)) the set of all unit decompositions of H over −→*Σ*(k, T) that are compatible with ω. Note that for each graph H, we have that

**Fig. 4.** A graph H, an ordering ω = (a, b, c, d) of the vertices of H, and all unit decompositions of H compatible with ω. For each unit decompositions **U**, **U** in *ud*(H, ω, −→*Σ*(k, T)), **U** is a twisting of **U**- .

$$\mathsf{u}\mathsf{d}(H,\overrightarrow{\Sigma}(k,T)) = \bigcup\_{\omega} \mathsf{u}\mathsf{d}(H,\omega,\overrightarrow{\Sigma}(k,T))\tag{4}$$

where ω ranges over all topological orderings of H.

**Definition 5 (Weak Saturation).** *We say that a slice automaton* A *is* weakly saturated *if for each DAG* H *in* LG(A)*, and each topological ordering* ω *of* H*,*

$$
\mathcal{L}(\mathcal{A}) \cap \mathfrak{ud}(H, \omega, \overrightarrow{\Sigma}(k, T)) \neq \emptyset.
$$

In other words, a slice automaton A is weakly saturated if for each graph H and each topological ordering ω of H there is at least one unit decomposition of H in L(A) which is compatible with ω. In Sect. 6 we will show that weak saturation is a decidable property. The following lemma states that each weakly saturated slice automaton can be transformed into a saturated slice automaton representing the same set of DAGs, and therefore the same set of partial-orders.

**Lemma 2.** *Let* <sup>A</sup> *be a weakly saturated slice automaton over* −→*Σ*(k, T)*. Then one can construct in time* <sup>2</sup><sup>O</sup>(<sup>k</sup> log <sup>k</sup>) · |A| *a saturated slice automaton* <sup>A</sup> *such that* LG(A) = LG(A )*.*

*Proof.* For w ≥ 0, let [w] = {1, ..., w}. We let [0] be the empty set ∅. A permutation of [w] is a bijective mapping *<sup>π</sup>* : [w] <sup>→</sup> [w]. We denote by <sup>∅</sup> the empty permutation *<sup>π</sup>* : [0] <sup>→</sup> [0]. Let **<sup>S</sup>** be a slice with frontiers (I,O) and let π : [|I|] → [|I|] and π : [|O|] → [|O|] be permutations. We denote by (π, **S**, π ) the slice that is obtained from **S** by permuting the labels of the in-frontier nodes according to π, and by permuting the labels of the out-frontier nodes according to π .

Let **<sup>U</sup>** <sup>=</sup> **<sup>S</sup>**1**S**2...**S**<sup>n</sup> be a unit decomposition over −→*Σ*(k, T), where each slice **<sup>S</sup>**<sup>i</sup> has frontiers (Ii, Oi). Let *<sup>π</sup>*1, ...,*π*<sup>n</sup>−<sup>1</sup> be a sequence where for each <sup>j</sup> ∈ {1, ..., n <sup>−</sup> <sup>1</sup>}, *<sup>π</sup>*<sup>j</sup> : [|O<sup>j</sup> <sup>|</sup>] <sup>→</sup> [|O<sup>j</sup> <sup>|</sup>] is a permutation. Then we say that the unit decomposition

$$\mathbf{U}' = (\emptyset, \mathbf{S}\_1, \pi\_1)(\pi\_1, \mathbf{S}\_2, \pi\_2)...(\pi\_n, \mathbf{S}\_n, \emptyset)$$

is a twisting of **U**. Note that ◦ **U**= ◦ **U** (see Fig. 4). In other words, if **U** is a twisting of **U** then both decompositions give rise to the same DAG. Conversely, if **U** and **U** are compatible with the same topological ordering of a graph H then **U** and **U** are twistings of each other. These remarks are formalized in the next proposition.

**Proposition 1.** *Let* H *be a DAG and* **U** *and* **U** *be unit decompositions of* H*. Then* **U** *is a twisting of* **U** *if and only if there is a topological ordering* ω *of* H *such that* **<sup>U</sup>**, **<sup>U</sup>** <sup>∈</sup> *ud*(H, ω, −→*Σ*(k, T))*.*

We say that a slice automaton <sup>A</sup> over −→*Σ*(k, T) is *twisted* if whenever a unit decomposition **U** belongs to L(A) then all its twistings also belong to L(A). Alternatively, in view of Proposition 1, A is twisted if whenever

$$\mathcal{L}\_{\mathcal{G}}(\mathcal{A}) \cap \mathsf{ud}(H, \omega, \overrightarrow{\Xi}(k, T)) \neq \emptyset$$

for a DAG H and a topological ordering ω of H, we have that *ud*(H, ω, −→*Σ*(k, T)) ⊆ L(A). Using Eq. <sup>4</sup> the notion of saturation can be redefined in terms of weak saturation and twisting.

**Proposition 2.** *Let* <sup>A</sup> *be a slice automaton over* −→*Σ*(k, T)*. Then* <sup>A</sup> *is saturated if and only if it is both twisted and weakly saturated.*

Therefore, to prove Lemma 2 it is enough to devise a procedure that takes a slice automaton A and returns a slice automaton *tw*(A) whose slice language L(*tw*(A)) consists of all twisted versions of unit decompositions in L(A). If A is weakly saturated, then *tw*(A) is (fully) saturated.

Let <sup>A</sup> = (Q, Δ, q0, F). We assume that all states of <sup>A</sup> can be reached from the initial state q<sup>0</sup>, and reach some final state in F. Let q be a state in Q. We say that the width of q is w if either there is a transition (q, **S**, q ) such that the infrontier of **S** has size w, or there is a transition (q , **S**, q) such that the out-frontier of **S** has size w. Note that conditions 1–3 of the definition of slice automaton (Definition 4) ensure that the notion of width of a state is well defined. Now the automaton *tw*(A)=(Q , Δ , r 0, F ) is defined as follows:

$$r\_0' = q\_\emptyset^0 \quad F' = \{ q\_\emptyset \mid q \in F \}$$

$$Q' = \{ q\_\pi \mid \pi : [w(q)] \to [w(q)] \text{ is a permutation.} \}\tag{5}$$

$$\Delta' = \{ (q\_\pi, (\pi, \mathbf{S}, \pi'), q\_{\pi'}) \mid (q, \mathbf{S}, q') \in \Delta, \ q\_\pi, q\_{\pi'} \in Q' \}$$

It is immediate to check that a unit decomposition **U** = **S**1**S**2...**S**<sup>n</sup> is accepted by A if and only each twisting **U** = (∅, **S**1, π1)(π1, **S**2, π2)...(π<sup>n</sup>−<sup>1</sup>, **S**n, ∅) is accepted by A . Therefore, the automaton *tw*(A) is twisted. Additionally, if A is weakly saturated, then by Eq. 4 we have that *tw*(A) is saturated. Finally, we note that the size of <sup>A</sup> is at most 2<sup>O</sup>(<sup>k</sup> log <sup>k</sup>) · |A|, since there can be at most O(k!) = 2<sup>O</sup>(<sup>k</sup> log <sup>k</sup>) permutations of a set of labels with at most k elements. -

#### **6 Slice Traces**

In this section we introduce the notion of slice traces, and use this notion to show that the weak-saturation property for slice automata is decidable. This notion will also be used in Sect. 3 to establish connections between MSC languages and saturated slice languages.

We say that two slice strings **<sup>U</sup>**, **<sup>U</sup>** <sup>∈</sup> −→*Σ*(k, T)<sup>∗</sup> are locally −→*Σ*(k, T) equivalent, and denote this fact by **U** <sup>k</sup> **U** , if there exist **<sup>W</sup>**,**W** <sup>∈</sup> −→*Σ*(k, T)<sup>∗</sup> and **S**1, **S** <sup>1</sup>, **S**2, **S** <sup>2</sup> <sup>∈</sup> −→*Σ*(k, T) with **<sup>S</sup>**<sup>1</sup> ◦ **<sup>S</sup>**<sup>2</sup> <sup>=</sup> **<sup>S</sup>** <sup>1</sup> ◦ **S** <sup>2</sup> such that **U** = **WS**1**S**2**W** and **U** = **WS** 1**S** <sup>2</sup>**W** (Fig. 5).

**Fig. 5.** Local Equivalence. **S**1**S**<sup>2</sup> is 4-equivalent to **S**- <sup>1</sup>, **S**- 2.

We let <sup>k</sup> ≡ ⊆ −→*Σ*(k, T)<sup>∗</sup> <sup>×</sup>−→*Σ*(k, T)<sup>∗</sup> be the equivalence relation defined on slice strings by taking the reflexive, symmetric and transitive closure of <sup>k</sup> . We note that if **U** is a unit decomposition in −→*Σ*(k, T) then any slice string **U** that is −→*Σ*(k, T)-equivalent to **U** is also a unit decomposition in −→*Σ*(k, T)-, and additionally, ◦ **U**= ◦ **U** . We note that there may exist unit decompositions in −→*Σ*(k, T)- which are not −→*Σ*(k, T)-equivalent but which are −→*Σ*(k , T)-equivalent for some k > k. Nevertheless, the following proposition states that for each k-coverable DAG H, <sup>k</sup> ≡-equivalence is already enough to relate any two unit decompositions of H.

**Proposition 3.** *Let* **U**<sup>1</sup> *and* **U**<sup>2</sup> *be unit decompositions in* −→*Σ*(k, T) *such that the DAGs* ◦ **U**<sup>1</sup> *and* ◦ **U**<sup>2</sup> *are* k*-coverable. Then* ◦ **U**1= ◦ **U**<sup>2</sup> *if and only if* **U**<sup>1</sup> k ≡ **U**2*.*

There is a substantial difference between our notion of independence, defined on slice alphabets and the notion of independence in Mazurkiewicz trace theory. While the independence relation on slices is determined solely based on the structure of the slices (Fig. 5), without taking into consideration the events that label their center vertices, the Mazurkiewicz independence relation is defined directly on events. As a consequence, once an independence relation I is fixed, the nature of the partial-orders that can be represented as traces with respect to I is restricted. This is valid even for more general notions of traces, such as Diekert's semi-traces [8] and the context dependent traces of [20], in which for instance, partial-orders containing auto-concurrency<sup>2</sup> cannot be represented. In

<sup>2</sup> Auto-concurrency is the process of firing two transitions with the same label simultaneously.

our setting, any partial order labeled over a set of events T may be represented by a slice trace: namely the set of unit decompositions of its Hasse diagram.

**Theorem 5.** *Let* <sup>A</sup> *be a slice automaton over a slice alphabet* −→*Σ*(k, T) *representing a set of* k*-partial-orders. Then we may effectively determine whether the slice language generated by* A *is weakly saturated.*

*Proof.* Assume without loss of generality that the slice automaton A is transitively reduced. Otherwise, just apply the transitive reduction algorithm from [34]. Since each partial-order ∈ L*po*(A) is a k-partial-order, the Hasse digram H of can be covered by k paths. Therefore, by Proposition 3, any unit decomposition of H has width at most k. Now let *tw*(A) be automaton obtained from A by applying the twisting procedure in the proof of Lemma 2. Then by Proposition 2 the automaton A is weakly saturated if and only if *tw*(A) is saturated. Therefore, it is enough to verify whether *tw*(A) is saturated. With this in mind, it is enough to test the following condition. If a slice word w**S**1**S**2u is generated by A then every word w**S** 1**S** <sup>2</sup>u satisfying **S** <sup>1</sup> ◦ **S** <sup>2</sup> = **S**<sup>1</sup> ◦ **S**<sup>2</sup> is generated by A as well. Let A be the minimal deterministic slice automaton generating the same slice language as *tw*(A). Then any unit decomposition **S**1**S**<sup>2</sup> ··· **S**<sup>n</sup> ∈ L(A ) corresponds to a unique computational path in A . Therefore to verify our condition, we just need to determine whether A is "diamond" closed. In other words we need to test whether for each pair of transition rules q**S**1r and r**S**2q of the A and each unit decomposition **S** 1**S** <sup>2</sup> of **S**<sup>1</sup> ◦ **S**2, A has a state r and transitions q**S** <sup>1</sup>r and r **S** 2q . Clearly this condition can be verified in polynomial time on the size of A , since **<sup>S</sup>**<sup>1</sup> ◦**S**<sup>2</sup> can have at most 2<sup>O</sup>(k·log <sup>k</sup>) possible decompositions. -

#### **7 From MSC Automata to Slice Automata**

In this section we define the notion of locally-synchronized slice automata. Let **S** be a slice (possibly with several vertices in the center) with k in-frontier vertices v1, ..., v<sup>k</sup> and k out-frontier vertices u1, ..., uk. For each i ∈ {1, ..., k}, we say that a path p<sup>i</sup> from v<sup>i</sup> to u<sup>i</sup> is *trivial* if v<sup>i</sup> and u<sup>i</sup> are the only vertices in pi. Let p1,...,p<sup>k</sup> be paths such that for each i, p<sup>i</sup> is a path from v<sup>i</sup> to ui. We let the communication graph comm(**S**, p1, ..., pk) be the directed graph whose vertices are paths in {p1, ..., pk}, and such that for each i, j ∈ {1, ..., k}, there is an edge from p<sup>i</sup> to p<sup>j</sup> if either these paths share a vertex or there is an edge with source in some vertex of p<sup>i</sup> and target in some vertex of p<sup>j</sup> . We say that **S** is *locally-synchronized* comm(**S**, p1, ..., pk) has at most one strongly connected component with more than one vertex. Note that trivial paths correspond to isolated vertices. This notion of local synchronization for slices, generalizes the notion of local synchronization for message sequence charts in the sense that processes correspond to paths, and isolated vertices in the communication graph of an MSC correspond to trivial paths.

**Definition 6 (Locally-Synchronized Slice Automaton).** *A slice automaton* A *is locally-synchronized if for every loop* q<sup>1</sup> **<sup>S</sup>**<sup>1</sup> −→ <sup>q</sup><sup>2</sup> **<sup>S</sup>**<sup>2</sup> −→ ... **<sup>S</sup>***n*−<sup>1</sup> −−−→ q<sup>n</sup> **<sup>S</sup>***<sup>n</sup>* −−→ <sup>q</sup><sup>1</sup> *in* A*, the slice* **S**<sup>1</sup> ◦ **S**2... ◦ **S**<sup>n</sup> *is locally synchronized.*

The next theorem states that any locally-synchronized slice automaton can be transformed further into a saturated slice automaton representing the same partial order language as the original one.

**Theorem 6.** *Let* <sup>T</sup> *be a finite set of symbols, and* <sup>k</sup> <sup>∈</sup> <sup>N</sup>*. Let* <sup>A</sup> *be a locallysynchronized slice automaton over* −→*Σ*(k, T)*. Then one can construct a saturated slice automaton* A *such that* L*po*(A ) = L*po*(A)*.*

The following theorem is the main result of this section. It states that MSC automata can be converted into slice automata representing the same partialorder language. Additionally, this conversion transforms locally-synchronized MSC automata into saturated slice automata.

**Theorem 7 (From MSC Automata to Slice Automata).** *Let* M *be an MSC automaton over* J *. Then one can construct a transitively-reduced slice automaton* A(M) *satisfying* L*po*(M) = L*po*(A(M))*. Furthermore, if* M *is locally-synchronized, then* A(M) *is saturated.*

*Proof.* Let <sup>J</sup> <sup>=</sup> {1, ..., k} be a set of processes. We let **<sup>S</sup>**<sup>ι</sup> (J ) be the slice with empty in-frontier I = ∅, k out-frontier vertices O = {v1, ..., vk} where each v<sup>i</sup> is labeled with the number i, and with a unique vertex v in the center which is connected to each vertex in O. We say that **S**<sup>ι</sup> (J ) is the *initial slice* of J .

Analogously, let **<sup>S</sup>**<sup>ε</sup>(<sup>J</sup> ) be the slice with empty out-frontier <sup>O</sup> <sup>=</sup> <sup>∅</sup>, <sup>k</sup> infrontier vertices I = {u1, ..., uk} where each u<sup>i</sup> is labeled with the number i, and with a unique center vertex v in the center. For each i there is an edge with source in <sup>u</sup><sup>i</sup> and target in <sup>v</sup>. We say that **<sup>S</sup>**<sup>ε</sup>(<sup>J</sup> ) is the *final slice* of <sup>J</sup> .

Now let M be an MSC over J . Then we let **S**(M) be the slice (not necessarily a unit slice) constructed as follows. **S**(M) has k in-frontier vertices x1, ..., xk, and k out-frontier vertices y1, ..., yk. Let H be the Hasse diagram of M. Then for each i ∈ {1, ..., k} proceed as follows. If H has no vertex labeled with an element of Σi, then add an edge from the in-frontier vertex x<sup>i</sup> to the out-frontier vertex yi. Otherwise, if such a vertex exists, then add an edge from x<sup>i</sup> to the (unique) minimal vertex of H labeled with an element of Σi, and an edge from the (unique) maximal vertex of H labeled with an element of Σ<sup>i</sup> to the out-frontier vertex yi. Note that the transitive closure of the slice **S**<sup>ι</sup> (<sup>J</sup> ) ◦**S**(M) ◦**S**<sup>ε</sup>(<sup>J</sup> ) is precisely the extension of the partial-order M (see Sect. 2). We let **W**(M) = **S**1**S**2...**S**<sup>n</sup> be an arbitrary sequence of unit slices such that **S**(M) = **S**<sup>1</sup> ◦ **S**<sup>2</sup> ◦ ... ◦ **S**n.

Now let M be an MSC automaton over J . We will show how to construct a slice automaton A (M) over −→*Σ*(|J |, Σ<sup>J</sup> ) such that <sup>L</sup>*po*(A (M)) = L*po*(M). The conversion is done as follows. Let M be an MSC with m nodes and let **W**(M) = **S**1**S**2...**S**m. We replace each transition (q,M, q ) in M by a sequence of transitions

$$(q, \mathbf{S}\_1, q\_1)(q\_1, \mathbf{S}\_2, q\_2)...(q\_{n-1}, \mathbf{S}\_n, q').$$

Now we create an initial state q<sup>ι</sup> and add the transition (qι, **S**<sup>ι</sup> (J ), q) for each initial state q of M. Analogously, we create a final state q<sup>ε</sup> and add the transition (q, **<sup>S</sup>**ε(<sup>J</sup> ), qε) for each final state of <sup>M</sup>. Now it is immediate to check that <sup>M</sup> accepts a sequence M1M2...M<sup>n</sup> of MSCs if and only if A (M) accepts the unit decomposition

$$\mathbf{U} = \mathbf{S}^{\iota}(\mathcal{I})\mathbf{W}(M\_1)\mathbf{W}(M\_2)...\mathbf{W}(M\_n)\mathbf{S}^{\varepsilon}(\mathcal{I})...$$

This implies that **tc**( ◦ **U**) = <sup>∗</sup> where = M<sup>1</sup> ◦ M<sup>2</sup> ◦ ... ◦ Mn. Therefore, L*po*(A (M)) = L*po*(M).

Now assume that L*po*(M) is linearization-regular. Then by Theorem 1, we may assume that M is locally-synchronized. Let **W**(M1)**W**(M2)...**W**(Mn) label a loop in A (M). Then M1M2...M<sup>n</sup> labels a loop in M. Since by assumption M is locally-synchronized, we have that the MSC M<sup>1</sup> ◦ M<sup>2</sup> ◦ ... ◦ M<sup>n</sup> is locallysynchronized. This implies that the slice **S**(M1) ◦ **S**(M2) ◦ ... ◦ **S**(Mn) is also locally-synchronized. Since a sequence of MSCs M1M2...M<sup>n</sup> labels a loop in M if and only if the sequence of unit slices **W**(M1)**W**(M2)...**W**(Mn) labels a loop in A (M), we have that A (M) is locally-synchronized. Therefore, as a last step we apply Theorem 6 to construct a slice automaton A(M) which is saturated and has the same partial-order language as A (M), and therefore, the same partial-order language as M. -

By combining Theorem 7 with Theorem 3 and Lemma 1, we have the following theorem.

**Theorem 8.** *Let* M *be an MSC automaton over* J *and* N *be* b*-bounded Petri net with transition set* T = J *.*


*Proof.* Let A be the slice automaton derived from M as in Theorem 7. Then A is transitively-reduced and L*po*(M) = L*po*(A). Let A be the slice automaton constructed from N as in Theorem 3. Then A is transitively-reduced, saturated and L*po*(A) = P*cau* (N). By Lemma 1, we have that it is decidable whether L*po*(A) ∩ L*po*(A ) = ∅ and whether L*po*(A) ⊆ L*po*(A ). Finally, if M is locally synchronized, then A is saturated, and therefore the inclusion L*po*(A ) ⊆ L*po*(A) is also decidable. -

By combining Theorem 7 with Theorem 9 we have the following theorem.

**Theorem 9 (Synthesis From MSC Automata).** *Let* M *be an MSC automaton over* <sup>J</sup> *. For each* <sup>b</sup> <sup>∈</sup> <sup>N</sup> *one can construct a* <sup>b</sup>*-bounded Petri net* N *satisfying the following properties.*


*Additionally, if* M *is locally synchronized, then one can decide whether* P*cau* (N) = L*po*(M)*.*

*Proof.* Let A be the slice automaton of Theorem 7. Then A is transitivelyreduced and L*po*(M) = L*po*(A). By Theorem 4, one can construct a b-bounded Petri net N such that L*po*(A) ⊆ P*cau* (N), and such that there is no b-bounded Petri net N with L*po*(A) ⊆ P*cau* (N ) -P*cau* (N).

Note that if there is a b-bounded Petri net whose causal behavior is equal to L*po*(A), then by minimality, we have that P*cau* (N) = L*po*(A). Nevertheless, in general it is not possible to verify whether equality is achieved. On the other hand, if M is locally synchronized, then by Item 3 of Theorem 8, one can also test whether the equality P*cau* (N) ⊆ L*po*(A) holds. -

#### **8 From Mazurkiewicz Traces to Slice Languages**

In Mazurkiewicz trace theory, partial-orders are represented as equivalence classes of words over an alphabet of events [28]. Given an alphabet T of events and a symmetric and anti-reflexive *independence relation* I ⊆ T × T, a string αabβ is defined to be similar to the string αbaβ (αabβ αbaβ) provided (a, b) ∈ I. A trace is then an equivalence class of the transitive reflexive closure <sup>∗</sup> of the relation . We denote by [α]<sup>I</sup> the trace corresponding to a string α ∈ T <sup>∗</sup>.

A partial-order <sup>I</sup> (α) is associated with a string α ∈ T <sup>∗</sup> of events in the following way: First we consider a dependence DAG *dep*<sup>I</sup> (α)=(V,E,l) that has one vertex v<sup>i</sup> ∈ V labeled by the event α<sup>i</sup> for each i ∈ {1, ..., |α|}. An edge connects v<sup>i</sup> to v<sup>j</sup> in E if and only if i<j and (αi, α<sup>j</sup> ) ∈/ I. Then <sup>I</sup> (α) is the transitive closure of dep<sup>I</sup> (α). One may verify that two strings induce the same partial-order if and only if they belong to the same trace. The trace language induced by a string language L ⊆ T <sup>∗</sup> with respect to an independence relation I is the set [L]<sup>I</sup> = {[α]<sup>I</sup> |α ∈ L} and the trace closure of L is the language <sup>L</sup><sup>I</sup> <sup>=</sup> <sup>∪</sup><sup>α</sup>∈L[α].

Given a finite automaton F over an alphabet T and an independence relation I ⊂ T × T, we denote by L(F) the regular language defined by F and by L*po*(F, I) = {<sup>I</sup> (α)|α ∈ L(F)} the partial-order language induced by (F, I). We call the pair (F, I) a Mazurkiewicz pair. We say that L(F) is trace-closed if [α]<sup>I</sup> ⊆ L(F) for each α ∈ L(F). As an abuse of terminology, we may say that the Mazurkiewicz pair (F, I) is trace-closed.

We let <sup>L</sup>ˆ*po*(F, <sup>I</sup>) = {ˆ <sup>|</sup> ∈ L*po*(F, <sup>I</sup>)} be the set of extensions of partialorders in L*po*(F, I). We note that extensions are only considered to make the construction of the automaton A(F, I) slightly cleaner. With some easy case analysis one can construct slice automata whose partial-order language is L*po*(F, I) instead of <sup>L</sup>ˆ*po*(F, <sup>I</sup>). The next theorem (Theorem 10) states that for any finite automaton F and independence relation I, one can construct a slice automaton <sup>A</sup>(F, <sup>I</sup>) whose partial-order language is equal to <sup>L</sup>ˆ*po*(F, <sup>I</sup>).

**Theorem 10 (From Traces to Slices).** *Let* F *be a finite automaton over an alphabet* T*, and* I ⊂ T ×T *an independence relation. Then for some* k ≤ |T| <sup>2</sup> *one can construct a transitively-reduced slice automaton* <sup>A</sup>(F, <sup>I</sup>) *over* −→*Σ*(k, T ∪{ι, ε}) *such that* <sup>L</sup>*po*(A(F, <sup>I</sup>)) = <sup>L</sup>ˆ*po*(F, <sup>I</sup>). *Additionally, if* (F, <sup>I</sup>) *is trace-closed, then* A(F, I) *is saturated.*

**Fig. 6.** Mapping an independence alphabet (T, <sup>I</sup>) to a slice alphabet −→*Σ*(T, <sup>I</sup>) <sup>⊆</sup> −→*Σ*(k, T) where <sup>k</sup> ≤ |T<sup>|</sup> 2.

In the remainder of this section we prove Theorem 10. We note that the difficulty in the construction of the automaton A(F, I) lies in showing that A(F, I) is saturated whenever L(F) is trace-closed. As a first step in the proof, we will use the independence alphabet (T, I) to construct a slice alphabet −→*Σ*(T, <sup>I</sup>) = {**S**a|<sup>a</sup> <sup>∈</sup> <sup>T</sup>}∪{**S**ι, **<sup>S</sup>**ε} with the following property: For each string α = α1α2...α<sup>n</sup> ∈ T <sup>∗</sup> the partial-order defined by the unit decomposition **U**<sup>α</sup> = **S**ι**S**<sup>α</sup><sup>1</sup> **S**<sup>α</sup><sup>2</sup> ··· **S**<sup>α</sup>*<sup>n</sup>* **S**<sup>ε</sup> is precisely the extension of the partial-order <sup>I</sup> (α) induced by α (Fig. 6).

Let ρ : T → {1, ..., |T|} be an arbitrary ordering of the elements of T. Let D = {ab | a, b∈T, ρ(a) ≤ ρ(b), (a, b) ∈/ I} be the set of pairs of non-independent elements of T. Let ρ : D → {1, ..., |D|} be the natural lexicographic ordering induced on D by the ordering ρ. For each symbol a ∈ T we define the slice **S**<sup>a</sup> as follows: Both the in-fronter I and the out-frontier O of **S**<sup>a</sup> have |D| vertices, and the center of **S**<sup>a</sup> has a unique vertex v<sup>a</sup> which is labeled by a. In symbols I = {Iab|ab ∈ D} and O = {Oab|ab ∈ D}. For each ab ∈ D, both the in-frontier vertex Iab and the out-frontier vertex Oab are labeled with the number ρ(ab). For each pair bc ∈ D with a = b and a = c we add an edge to **S**<sup>a</sup> with source in Ibc and target in Obc, and for each pair ax ∈ D (xa ∈ D) we add an edge with source in Iax (Ixa) and target in va, and an edge with source in v<sup>a</sup> and target in Oax (Oxa) (Fig. 6). We associate with the symbol ι an initial slice **S**ι, with center vertex v<sup>ι</sup> labeled by ι, and out-frontier O. Analogously, with the symbol ε, we associate a final slice **S**<sup>ε</sup> with center vertex v<sup>ε</sup> labeled by ε, and in-frontier <sup>I</sup>. We note that the slice alphabet −→*Σ*(T, <sup>I</sup>) is a subset of −→*Σ*(k, T ∪ {ι, ε}) where k = |D|≤|T| 2.

Now let α = α1α2...α<sup>n</sup> be a string in T, *dep*<sup>I</sup> (α) be the dependence graph of α and **U**<sup>α</sup> = **S**ι**S**<sup>α</sup><sup>1</sup> ...**S**<sup>α</sup>*<sup>n</sup>* **S**ε. Let v<sup>i</sup> be the i-th vertex of *dep*<sup>I</sup> (α), and u<sup>i</sup> be the center vertex of the slice **S**<sup>α</sup>*<sup>i</sup>* . Then it is straightforward to check that for each i, j ∈ {1, ..., n} with i<j, there is a path from <sup>u</sup><sup>i</sup> to <sup>u</sup><sup>j</sup> in the graph ◦ **U**<sup>α</sup> if and only if there is a path from v<sup>i</sup> to v<sup>j</sup> in *dep*<sup>I</sup> (α). This implies that the partial-order **tc**( ◦ **U**α) is the extension of the partial-order <sup>I</sup> (α) induced by α. In other words, **tc**( ◦ **U**α) = ˆ<sup>I</sup> (α).

Now, from the pair (F, I)=(Q, R, Q0, F) we construct an auxiliary slice automaton A (F, I)=(Q , R , Q 0, F ) as follows. We let Q = Q ∪ {qι, qε}, Q <sup>0</sup> = {qι}, F = {qε}, and R = {(qι, **S**ι, q) | q ∈ Q0}∪{(q, **S**ε, qε) | q ∈ F}∪{(q, **S**a, q ) | (q, a, q ) ∈ R}. Then we have that F accepts a string α if and only if A (F, I) accepts the unit decomposition **U**α. This implies that L*po*(A (F, <sup>I</sup>)) = <sup>L</sup>ˆ*po*(F, <sup>I</sup>).

Now assume that L(F) is trace closed. Then for each string γ ∈ [α]<sup>I</sup> , we have that γ ∈ L(F) and therefore **U**<sup>γ</sup> ∈ L(A(F, I)). Since for each topological ordering ω of the graph ◦ **U**α, there is a γ ∈ [α]<sup>I</sup> such that **U**<sup>γ</sup> is compatible with ω, we have that A (F, I) is weakly saturated. Therefore, by Lemma 2 we can construct a saturated slice automaton A(F, I) with L*po*(A(F, I)) = L*po*(A (F, I)). -

#### **9 Conclusion**

In this work, we have established connections between the causal semantics of Petri nets and message sequence chart languages. In particular, we showed that message sequence chart automata can be used as a tool for the study of the causal behavior of Petri nets. Despite the fact that each of these formalisms have been defined several decades ago, the connections established in our work were unknown. In order to prove our results we have introduced new slice theoretic machinery of independent interest. In particular, our techniques pave the way for the use of slice automata as a bridge between bounded Petri nets and behavioral formalisms. Further evidence for this assessment is given in Sect. 8, where we show how to map Mazurkiewicz Trace languages to trace languages in such a way that trace closure implies saturation. This means that the results in Theorems 8 and 9 also hold if instead of MSC automata we use Mazurkiewicz pairs.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Verifying Generalised and Structural Soundness of Workflow Nets via Relaxations**

Michael Blondin<sup>1</sup> , Filip Mazowiecki<sup>2</sup> , and Philip Offtermatt1,2(B)

<sup>1</sup> Universit´e de Sherbrooke, Sherbrooke, Canada philip.offtermatt@usherbrooke.ca <sup>2</sup> Max Planck Institute for Software Systems, Saarbr¨ucken, Germany

**Abstract.** Workflow nets are a well-established mathematical formalism for the analysis of business processes arising from either modeling tools or process mining. The central decision problems for workflow nets are k-soundness, generalised soundness and structural soundness. Most existing tools focus on k-soundness. In this work, we propose novel scalable semi-procedures for generalised and structural soundness. This is achieved via integral and continuous Petri net reachability relaxations. We show that our approach is competitive against state-of-the-art tools.

## **1 Introduction**

*Workflow nets* are a well-established mathematical formalism for the description of business processes arising from software modelers and process mining (*e.g.*, see [2,3]), and further notations such as UML activity diagrams [4]. More precisely, a workflow net consists of *places* that contain resources, and *transitions* that can consume, create and move resources concurrently. Two designated places, denoted i and f, respectively model the initialization and completion of a process. Workflow nets, which form a subclass of Petri nets, enable the automatic formal verification of business processes. For example, 1*-soundness* states that from the initial configuration {i: 1}, every reachable configuration can reach the final configuration {f : 1}. Informally, this means that given any partial execution of a business process, it is possible to complete it properly.

*Soundness.* The main decision problems concerning workflow nets revolve around soundness properties. The generalisation of 1-soundness to several resources is k*soundness*. It asks whether from {i: k}, every reachable configuration can reach {f : k} (here, {p: k} indicates that place p contains k resources). Intuitively, 1 soundness guarantees that every initialised process terminates, and k-soundness guarantees that k initialised processes working in parallel will all terminate (see

An extended version of this paper with an appendix containing the missing proofs can be obtained from https://arxiv.org/abs/2206.02606.

c The Author(s) 2022

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 468–489, 2022. https://doi.org/10.1007/978-3-031-13188-2\_23

*e.g.* [1,2]). *Generalised soundness* asks whether k-soundness holds for all k ≥ 1. Unlike k-soundness, generalised soundness preserves desirable properties like composition and has other desirable properties for business applications [20]. *Structural soundness* is the existential counterpart of generalised soundness, *i.e.* it asks whether k-soundness holds for some k ≥ 1. Structural soundness gives information on how many processes can be controlled in parallel [31], moreover, by applying results about structural soundness, one can compute the set of all k for which the workflow net is k-sound [9, Section 7].

These problems are all decidable [1,21,31], but with high complexity: either PSPACE- or EXPSPACE-complete [9]. Most of the (software) tools focus on ksoundness, with an emphasis on k = 1. Existing algorithms for generalised and structural soundness rely on Petri net reachability [19,21,31], which was recently shown Ackermann-complete [13,24], so not primitive recursive. In this work, we describe *novel scalable semi-procedures for generalised and structural soundness*.

We focus on "negative instances", *i.e.* where soundness does *not* hold. Let us motivate this. It is known that given a workflow net N , one can iteratively apply simple reduction rules to N . The resulting workflow net N is sound iff N is as well [10,22]. In practice, one infers that N is sound from the fact that N has been reduced to a trivial workflow net where only i and f remain. However, if N is *not* sound, one obtains some nontrivial N that must be verified via some other approach such as model checking. In this work, we provide algorithmic building blocks for this case, where state-space exploration is prohibitive.

*Relaxations.* This is achieved by considering two reachability relaxations, namely integer reachability and continuous reachability. As their name suggests, these two notions relax some forbidden behaviour of workflow nets. Informally, integer reachability allows for the amount of resources to become temporarily negative, while continuous reachability allows the fragmentation of resources into pieces. Such relaxations possibly introduce spurious behaviour, but enjoy significantly better algorithmic properties (*e.g.*, see [7]). For example, they have been successfully employed for the verification of multi-threaded program skeletons [5,8,15].

*Generalised Soundness.* Based on these relaxations, we provide two necessary conditions for generalised soundness: *integer boundedness* and *continuous soundness*. The former states that the state-space of a given workflow net is bounded (from above) even under integer reachability. The latter states that a given workflow net is 1-sound under continuous reachability. We show the following for integer boundedness and continuous soundness:


Thus, altogether, in order to check whether a workflow net N is generalised *unsound*, one may first use classical reduction rules to obtain a smaller workflow net N ; test integer *unboundedness* in polynomial time; and, if needed, move onto testing continuous *unsoundness*.

The fact that continuous reachability can be used to semi-decide generalised soundness is arguably surprising. Using the notation of computation temporal logic (CTL), k-soundness can be rephrased as {i: k} |= ∀G∃F {f : k}. Some other well-studied properties have a similar structure, *e.g.* liveness and home-stateness amount to "*m*init |= - <sup>t</sup>∈<sup>T</sup> <sup>∀</sup>G∃<sup>F</sup> (<sup>t</sup> is enabled)" and "*m*init <sup>|</sup><sup>=</sup> <sup>∀</sup>G∃F*m*home". It is known that liveness, home-stateness, and other properties such as boundedness and inclusion, *cannot* be approximated continuously [8, Sect. 4]. Yet, generalised soundness quantifies k-soundness universally, and this enables a continuous over-approximation. Consequently, we provide a novel application of continuous relaxations for the efficient verification of properties beyond reachability.

*Structural Soundness.* The authors of [31] have observed that a property called structural quasi-soundness is a necessary condition for structural soundness. The former states that {i: k} can reach {f : k} for some k ≥ 1. In [31], structural quasi-soundness is reduced to Petri net reachability, which has non primitive recursive complexity. In this work, we show that structural quasi-soundness can be rephrased as continuous reachability. Since the latter can be tested in polynomial time [18], or alternatively via SMT solving [8], this vastly improves the practicability of structural quasi-soundness. We further show that this approach can be adapted so that it provides a lower bound on the first k such that {i: k} can reach {i: f}. From a practical point of view, this is useful as it can vastly reduce the number of reachability queries to decide structural soundness.

*Free-Choice Nets.* Many real-world workflow nets have a specific structure where concurrency is restricted. Such nets are known as *free-choice* workflow nets (*e.g.*, see [14] for a book). In particular, free-choice workflow nets allow for the modeling of many features present in common workflow management systems [2]. Generalised soundness is equivalent to 1-soundness for free-choice workflow nets [28]. In this work, we prove that continuous soundness is equivalent to generalised soundness. As a byproduct of our proof, we show that structural soundness is also equivalent to continuous soundness. Altogether, the notions of {1-, generalised, structural, continuous} soundness *all coincide* for free-choice nets. In particular, this means that the continuous relaxation is *exact* and can serve as an efficient addition to the existing algorithmic toolkit.

*Experimental Results.* To demonstrate the viability of our approach, we have implemented and experimentally evaluated a prototype. As part of our evaluation, we propose several new synthetic instances for generalised and structural soundness, which are hard to decide with naive approaches. Some of these instances involve the composition of workflow nets arising from the modeling of business processes in the IBM WebSphere Business Modeler. Our prototype is competitive against both a state-of-the-art Petri net model checker, and a workflow net analyzer. In particular, our approach exhibits better signs of scalability. *Organization.* The paper follows the structure of this introduction. Section 2 introduces notation, workflow nets and some properties. Section 3 defines integer and continuous relaxations, and further shows that they are preserved under reduction rules. Sections 4, 5, 6 present the aforementioned results on generalised soundness, structural soundness and free-choice nets. Section 7 provides experimental results. Section 8 concludes. Some proofs are deferred to an appendix.

#### **2 Preliminaries**

We use <sup>Z</sup>, <sup>N</sup>, <sup>Q</sup> and <sup>Q</sup>≥<sup>0</sup> to respectively denote the integers, the naturals (including 0), the rationals and the nonnegative rationals (including 0). Let *<sup>x</sup>*, *<sup>y</sup>* <sup>∈</sup> <sup>Q</sup><sup>S</sup> be vectors over a finite set S. We write *x* ≤ *y* if *x*[s] ≤ *y*[s] for all s ∈ S. We write *x* < *y* if *x* ≤ *y* and *x*[s] < *y*[s] for some s ∈ S. We extend addition and subtraction to vectors, *i.e.* (*x* + *y*)[s] := *x*[s] + *y*[s] and (*x* − *y*)[s] := *x*[s] − *y*[s] for all <sup>s</sup> <sup>∈</sup> <sup>S</sup>. We define supp(*x*) = {<sup>s</sup> <sup>∈</sup> <sup>S</sup> <sup>|</sup> *<sup>x</sup>*[s] = 0}. Given <sup>c</sup> <sup>∈</sup> <sup>Q</sup>, *<sup>c</sup>* <sup>∈</sup> <sup>Q</sup><sup>S</sup> denotes the vector such that *c*[s] = c for all s ∈ S.

#### **2.1 Petri Nets**

A *Petri net* N is a triple (P, T, F), where P is a finite set of *places*; T is a finite set of *transitions*, such that T ∩ P = ∅; and F : ((P × T) ∪ (T × P)) → {0, 1} is a set of *arcs*. For readers familiar with Petri nets, note that arc weights are not allowed, *i.e.* the weights are always 1. A *marking* is a vector *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> such that *m*[p] denotes the number of *tokens* in place p. We denote markings listing nonzero values, *e.g. m* = {p<sup>1</sup> : 1} means *m*[p1] = 1 and *m*[p] = 0 for p = p1.

Let <sup>t</sup> <sup>∈</sup> <sup>T</sup>. We define the *pre-vector* of <sup>t</sup> as •<sup>t</sup> <sup>∈</sup> <sup>N</sup><sup>P</sup> , where •t[p] := <sup>F</sup>(p, t). We define its *post-vector* symmetrically with t •[p] := F(t, p). The *effect* of t is denoted as Δ(t) := t • −•t. We say that a transition t is *enabled* at a marking *m* if *m* ≥ •t. If this is the case, then t can be *fired* at *m*, which results in a marking *<sup>m</sup>* such that *<sup>m</sup>* := *<sup>m</sup>*+Δ(t). We write *<sup>m</sup>* −→<sup>t</sup> to denote that <sup>t</sup> is *enabled* at *<sup>m</sup>*, and we write *<sup>m</sup>* −→<sup>t</sup> *<sup>m</sup>* whenever we care about the marking *<sup>m</sup>* resulting from the firing. We further write *<sup>m</sup>* −→ *<sup>m</sup>* to denote that *<sup>m</sup>* −→<sup>t</sup> *<sup>m</sup>* for some <sup>t</sup> <sup>∈</sup> <sup>T</sup>.

We say that a sequence of transitions π = t<sup>1</sup> ···t<sup>n</sup> is a *run*. We extend the notion of effect, enabledness and firing from transitions to runs in a straightforward way. The *effect* of a run is defined as the sum of the effects of its transitions, that is, Δ(π) := Δ(t1) + ... + Δ(tn). The run π is enabled at *m*, denoted as *<sup>m</sup>* −→<sup>π</sup>, if *<sup>m</sup>* −→<sup>t</sup><sup>1</sup> *<sup>m</sup>*<sup>1</sup> −→<sup>t</sup><sup>2</sup> *<sup>m</sup>*<sup>2</sup> ··· −→<sup>t</sup>n−<sup>1</sup> *<sup>m</sup>*<sup>n</sup>−<sup>1</sup> −→<sup>t</sup><sup>n</sup> for some markings *m*1,*m*2,...,*m*<sup>n</sup>−<sup>1</sup>. Furthermore, firing π from *m* leads to *m* , denoted as *<sup>m</sup>* −→<sup>π</sup> *<sup>m</sup>* , if *<sup>m</sup>* −→<sup>π</sup> and *<sup>m</sup>* <sup>=</sup> *<sup>m</sup>*+Δ(π). We denote the reflexive and transitive closure of −→ by −→<sup>∗</sup>.

A pair (N ,*m*), where N is a Petri net and *m* is a marking of N , is called a *marked Petri net*. We write Reach(N ,*m*) := {*m* | *m* −→<sup>∗</sup> *m* } to denote the set of markings reachable from *m* in N .

A marked Petri net (<sup>N</sup> ,*m*) is *bounded* if there exists <sup>b</sup> <sup>∈</sup> <sup>N</sup> such that *<sup>m</sup>* <sup>∈</sup> Reach(N ,*m*) implies *m* [p] ≤ b for all p ∈ P. It is further *safe* if b = 1. We say *unbounded* and *unsafe* for "not bounded" and "not safe".

Sometimes, we argue about transformations on Petri nets which take as an input a Petri net N and output a Petri net N . We say that such a transformation *preserves* some property if N satisfies that property iff N satisfies it.

**Fig. 1.** Example of two Petri nets: respectively Nleft and Nright.

*Example 1.* The left-hand side of Fig. 1 illustrates a Petri net Nleft = (P, T, F) where P := {i, p1, p2, q1, q2, f}, T := {s, t1, t2, u}, and F is depicted by arcs, *e.g.* F[i, s] = 1 and F[s, i] = 0. The Petri net is marked by {i: 1}, *i.e.* with one token in place <sup>i</sup>. We have {i: 1} −→<sup>s</sup> {p<sup>1</sup> : 1, p<sup>2</sup> : 1} −→<sup>t</sup>1t<sup>2</sup> {q<sup>1</sup> : 1, q<sup>2</sup> : 1} −→<sup>u</sup> {<sup>f</sup> : 1}. -

#### **2.2 Workflow Nets**

A workflow net N is a Petri net [1] such that:


We say that N is:


*Example 2.* Figure 1 depicts two workflow nets: Nleft and Nright. The former is generalised sound, but the latter is not. Indeed, from {i: 1}, transition t cannot be enabled (as transitions preserve the sum of all tokens). Both workflow nets are structurally sound. Indeed, Nright is 2-sound as it is always possible to redistribute the two tokens so that t can be fired in order to reach {f : 2}. -

#### **3 Reachability Relaxations**

Fix a Petri net N = (P, T, F). We describe the two aforementioned relaxations.

*Integer Reachability.* An *integral marking* is a vector *<sup>m</sup>* <sup>∈</sup> <sup>Z</sup><sup>P</sup> . Any transition <sup>t</sup> <sup>∈</sup> <sup>T</sup> is *enabled* in *<sup>m</sup>* <sup>∈</sup> <sup>Z</sup><sup>P</sup> , and *firing* <sup>t</sup> leads to *<sup>m</sup>* := *<sup>m</sup>*+Δ(t), denoted *<sup>m</sup>* −→<sup>t</sup> Z *m* . We define *m* −→<sup>Z</sup> *m* and *m* −→<sup>∗</sup> <sup>Z</sup> *m* analogously to the standard setting but w.r.t. −→<sup>t</sup> <sup>Z</sup> rather than −→<sup>t</sup> . Similarly, <sup>Z</sup>-Reach(<sup>N</sup> ,*m*) := {*m* <sup>∈</sup> <sup>Z</sup><sup>P</sup> <sup>|</sup> *<sup>m</sup>* −→<sup>∗</sup> <sup>Z</sup> *m* }. As transitions are always enabled, the order of a firing sequence is irrelevant. In particular, *m* −→<sup>∗</sup> <sup>Z</sup> *<sup>m</sup>* iff there exists *<sup>x</sup>* <sup>∈</sup> <sup>N</sup><sup>T</sup> such that *<sup>m</sup>* <sup>=</sup> *<sup>m</sup>*+ <sup>t</sup>∈<sup>T</sup> *<sup>x</sup>*[t]·Δ(t). Thus, integer reachability amounts to integer linear programming. Moreover, it is NP-complete [12].

*Continuous Reachability.* <sup>A</sup> *continuous marking* is a vector *<sup>m</sup>* <sup>∈</sup> <sup>Q</sup><sup>P</sup> <sup>≥</sup><sup>0</sup>. Let <sup>λ</sup> <sup>∈</sup> (0, 1]. We say that λt is *enabled* in *<sup>m</sup>*, denoted *<sup>m</sup>* −→λt <sup>Q</sup>≥<sup>0</sup> , if *<sup>m</sup>* <sup>≥</sup> <sup>λ</sup> · •t. In this context, <sup>λ</sup> is called the *scaling factor*. Furthermore, we denote by *<sup>m</sup>* −→λt <sup>Q</sup>≥<sup>0</sup> *m* that λt is enabled in *m*, and that its *firing* results in *m* := *m* + λ · Δ(t). A sequence of pairs of scaling factors and transitions is called a *continuous run*.

The notations *<sup>m</sup>* −→<sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* and *<sup>m</sup>* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* are defined analogously to the discrete case but with respect to −→λt <sup>Q</sup>≥<sup>0</sup> rather than −→<sup>t</sup> (the internal factors <sup>λ</sup> can differ). Similarly, <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> ,*m*) := {*m* <sup>|</sup> *<sup>m</sup>* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *m* } denotes the markings continuously reachable from *m*. For example, for Nleft from Fig. 1 and π := <sup>1</sup> <sup>2</sup> <sup>s</sup> <sup>1</sup> <sup>4</sup> <sup>t</sup>1, we have {i: 1} −→<sup>π</sup> <sup>Q</sup>≥<sup>0</sup> {i: 1/2, p<sup>1</sup> : 1/4, p<sup>2</sup> : 1/2, q<sup>1</sup> : 1/4}. It is known that continuous reachability, namely determining whether *m* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *m* , given *<sup>m</sup>*,*m* <sup>∈</sup> <sup>Q</sup><sup>P</sup> <sup>≥</sup><sup>0</sup>, can be checked in polynomial time [18].

Let us establish the following helpful lemma similar to [18, Lemma 12(1)].

**Lemma 1.** *Let m, m be continuous markings. It is the case that m* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *m iff there exists* <sup>b</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> *such that* <sup>b</sup> · *<sup>m</sup>* −→<sup>∗</sup> <sup>b</sup> · *<sup>m</sup> .*

#### **3.1 Preservation Under Reduction Rules**

In [10], the authors present six reduction rules, denoted R1,...,R6, that generalize the existing reduction rules of [27]. In the following, we show that these reduction rules preserve natural properties for the two reachability relaxations. This means we will be able to check these properties on a reduced workflow net and get the same results as on the original one.

Formally, the rules simplify a given workflow net N = (P, T, F). In particular, the places of the resulting workflow net N = (P ,T,F ) form a subset of P. Let us fix a domain <sup>D</sup> ∈ {N,Z, <sup>Q</sup>≥<sup>0</sup>} and let <sup>P</sup> <sup>⊆</sup> <sup>P</sup>. For ease of notation, we write P = P \ P to denote the (possibly empty) set of removed places. Rules never remove the initial and output places, *i.e.* i, f ∈ P . We denote by <sup>π</sup> : <sup>D</sup><sup>P</sup> <sup>→</sup> <sup>D</sup><sup>P</sup> the obvious projection function, and by π<sup>0</sup> : D<sup>P</sup> <sup>→</sup> <sup>D</sup><sup>P</sup> the "reverse projection" which fills new places with 0. Formally, π0(*m*)[p ] := *m*[p ] for all p ∈ P and π0(*m*)[p] := 0 for all p ∈ P.

In [10], the authors prove that the rules preserve generalised soundness. This of course implies that they preserve k-soundness for all k. The technical proposition below will be helpful in the forthcoming sections to show the preservation of useful properties based on reachability relaxations.

**Proposition 1.** *Let* <sup>N</sup> = (P, T, F) *be a workflow net, and let* <sup>D</sup> ∈ {N,Z, <sup>Q</sup>≥<sup>0</sup>}*. Let* N = (P , T , F ) *be a workflow net obtained by applying a reduction rule* R<sup>i</sup> *to* N *, where* P = P ∪ P*. The following holds.*


#### **4 Using Relaxations For Generalised Soundness**

In this section, we explain how reachability relaxations can be leveraged in order to semi-decide generalised soundness of workflow nets. More precisely, we state two necessary conditions for a workflow net to be generalised sound: one phrased in terms of integer reachability, and one in terms of continuous reachability. Furthermore, for each condition we: (1) show that it is preserved under reduction rules, and (2) establish its computational complexity. Overall, this means that to conclude that a given workflow net N is *not* generalised sound, one may first reduce N , and *then* efficiently test for one of these two necessary conditions.

For integer boundedness, we need the mild assumption of nonredundancy. Let <sup>N</sup> = (P, T, F) be a workflow net. We say that a place <sup>p</sup> <sup>∈</sup> <sup>P</sup> is *nonredundant*<sup>1</sup> if there exist <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> and *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> such that {i: <sup>k</sup>} −→<sup>∗</sup> *<sup>m</sup>* and *<sup>m</sup>*[p] <sup>≥</sup> 1. It is known (and simple to see) that redundant places can be removed from a workflow net without changing whether it is generalised sound. Moreover, testing whether a place is nonredundant can be done in polynomial time. Indeed, by Lemma 1, it amounts to testing for the existence of some *<sup>m</sup>* <sup>∈</sup> <sup>Q</sup><sup>P</sup> <sup>≥</sup><sup>0</sup> such that {i: 1} −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *m* and *m*[p] > 0. The latter is known as a *coverability* query and it can be checked in polynomial time [18]. Thus, in order to test whether a given workflow net is generalised sound, one can first remove its redundant places. We call a workflow net without redundant places a *nonredundant workflow net*.

#### **4.1 Integer Unboundedness**

Recall that a marked Petri net (<sup>N</sup> ,*m*) is *bounded* if there exists <sup>b</sup> <sup>∈</sup> <sup>N</sup> such that *m* ∈ Reach(N ,*m*) implies *m* ≤ *b*. It is well-known that any 1-sound workflow

<sup>1</sup> This notion is adapted from batch workflow nets considered in [21].

net must be bounded from {i: 1} [1]. In particular, this means that boundedness is a necessary condition for generalised soundness. However, testing boundedness has extensive computational cost as it is EXPSPACE-complete [11,29]. Consider the relaxed property of *integer boundedness*. It is defined as boundedness, but where "*m* <sup>∈</sup> Reach(<sup>N</sup> ,*m*)" is replaced with "*m* <sup>∈</sup> <sup>Z</sup>-Reach(<sup>N</sup> ,*m*) <sup>∩</sup> <sup>N</sup><sup>P</sup> ".

**Proposition 2 (**[9, Lemma 5.9]**).** *Let* N *be a nonredundant workflow net. If* (N , {i: 1}) *is integer unbounded, then* N *is not generalised sound.*

**Proposition 3.** *The reduction rules from [10] preserve integer unboundedness.*

Next, we establish the complexity of integer unboundedness in two steps. The first step, in the next proposition, shows that testing integer boundedness amounts to a simple condition, independent of the initial marking. The second step shows the condition can be translated into a linear program over Q, rather than N. As a corollary, integer unboundedness is testable in polynomial time.

**Proposition 4.** *A marked Petri net* (N ,*m*) *is integer unbounded iff there exists a marking m* > **0** *such that* **0** −→<sup>∗</sup> <sup>Z</sup> *m (independent of m).*

*Proof.* Let <sup>N</sup> = (P, F, T) be a Petri net and let *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> .

<sup>⇒</sup>) By assumption, there exist *<sup>m</sup>*0,*m*1,... <sup>∈</sup> <sup>Z</sup>-Reach(<sup>N</sup> ,*m*) <sup>∩</sup> <sup>N</sup><sup>P</sup> such that, for every <sup>i</sup> <sup>∈</sup> <sup>N</sup>, it is the case that <sup>m</sup><sup>i</sup> ≤ *<sup>i</sup>*. Since (N<sup>P</sup> , <sup>≤</sup>) is well-quasiordered, there exist indices i0, i1,... such that *m*<sup>i</sup><sup>j</sup> ≤ *m*<sup>i</sup><sup>k</sup> for all j<k. Without loss of generality, we can assume that *m*<sup>i</sup><sup>j</sup> < *m*<sup>i</sup><sup>k</sup> for all j<k, as we could otherwise extract such a subsequence. Recall that each *m*<sup>i</sup>- <sup>∈</sup> <sup>Z</sup>-Reach(<sup>N</sup> ,*m*). Let <sup>π</sup> <sup>∈</sup> <sup>T</sup> <sup>∗</sup> be such that *<sup>m</sup>* −→<sup>π</sup>- <sup>Z</sup> *m*<sup>i</sup>- . Let *<sup>x</sup>* <sup>∈</sup> <sup>N</sup><sup>T</sup> be the vector such that *<sup>x</sup>*(t) indicates the number of occurrences of transition <sup>t</sup> in <sup>π</sup>. Since (N<sup>T</sup> , <sup>≤</sup>) is well-quasi-ordered, there exist j<k such that *x*<sup>j</sup> ≤ *x*k. Let *m* := *m*<sup>i</sup><sup>k</sup> − *m*<sup>i</sup><sup>j</sup> and π := <sup>t</sup>∈<sup>T</sup> <sup>t</sup> (*x*k[t]−*x*-[t]). We have **<sup>0</sup>** −→<sup>π</sup> <sup>Z</sup> *m* > **0** as desired since:

$$\begin{aligned} m' &= m\_{i\_k} - m\_{i\_j} = (m + \Delta(\pi\_k)) - (m + \Delta(\pi\_\ell)) = \Delta(\pi\_k) - \Delta(\pi\_\ell) \\ &= \sum\_{t \in T} \mathbf{x}\_k[t] \cdot \Delta(t) - \sum\_{t \in T} \mathbf{x}\_\ell[t] \cdot \Delta(t) = \sum\_{t \in T} (\mathbf{x}\_k - \mathbf{x}\_\ell)[t] \cdot \Delta(t) = \Delta(\pi). \end{aligned}$$

⇐) By assumption **<sup>0</sup>** −→<sup>π</sup> <sup>Z</sup> *<sup>m</sup>* <sup>&</sup>gt; **<sup>0</sup>**. In particular, this means that *<sup>m</sup>* −→<sup>π</sup> Z *<sup>m</sup>* <sup>+</sup> *<sup>m</sup>* −→<sup>π</sup> <sup>Z</sup> *m* + 2*m* −→<sup>Z</sup> ··· . Therefore, (N ,*m*) is not integer bounded.

**Proposition 5.** *A marked Petri net* (N ,*m*)*, where* N = (P, T, F)*, is integer unbounded iff this system has a solution:* <sup>∃</sup>*<sup>x</sup>* <sup>∈</sup> <sup>Q</sup><sup>T</sup> <sup>≥</sup><sup>0</sup> : <sup>t</sup>∈<sup>T</sup> *<sup>x</sup>*[t] · <sup>Δ</sup>(t) <sup>&</sup>gt; **<sup>0</sup>***. In particular, given a workflow net* N *, testing integer boundedness of* (N , {i: 1}) *can be done in polynomial time.*

#### **4.2 Continuous Soundness**

Let us now introduce a continuous variant of 1-soundness based on continuous reachability. We prove that this variant, which we call *continuous soundness*, is a necessary condition for generalised soundness, and preserved by reduction rules. Moreover, we show that continuous soundness is coNP-complete, and relates to integer boundedness.

We say that a workflow net N is *continuously sound* if for all continuous markings *<sup>m</sup>* <sup>∈</sup> <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> , {i: 1}) it is the case that *<sup>m</sup>* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> {<sup>f</sup> : 1}.

**Theorem 1.** *Continuous unsoundness implies generalised unsoundness.*

*Proof.* Let N = (P, T, F) be a workflow net that is not continuously sound. By definition of continuous soundness, there exists some continuous marking *m* ∈ Q<sup>P</sup> <sup>≥</sup><sup>0</sup> such that {i: 1} −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* and *<sup>m</sup>* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> {<sup>f</sup> : 1}. By Lemma 1, there exists <sup>b</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> such that {i: <sup>b</sup>} −→<sup>∗</sup> <sup>b</sup> · *<sup>m</sup>*. Furthermore, by Lemma 1, <sup>b</sup> · *<sup>m</sup>* −→<sup>∗</sup> {<sup>f</sup> : <sup>b</sup>}. This means that N is not b-sound, and consequently not generalised sound.

**Proposition 6.** *The reduction rules from [10] preserve continuous soundness.*

**Theorem 2.** *Continuous soundness is coNP-complete. Moreover, coNPhardness holds even if the underlying graph of the given workflow net is acyclic.*

*Proof (of membership in coNP).* The *inclusion problem* consists in determining whether, given Petri nets N and N over a common set of places, and markings *m* and *m* , it is the case that <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> ,*m*) <sup>⊆</sup> <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> ,*m* ). The inclusion problem is known to be coNP-complete [8, Prop. 4.6].

Let <sup>N</sup> = (P, T) be a workflow net. Let <sup>N</sup> <sup>−</sup><sup>1</sup> = (P, T <sup>−</sup><sup>1</sup>) be defined as <sup>N</sup> but with its transitions reversed, *i.e.* where <sup>T</sup> <sup>−</sup><sup>1</sup> := {<sup>t</sup> <sup>−</sup><sup>1</sup> <sup>|</sup> <sup>t</sup> <sup>∈</sup> <sup>T</sup>} with •(<sup>t</sup> <sup>−</sup><sup>1</sup>) := t • and (t <sup>−</sup><sup>1</sup>)• := •t. It is the case that *<sup>m</sup>* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* in <sup>N</sup> iff *<sup>m</sup>* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* in <sup>N</sup> <sup>−</sup><sup>1</sup>. Observe that N is continuously sound iff the following holds for all *m*:

$$
\mathfrak{m} \in \mathbb{Q}\_{\geq 0}\text{-Reach}(\mathcal{N}, \{\mathfrak{i} \text{- } 1\}) \implies \{\mathfrak{f} \text{-} 1\} \in \mathbb{Q}\_{\geq 0}\text{-Reach}(\mathcal{N}, \mathfrak{m}).
$$

So, as {<sup>f</sup> : 1} ∈ <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> ,*m*) is equivalent to *<sup>m</sup>* <sup>∈</sup> <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> <sup>−</sup><sup>1</sup>, {<sup>f</sup> : 1}), continuous soundness holds iff <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> , {i: 1}) <sup>⊆</sup> <sup>Q</sup>≥<sup>0</sup>-Reach(<sup>N</sup> <sup>−</sup><sup>1</sup>, {f : 1}). As inclusion can be tested in coNP, membership follows.

*Proof (of coNP-hardness).* We give a reduction from the problem of determining whether a Boolean formula in disjunctive normal form (DNF) is a tautology. We adapt a construction from [30] used to show that soundness in acyclic workflow nets is coNP-hard. The proof is more challenging under the continuous semantics as several variable valuations and clauses can be simultaneously used.

The reduction is depicted in Fig. 2 for ϕ = (x<sup>1</sup> ∧ x<sup>2</sup> ∧ ¬x4) ∨ (¬x<sup>1</sup> ∧ x<sup>3</sup> ∧ x4). In general, let ϕ = <sup>j</sup>∈[1..k] <sup>C</sup><sup>j</sup> be a Boolean formula in DNF with <sup>k</sup> clauses over variables x1,...,xm. We define a workflow net N<sup>ϕ</sup> = (P, T, F).

**Fig. 2.** A workflow net N<sup>ϕ</sup> such that N<sup>ϕ</sup> is continuously sound iff ϕ = (x1∧x2∧¬x4)∨ (x<sup>1</sup> ∧ x<sup>3</sup> ∧ x4) is a tautology. Places and transitions contain their names (not values). Arcs corresponding to the first and second clauses are respectively dotted and dashed.

*Definition.* The places are defined as P := {i, pcl, f} ∪ Pvar ∪ Pclean, where Pvar := i∈[1..m] {pi,?, pi,<sup>1</sup>, pi,<sup>0</sup>} and Pclean := i∈[1..m] {qi, ri}. The transitions are defined as T := {tinit, tfin} ∪ Tvar ∪ Tclauses ∪ Tvar, where

$$T\_{\mathrm{var}} := \bigcup\_{i \in [1..m]} \{v\_{i,1}, v\_{i,0}\},\\ T\_{\mathrm{clauses}} := \{c\_i \mid i \in [1..k] \} \text{ and } T\_{\overline{\mathrm{var}}} := \bigcup\_{i \in [1..m]} \{\overline{v}\_{i,1}, \overline{v}\_{i,0}\}.$$

Let us explain how N<sup>ϕ</sup> is *intended* to work. Transition tinit enables the initialization of variables and the selection of a clause that satisfies ϕ, i.e. •tinit := {i: 1} and t • init := {pi,? : 1 | i ∈ [1..m]} + {pcl : 1}. A token in place pi,b indicates that variable x<sup>i</sup> has been assigned value b (where "?" indicates "none"). Consequently, we have •vi,b := pi,? and v• i,b := pi,b for each i ∈ [1..m] and b ∈ {0, 1}.

Transition c<sup>j</sup> consumes a token associated to each literal of clause C<sup>j</sup> , *i.e.* •c<sup>j</sup> := {vi,<sup>1</sup> | x<sup>i</sup> ∈ Cj} + {vi,<sup>0</sup> | ¬x<sup>i</sup> ∈ Cj}. A token in place q<sup>i</sup> indicates that variable x<sup>i</sup> is not needed anymore (due to some satisfied clause). A token in place r<sup>i</sup> indicates that variable x<sup>i</sup> has been discarded. Therefore, transition c<sup>j</sup> produces these tokens: c• <sup>j</sup> := {q<sup>i</sup> | x<sup>i</sup> ∈/ Cj∧¬x<sup>i</sup> ∈/ Cj}+{r<sup>i</sup> | x<sup>i</sup> ∈ Cj∨¬x<sup>i</sup> ∈ Cj}.

Transition vi,b discards variable xi, i.e. •vi,b := {pi,b, qi} and •vi,b := {qi}. Once each variable is discarded, transition tfin terminates the execution, i.e. •tfin := {r<sup>i</sup> | i ∈ [1..m]} and t • fin := {f : 1}.

*Correctness.* Note that under −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> , the workflow net needs not to proceed as described. Indeed, it could, e.g., assign half a token to pi,<sup>0</sup> and half a token to pi,<sup>1</sup>. Similarly, several clauses can be used, with distinct scaling factors. Nonetheless, N<sup>ϕ</sup> is continuously sound iff ϕ is a tautology.

<sup>⇒</sup>) Let <sup>b</sup>1,...,b<sup>m</sup> ∈ {0, <sup>1</sup>}. Let <sup>π</sup> := <sup>t</sup>initv<sup>1</sup>,b<sup>1</sup> ··· <sup>v</sup>m,bm. We have: {i: 1} −→<sup>π</sup> <sup>Q</sup>≥<sup>0</sup> {vi,b<sup>i</sup> : 1 | i ∈ [1..m]} + {pcl : 1}. Since N<sup>ϕ</sup> is continuously sound by assumption, there must exists some j ∈ [1..k] such that c<sup>j</sup> is enabled. This implies that clause C<sup>j</sup> is satisfied by the assignment. Hence, ϕ is a tautology.

⇐) The proof is technical and involves several invariants (see appendix).

We may now prove that any nonredundant workflow net that is integer unbounded is also continuously unsound (the reverse is not necessarily true). Therefore, integer unboundedness relates to continuous soundness much like continuous unsoundness relates to generalised soundness.

**Proposition 7.** *Let* <sup>N</sup> *be a nonredundant workflow net and <sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> *. If* (<sup>N</sup> ,*m*) *is integer unbounded, then* N *is not continuously sound.*

*Proof.* Let <sup>N</sup> = (P, T, F) and *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> be such that (<sup>N</sup> ,*m*) is not integer bounded. By Proposition 4, there exists *m* > **0** such that **0** −→<sup>∗</sup> <sup>Z</sup> *m* . By nonredundancy, there exist <sup>λ</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> and *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> such that {i: <sup>λ</sup>} −→<sup>∗</sup> {<sup>f</sup> : 1} <sup>+</sup> *<sup>m</sup>*.

In [21, Lemma 12], it is shown that {i: k} −→<sup>∗</sup> <sup>Z</sup> *n* implies the existence of some <sup>∈</sup> <sup>N</sup> such that {i: <sup>k</sup> <sup>+</sup> } −→<sup>∗</sup> {<sup>f</sup> : } <sup>+</sup> *<sup>n</sup>*. By invoking this lemma with k := 0 and *n* := *m* , we obtain {i: } −→<sup>∗</sup> {<sup>f</sup> : } <sup>+</sup> *<sup>m</sup>* for some <sup>∈</sup> <sup>N</sup>.

Altogether, {i: λ + } −→<sup>∗</sup> {f : λ + } + *m* + *m*. Since λ + ≥ 1, Lemma 1 yields {i: 1} −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> {<sup>f</sup> : 1}+*m* where *<sup>m</sup>* := (1/(λ+))*m* . As every transition of a workflow net produces at least one token, this contradicts the fact that N is continuously sound. Indeed, it is impossible to fully get rid of *m* > **0**.

#### **5 Using Relaxations For Structural Soundness**

A workflow net N is k*-quasi-sound* if {i: k} −→<sup>∗</sup> {f : k}. Furthermore, N is *structurally quasi-sound* if it is <sup>k</sup>-quasi-sound for some <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>.

As observed in [31], structural quasi-soundness is a necessary condition for structural soundness. The notion of structural quasi-soundness is naturally generalised to an arbitrary Petri net <sup>N</sup> = (P, T, F). Given markings *<sup>m</sup>*,*m* <sup>∈</sup> <sup>N</sup><sup>P</sup> , we say that *<sup>m</sup> structurally reaches <sup>m</sup>* in <sup>N</sup> if <sup>k</sup> ·*<sup>m</sup>* −→<sup>∗</sup> <sup>k</sup> ·*m* for some <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>. A workflow net is structurally quasi-sound iff *m* := {i: 1} structurally reaches *m* := {f : 1}. So, the observation of [31] can be rephrased as follows.

**Proposition 8.** *Let* N *be a workflow net. If* {i: 1} *does not structurally reach* {f : 1} *in* N *, then* N *is not structurally sound.*

The problem of structural quasi-soundness can be reduced to an instance of the Petri net reachability problem [31, Lemma 2.1]. Intuitively, the reduction produces a Petri net that nondeterministically chooses multiples of {i: 1} and {f : 1} for which to check reachability. Such an approach has a prohibitive computational cost as Petri net reachability is Ackermann-complete. However, we observe that structural reachability, and hence structural quasi-soundness, is equivalent to continuous reachability by Lemma 1.

**Proposition 9.** *Let* <sup>N</sup> = (P, T, F) *be a Petri net, and let <sup>m</sup>*,*m* <sup>∈</sup> <sup>N</sup><sup>P</sup> *be markings. It is the case that m structurally reaches m iff m* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *m .*

For a workflow net <sup>N</sup> = (P, T, F), let <sup>k</sup><sup>N</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>∪{∞} be the smallest number for which N is k<sup>N</sup> -quasi-sound. Then N is structurally sound iff k<sup>N</sup> = ∞ and N is k<sup>N</sup> -sound [31, Thm 2.1]. By Proposition 9, k<sup>N</sup> = ∞ can be checked in polynomial time via a continuous reachability query. Moreover, a lower bound on k<sup>N</sup> can be obtained by computing <sup>k</sup>N,<sup>Z</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> ∪ {∞}, defined as the smallest value such that {i: k} −→<sup>∗</sup> <sup>Z</sup> {<sup>f</sup> : <sup>k</sup>}. We obtain a better bound by defining <sup>k</sup>N,Q≥<sup>0</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>∪{∞} as the smallest value for which there is a continuous run π = λ1t<sup>1</sup> ··· λnt<sup>n</sup> such that {i: <sup>k</sup>} −→<sup>π</sup> <sup>Q</sup>≥<sup>0</sup> {<sup>f</sup> : <sup>k</sup>} and *<sup>π</sup>* <sup>∈</sup> <sup>N</sup><sup>T</sup> , where *<sup>π</sup>*[t] := <sup>i</sup>∈[1..n]:ti=<sup>t</sup> <sup>λ</sup>i. Values kN,<sup>Z</sup> and kN,Q≥<sup>0</sup> can respectively be computed by a translation to integer linear programming, and a decidable optimization modulo theory.

**Proposition 10.** *Let* N *be a workflow net. It is the case that* kN,<sup>Z</sup> ≤ kN,Q≥<sup>0</sup> ≤ k<sup>N</sup> *. Moreover,* kN,<sup>Z</sup> *can be computed from an integer linear program* P*;* kN,Q≥<sup>0</sup> *can be obtained by computing* min <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> : <sup>ϕ</sup>(k) *where* <sup>ϕ</sup> *is a formula from the existential fragment of mixed linear arithmetic* <sup>ϕ</sup>*, i.e.* <sup>∃</sup>FO(Q,Z, <, +)*; and both* P *and* ϕ *are constructible in polynomial time from* N *.*

#### **6 Free-Choice Workflow Nets**

Let N = (P, T, F) be a Petri net. We say that N is *free-choice* if for any s, t ∈ T, it is the case that either supp(•s) ∩ supp(•t) = ∅ or •s = •t. For example, the nets Nleft and Nright from Fig. 1 are respectively free-choice and not free-choice.

It is known that generalised soundness is equivalent to 1-soundness in freechoice workflow nets [28]. We will show that the same holds for structural soundness, and that, surprisingly, for continuous soundness as well. This means that notions of soundness collapse for free-choice nets. This is proven in the forthcoming Lemma 2 and Theorem 3, which form one of the main theoretical contributions of this work.

Let (N ,*m*) be a marked Petri net. We say that a transition t is *quasi-live* in (<sup>N</sup> ,*m*) if there exists *<sup>m</sup>* such that *<sup>m</sup>* −→<sup>∗</sup> *<sup>m</sup>* −→<sup>t</sup> . Similarly, we say that a transition t is *live* in (N ,*m*) if for all *m* such that *m* −→<sup>∗</sup> *m* , t is quasilive in (N ,*m* ). In words, quasi-liveness states that there is at least one way to enable t, and liveness states that t can always be re-enabled. The set of *quasilive* and *live* transitions of (N ,*m*) are defined respectively as F(m) := {t ∈ T | t is quasi-live in (N ,*m*)} and L(m) := {t ∈ T | t is live in (N ,*m*)}.

**Lemma 2.** *Let* <sup>N</sup> = (P, T, F) *be a free-choice Petri net, let* <sup>c</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>*, and let <sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> *. The following statements hold.*


**Lemma 3.** *Let* N *be a workflow net. If* N *is continuously sound, then* (<sup>N</sup> , {i: <sup>k</sup>}) *is bounded for all* <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>*.*

**Theorem 3.** *Let* N *be a free-choice workflow net. These statements are equivalent: (1)* N *is* 1*-sound, (2)* N *is generalised sound, (3)* N *is structurally sound, and (4)* N *is continuously sound.*

*Proof.* (1) ⇒ (2). This was shown in [28].


(3) <sup>⇒</sup> (1). Let <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> be such that <sup>N</sup> is <sup>k</sup>-sound. Let *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> be such that {i: 1} −→<sup>∗</sup> *<sup>m</sup>*. By Lemma 2(1), there is a marking *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> such that *<sup>m</sup>* −→<sup>∗</sup> *<sup>m</sup>* and F(*m* ) = L(*m* ). By Lemma 2(2), we have L(k · *m* ) = F(k · *m* ) = F(*m* ).

By k-soundness, (N , {i: k}) must be bounded [9, Proposition 3.2 and Lemma 3.6]. Thus, since {i: k} −→<sup>∗</sup> k · *m* −→<sup>∗</sup> k · *m* , it is also the case that (N , k · *m* ) is bounded. By k-soundness, k · *m* −→<sup>∗</sup> {f : k}. By invoking Lemma 2(3) with c := k, we conclude that *m* = {f : 1}. So, N is 1-sound as {i: 1} −→<sup>∗</sup> *m* −→<sup>∗</sup> *m* = {f : 1}.

(4) <sup>⇒</sup> (1). Assume that <sup>N</sup> is continuously sound. Let *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> be a marking such that {i: 1} −→<sup>∗</sup> *<sup>m</sup>*. By Lemma 2(1), there exists *<sup>m</sup>* <sup>∈</sup> <sup>N</sup><sup>P</sup> such that *<sup>m</sup>* −→<sup>∗</sup> *m* and L(m ) = F(m ). Clearly, {i: 1} −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* and by continuous soundness *m* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> {<sup>f</sup> : 1}. By Lemma 1, there exists <sup>b</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> such that <sup>b</sup> · *<sup>m</sup>* −→<sup>∗</sup> {<sup>f</sup> : <sup>b</sup>}.

By Lemma 3, continuous soundness of N implies that (N , b ·*m* ) is bounded, as {i: b} −→<sup>∗</sup> b · *m* . Since L(m ) = F(m ), it follows from Lemma 2(2) that L(b·*m* ) = F(b·*m* ). By invoking Lemma 2(3) with c := b, we derive *m* = {f : 1}. Therefore, N is 1-sound as {i: 1} −→<sup>∗</sup> *m* −→<sup>∗</sup> *m* = {f : 1}.

#### **7 Experimental Evaluation**

We implemented our approaches for generalised and structural soundness in C#.<sup>2</sup> We test continuous soundness via SMT solving. More precisely, we use an existential <sup>ψ</sup><sup>N</sup> formula of linear arithmetic, i.e. FO(Q, <, +), from [8]. This formula is such that ψ(*m*,*m* ) holds iff *m* −→<sup>∗</sup> <sup>Q</sup>≥<sup>0</sup> *<sup>m</sup>* in <sup>N</sup> . Continuous soundness amounts to the ∃∀-formula ψ<sup>N</sup> ({i: 1},*m*) ∧ ¬ψ<sup>N</sup> (*m*, {f : 1}). To solve such formulas, we use Z3 [26]. We further use Z3 to decide structural quasi-soundness and compute kN,Q≥<sup>0</sup> (see Proposition 10), again via the formulas of [8].

We evaluated our prototype implementation on a standard benchmark suite used regularly in the literature, and a novel suite of synthetic instances where generalised or structural soundness are hard to decide with a naive approach.

We compared with two established tools for soundness: LoLA (v2.0) [35], and Woflan [33].<sup>3</sup> The latter can only decide *classical* soundness (1-soundness + quasi-liveness). Nonetheless, we use quasi-live instances, so for which 1-soundness and classical soundness are equivalent. We further use a transformation to reduce the verification of k-soundness to the one of 1-soundness [9, Lemma 3.6]. On the

<sup>2</sup> The implementation can be obtained from https://doi.org/10.6084/m9.figshare. 19721674.v2.

<sup>3</sup> A version of Woflan suitable for running without user interaction was provided, via personal communication, by its maintainer.

other hand, LoLA can directly decide k-soundness. To do so, we start from {i: k} and check a CTL formula of the form ∀G∃F ((*m*[f] = k) ∧ - <sup>p</sup>=<sup>f</sup> *<sup>m</sup>*[p] = 0).

Experiments were run on an 8-Core Intel<sup>R</sup> CoreTM i7-7700 CPU @ 3.60 GHz with Ubuntu 18.04. We limited memory to ∼8 GB, and time to 120 s for each instance. Tools were called from a Python script. For LoLA and our implementation, we used the *time* module to measure time. Running Woflan involves some overhead, so we instead take the total verification time reported by Woflan itself.

#### **7.1 Free-Choice Benchmark Suite**

The benchmark suite encompasses 1386 free-choice Petri nets that represent business processes modeled in the IBM WebSphere Business Modeler. It was originally presented in [16], and has been studied frequently in the literature [10,17]. These nets are not workflow nets by our definition, but can be transformed using a known procedure [23]. Intuitively, the nets are workflow nets with multiple final places, and the procedure adds a dedicated output place and ensures that the resulting workflow net represents the desired behaviour. However, roughly 1% of the nets are not workflow nets by our definition even after the procedure, as they contain nodes that are not on a path from i to f. We removed these nets.

We further checked each net for safety using LoLA and dropped unsafe nets. Recall that (N , {i: 1}) is sound if each reachable marking has at most one token per place. Unsafe instances can be dropped as unsafety implies 1-unsoundness in free-choice nets [34, Thm. 4.2 and 4.4], and as existing methods for checking safety, *e.g.* via state-space exploration with partial order reductions, are very efficient (here needing a mean of 3 ms). Thus, we considered safe instances only. Among the 1386 instances, 1382 are workflow nets, and 977 are further safe.

We also invoked an implementation of the reduction rules of [10] to reduce the size of all instances.<sup>4</sup> As discussed in the introduction, the rules can reduce some instances to trivially sound nets. However, even the size of nontrivial reduced instances tends to be small, with an average number of places and transitions of roughly 14, while three quarters of nets have at most 18 places and transitions. This is small enough that a complete state-splace enumeration is often feasible, in particular as the nets are safe and especially LoLA utilizes powerful partial order reductions for such nets. As we want to focus on scalability, we chained instances to produce challenging synthetic nets based on real-world instances. This is a natural way of constructing workflow nets, intuitively, the final process can be composed of many subtasks. It can be seen as a special case of refinement operations, studied in the context of generalised soundness [20].

The chaining procedure merges two workflow nets N = (P, T, F) and N = (P , T , F ) into N := (P, T, F) where P := P ∪ P , T := T ∪ T ∪ {taux} with F as F + F extended with •taux[f] := 1, t • aux[i ] := 1, and •taux[p] = t • aux[p ] := 0 for other entries. It is readily seen that this construction (1) produces a free-choice net if both N and N are free-choice; and (2) preserves safety.

<sup>4</sup> At time of writing, an implementation is available at https://github.com/LoW12/ Hadara-AdSimul.

This way, we generated large instances by using ∈ {1, 21, 41,..., 401} randomly chosen unreduced safe instances from the benchmark suite as inputs to be chained into one instance, then reduced that instance. For each number , we produced 20 combined nets, with a fresh random choice each time, in order to have a more representative collection of nets for . This resulted in 420 instances, of which 405 are nontrivial after applying reduction rules.

A caveat is that such large nets may seem unlikely to arise in practice. It seems a human designer would avoid designing highly complex processes corresponding to Petri nets with thousands of places. However, process models are not only explicitly written by humans, but also machine-generated, *e.g.* by mining event logs (see [32] for a book on the topic). In particular, being free-choice is preserved by chaining, so a large free-choice net may "hide" and combine several less complex processes, which might necessitate analyzing large workflow nets.

**Results.** We checked the safe free-choice instances obtained as explained above for 1-soundness using LoLA, Woflan and our implementation of continuous soundness. The results are shown on the left of Fig. 3. The right-hand side of the figure provides an overview over the sizes of the nets. In each case, N refers to the number of original instances that were chained to create each instance.

**Fig. 3.** Experiments on chained free-choice instances. The x-value denotes the number N of chained nets. Dark thick lines denote the mean, and light thin lines of the same color denote the minimum and maximum, respectively. For Woflan, the minimum line is slightly below the line of this work. For this work, the minimum and maximum lines are very close to the mean. *Left:* The y-value denotes time for checking soundness of the 20 nets for each N. Marks on the gray line at 120 s denote timeouts. *Right:* The y-value denotes the size of generated nets. (Color figure online)

The results show that state-space exploration via LoLA is very fast for moderate sizes, but does not scale as well. Continuous soundness is in fact outperformed by LoLA for N ≤ 100, but scales much better, showing essentially linear growth in the given data range. For instance, continuous soundness takes a mean of 0.25 s for N = 1, a mean of 1.07 s for N = 201, and a mean of 2.28 s for N = 401.

Woflan performs very well on the original instances, but times out frequently for larger instances. Woflan checks so-called S-coverability [34]. This is fast on many instances, even large ones, but starts running into the exponential-time worst case when instances get larger. For N = 1 and N = 21, Woflan does not ever time out, while it times out for roughly half of the instances in the range from N = 201 to N = 401. Overall, we infer that for large free-choice workflow nets, deciding soundness by checking continuous soundness can outperform existing techniques, while the procedure is still competitive on moderate instances.

#### **7.2 Synthetic Instances**

In the previously discussed benchmark suite, nets are free-choice. So structural and generalised soundness are equivalent by Theorem 3. We considered including a second suite of 590 non-free-choice Petri nets that represent processes of the SAP reference model [25]. However them turn out to be 1-quasi-sound but not 1-sound, so they represent trivial cases for generalised and structural soundness: simply checking 1-soundness, or 1-quasi-soundness and then 1-soundness, decides all instances. It's also worth mentioning that none of the 590 SAP instances are continuously sound, so all of them can be shown to not be generalised sound by checking continuous soundness, without having to check 1-soundness.

In order to have a wider variety of challenging instances, we introduce several families of synthetic workflow nets. The nets are simple to understand, but have large numbers of reachable marking, so are challenging for approaches relying on state-space exploration, *e.g.* model checking.

*Encoding Arc Weights.* To simplify the presentation, we describe synthetic instances utilizing arcs with weights. For benchmarking, we removed the arc weights and instead input equivalent weightless nets. To do so, we used an encoding that simulates exponentially large weights by polynomially many transitions and places (the encoding is explained in ??). It preserves (quasi-)soundness, but significantly increases the number of reachable markings. Indeed, our synthetic instances are mostly trivial to solve by enumerating reachable markings when arcs have weights, but become much harder to decide when the encoding is used.<sup>5</sup> While much of the literature on workflow nets does not consider nets with arc weights, implicit structural encodings can occur in practice.

#### **Generalised Soundness**

*Benchmark Instances.* We introduce a synthetic family of nets where generalised soundness appears to be challenging. The family {Nc}<sup>c</sup>∈N≥<sup>1</sup> is defined at the top of Fig. 4. Parameter <sup>c</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup> is the smallest value for which <sup>N</sup><sup>c</sup> is <sup>c</sup>-unsound. From {i: c}, the sequence t c i t c+1 <sup>r</sup> can be fired, which leads to the deadlock {r : c+

<sup>5</sup> It is deliberately used to make instances challenging, not to ensure compatibility with LoLA or Woflan, as both support arc weights.

1}. Yet, when starting with k<c tokens in i, and firing t k <sup>i</sup> , transitions t<sup>r</sup> and t<sup>f</sup> can only be fired exactly k times, and {f : k} will be reached.

**Fig. 4.** *Top:* A workflow net N<sup>c</sup> that is c-unsound and k-sound for all k ∈ [1..c − 1]. *Bottom*: Three families of instances. *Bottom left:* Nsound-<sup>c</sup> is quasi-sound and c-sound for all <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>. *Bottom center:* <sup>N</sup>¬quasi-<sup>c</sup> is not structurally quasi-sound. *Bottom right:* <sup>N</sup>¬sound-<sup>c</sup> is c-quasi-sound for all <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>, but not structurally sound.

The naive approach to decide generalised soundness is to check k-soundness for all k until a counterexample is found or a bound is exceeded. It is known that if a counterexample exists, then there also is one of size at most exponential [9, Lemma 5.6 and 5.8]. The approach we chose for semi-deciding generalised soundness is to check continuous soundness. Recall that continuous soundness is a necessary (albeit not sufficient) condition, as shown in Theorem 1.

In our evaluation, we used Woflan and LoLA to check generalised soundness of the family for different c by checking 1-sound, . . . , c-soundness, and compared the result to the time needed for testing continuous soundness. Our main goal is to evaluate whether checking continuous soundness is efficient enough to serve as an inexpensive way to witness generalised unsoundness for nontrivial instances.

*Results.* Figure 5 depicts the results. Woflan and LoLA show good performance for small values of c, but do not scale well to larger values. They respectively time out for c ≥ 5 and c ≥ 8. The instances are not free-choice, so LoLA and Woflan need to explore the state-space for each k ≤ c, which becomes infeasible. For c ≥ 14, Woflan cannot even check 1-soundness within the time limit. LoLA can check 1- and 2-soundness for c ≤ 28, but cannot handle 2-soundness for larger c. Continuous soundness is efficiently verifiable even for c = 40. In particular, we need less than 5 s on all instances. The greatest time is at c = 33. Further, at most 1 s is needed on 34 out of 40 instances (mean of 0.6 s).

**Fig. 5.** Time to check generalised soundness of N<sup>c</sup> for different values of c. Marks on the gray line at 120 s denote timeouts. (Color figure online)

#### **Structural Soundness**

*Benchmark Instances.* For structural soundness, recall that our decision procedure is based on checking structural quasi-soundness and obtaining some lower bound for the smallest number for which the net is quasi-sound. Thus, we want to test on both benchmark instances that are structurally quasi-sound and those that are not. We introduce three families of non-free-choice nets for which structural soundness appears challenging. These instances are defined at the bottom of Fig. 4. We respectively denote them Nsound-<sup>c</sup> (left), N¬quasi-<sup>c</sup> (center) and <sup>N</sup>¬sound-<sup>c</sup> (right). We claim that: <sup>N</sup>sound-<sup>c</sup> is c-sound for all <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>; <sup>N</sup>¬quasi-<sup>c</sup> is not structurally quasi-sound; <sup>N</sup>¬sound-<sup>c</sup> is c-quasi-sound for all <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>, not <sup>k</sup>-quasi-sound for any other number <sup>k</sup> <sup>∈</sup> <sup>N</sup>≥<sup>1</sup>, and not structurally sound.

For the experiments, our goal is twofold. First, we want to evaluate whether utilizing continuous reachability to decide structural quasi-soundness is more efficient than using the known reduction to reachability described in [31, Lemma 2.1]. Woflan does not directly support checking reachability, so we only compare with LoLA. Second, we want to evaluate whether the lower bound for the smallest number for which the net is quasi-sound, which we dubbed kN,Q≥<sup>0</sup> towards the end of Sect. 5, is close to the actual smallest number, dubbed k<sup>N</sup> .

A caveat of this evaluation is that we evaluate only on our synthetic instances, and that computing kN,Q≥<sup>0</sup> is only one step in deciding structural soundness. However, we think that the evaluation on these hard synthetic instances can give insights into the applicability on nontrivial real-world instances.

*Results.* Figure 6 compares the time needed to verify structural reachability for LoLA and our prototype. For small instances, LoLA sometimes performs very well, but we scale better for large values. Of particular note is that in the absence of quasi-soundness, LoLA will generate an infinite state-space, so will generally run out of time or memory. In particular, LoLA times out for all <sup>c</sup> on <sup>N</sup>¬quasi-c. It also times out for <sup>c</sup> <sup>≥</sup> 32 on <sup>N</sup>¬sound-c. On the other hand, continuous soundness never times out for the given values of c. In fact, when we tested continuous soundness for much larger values of c, we found that our implementation of continuous reachability decides structural quasi-soundness for N¬quasi-<sup>c</sup> in under 2s for c = 20 000 000.

We further found that for all instances, kN,Q≥<sup>0</sup> = k<sup>N</sup> , that is, our lower bound exactly matches the smallest number for which the net is quasi-sound. Thus, it only remains to decide kN,Q≥<sup>0</sup> -quasi-soundness and kN,Q≥<sup>0</sup> -soundness in order to decide structural soundness. This is in contrast to the naive approach, which starts at k = 1 and checks k-quasi-soundness for each value up to kN,Q≥<sup>0</sup> .

**Fig. 6.** Time taken vs parameter c for checking structural quasi-soundness using the reduction to reachability, and utilizing our approach to compute kN,Q≥<sup>0</sup> , for each of the three families at the bottom of Fig. 4: Nsound-<sup>c</sup> (*left*),N¬quasi-<sup>c</sup> (*center*), N¬sound-<sup>c</sup> (*right*). Note that the axis ranges differ. Marks on the gray line at 120 s denote timeouts. (Color figure online)

#### **8 Conclusion**

In this work, we have shown how reachability relaxations allow to efficiently semidecide generalised and structural soundness. Our approach combines nicely with reduction rules, as they all preserve relaxations. In particular, we have introduced continuous soundness as an approximation of generalised soundness, and shown that it coincides with other types of soundness for free-choice nets.

As part of future work, we plan to migrate our prototype into the process mining framework ProM, to make the algorithms available to practitioners.

**Acknowledgements.** We thank Dirk Fahland and Eric Verbeek for their help with Woflan. Michael Blondin was supported by a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (NSERC), and by the Fonds de recherche du Qu´ebec - Nature et technologies (FRQNT).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Capture, Analyze, Diagnose: Realizability Checking Of Requirements in FRET**

Andreas Katis1(B) , Anastasia Mavridou<sup>1</sup>, Dimitra Giannakopoulou<sup>2</sup>, Thomas Pressburger2(B) , and Johann Schumann<sup>1</sup>

<sup>1</sup> Employed by KBR; NASA Ames Research Center, Moffett Field, CA, USA andreas.katis@nasa.gov <sup>2</sup> NASA Ames Research Center, Moffett Field, CA, USA tom.pressburger@nasa.gov

**Abstract.** Requirements formalization has become increasingly popular in industrial settings as an effort to disambiguate designs and optimize development time and costs for critical system components. Formal requirements elicitation also enables the employment of analysis tools to prove important properties, such as consistency and realizability. In this paper, we present the realizability analysis framework that we developed as part of the Formal Requirements Elicitation Tool (FRET). Our framework prioritizes usability, and employs state-of-the-art analysis algorithms that support infinite theories. We demonstrate the workflow for realizability checking, showcase the diagnosis process that supports visualization of conflicts between requirements and simulation of counterexamples, and discuss results from industrial-level case studies.

#### **1 Introduction**

Requirements elicitation is a proactive process which, by capturing the intended behavior of a system at an early stage, safeguards against decisions that could lead to increased development costs and even catastrophic failures. Formal requirements analysis can solidify engineers' confidence in the expressed specification. Our work is concerned with ensuring requirements consistency for system components, as a pre-requisite for subsequent system-level analysis. In particular, we focus on the notion of *realizability*: a realizable set of requirements guarantees that an implementation exists, such that it always behaves in a manner consistent with the specification, no matter what input it receives from its environment. The notion of realizability, first described as implementability by Pnueli and Rosner [47], has since then shaped an entire research area over the specification and synthesis of *reactive systems*.

This paper presents the realizability analysis framework that we have developed as part of NASA's open source tool FRET [3] for writing, understanding, and formalizing requirements. FRET is designed with a strong focus on usability, and is used by several NASA projects to explore the benefits of writing

Dimitra Giannakopoulou contributed to this work prior to joining AWS.

This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2022

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 490–504, 2022. https://doi.org/10.1007/978-3-031-13188-2\_24

requirements that can be processed by formal analysis tools [10,17,42,45]. Additionally, FRET has been used by external (to NASA) industrial and research teams, e.g., for the formalization of aircraft engine controller requirements [19]. FRET's realizability framework has two main goals: 1) to implement efficient algorithms for checking realizability, and 2) to provide user support in understanding and correcting sources of unrealizability. With these features, FRET provides an end-to-end solution to capturing, analyzing, and diagnosing requirements.

FRET's realizability framework provides a user-friendly interface for analyzing the requirements of system components. We have designed a graphic environment, in which the user can observe a (potentially) decomposed version of the specification that is sound with respect to realizability, as well as further dive into the task of *diagnosing* unrealizable requirements. Compositional analysis is based on our theoretical framework for checking realizability of a global specification through smaller, more tractable parts [25,43]. The diagnosis process is based on the theoretical work by K¨onighofer et al. [33,34] on generating minimal conflicts of unrealizability. We adjusted the diagnosis algorithm to support the discovery of all minimal conflicts in a contract, accompanied by a counterexample of unrealizability. The computed artifacts can be visualized as an interactive diagram that depicts the dependencies between requirements and conflicts. Counterexample traces that originate from these conflicts can also be simulated to enhance the understanding of unrealizability sources. For the analysis, we have integrated in FRET state-ofthe-art tools with respect to realizability checking modulo infinite theories.

In particular, the contributions of this work are:


#### **2 Related Work**

Table 1 provides a comparison between prominent requirements specification tools that support realizability checking with respect to various aspects, such as support for liveness properties, specification decomposition, algorithms.

Spectra Tools [37] and RATSY [8] are requirements specification tools for reactive synthesis over the General Reactivity of Rank 1 (GR(1)) fragment of LTL. The GR(1) fragment is particularly appealing, because it subsumes a subset of requirements that may appear in real world problems, adheres to the popular Assume-Guarantee paradigm, and a polynomial-time synthesis algorithm exists for it [9,46]. Both tools are limited to finite-state problems, and provide the ability to diagnose unrealizable specifications, primarily through the computation of minimal unrealizable cores [33,40] and counterstrategy synthesis, where an


**Table 1.** Comparison of requirements specification tools w.r.t. realizability checking.

implementation for the environment is generated, such that its actions always lead to the violation of the specification [34,39]. Furthermore, Spectra Tools provide the ability to repair unrealizable specifications [38].

SpeAR [22] and AGREE [14] are tools developed at Collins Aerospace for the purpose of requirements specification and analysis. Realizability checking is provided as a feature in both tools with limited support. Both tools depend on JKind's *k*-induction algorithm for realizability checking, which supports infinitestate problems, but is not sound with respect to unrealizable results [24].

EARS-CTRL [36] is yet another requirements specification platform that enables analysis of requirements written in Easy Approach to Requirements Syntax (EARS) [41]. Its realizability checking implementation relies upon autoCode4 [13], and is limited to the GXW subset of LTL [12]. Similar to Spectra Tools and RATSY, its analysis is limited to finite-state problems.

FRET's realizability-checking framework encapsulates desirable features of the aforementioned tools into an interface that is designed for users of varying backgrounds in formal methods. Additionally, it is the only requirements specification tool that provides a powerful decomposition approach to help with analysis performance [25,43]. FRET's realizability framework is powered by the algorithms in JKind and Kind 2. As such, it can analyze requirements that are as expressive as arbitrary discrete past-time metric LTL (pmLTL) formulas, and which may involve arithmetic expressions over the Linear Integer and Real Arithmetic SMT-LIB logics [7]. In practice, the framework targets analysis of formulas corresponding to requirements written in FRETish, as presented in the next section. FRETish requirements correspond to templates that form only a subset of all pmLTL formulas. As long as future FRETish extensions can be translated into pmLTL, analysis will be supported by the realizability backend.

#### **3 The FRETish Language**

In FRET, requirements are written in a restricted natural language called FRETish [27]. FRET formalizes FRETish requirements in pmLTL and then



into Lustre. A FRETish requirement is described using up to six distinct fields (the \* symbol designates mandatory fields): 1) scope specifies the time intervals where the requirement is enforced, 2) condition is a Boolean expression that triggers the response to occur at the time the expression's value becomes true, or is true at the beginning of the scope interval, 3) component\* is the system component that the requirement is levied upon, 4) shall\* is used to express that the component's behavior must conform to the requirement, 5) timing specifies when the response shall happen, subject to the constraints defined in scope and condition and 6) response\* is the Boolean expression that the component's behavior must satisfy.

FRETish provides 8 scopes: *global*, *in*, *before*, *after* , *notin*, *only in*, *only before*, and *only after* . The scope *global* means *always*; the others are with respect to when the system is in a mode or satisfies a Boolean expression. For example, *In mode M* means the requirement is enforced when the system is in mode *M*, as determined by the Boolean variable *M*. Also allowed for scope in place of a single Boolean variable is a Boolean expression, except for *in* which in the expression case is written with *while*; e.g., *While vehicle mode = hover* . In FRETish, the optional condition field is introduced by the words *upon*, *when*, or *if* , which are synonymous in FRETish, or the word *unless*, which is the same as *when !*. FRETish provides 10 timings: *immediately*, *at the next timepoint*, *always*, *eventually*, *never* , *for N* time steps, *within N* time steps, *after N* time steps, *until bool expr*, and *before bool expr*. When the scope is omitted it is taken as *global*; when the condition is omitted, it is taken as true; when the timing is omitted, it is taken as *eventually*. If we consider the condition being omitted as a separate case, there are 8 × 2 × 10 = 160 possible combinations of *scope*, *condition*, *timing*, each formalized as a distinct pmLTL formula template. The templates are generated by an algorithm that has been formally proven to generate formalizations with the intended semantics [15].

Boolean expressions can use the standard logical connectives (!, &, |) and can involve arithmetic relations (=,!=,<,<=,>,>=) and operators (+,−,∗,*/*) over integer and real variables. There are two predefined predicates *preInt* and *preReal* that refer to previous values: the expression *preInt*(*init, n*), for integer expression *n*, returns the value of *n* at the previous timepoint; if at the beginning of the trace where there is no previous value, then the value of *init* is returned. Currently, FRETish does not allow arbitrary nesting of temporal operators, e.g. "*In mode m, before q the system shall . . .* ". Timed operators with intermediate bounds are also not currently expressible; e.g., the equivalent of H[i,j] p, where i <sup>=</sup> 0.

**Fig. 1.** Implementation views for realizability checking in FRET.

For the remainder of the paper we use a running example, namely Finite State Machine (FSM), to demonstrate the various aspects of our framework. FSM contains 13 requirements for an abstracted version of an advanced autopilot system, and is part of the Lockheed-Martin Cyber-Physical Challenge Problems [18,32,42]. The requirements capture safety expectations with regards to the autopilot system's state transitions. Table <sup>2</sup> contains two FSM requirements written in FRETish and their pmLTL formulas, which are generated by FRET.

#### **4 Implementation**

Figure 1a shows the architectural components of FRET that communicate with or belong to the *Realizability Analysis* framework.<sup>1</sup> Grayed components illustrate the contributions of this paper. The asterisks in *Simulator* and *JKind* indicate that their existing implementation and features were considerably extended for this work. Arrows show the flow of data between components. All components are implemented in JavaScript using the React, Material-UI and D3 libraries [2,5,6].

FRET requirements are written using the *Editor/Elicitor* component, which also provides semantic explanations in various forms to assist users to clarify subtle semantic issues. The Simulator component provides an interactive visualizer based on graphical signal representation. Given a FRET requirement, it shows

<sup>1</sup> The FRET architecture is described in previous work by Giannakopoulou et al. [26].

temporal traces of each of the variables involved as well as the valuation of the requirement for each point in time. The user can interactively modify the input signals, which results in automatically updating the valuation of the requirement and thus, visually inspecting the temporal behavior of the requirement. As part of this work, we extended the Simulator with the following features: 1) the ability to import and export simulation traces, 2) support for numerical expressions, and 3) simultaneous visualization of multiple requirements. We integrated the Simulator in our realizability analysis workflow, to provide the ability to inspect and interact with counterexample traces in unrealizable specifications.

The *Variable Mapping* component collects essential information provided by the user regarding the variables of the requirements, e.g., data types and correspondence to system inputs or outputs. *Realizability Analysis* consists of three sub-components. The *Realizability Checking Engine* is responsible for checking realizability of requirement sets either monolithically or compositionally. Given an unrealizable set of requirements, the *Realizability Diagnosis Engine* implements the algorithm proposed by K¨onighofer et al. [33,34] to compute all minimal unrealizable sets of requirements, called *minimal unrealizable cores*. For each such core, a counterexample trace is computed that depicts a case under which the environment can lead the system into a deadlocking state. For the computation of minimal conflicts, our implementation uses the *delta-debugging* algorithm [49]. The *Visualizer* implements the user interface that displays analysis results as well as diagnostic results in the case of unrealizable specifications. These results are typically hard to digest in their original form. As such, the visualizer translates the information into an interactive diagram that allows the user to focus on unrealizable cores and inspect or simulate conflicting requirements.

We have integrated into FRET the JKind [23] and Kind 2 [11] tools for checking realizability. We actively maintain a fork of JKind [30], because the original repository lacks an implementation for the fixpoint algorithm by Katis et al. [31]. Formerly, the fork implementation relied on the AE-VAL solver's Model-Based Projection algorithm to perform quantifier elimination over forallexists formulas [20,21]. As part of this work, we have improved its performance by utilizing Z3's [16] quantifier elimination tactics. For instance, for the analysis of FSM the version of JKind using AE-VAL took 1524*.*82 s [43], whereas our optimization through Z3 dramatically decreased the time to 0*.*6 s.

The flow of usage of our framework is as follows (Fig. 1b). Once requirements are written in FRETish and variable information is provided, the user may start the analysis. Realizability can be performed through two different modes: 1) monolithic and 2) compositional, i.e., through the computation of independent sub-specifications, namely connected components. Each connected component is an undirected dependency graph with requirements as vertices and system outputs as edges. Compositional analysis has been proved faster and more prone to return result, compared to the monolithic option [43]. At the next step, the specification is translated to Lustre [29] and fed into JKind and Kind 2 to perform realizability checking. If the specification is unrealizable, the user can diagnose it using the generated counterexamples, and the FRET simulator.

## **5 Features Walkthrough**

We next demonstrate the features of framework through our running example.

**Fig. 2.** The realizability checking interface in FRET.

**Realizability Checking.** Figure 2 provides a snapshot of the overall graphical user interface (GUI) for realizability checking in FRET. As soon as the system component is selected, its connected components (CC) are computed. In the case of FSM, three CCs are identified. The GUI provides a focused view for each one ('CC*X*' tabs, with *X* being the corresponding index value), where the user can see which requirements participate in each CC via a table that dynamically grays out unrelated requirements. As soon as the CCs are computed, the realizability checking options become available, i.e., compositional and monolithic.

To check realizability, the user clicks the 'Check' button. Depending on the input specification, four possible answers may be given i.e., the specification is realizable, unrealizable, inconsistent, or the analysis is inconclusive ("unknown" result). Figure <sup>2</sup> shows the results of a compositional check for FSM, where connected components CC0 and CC1 are unrealizable, and CC2 is realizable.

**Diagnosing Unrealizability.** The compositional results above suggest that the FSM requirements are, as a whole, unrealizable. The next step in the process is to try and understand the source(s) of unrealizability. Since only CC0 and CC1 are unrealizable, it suffices to diagnose these independently. Following Fig. 2, the user selects the 'CC0' tab and clicks the 'Diagnose' button. The computation of minimal unrealizable cores kicks in, as outlined in Sect. 4, identifying 4 cores.

**Visualizing Unrealizability.** The raw artifacts produced by realizability checking and diagnosis are difficult for the users to digest. Therefore, the ability to visualize data in a user-friendly format is necessary, especially for unrealizable specifications. The core of our proposed solution to visualize unrealizability relies on the use of *chord diagrams* [1]. A chord diagram is a graphic representation of interrelationships between data, where each individual element is placed along the perimeter of a circular construct and relationships are depicted through edges between elements. An important feature of chord diagrams is the ability to maintain a clear representation of dependencies through *hierarchical edge bundling* [28], even when the size of data is large.

**Fig. 3.** (a) Chord Diagram for connected component CC0 in FSM. (b) Chord Diagram for Infusion Manager. (c) Focused view (one core) for Infusion Manager. (d) Focused view (one requirement) for Infusion Manager.

Figure 3a shows the chord diagram that is generated for connected component CC0 in FSM. Requirements and conflicts (i.e., unrealizable cores) define the input data to the chord diagram, which depicts each set using a distinguishable arc on the circular pattern (left and right arc, respectively). Chords, i.e., edges, connect each requirement to the conflicts that it appears in, with each edge being assigned a distinct color that matches the color-coded conflicts.

While hierarchical edge bundling helps us maintain a clear total view, it may be the case that the engineer would like to focus on a particular subset of dependencies, related to either a particular requirement or a specific conflict. We enable this through interactive means where parts of the interface that are not

**Table 3.** Counterexample for conflicting requirements **[FSM-006]** and **[FSM-007]**.


related to the selected element can be filtered out. Figure 2 shows an instance where the user has already interacted with the chord diagram for CC0, focusing on the unrealizable core containing **[FSM-006]** and **[FSM-007]**. The table of requirements is dynamically sorted so that relevant requirements appear on the top, and are outlined with the color of the corresponding conflict. Additionally, a counterexample witnessing the unrealizability of the conflict is displayed. Table 3 shows the counterexample for requirements **[FSM-006]** and **[FSM-007]**.

**Fig. 4.** Simulation of conflicting requirements **[FSM-006]** and **[FSM-007]**.

**Simulating Conflicting Requirements.** Our experience with counterexamples has indicated that a single execution trace is not enough to truly understand interactions between requirements. Therefore, we provide the ability for the user to interact with the set of conflicting requirements by using the FRET simulator, which we have substantially extended to meet our needs in visualizing conflicting requirements. Figure 4 shows how the counterexample (Table 3) for **[FSM-006]** and **[FSM-007]** is displayed in the simulator window: each line shows the values of the input signals as well as the valuation of each of the requirements.

The counterexample in Table 3 is not the only witness to the unrealizability of these requirements. Another example is a trace where requirement **[FSM-006]** holds for 5 consecutive ticks, leading to a violation of requirement **[FSM-007]** at the last tick, assuming that the antecedent of the latter was true at least once within the last 5 ticks. By modifying the values of the input variables, a user may identify additional witnesses to unrealizability causes. Combined with the ability to store and review traces, the simulator makes for an integral element towards understanding and repairing unrealizable specifications.

#### **6 Case Studies**

#### **6.1 Lift Plus Cruise Aircraft**

This study reports preliminary results on requirements for an autonomous 'lift plus cruise' concept aircraft.<sup>2</sup> This aircraft has a hovering vehicle mode, using its lifting rotors. From the hover mode, it can transition to a flying forward mode, eventually using its rear pusher propeller, and where lift is provided by the wing instead of the lifting rotors. Inbetween the hover and forward modes is a transitional mode which is a phase of concern for the aircraft engineers.



As of this paper, 11 requirements have been formalized in FRET [32]. A subset is shown in Table 4, describing the transition relations and constraints among various vehicle modes and vehicle motion. Requirement **[LPC01]** states that the vehicle starts in hover mode. Requirement **[LPC04]** specifies that if the previous mode is hover, and ground speed is greater than 15 knots, then the vehicle enters transitional mode. Requirement **[LPC09]** states the conditions for transitioning to forward mode. Variables hover, transitional and forward are specified as distinct integer constants. All of the other variables, e.g., airspeed, rear propeller, are outputs.

<sup>2</sup> We acknowledge discussions with John Kaneshige, Michael Feary and the Revolutionary Vertical Lift Technology team.

The first complete set of FRETish requirements raised concerns, as realizability checking yielded non-sensical counterexamples, where at least one requirement between **[LPC04]** and **[LPC09]** was violated in the initial state. We quickly identified the issue: both requirements were written using a version of the 'previous' operator pre which is undefined at the initial state. We addressed this by introducing the preInt and preReal operators, which at the initial state return the value of their first argument.

The resulting 11 requirements are in one CC, so we ran analysis in monolithic mode. The requirements are shown to be realizable in about 8 s. As a sanity check for realizability, we experimented with various subsets of the original requirements, as well as adding contradictions. A notable example was omitting **[LPC01]**, while modifying **[LPC03]** so that in hover mode, the vehicle must fly faster than 30 knots. This experiment, unexpectedly to us, led to realizability. Further inspection quickly revealed how omitting **[LPC01]** allows the controlled variable vehicle mode to never enter the hover mode. Including **[LPC01]** led to unrealizability with minimal conflict **[LPC01]**, **[LPC02]** and **[LPC03]**.

#### **6.2 Generic Infusion Pump**

This study explores 12 formalized requirements, proven unrealizable by Gacek et al. [24], of the Infusion Manager subcomponent for a Generic Patient Controlled Analgesic (GPCA) infusion pump [44]. The GPCA system originates from the Generic Infusion Pump Research project, a joint effort to identify best software engineering practices in the development of medical devices [4].

Taking advantage of FRETish's support for system modes (scope field), we derived 26 requirements, as opposed to the original 12 [32]. The increased number is a direct product of the declaration of 8 distinct modes, stemming from the system variable *Current System Mode*, which was originally of integer type. For example, requirement **G1** from Gacek et al.:

$$\begin{array}{c} \mathbf{G1} \stackrel{\text{def}}{=} (\text{Current}.\text{System}.\text{Mode}' \ge 0) \ \land (\text{Current}.\text{System}.\text{Mode}' \le 8) \ \land \\ (\text{Current}.\text{System}.\text{Mode}' = 0 \Rightarrow \text{Commanded}.\text{Flow}.\text{Rate}' = 0) \ \land \\ (\text{Current}.\text{System}.\text{Mode}' = 1 \Rightarrow \text{Commarked}.\text{Flow}.\text{Rate}' = 0) \end{array}$$

was rewritten into three requirements: **G1<sup>1</sup>** ensures that the system is in at least one of the 8 modes at any time, while requirements **G1<sup>2</sup>** and **G1<sup>3</sup>** ensure that the pump's flow rate is equal to 0 when the system is in mode 0 or 1, respectively. We additionally introduced requirements to ensure mutual exclusion between modes, something that was not needed with a single mode variable. We used Kind 2 to show equivalence between our requirements and the original specification.

Gacek et al. had already shown that the Infusion Manager requirements are unrealizable, verbally attributing unrealizability to a conflict between **G1** and requirement **G7**:

$$\mathbf{G7} \overset{\text{def}}{=} (System\text{\textbullet}\text{\textbullet}\text{\textbullet}\text{\textbullet}\text{\textbullet}\text{\textbullet}\text{\textbullet}\text{\textbullet}) \Rightarrow (Comm2vel\text{\textbullet}\text{\textbullet}' = Flow\text{\textbullet}\text{\textbullet}\text{\textbullet}\text{\textbullet})$$

The authors claimed that the requirements are unrealizable because they disagree on the value of output *Commanded Flow Rate* under specific conditions. However, FRET's diagnostic procedure provided a different answer, identifying 8 minimal unrealizable cores. Furthermore, the assumed conflict between requirements **G1** and **G7** does not really exist. While the two requirements do disagree on the value for the system output *Commanded Flow Rate* under specific circumstances, a realization still exists: one which would never exercise modes 0 or 1! Nevertheless, the report by Gacek et al. was still on the right track, as part of **G1** (FRETish requirement **G13**) and **G7** participate in at least one minimal unrealizable core with requirement **G11**, the latter enforcing the system to enter mode 1, given specific system input values:

> **G11** def = (*System On* ∧ *Configured <* 1) ⇒ *Current System Mode' = 1*

Figure 3b shows the chord diagram for Infusion Manager, depicting the 8 minimal unrealizable cores. Figures 3c and 3d show resulting states of the diagram after the user interacted with it in order to focus on a specific core, or a specific requirement, respectively.

#### **7 Conclusion**

We presented the realizability analysis framework in FRET and demonstrated its interactive GUI, which helps users diagnose unrealizable specifications through visualizations and simulation of conflicts. The framework employs stateof-the-art analysis algorithms that support infinite theories. In the future, we plan to extend the tool with recommendations in the form of environment assumptions.

#### **References**


2021. LNCS, vol. 13047, pp. 503–523. Springer, Cham (2021). https://doi.org/10. 1007/978-3-030-90870-6 27


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Information Flow Guided Synthesis**

Bernd Finkbeiner<sup>1</sup> , Niklas Metzger1(B) , and Yoram Moses<sup>2</sup>

<sup>1</sup> CISPA Helmholtz Center of Information Security, Saarbr¨ucken, Germany

*{*finkbeiner,niklas.metzger*}*@cispa.de <sup>2</sup> The Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering and the Taub Faculty of Computer Science, Technion, Haifa, Israel moses@technion.ac.il

**Abstract.** Compositional synthesis relies on the discovery of assumptions, i.e., restrictions on the behavior of the remainder of the system that allow a component to realize its specification. In order to avoid losing valid solutions, these assumptions should be *necessary* conditions for realizability. However, because there are typically many different behaviors that realize the same specification, necessary behavioral restrictions often do not exist. In this paper, we introduce a new class of assumptions for compositional synthesis, which we call *information flow assumptions*. Such assumptions capture an essential aspect of distributed computing, because components often need to act upon information that is available only in other components. The presence of a certain flow of information is therefore often a necessary requirement, while the actual behavior that establishes the information flow is unconstrained. In contrast to behavioral assumptions, which are properties of individual computation traces, information flow assumptions are *hyperproperties*, i.e., properties of sets of traces. We present a method for the automatic derivation of information-flow assumptions from a temporal logic specification of the system. We then provide a technique for the automatic synthesis of component implementations based on information flow assumptions. This provides a new compositional approach to the synthesis of distributed systems. We report on encouraging first experiments with the approach, carried out with the BoSyHyper synthesis tool.

#### **1 Introduction**

In *distributed synthesis*, we are interested in the automatic translation of a formal specification of a distributed system's desired behavior into an implementation that satisfies the specification [22]. What makes distributed synthesis far more interesting than the standard synthesis of reactive systems, but also more challenging, is that the result consists of a set of implementations of subsystems, each of which operates based only on partial knowledge of the global system state. While algorithms for distributed synthesis have been studied since the

This work was funded by the German Israeli Foundation (GIF) Grant No. I-1513- 407./2019. and by DFG grant 389792660 as part of TRR 248 – CPEC.

1990s [10,18,22], their high complexity has resulted in applications of distributed synthesis being, so far, very limited.

One of the most promising approaches to making distributed synthesis more scalable is *compositional synthesis* [7,9,14,19,23]. The compositional synthesis of a distributed system with two processes, p and q, avoids the construction of the product of p and q and instead focuses on one process at a time. Typically, it is impossible to realize one process without making certain assumptions about the other process. Compositional synthesis therefore critically depends on finding the assumption that p must make about q, and vice versa: once the assumptions are known, one can build each individual process, relying on the fact that the assumption will be satisfied by the synthesized implementation of the other process. Ideally, the assumptions should be both *sufficient* (i.e., the processes are realizable under the assumptions) and *necessary* (i.e., any implementation that satisfies the specification would also satisfy the assumptions). Without sufficiency, the synthesis cannot find a compositional solution; without necessity, the synthesis loses valid solutions. While sufficiency is obviously checked as part of the synthesis process, it is often impossible to find necessary conditions, because the specifications can be realized by many different behaviors. Any concrete implementation would lead to a specific assumption; however, this implementation is only known once the synthesis is complete, and an assumption that is satisfied by *all* implementations often does not exist.

In this paper, we propose a way out of this chicken-and-egg type of situation. Previous work on generating assumptions for compositional synthesis has focused on *behavioral* restrictions on the environment of a subsystem. We introduce a new class of more abstract assumptions that, instead, focus on the *flow of information*. Consider a system architecture (depicted in Fig. 1a) where two processes a and b are linked by a communication channel c, such that a can write to c and b can read from c. Suppose also that a reads a boolean input in from the environment that is, however, not directly visible to b. We are interested in a distributed implementation for a specification that demands that b should eventually output the value of input in. Since b cannot observe in, its synthesis must rely on the assumption that the value of in will be communicated over the channel c by process <sup>a</sup>. Expressing this as a *behavioral assumption* is difficult, because there are many different behaviors that accomplish this. Process a could, for example, literally copy the value of in to c. It could also encode the value, for example by writing to c the negation of the value of in. Alternatively, it could delay the transmission of in by an arbitrary number of steps, and even use the length of the delay to encode information about the value of in. Fixing any such communication protocol, by a corresponding behavioral assumption on a, would unnecessarily eliminate potential implementations of b. The minimal assumption that subsystem a must satisfy is in fact an information-flow assumption, namely that b will eventually be able to determine the value of in.

We present a method that derives necessary information flow assumptions automatically. A fundamental difference between behavioral and information flow assumptions is that behavioral assumptions are *trace properties*, i.e., properties of individual traces; by contrast, information flow assumptions are *hyperproperties*, i.e., properties of *sets* of traces. In our example, the assumption that a will eventually communicate the value of in to b is the hyperproperty that any two traces that differ in the value of in must eventually also differ in c. The precise difference between the two traces depends on the communication protocol chosen in the implementation of a; however, any correct implementation of a must ensure that some difference in b's input (on channel c) in the two traces occurs, so that b can then respond with a different output.

Once we have obtained information flow assumptions for all of the subsystems, we proceed to synthesize each subsystem under the assumption generated for its environment. It is important to note that, at this point, the implementation of the environment is not known yet; as a result, we only know *what* information will be provided to process b, but not *how*. This also means that we cannot yet construct an executable implementation of the process under consideration; after all, this implementation would need to correctly decode the information provided by its partner processes. Clearly, we cannot determine how to *decode* the information before we know how the implementation of the sending process *encodes* the information!

Our solution to this quandary is to synthesize a prototype of an implementation for the process that works with *any* implementation of the sender, as long as the sender satisfies the information flow requirement. The prototype differs from the actual implementation in that it has access to the original (unencoded) information. Because of this information the prototype, which we call a *hyper implementation*, can determine the correct output that satisfies the specification. Later, in the actual implementation, the information is no longer available in its original, unencoded form, but must instead be decoded from the communication received from the environment. However, the information flow assumption guarantees that this is actually possible, and access to the original information is, therefore, no longer necessary.

In Sect. 2, we explain our approach in more detail, continuing the discussion of the bit transmission example mentioned above. The paper then proceeds to make the following contributions:


**Fig. 1.** The distributed system of the *bit transmission* protocol. The architecture is given in (a), the hyper implementation of *b* in (b), the hyper implementation of *a* in(c), and the resulting local implementation of *b* in (d).


#### **2 The Bit Transmission Problem**

We use the *bit transmission* example from the introduction to motivate our approach. The example consists of two processes a and b that are combined into the distributed architecture shown in Fig. 1a. Process a observes the (binary) input of the environment through variable in and can communicate with the second process b via a channel (modeled by the shared variable c). Process b observes its own local input from a and has a local output out. We are interested in synthesizing an implementation for our distributed system consisting of two strategies, one for each process, whose combined behavior satisfies the specification. In this example, the specification for process b is to transmit the initial value of in, an input of a, to b's own output; this is expressed by the linear-time temporal logic (LTL) formula <sup>ϕ</sup><sup>b</sup> <sup>=</sup> in <sup>↔</sup> out. The specification does not restrict <sup>a</sup>'s behavior, and so <sup>ϕ</sup><sup>a</sup> <sup>=</sup> *true*. Since the value of out is controlled by b, whereas in is determined by the environment and observed by a, this specification forces b to react to an input that b neither observes nor controls. To satisfy the goal, out must remain *false* forever if in is initially *false*, while out must eventually become *true* at least once if in starts with value *true*. Indeed, in order to set out to *true*, process b must *know* that in is initially *true*, which can only be satisfied via information flow from a to b. We can capture this information flow requirement as the following hyperproperty: For every pair of traces that disagree on the initial value of in, process a must (eventually) behave differently on c. The requirement can be expressed in HyperLTL by the formula Ψ <sup>=</sup> <sup>∀</sup>π, π .(in<sup>π</sup> inπ- ) <sup>→</sup> (c<sup>π</sup> cπ- ). The information flow requirement does not restrict a to behave in a particular manner; the *encoding* of the information about in on the channel c depends on a's behavior. Under the assumption that a will behave according to the information flow requirement Ψ, one can synthesize a solution of b that is correct for every implementation of a. Given its generality, we call such a solution a *hyper implementation*. The hyper implementation of process b is shown in Fig. 1b. Since the point in time when the information is received by b is unknown during the local synthesis process, an additional auxiliary boolean variable t is added to the specification of b. This variable signals that the information has been transmitted and is later derived by a's implementation. Setting out to *true* is only allowed after t is observed by process b. When the hyper implementation is composed with the actual implementation of a, as shown in Fig. 1c, both local specifications are satisfied. The resulting local implementation of b, depicted in Fig. 1d, branches only on local inputs and, together with <sup>a</sup>, satisfies the specification. While changing state <sup>b</sup><sup>0</sup> to b<sup>1</sup>, process b cannot distinguish in from <sup>¬</sup>in. It has to wait for one time step, i.e., the first difference in outputs of process a, to observe the difference in the shared communication channel. The value of t is obtained from a's implementation and set to *true* with the first difference in c, forbidding the edge from h<sup>b</sup> 0 to h<sup>b</sup> <sup>3</sup> in the local implementation of <sup>b</sup>.

#### **3 Preliminaries**

*Architectures.* For ease of exposition we focus in this paper on systems with two processes. Let V be a set of variables. An architecture with two black-box processes <sup>p</sup> and <sup>q</sup> is given as a tuple (I<sup>p</sup>, I<sup>q</sup>, O<sup>p</sup>, O<sup>q</sup>, I<sup>e</sup>), where <sup>I</sup><sup>p</sup>, I<sup>q</sup>, O<sup>p</sup>, O<sup>q</sup>, and <sup>I</sup><sup>e</sup> are all subsets of <sup>V</sup>. <sup>O</sup><sup>p</sup> and <sup>O</sup><sup>q</sup> are the *output variables* of <sup>p</sup> and <sup>q</sup>. <sup>O</sup><sup>e</sup> are the output variables of the uncontrollable environment. The three sets <sup>O</sup><sup>p</sup>, O<sup>q</sup> and <sup>O</sup><sup>e</sup> form a partition of <sup>V</sup>. <sup>I</sup><sup>p</sup> and <sup>I</sup><sup>q</sup> are the *input variables* of processes <sup>p</sup> and <sup>q</sup>, respectively. For each black-box process, the inputs and outputs are disjoint, i.e., <sup>I</sup><sup>p</sup>∩O<sup>p</sup> <sup>=</sup> <sup>∅</sup> and <sup>I</sup><sup>q</sup> <sup>∩</sup> <sup>O</sup><sup>q</sup> <sup>=</sup> <sup>∅</sup>. The inputs <sup>I</sup><sup>p</sup> and <sup>I</sup><sup>q</sup> of the black-box processes are all either outputs of the environment or outputs of the other black-box process, i.e., <sup>I</sup><sup>p</sup> <sup>⊆</sup> <sup>O</sup><sup>q</sup> <sup>∪</sup> <sup>O</sup><sup>e</sup> and <sup>I</sup><sup>q</sup> <sup>⊆</sup> <sup>O</sup><sup>p</sup> <sup>∪</sup> <sup>O</sup><sup>e</sup>. We assume that all variables are of boolean type. For a set V ⊆ V, every subset V <sup>⊆</sup> <sup>V</sup> defines a *valuation* of <sup>V</sup> , where the variables in <sup>V</sup> have value *true* and the variables in V \ V have value *false*.

*Implementations.* An implementation of an architecture (I<sup>p</sup>, I<sup>q</sup>, O<sup>p</sup>, O<sup>q</sup>, I<sup>e</sup>) is a pair (s<sup>p</sup>, s<sup>q</sup>), consisting of a strategy for each of the two black-box processes. A *strategy* for a black-box process <sup>p</sup> is a function <sup>s</sup><sup>p</sup> : (2<sup>I</sup><sup>p</sup> )<sup>∗</sup> <sup>→</sup> (2<sup>O</sup><sup>p</sup> ) that maps finite sequences of valuations of p's input variables (i.e., *histories* of inputs) to a valuation of p's output variables. The (synchronous) *composition* <sup>s</sup><sup>p</sup>||s<sup>q</sup> of the two strategies is the function <sup>s</sup> : (2<sup>O</sup><sup>e</sup> )<sup>∗</sup> <sup>→</sup> (2<sup>V</sup> ) that maps finite sequences of valuations of the environment's output variables to valuations of all variables: we define s() = sp() <sup>∪</sup> <sup>s</sup>q() and, for <sup>v</sup> <sup>∈</sup> (2O<sup>e</sup> )∗, x <sup>∈</sup> <sup>2</sup>O<sup>e</sup> , <sup>s</sup>(<sup>v</sup> · <sup>x</sup>)=(sp(fp(v)) <sup>∪</sup> <sup>s</sup>q(fq(v)) <sup>∪</sup> <sup>x</sup>), where <sup>f</sup><sup>p</sup> and <sup>f</sup><sup>q</sup> map sequences of environment outputs to sequences of process inputs with fp() = , fp(<sup>v</sup> · <sup>x</sup>) = fp(v)·((x <sup>∪</sup> sq(fq(v))) <sup>∩</sup> Ip) and fq() = , fq(v·x) = fp(v)·((x <sup>∪</sup> sp(fp(v))) <sup>∩</sup> Iq).

*Specifications.* Our specifications refer to traces over the set V of all variables. In general, for a set V ⊆ V of variables, a *trace* over V is an infinite sequence <sup>x</sup>0x1x<sup>2</sup> ... <sup>∈</sup> (2<sup>V</sup> )<sup>ω</sup> of valuations of <sup>V</sup> . A *specification* <sup>ϕ</sup> <sup>⊆</sup> (2<sup>V</sup> )<sup>ω</sup> is a set of traces over <sup>V</sup>. Two traces of disjoint sets V,V ⊂ V can be *combined* by forming the union of their valuations at each position, i.e., <sup>x</sup>0x1x<sup>2</sup> ... <sup>y</sup>0y1y<sup>2</sup> ... <sup>=</sup> (x<sup>0</sup> <sup>∪</sup> <sup>y</sup><sup>0</sup>)(x<sup>1</sup> <sup>∪</sup> <sup>y</sup><sup>1</sup>)(x<sup>2</sup> <sup>∪</sup> <sup>y</sup><sup>2</sup>).... Likewise, the *projection* of a trace onto a set of variables V ⊆ V is formed by intersecting the valuations with <sup>V</sup> at each position: <sup>x</sup>0x1x<sup>2</sup> ... <sup>↓</sup><sup>V</sup> -= (x<sup>0</sup> <sup>∩</sup> <sup>V</sup> )(x<sup>1</sup> <sup>∩</sup> <sup>V</sup> )(x<sup>2</sup> <sup>∩</sup> <sup>V</sup> )....

For our specification language, we use propositional linear-time temporal logic (LTL) [21], with the set V of variables as atomic propositions and the usual temporal operators Next , Until U, Globally , and Eventually . System specifications are given as a conjunction <sup>ϕ</sup><sup>p</sup> <sup>∧</sup> <sup>ϕ</sup><sup>q</sup> of two LTL formulas, where <sup>ϕ</sup><sup>p</sup> refers only to variables in <sup>O</sup><sup>p</sup> <sup>∪</sup> <sup>O</sup><sup>e</sup>, i.e., the formula relates the outputs of process <sup>p</sup> to the outputs of the environment, and <sup>ϕ</sup><sup>q</sup> refers only to variables in O<sup>q</sup>∪O<sup>e</sup>. The two formulas represent the *local specifications* for the two black-box processes. An implementation s = (s<sup>p</sup>, s<sup>q</sup>) defines a set of traces

$$Transes(s\_p, s\_q) = \{x\_0 x\_1 \ldots \in (2^O)^\omega \mid x\_k = s(i\_0 i\_1 \ldots i\_{k-1}) \text{ for all } k \in \mathbb{N}\_{\text{for some}}\}$$

We say that an implementation *satisfies* the specification if the traces of the implementation are contained in the specification, i.e., *Traces*(s<sup>p</sup>, s<sup>q</sup>) <sup>⊆</sup> <sup>ϕ</sup>.

*The Synthesis Problem.* Given an architecture and a specification ϕ, the synthesis problem is to find an implementation s = (s<sup>p</sup>, s<sup>q</sup>) that satisfies <sup>ϕ</sup>. We say that a specification ϕ is *realizable* in a given architecture if such an implementation exists, and *unrealizable* if not.

*Hyperproperties.* We capture information-flow assumptions as hyperproperties. <sup>A</sup> *hyperproperty over* <sup>V</sup> is a set H <sup>⊆</sup> <sup>2</sup>(2<sup>V</sup> )<sup>ω</sup> of sets of traces over V [6]. An implementation (s<sup>p</sup>, s<sup>q</sup>) satisfies the hyperproperty H iff its traces are an element of H, i.e., *Traces*(s<sup>p</sup>, s<sup>q</sup>) <sup>∈</sup> <sup>H</sup>. A convenient specification language for hyperproperties is the temporal logic HyperLTL [5]. HyperLTL extends LTL with quantification over trace variables. The syntax of HyperLTL is given by the following grammar <sup>ϕ</sup> := <sup>∀</sup>π.ϕ | ∃π.ϕ <sup>|</sup> <sup>ψ</sup> and <sup>ψ</sup> := <sup>v</sup><sup>π</sup> | ¬<sup>ψ</sup> <sup>|</sup> <sup>ψ</sup> <sup>∧</sup> <sup>ψ</sup> <sup>|</sup> <sup>ψ</sup> <sup>|</sup> <sup>ψ</sup> <sup>U</sup> <sup>ψ</sup> where <sup>v</sup><sup>π</sup> ∈ V is a variable and π ∈ T is a trace variable. Note that the output variables are indexed by trace variables. The quantification over traces makes it possible to express properties like "ψ *must hold on all traces*", which is expressed by <sup>∀</sup>π. ψ . Dually, one can express that "*there exists a trace on which* ψ *holds*", denoted by <sup>∃</sup>π. ψ . The temporal operators are defined as in LTL.

In some cases, a hyperproperty can be expressed in terms of a binary relation on traces. A relation R <sup>⊆</sup> (2<sup>V</sup> )<sup>ω</sup> <sup>×</sup> (2<sup>V</sup> )<sup>ω</sup> of pairs of traces defines the hyperproperty H, where a set T of traces is an element of H iff for all pairs π, π <sup>∈</sup> T of traces in T it holds that (π, π ) <sup>∈</sup> R. We call a hyperproperty defined in this way a *2-hyperproperty*. In HyperLTL, 2-hyperproperties are expressed as formulas with two universal quantifiers and no existential quantifiers. A 2 hyperproperty can equivalently be represented as a set of infinite sequences over the product alphabet Σ<sup>2</sup>: for a given 2-hyperproperty <sup>R</sup> <sup>⊆</sup> <sup>Σ</sup><sup>ω</sup> <sup>×</sup> <sup>Σ</sup>ω, let R <sup>=</sup> {(σ0, σ <sup>0</sup>)(σ1, σ <sup>1</sup>)... <sup>|</sup> (σ0σ<sup>1</sup> ...,σ 0σ <sup>1</sup> ...) <sup>∈</sup> <sup>R</sup>}. This representation is convenient for the use of automata to recognize 2-hyperproperties.

#### **4 Necessary Information Flow in Distributed Systems**

In reactive synthesis it is natural that the synthesized process reacts to different environment outputs. This is also the case for distributed synthesis, where some outputs of the environment are not observable by a local process and the hidden values must be communicated to the process. In the following we show when such information flow is necessary.

#### **4.1 Necessary Information Flow**

Our analysis focuses on pairs of situations for which the specification dictates <sup>a</sup> *different* reaction from a given black-box process p. Such pairs imply the need for information flow that will enable p to distinguish the two situations: if p cannot distinguish the two situations, it will behave in the same manner in both. Consequently, the specification will be violated, no matter how p is implemented, in at least one of the two situations. A process p needs to satisfy a local specification <sup>ϕ</sup><sup>p</sup>, which relates its outputs <sup>O</sup><sup>p</sup> to the outputs <sup>O</sup><sup>e</sup> of the environment. (Recall that <sup>O</sup><sup>e</sup> may contain inputs to the other black-box process.) We are therefore interested in pairs of traces over <sup>O</sup><sup>e</sup> for which <sup>ϕ</sup><sup>p</sup> does *not* admit a common valuation of O<sup>p</sup>. We collect such pairs of traces in a *distinguishability relation*, denoted by Δ<sup>p</sup>:

**Definition 1 (Distinguishability).** *Given a local specification* <sup>ϕ</sup><sup>p</sup> *for process* <sup>p</sup>*, the* distinguishability relation <sup>Δ</sup><sup>p</sup> *is the set of pairs of traces over* <sup>O</sup><sup>e</sup> *(environment outputs) such that no trace over* <sup>O</sup><sup>p</sup> *satisfies* <sup>ϕ</sup><sup>p</sup> *in combination with both traces in the pair. Formally:*

$$\begin{aligned} \Delta\_p &= \{ (\pi\_e, \pi'\_e) \in (2^{O\_e})^\omega \times (2^{O\_e})^\omega \mid \\ &\forall \pi\_p \in (2^{O\_p})^\omega. \text{ if } \; \pi\_e \sqcup \pi\_p \models \varphi\_p \quad \text{then} \; \; \pi'\_e \sqcup \pi\_p \nvdash \varphi\_p \} \end{aligned}$$

By definition of <sup>Δ</sup><sup>p</sup>, process <sup>p</sup> must distinguish <sup>π</sup><sup>e</sup> from <sup>π</sup> <sup>e</sup>, because it cannot respond to both in the same manner. In our running example, <sup>Δ</sup><sup>b</sup> consists of all pairs of sequences of values of in that differ in the first value of in. Process b must act differently in such situations: if in is initially *true* then b must eventually set out to *true*, while if it starts as *false*, then b must keep out always set to *false*.

In general, a black-box process <sup>p</sup> must satisfy its specification <sup>ϕ</sup><sup>p</sup> despite having only partial access to Oe. The distinguishability relation therefore directly defines an *information flow* requirement: In order to satisfy ϕp, enough information about <sup>O</sup><sup>e</sup> must be communicated to <sup>p</sup> via its local inputs <sup>I</sup><sup>p</sup> to ensure that p can distinguish any pair of traces in Δp. We formalize this information flow assumption as the following 2-hyperproperty, which states that if the outputs of the environment in the two traces must be distinguished, i.e., the projection on <sup>O</sup><sup>e</sup> is in <sup>Δ</sup>p, then there must be a difference in the local inputs <sup>I</sup>p:

**Definition 2 (Information flow assumption).** *The* information flow assumption <sup>ψ</sup><sup>p</sup> *induced by* <sup>Δ</sup><sup>p</sup> *is the 2-hyperproperty defined by the relation*

$$R\_{\psi\_p} = \{ (\pi, \pi') \in (2^\mathcal{V})^\omega \times (2^\mathcal{V})^\omega \mid (\pi \downarrow\_{O\_e}, \pi' \downarrow\_{O\_e}) \in \Delta\_p \text{ then } \pi \downarrow\_{I\_p} \neq \pi' \downarrow\_{I\_p} \} $$

In our running example, the information flow assumption for process b requires that on any two executions that disagree on the initial value of in, the values communicated to b over the channel c must differ at some point. Observe that the information flow assumption <sup>ψ</sup><sup>p</sup> specifies neither how the information is to be encoded on c nor the point in time when the different communication occurs. However, <sup>ψ</sup><sup>p</sup> requires that the communication differs eventually if the initial values of in are different. Moreover, notice that both <sup>Δ</sup><sup>p</sup> and <sup>ψ</sup><sup>p</sup> are determined by p's specification ϕ<sup>p</sup>. The following theorem shows that the information flow assumption <sup>ψ</sup><sup>p</sup> is a necessary condition, the proof can be found in the full version of this paper [12].

**Theorem 1.** *Every implementation that satisfies the local specification* <sup>ϕ</sup><sup>p</sup> *for* <sup>p</sup> *also satisfies the information flow assumption* ψ<sup>p</sup>*.*

#### **4.2 Time-Bounded Information Flow**

We now introduce a strengthened version of the information flow assumption. As shown in Theorem 1, the information flow assumption is a necessary condition for the existence of an implementation that satisfies the specification. Often, however, the information flow assumption is not strong enough to allow for the separate synthesis of individual components in a compositional approach.

Consider again process b in our motivating example. The information flow assumption guarantees that any pair of traces that differ in the initial value of the global input in will differ at some point in the value of the channel c. This assumption is not strong enough to allow process b to satisfy the specification that b must eventually set out to *true* iff the initial value of in is *true*. Suppose that in is *true* initially. Then b must at some point set out to *true*. Process b can only do so when it *knows* that the initial value of in is *true*. The information flow assumption is, however, too weak to guarantee that process b will eventually obtain this knowledge. To see this, consider a hypothetical behavior of process a that sets c forever to *true*, if in is *true* in the first position, and if in is *false* then a keeps c true for n <sup>−</sup> 1 steps, where n > 0 is some fixed natural number, before it sets c to *false* at the nth step. This behavior of process <sup>a</sup> satisfies the information flow assumption for any number n; however, without knowing n, process b does not know how many steps it should wait for in to become *false*. If, at any point in time t, the channel c has not yet been set to *false*, process b can never rule out the possibility that the initial value of in is *true*; it might simply be the case that t<n and, hence, the time when c will be set to *false* still lies in the future of t! Hence, process b can never actually set out to *true*.

To address this, we present a finer version of the distinguishability relation from Definition 1 that we call *time-bounded distinguishability*. Recall that by Definition 1, a pair (π<sup>e</sup>, π <sup>e</sup>) is in the distinguishability relation <sup>Δ</sup><sup>p</sup> if every output sequence <sup>π</sup><sup>p</sup> for <sup>p</sup> violates <sup>p</sup>'s specification <sup>ϕ</sup><sup>p</sup> when combined with at least one of the input sequences <sup>π</sup><sup>e</sup> or <sup>π</sup> <sup>e</sup>. Equivalently, if <sup>ϕ</sup><sup>p</sup> is satisfied by <sup>π</sup><sup>p</sup> combined with <sup>π</sup><sup>e</sup>, then it is violated when <sup>π</sup><sup>p</sup> is combined with <sup>π</sup> <sup>e</sup>. Observe that for p to behave differently in two scenarios, a difference must occur at a finite time t. Clearly, this will only happen if p's input shows a difference in finite time. To capture this, we say that a pair (π<sup>e</sup>, π <sup>e</sup>) of environment output sequences is in the *time-bounded* distinguishability relation if the violation with π <sup>e</sup> is guaranteed to happen in finite time. In order to avoid this violation, process p must act in finite time, before the violation occurs on π <sup>e</sup>. We say that a trace π *finitely violates* an LTL formula <sup>ϕ</sup>, denoted by <sup>π</sup> <sup>f</sup> <sup>ϕ</sup>, if there exists a finite prefix <sup>w</sup> of <sup>π</sup> such that every (infinite) trace extending w violates ϕ.

**Definition 3 (Time-bounded distinguishability).** *Given a local specification* <sup>ϕ</sup><sup>p</sup> *for process* <sup>p</sup>*, the* time-bounded distinguishability relation <sup>Λ</sup><sup>p</sup> *is the set of pairs* (π<sup>e</sup>, π <sup>e</sup>) <sup>∈</sup> (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> <sup>×</sup> (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> *of traces of global inputs such that every trace of local outputs* <sup>π</sup><sup>p</sup> <sup>∈</sup> <sup>O</sup><sup>p</sup> *either violates the specification* <sup>ϕ</sup><sup>p</sup> *when combined with* <sup>π</sup><sup>e</sup>*, or finitely violates* <sup>p</sup>*'s local specification* <sup>ϕ</sup><sup>p</sup> *when combined with* <sup>π</sup> e*:*

$$\begin{aligned} A\_p &= \{ (\pi\_e, \pi'\_e) \in (2^{O\_e})^\omega \times (2^{O\_e})^\omega \mid \\ &\quad \forall \pi\_p \in (2^{O\_p})^\omega . \quad \text{if} \ \pi\_e \sqcup \pi\_p \models \varphi\_p \quad \text{then} \ \pi'\_e \sqcup \pi\_p \not\models \varphi\_p \ \} \end{aligned}$$

Note that, unlike the distinguishability relation <sup>Δ</sup><sup>p</sup>, the *time-bounded* distinguishability relation <sup>Λ</sup><sup>p</sup> is not symmetric: For (π<sup>e</sup>, π <sup>e</sup>), the trace π <sup>e</sup> <sup>π</sup><sup>p</sup> has to finitely violate <sup>ϕ</sup><sup>p</sup>, while the trace <sup>π</sup><sup>e</sup> <sup>π</sup><sup>p</sup> only needs to violate <sup>ϕ</sup><sup>p</sup> in the infinite evaluation. As a result, the corresponding *time-bounded* information flow assumption will also be asymmetric: we require that on input π<sup>e</sup>, process <sup>p</sup> eventually obtains the knowledge that the input is different from π <sup>e</sup>. For input π <sup>e</sup> we do not impose such a requirement. The intuition behind this definition is that on environment output π <sup>e</sup>, process p must definitely produce some output that does *not* finitely violate ϕ<sup>p</sup>. This output can safely be produced without ever knowing that the input is π <sup>e</sup>. However, on input π<sup>e</sup>, it becomes necessary for process p to eventually deviate from the output that would work for π <sup>e</sup>. In order to safely do so, p needs to realize after some finite time that the input is not π <sup>e</sup>. In our running example, <sup>π</sup><sup>e</sup> would be an input in which in is initially *true*, while π <sup>e</sup> will be one in which it starts out being *false*.

Suppose we have a function t : (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> <sup>→</sup> <sup>N</sup> that identifies, for each environment output π<sup>e</sup>, the time <sup>t</sup>(π<sup>e</sup>) by which process <sup>p</sup> is guaranteed to know that the environment output is not π <sup>e</sup>. We define the information flow assumption for this particular function t as a 2-hyperproperty. Since we do not know t in advance, the time-bounded information flow assumption is the (infinite) union of all 2-hyperproperties corresponding to the different possible functions t.

**Definition 4 (Time-bounded information flow assumption).** *Given the time-bounded distinguishability relation* <sup>Λ</sup><sup>p</sup> *for process* <sup>p</sup>*, the* time-bounded information flow assumption <sup>χ</sup><sup>p</sup> *for* <sup>p</sup> *is the (infinite) union over the 2-hyperproperties induced by the following relations* R<sup>t</sup>*, for all possible functions* <sup>t</sup> : (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> <sup>→</sup> <sup>N</sup>*:*

$$\begin{aligned} R\_t &= \{ (\pi, \pi') \in (2^\mathcal{V})^\omega \times (2^\mathcal{V})^\omega \mid \\ &\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \notin \left( \pi \downarrow\_{O\_e}, \pi' \downarrow\_{O\_e} \right) \in \Lambda\_p, \text{ then } \pi[0..t(\pi \downarrow\_{O\_e})] \downarrow\_{I\_p} \neq \pi'[0..t(\pi \downarrow\_{O\_e})] \downarrow\_{I\_p} \} \end{aligned}$$

Unlike the information flow assumption (cf. Theorem 1), the *time-bounded* information flow assumption is not in general a necessary assumption. Consider a modification of our motivating example, where there is an additional environment output start, which is only visible to process a, not to process b. The previous specification <sup>ϕ</sup><sup>b</sup> is modified so that if in is *true* initially, then out must be *true* two steps after start becomes *true* for the first time; if in is *false* initially, then out must become *false* after two positions have passed since the first time start has become *true*. The specification <sup>ϕ</sup><sup>a</sup> ensures that the channel <sup>c</sup> is set to *true* until start becomes *true*. Clearly, this is realizable: if in is *false* initially, process a sets c to *false* once start becomes *true*, otherwise c stays *true* forever. Process b starts by setting out to *true*. It then waits for c to become *false*, and, if and when that happens, sets out to *false*. In this way, process b accomplishes the correct reaction within two steps after start has occurred. However, the function t required by the time-bounded information flow assumption does not exist, because the time of the communication depends on the environment: the prefix needed to distinguish an environment output π<sup>e</sup>, where in is *true* initially from an environment output π <sup>e</sup>, where in is *false* initially, depends on the time when start becomes *true* on π e.

We now characterize a set of situations in which the time-bounded information flow requirement is still a necessary requirement. For this purpose we consider time-bounded distinguishability relations where the safety violation occurs after a bounded number of steps. We call such time-bounded distinguishability relations *uniform*; the formal definition follows below.

**Definition 5 (Uniform distinguishability).** *A time-bounded distinguishability relation* <sup>Λ</sup><sup>p</sup> *is* uniform *if for every trace* <sup>π</sup><sup>e</sup> <sup>∈</sup> (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> *of global inputs, and every trace* <sup>π</sup><sup>p</sup> <sup>∈</sup> (2<sup>O</sup><sup>p</sup> )<sup>ω</sup> *of local outputs of* <sup>p</sup>*, there exists a natural number* <sup>n</sup> <sup>∈</sup> <sup>N</sup> *such that for all* π <sup>e</sup> <sup>∈</sup> (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> *s.t.* (π<sup>e</sup>, π <sup>e</sup>) <sup>∈</sup> <sup>Λ</sup><sup>p</sup> *if* <sup>π</sup><sup>e</sup><sup>π</sup><sup>p</sup> <sup>ϕ</sup><sup>p</sup> *then* <sup>π</sup> <sup>e</sup><sup>π</sup><sup>p</sup> <sup>n</sup> <sup>ϕ</sup><sup>p</sup>*.*

**Theorem 2.** *Let* <sup>Λ</sup><sup>p</sup> *be a uniform time-bounded distinguishability relation derived from process* p*'s local specification* ϕ<sup>p</sup>*. Every computation tree that satisfies* <sup>ϕ</sup><sup>p</sup> *also satisfies the time-bounded information flow assumption* <sup>χ</sup><sup>p</sup>*.*

The proof of Theorem 2 can be found in the full version of this paper [12]. The relations presented in this section as well as the uniformity check can be represented by and verified with automata, also shown in [12].

#### **5 Compositional Synthesis**

We now use the time-bounded information flow assumptions to split the distributed synthesis problem for an architecture (Ip, Iq, Op, Oq, Ie) into two separate synthesis problems. The local implementations are then composed and form a correct system, whose decomposition returns the solution for each process.

#### **5.1 Constructing the Hyper Implementations**

We begin with the synthesis of local processes. Let <sup>Λ</sup><sup>p</sup> and <sup>Λ</sup><sup>q</sup> be the timebounded distinguishability relations for <sup>p</sup> and <sup>q</sup>, and let <sup>χ</sup><sup>p</sup> and <sup>χ</sup><sup>q</sup> be the resulting time-bounded information flow assumptions. In the individual synthesis problems, we ensure that process p provides the information needed by process q, i.e., that the implementation of p satisfies χ<sup>q</sup>, and, similarly, that <sup>q</sup> provides the information needed by p, i.e., q's implementation satisfies χ<sup>p</sup>.

We carry out the individual synthesis of a process implementation on trees that branch according to the input of the process (including t<sup>p</sup>) *and* the environment's output. In such a tree, the synthesized process thus has access to full information. We call this tree a *hyper implementation*, rather than an implementation, because the hyper implementation describes how the process will react to certain information, without specifying *how* the process will receive information. This detail is left open until we know the other process' hyper implementation: at that point, both hyper implementations can be turned into standard strategies, which are trees that branch according to the process' own inputs.

**Definition 6 (Hyper implementation).** *Let* p *and* q *be processes and* e *be the environment. A* <sup>2</sup><sup>O</sup>e∪Ip∪{tp}*-branching* <sup>2</sup><sup>O</sup>p∪{tq}*-labeled tree* <sup>h</sup><sup>p</sup> *is a* hyper implementation *of p.*

Since the hyper implementation has access to the full global information, while the time-bounded information flow assumption only guarantees that the relevant information arrives after some bounded time, the strategy has "too much" information. We compensate for this by introducing a *locality condition*: on two traces (π<sup>e</sup>, π <sup>e</sup>) <sup>∈</sup> <sup>Λ</sup><sup>p</sup> in the distinguishability relation of process <sup>p</sup>, as long as the input to the process from the external environment is identical, process <sup>p</sup>'s output must be identical until <sup>t</sup><sup>p</sup> happens (which signals that the bound for the transmission of the information has been reached). For traces (π<sup>e</sup>, π <sup>e</sup>) ∈ <sup>Λ</sup><sup>p</sup> outside the distinguishability relation, process p's output must be identical until there is a difference in the input to process p or in the value of t<sup>p</sup>.

**Definition 7 (Locality condition).** *Given the time-bounded distinguishability relation* <sup>Λ</sup><sup>p</sup> *for process* <sup>p</sup>*, the* locality condition <sup>η</sup><sup>p</sup> *for* <sup>p</sup> *is the 2-hyperproperty induced by the following relation* R*:*

$$\begin{split} R &= \{ (\pi, \pi') \in (2^{O\_e \cup I\_p \cup \{\mathtt{t}\_p\}})^\omega \times (2^{O\_e \cup I\_p \cup \{\mathtt{t}\_p\}})^\omega \mid \\ &\quad \text{if } (\pi \downarrow\_{O\_e}, \pi' \downarrow\_{O\_e}) \in A\_p, \text{ then } \pi[0..t] \downarrow\_{O\_p} = \pi'[0..t] \downarrow\_{O\_p} \text{ and} \\ &\quad \text{if } (\pi \downarrow\_{O\_e}, \pi' \downarrow\_{O\_e}) \notin A\_p, \text{ then } \pi[0..t'] \downarrow\_{O\_p} = \pi'[0..t'] \downarrow\_{O\_p} \} \end{split}$$

*where* <sup>t</sup> *is the smallest natural number such that* <sup>t</sup><sup>p</sup> <sup>∈</sup> <sup>π</sup>[0...t] *or* <sup>π</sup>[0...t] <sup>↓</sup>I<sup>p</sup> <sup>=</sup> π [t] <sup>↓</sup>I<sup>p</sup> *(and* <sup>∞</sup> *if no such* <sup>t</sup> *exists), and* <sup>t</sup> *is the smallest natural number such that* π[0...t ] <sup>↓</sup>I<sup>p</sup> <sup>=</sup> <sup>π</sup> [0...t ] <sup>↓</sup>I<sup>p</sup> *or* <sup>π</sup>[0...t ] <sup>↓</sup>{tp}<sup>=</sup> π [0...t ] ↓{tp} *(and* ∞ *if no such* t *exists).*

We now use HyperLTL to formulate the locality condition for process b in our running example. Based on the time-bounded distinguishability relation <sup>Λ</sup>b, which relates every trace with in = true in the first step to all traces on which in = false holds there, we can write the locality condition:

$$\begin{split} & (\forall \pi, \pi'. (\mathsf{in}\_{\pi} \wedge \neg \mathsf{in}\_{\pi'}) \to ((\mathsf{t}\_{\pi} \vee \mathsf{c}\_{\pi} \leftrightarrow \mathsf{c}\_{\pi'}) \, \mathsf{R}(\mathsf{out}\_{\pi} \leftrightarrow \mathsf{out}\_{\pi'})) \\ & \qquad \land (\neg(\mathsf{in}\_{\pi} \wedge \neg \mathsf{in}\_{\pi'})) \to (\mathsf{t}\_{\pi} \rightsquigarrow \mathsf{t}\_{\pi'} \vee \mathsf{c}\_{\pi} \leftrightarrow \mathsf{c}\_{\pi'}) \, \mathsf{R}(\mathsf{out}\_{\pi} \leftrightarrow \mathsf{out}\_{\pi'})) \end{split}$$

The order in the formula is analogous to the order in Definition 7. For all pairs of traces that are in the distinguishability relation, i.e., in is *true* on π and *false* on π , the outputs being equivalent on both traces can only be released by t on trace π or by a difference in the local inputs (c). Moreover, if the traces are not in the distinguishability relation, i.e., <sup>¬</sup>(in<sup>π</sup> ∧ ¬in<sup>π</sup>- ), then only a difference in t or c can release out to be equivalent on both traces. With the locality condition at hand, we define when a hyper implementation is locally correct:

**Definition 8 (Local correctness of hyper implementations).** *Let* p *and* <sup>q</sup> *be processes, let* <sup>ϕ</sup><sup>p</sup> *be the local specification of* <sup>p</sup>*, let* <sup>η</sup><sup>p</sup> *be its locality condition, and let* <sup>χ</sup><sup>q</sup> *be the information flow assumption of* <sup>q</sup>*. The hyper implementation* <sup>h</sup><sup>p</sup> *of* <sup>p</sup> *is* locally correct *if it satisfies* <sup>ϕ</sup><sup>p</sup>*,* <sup>η</sup><sup>p</sup>*, and* <sup>χ</sup><sup>q</sup>*.*

The specification <sup>ϕ</sup><sup>p</sup> is a trace property, while <sup>η</sup><sup>p</sup> and <sup>χ</sup><sup>q</sup> are hyperproperties. Since all properties that need to be satisfied by the process are guarantees, it is not necessary to assume explicit behaviour of process q to realize process p. Local correctness relies on the guarantee that the other process satisfies the current process' own information flow assumption. Note that both the locality condition and the information flow assumption for p build on the time-bounded distinguishability relation of p.

#### **5.2 Composition of Hyper Implementations**

The hyper implementations of each of the processes are locally correct and satisfy the information flow assumptions of the other process respectively. However, the hyper implementations have full information of the inputs and are dependent on the additional variables <sup>t</sup><sup>p</sup> and <sup>t</sup><sup>q</sup>. To construct practically executable local implementations, we first compose the hyper implementations into one strategy.

**Definition 9 (Composition of hyper implementations).** *Let* p *and* q *be two processes with hyper implementations given as infinite* 2<sup>O</sup>e∪Ip∪{tp}*-branching* <sup>2</sup><sup>O</sup>p∪{tq}*-labeled tree* <sup>h</sup><sup>p</sup> *for process* <sup>p</sup>*, and an infinite* <sup>2</sup><sup>O</sup>e∪Iq∪{tq}*-branching* <sup>2</sup><sup>O</sup>p∪{tp}*-labeled tree* <sup>h</sup><sup>q</sup> *for process* <sup>q</sup>*.*

**Fig. 2.** The composition of the hyper implementations of *a* in Fig. 1c and *b* in Fig. 1d. The states are labeled with the combination of states that can be reached for both processes.

*Given two hyper implementations* <sup>h</sup><sup>p</sup> *and* <sup>h</sup><sup>q</sup>*, we define the composition* <sup>h</sup> <sup>=</sup> <sup>h</sup><sup>p</sup>||h<sup>q</sup> *to be a* <sup>2</sup><sup>O</sup><sup>e</sup> *-branching* <sup>2</sup><sup>O</sup>p∪O<sup>q</sup> *-labeled tree, where* <sup>h</sup>(v)=(h<sup>p</sup>(f<sup>p</sup>(v)) <sup>∪</sup> <sup>h</sup><sup>q</sup>(f<sup>q</sup>(v))) <sup>∩</sup> (O<sup>p</sup> <sup>∪</sup> <sup>O</sup><sup>q</sup>) *and* <sup>f</sup><sup>p</sup>, f<sup>q</sup> *are defined as follows:*

$$\begin{aligned} f\_p(\epsilon) &= \epsilon & f\_p(v \cdot x) &= f\_p(v) \cdot ((x \cap I\_p) \cup (h\_q(f\_q(v)) \cap (I\_p \cup \{\mathfrak{t}\_p\}))) \\ f\_q(\epsilon) &= \epsilon & f\_q(v \cdot x) &= f\_q(v) \cdot ((x \cap I\_q) \cup (h\_p(f\_p(v)) \cap (I\_q \cup \{\mathfrak{t}\_q\}))) \end{aligned}$$

If each hyper implementation satisfies the time-bounded information flow assumption of the other process, then there exists a strategy for each process (given as a tree that branches according to the local inputs of the process), such that the combined behavior of the two strategies corresponds exactly to the composition of the hyper implementations.

The composition of the hyper implementations of the bit transmission protocol is shown in Fig. 2. The initial state is the combination of both process's initial states with the corresponding outputs. We change the state after the value of in is received. While process a directly reacts to in, process b cannot observe its value, and the composition can either be in h<sup>b</sup> <sup>0</sup> or <sup>h</sup><sup>b</sup> <sup>1</sup>. Both states have the same output. In the next step, process a communicates the value of in by setting c to *true* or *false*, such that the loop states h<sup>a</sup> <sup>1</sup>, h<sup>a</sup> <sup>1</sup> and <sup>h</sup><sup>a</sup> 2, hb <sup>3</sup> are reached.

The local strategies of the processes are constructed from the composed hyper implementations. As an auxiliary notion we introduce the *knowledge set*: the set of finite traces in the composition that cannot be distinguished by a process.

**Definition 10 (Knowledge set).** *Let* p *and* q *be two processes with composed hyper implementations* h <sup>=</sup> h<sup>p</sup>||h<sup>q</sup>*. For a finite trace* <sup>v</sup> <sup>∈</sup> (2<sup>I</sup><sup>p</sup> )<sup>∗</sup> *of inputs to* <sup>p</sup>*, we define the knowledge set* K<sup>p</sup>(v) *to be*

<sup>K</sup><sup>p</sup>(v) {w <sup>|</sup> w *is a finite trace of* (2<sup>O</sup><sup>e</sup> )<sup>∗</sup> *and* f<sup>p</sup>(w) = v}.

**Lemma 1.** *For all s* v, v <sup>∈</sup> (2<sup>I</sup><sup>p</sup> ) ∗ *, if* K<sup>p</sup>(v) = K<sup>p</sup>(v ) *then* h(v) <sup>↓</sup><sup>O</sup>p<sup>=</sup> h(v ) ↓<sup>O</sup><sup>p</sup> *.*

The proof of Lemma 1 can be found in the full version of this paper [12]. The local strategies from the composed hyper implementations are then defined as follows:

**Definition 11 (Local strategies from hyper implementations).** *Let* p *and* <sup>q</sup> *be two processes with time-bounded information flow assumptions* <sup>χ</sup><sup>p</sup> *and* <sup>χ</sup>q*, and* <sup>h</sup> <sup>=</sup> <sup>h</sup>p||h<sup>q</sup> *be the composition of their hyper implementations. For* <sup>j</sup> ∈ {p, q} *the strategy* <sup>s</sup><sup>j</sup> *, represented as a* <sup>2</sup>I<sup>j</sup> *-branching* <sup>2</sup>O<sup>j</sup> *-labeled tree for process* j*, is defined as follows:*

$$s\_j(\epsilon) = \epsilon \qquad s\_j(v) = \begin{cases} \emptyset & \text{if } |K\_j(v)| = 0\\ h(\min(K\_j(v))) \downarrow\_{O\_j} & \text{if } |K\_j(v)| > 0 \end{cases}$$

*where* min(K<sup>j</sup> (v)) *is the smallest trace based on an arbitrary order over* <sup>K</sup><sup>j</sup> (v)*.*

The base case of the definition inserts a label for unreachable traces in the composed hyper implementation. For example, the local inputs <sup>I</sup><sup>p</sup>\O<sup>e</sup> are determined by s<sup>q</sup>, and not all input words in (2<sup>I</sup><sup>q</sup> )<sup>∗</sup> are possible. Process <sup>p</sup>'s local strategy <sup>s</sup><sup>p</sup> can discard these input words. The second case of the definition picks the smallest trace in the knowledge set and computes the outputs from h that are local to a process. Intuitively, the outputs of h have to be the same for every trace that a process considers possible in the composed hyper implementations. We therefore pick one of them, compute the output of the composed hyper-strategy, and restrict the output to the local outputs of the process. The following theorem states the correctness of the construction in Definition 11.

**Theorem 3.** *Let* p *and* q *be two processes with time-bounded information flow assumptions* <sup>χ</sup><sup>p</sup> *and* <sup>χ</sup><sup>q</sup>*, let* <sup>h</sup> <sup>=</sup> <sup>h</sup><sup>p</sup>||h<sup>q</sup> *be the composition of their hyper implementations, and* <sup>s</sup><sup>p</sup> *and* <sup>s</sup><sup>q</sup> *be their local strategies. Then, for all* <sup>v</sup> <sup>∈</sup> (2<sup>O</sup><sup>e</sup> )<sup>∗</sup> *it holds that* <sup>h</sup>(v) = <sup>s</sup><sup>p</sup>(g<sup>p</sup>(v)) <sup>∪</sup> <sup>s</sup><sup>q</sup>(g<sup>q</sup>(v)) *where* <sup>g</sup><sup>p</sup>, g<sup>q</sup> *are defined as follows:*

$$\begin{cases} g\_p(\epsilon) = \epsilon & g\_p(v \cdot x) = g\_p(v) \cdot ((x \cap I\_p) \cup (s\_q(g\_q(v)) \cap I\_p)) \\ g\_q(\epsilon) = \epsilon & g\_q(v \cdot x) = g\_q(v) \cdot ((x \cap I\_q) \cup (s\_p(g\_p(v)) \cap I\_q)) \end{cases}$$

The proof is inductive over the words v <sup>∈</sup> (2<sup>O</sup><sup>e</sup> )<sup>∗</sup> and can be found in the full version of this paper [12]. Combining all definitions and theorems of the previous sections, we conclude with the following corollary.

**Corollary 1.** *Let* (I<sup>p</sup>, I<sup>q</sup>, O<sup>p</sup>, O<sup>q</sup>, I<sup>e</sup>) *be an architecture and* <sup>ϕ</sup> <sup>=</sup> <sup>ϕ</sup><sup>p</sup> <sup>∧</sup> <sup>ϕ</sup><sup>q</sup> *be a specification. If the hyper-strategies* <sup>h</sup><sup>p</sup> *and* <sup>h</sup><sup>q</sup> *are locally correct, then the implementation* (s<sup>p</sup>, s<sup>q</sup>) *satisfies* <sup>ϕ</sup>*.*

#### **6 A More Practical Approach**

A major disadvantage of the synthesis approach of the preceding sections is that the hyper implementations are based on the full set of environment outputs; as a result, hyper implementations branch according to inputs that are not actually available; this, in turn, results in our introduction of the locality condition.

In this section, we develop a more practical approach, where the branching is limited to the information that is actually available to a process: this includes any environment output directly visible to the process and, additionally, the information the process is guaranteed to receive according to the information flow assumption. As a result, the synthesis of the process is sound without need for a locality condition. We develop this approach under two assumptions: First, we assume that the time-bounded information flow assumption only depends on environment outputs the sending process can actually see; second, we assume that the time-bounded information flow assumption can be decomposed into a finite set of classes in the following sense: For a trace π of environment outputs, the information class [π]<sup>p</sup> describes that, on the trace <sup>π</sup>, the process <sup>p</sup> eventually needs to become aware that the current trace is in the set [π]. The information class is obtained by collecting all traces that are *not* related to π in the timebounded distinguishability relation.

**Definition 12 (Information classes).** *Given a time-bounded distinguishability relation* <sup>Λ</sup><sup>p</sup> *for process* <sup>p</sup>*, the* information class [π]<sup>p</sup> *of a trace* <sup>π</sup> *over* <sup>O</sup><sup>e</sup> *is the following set of traces:* [π]<sup>p</sup> = (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> \ {π <sup>∈</sup> (2<sup>O</sup><sup>e</sup> )<sup>ω</sup> <sup>|</sup> (π, π ) <sup>∈</sup> Λ<sup>p</sup>}

The next definition relativizes the specification of the processes for a particular information class, reflecting the fact that the process does not know the actual environment output, but only its information class; hence, the process output needs to be correct for all environment outputs in the information class.

**Definition 13 (Relativized specification).** *For a process* p *with specification* <sup>ϕ</sup><sup>p</sup> *and an information class* <sup>c</sup>*, the relativized specification* <sup>ϕ</sup>p,c *is the following trace property over* (I<sup>p</sup> <sup>∩</sup> <sup>O</sup><sup>e</sup>) <sup>∪</sup> <sup>O</sup><sup>p</sup>*:*

$$\varphi\_{p,c} = \{ \pi\_e \sqcup \pi\_p \mid \pi\_e \in (2^{I\_p \cap O\_e})^\omega, \pi\_p \in (2^{O\_p})^\omega \text{ s.t. } \forall \pi'\_e \in c. \ \pi'\_e \sqcup \pi\_p \models \varphi\_p \} $$

The component specification, which is the basis for the synthesis of the process, must take into account that the process does not know the information class in advance; the behavior of the other process will only eventually reveal the information class. Let IC be the set of information classes for process p. Assume that this set is finite. We now replace the inputs of the process that come from the other process with new auxiliary input channels IC as new inputs. In the hyper implementation, receiving such an input reveals the information class to the process. In the actual implementation, the information class will be revealed by the actual outputs of the other process that are observable for p. The component specification requires that the processes satisfy the relativized specification under the assumption that the information class is eventually received. We encode this assumption as a trace condition ψ, which requires that exactly one of the elements of IC eventually occurs.

**Definition 14 (Component specification).** *For process* p *with specification* <sup>ϕ</sup><sup>p</sup>*, the component specification* ϕ<sup>p</sup> *over* (I<sup>p</sup> <sup>∩</sup> <sup>O</sup><sup>e</sup>) <sup>∪</sup> IC <sup>∪</sup> <sup>O</sup><sup>p</sup> *is defined as*

$$<\langle \varphi\_p \rangle = \{ \pi \in (2^{(I\_p \cap O\_e) \cup IC \cup O\_p})^{\omega} \mid \text{ if } \pi \models \psi \text{ then } \pi \models \bigwedge\_{c \in IC} (\lozenge c \to \varphi\_{p,c}) \} $$

**Fig. 3.** The architecture used for our experiments in (a) where the number outputs, inputs, and communication channels can vary. Figure 3b shows the implementation of process *b* for its bit transmission component specification.

*where* <sup>ψ</sup> *is the following trace property over* (I<sup>p</sup> <sup>∩</sup> <sup>O</sup><sup>e</sup>) <sup>∪</sup> IC <sup>∪</sup> <sup>O</sup><sup>p</sup>*:*

$$\psi = \{ \pi \in (2^{(I\_p \cap O\_e) \cup IC \cup O\_p})^{\omega} \mid \exists \pi' \in (2^{O\_e})^{\omega} \cdot \pi \downarrow\_{I\_p \cap O\_e} = \pi' \downarrow\_{I\_p \cap O\_e} $$
 
$$ and \ \pi \models \mathsf{Q}[\pi'] \text{ and exactly one element of } IC \text{ occurs on } \pi \} $$

The component specification allows us to replace the locality condition (Definition 7), which is a hyperproperty, with a trace property. Note, however, that the process additionally needs to satisfy the information flow assumption of the other process, which may in general depend on the full set <sup>O</sup><sup>e</sup> of environment outputs. This would require us to synthesize the process on the full set O<sup>e</sup>, and to re-introduce the locality condition. In practice, however, the information flow assumption of one process often only depends on the information of the other process. In this case, it suffices to synthesize each process based only on the locally visible environment outputs.

Figure 3b shows the implementation of b for its component specification ϕ<sup>b</sup>. In contrast to its hyper implementation (cf. Fig. 1b), it does not branch according to in and t<sup>p</sup>, but only variables in IC. The specification is encoded as the following LTL formula:

$$\begin{aligned} \langle \varphi\_b \rangle &= \left( \Box \neg \mathbf{i} \mathbf{c}\_0 \lor \Box \neg \mathbf{i} \mathbf{c}\_1 \right) \land \bigotimes \left( (\mathbf{i} \mathbf{c}\_0 \lor \mathbf{i} \mathbf{c}\_1) \right) \\ &\to \left( (\mathsf{Q} \lor \mathbf{i} \mathbf{c}\_0 \to \mathsf{Q} \bullet \mathsf{ut}) \land (\mathsf{Q} \lor \mathbf{i} \mathbf{c}\_1 \to \square \neg \mathsf{out}) \right) \end{aligned}$$

The left hand side of the implication represents the assumption ψ, while the right hand side specifies the guarantee for each information class. The composition and decomposition can be performed analogously to the hyper implementations, where we map the value of ic to the values of the communication variables. We construct the automata for component specifications in the full version of this paper [12].

#### **7 Experiments**

The focus of our experiments is on the performance of the compositional synthesis approach compared to non-compositional synthesis methods for distributed systems. While the time-bounded information flow assumptions and the component specification can be computed automatically by automata constructions, we have, for the purpose of these experiments, built them manually and encoded them as formulas in HyperLTL or LTL, which were then entered to the BoSy/BoSyHyper [11] synthesis tool<sup>1</sup>. Our experiments are based on the following benchmarks:


Table 1 shows the performance of the compositional synthesis approach. The column architecture (Arch.) determines for each benchmark if the information flow is directional (dir.) or bidirectional (bidir.). Column (Inflow send) indicates the running time for the sending process; where applicable, column (Inflow rec.) indicates the running time for the synthesis of the process that only receives information. We compare the compositional approach to BoSyHyper, based on a standard encoding of distributed synthesis in HyperLTL (Inc. BoSy), and a specialized tool for distributed synthesis [2] (Distr. BoSy). All experiments were performed on a MacBook Pro with a 2,8 GHz Intel Quad Core processor and 16 GB of RAM. The timeout was 30 min.

Information flow guided synthesis outperforms the standard approaches, especially for more complex components. For example, in the atomic commitment benchmark, scaling in the number of inputs does not impact the synthesis of the local processes, while Distr. BoSy eventually times out, and the running time of Inc. BoSy increases faster than for the information flow synthesis. For all approaches, the Send All benchmark is the hardest one to solve. Here, each input that will eventually be set needs to be eventually sent, which leads to nontrivial communication over the shared variables and an increased state space to memorize the individual inputs. Nevertheless, the information flow guided synthesis outperforms the other approaches and times out with parameter 3

<sup>1</sup> The experiments are available at https://doi.org/10.6084/m9.figshare.19697359.



because BoSyHyper cannot cope with the number of states needed. Synthesizing a receiver that does not satisfy an information flow assumption is close to irrelevant for every benchmark run. Since these processes are synthesized with local LTL specifications, scaling only in the number of local inputs or information that will eventually be received is easily possible. Notably, these receivers are compatible with any implementation of the sender, whereas the solutions of the other approaches are only compatible for the same synthesis run.

#### **8 Related Work**

Compositional synthesis is often studied in the setting of *complete information*, where all processes have access to all environment outputs [9,14,17,19]. In the following, we focus on compositional approaches for the synthesis of distributed systems, where the processes have incomplete information about the environment outputs. Compositionality has been used to improve distributed synthesis in various domains, including reactive controllers [1,16]. Closest to our approach is assume-guarantee synthesis [3,4], which relies on behavioral guarantees of the processs behaviour and assumptions about the behavior of the other processes. Recently, an extension of assume-guarantee synthesis for distributed systems was proposed [20], where the assumptions are iteratively refined. Using a weaker winning condition for synthesis, remorse-free dominance [7] avoids the explicit construction of assumptions and guarantees, resulting in implicit assumptions. A recent approach [13] uses behavioral guarantees in the form of certificates to guide the synthesis process. Certificates specify partial behaviour of each component and are iteratively synthesized. The fundamental difference between all these approaches to the current work is that the assumptions are behavioral. To the best of our knowledge, this is the first synthesis approach based on information-flow assumptions. While there is a rich body of work on the verification of information-flow properties (cf. [8,15,24]), and the synthesis from information-flow properties and other hyperproperties has also been studied before (cf. [11]), the idea of utilizing hyperproperties as assumptions for compositional synthesis of distributed systems is new.

#### **9 Conclusion**

The approach introduced in this paper provides the foundation for a new class of distributed synthesis algorithms, where the assumptions refer to the flow of information and are represented as hyperproperties. In many situations, necessary information flow assumptions exist even if there are no necessary behavioral assumptions. There are at least two major directions for future work. The first direction concerns the insight that compositional synthesis profits from the generality of hyperproperties; at the same time, synthesis from hyperproperties is much more challenging than synthesis from trace properties. To address this issue, we have presented the more practical method in Sect. 6, which replaces locality, a hyperproperty, with the component specification, a trace property. However, this method is limited to information flow assumptions that refer to a finite amount of information. It is very common for the required amount of information to be infinite in the sense that the same type of information must be transmitted again and again. We conjecture that our method can be extended to such situations.

A second major direction is the extension to distributed systems with more than two processes. The two-process case has the advantage that the assumptions of one process must be guaranteed by the other. With more than two processes, the localization of the assumptions becomes more difficult or even impossible, if multiple processes have access to the required information.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Randomized Synthesis for Diversity and Cost Constraints with Control Improvisation**

Andreas Gittis, Eric Vin, and Daniel J. Fremont(B)

University of California, Santa Cruz, USA {agittis,evin,dfremont}@ucsc.edu

**Abstract.** In many synthesis problems, it can be essential to generate implementations which not only satisfy functional constraints but are also *randomized* to improve variety, robustness, or unpredictability. The recently-proposed framework of control improvisation (CI) provides techniques for the correct-by-construction synthesis of randomized systems subject to hard and soft constraints. However, prior work on CI has focused on qualitative specifications, whereas in robotic planning and other areas we often have quantitative quality metrics which can be traded against each other. For example, a designer of a patrolling security robot might want to know by how much the average patrol time needs to be increased in order to ensure that a particular aspect of the robot's route is sufficiently diverse and hence unpredictable. In this paper, we enable this type of application by generalizing the CI problem to support quantitative soft constraints which bound the expected value of a given cost function, and randomness constraints which enforce diversity of the generated traces with respect to a given label function. We establish the basic theory of labelled quantitative CI problems, and develop efficient algorithms for solving them when the specifications are encoded by finite automata. We also provide an approximate improvisation algorithm based on constraint solving for any specifications encodable as Boolean formulas. We demonstrate the utility of our problem formulation and algorithms with experiments applying them to generate diverse near-optimal plans for robotic planning problems.

#### **1 Introduction**

Correct-by-construction synthesis of systems from high-level specifications has become a popular paradigm in fields ranging from circuit design [5] to robotic task planning [25]. Synthesis techniques for many different types of specifications have been developed, especially for temporal logic formulas, which can encode many properties of interest [14]. One less-studied type of specification are *randomness constraints* that require the system's behavior to be sufficiently random, for instance by being close to a uniform distribution over the set of

A. Gittis and E. Vin—The two first authors contributed equally to the paper.

c The Author(s) 2022

S. Shoham and Y. Vizel (Eds.): CAV 2022, LNCS 13372, pp. 526–546, 2022. https://doi.org/10.1007/978-3-031-13188-2\_26

allowed behaviors. Such specifications are useful in many applications, as randomness can provide robustness, variety, and unpredictability to a system. For example, fuzz testing tools often use constraints to select classes of inputs which are more likely to trigger bugs, but then search randomly within that class to prevent bias [29]. In robotic planning, a patrolling security robot that uses a fixed plan satisfying its requirements might be vulnerable to exploitation; adding randomness to make its route unpredictable can make exploitation more difficult.

While there has been substantial work on synthesis with stochastic environments (e.g. [2,9]), randomness constraints require the system itself to behave randomly even if the environment is deterministic. Furthermore, unlike most specifications used in synthesis, randomness constraints are properties not of individual behaviors but rather of their distribution, and they cannot be concisely encoded into existing specification formalisms like PCTL [22] and SGL [3]. As a result, synthesis of systems under such constraints requires new techniques.

A recently-proposed paradigm for the correct-by-construction synthesis of systems under randomness constraints is *algorithmic improvisation* [13,15,16]. Algorithmic improvisation comprises a class of synthesis problems whose goal is to construct a randomized algorithm, an *improviser*, satisfying three kinds of constraints: *hard constraints* that the improviser's output must always satisfy, *soft constraints* that need only be satisfied to a certain (tunable) extent, and *randomness constraints* requiring the output to be sufficiently random. These types of constraints correspond to natural requirements arising for example in robot planning: the hard constraints can encode safety or other functional requirements, the soft constraints can encode notions of efficiency or optimality, and the randomness constraints enforce diversity or unpredictability. The original and most-studied form of algorithmic improvisation is the *control improvisation* (CI) problem (introduced in [12] and formalized in [16,17]), where the improviser generates finite sequences of symbols, the hard constraint is a trace property, the soft constraint requires some trace property hold with at least a desired probability, and the randomness constraint puts upper and lower bounds on the probability of individual outputs. Control improvisation and its extensions have been successfully used for musical improvisation [13], robotic planning [19], human modeling subject to constraints [1], and generating synthetic datasets for testing and training cyber-physical systems with machine learning components [18].

However, the prior work on CI is not general enough to cover many randomized synthesis problems of interest, for two reasons. First, many planning, design space exploration, and other problems come with a *cost function* expressing how optimal a particular solution is; in the setting of generating randomized solutions, the most natural soft constraint would be to require that the *expected cost* of a solution should be low, so that we can obtain a diverse set of near-optimal solutions. In a patrolling robot application, for example, the fastest patrol route might be unique and so predictable, and we then want to know by how much we would need to increase the average patrol time in order to enable a sufficientlydiverse set of routes. The prior work on CI cannot provide such an analysis.

Second, while the CI randomness constraint is sufficient to make the improviser's exact output unpredictable, it is *not* sufficient to ensure diversity when many outputs are similar to each other. Continuing our patrolling robot example, suppose that the robot has a choice of two rooms to go through: one room is larger, and so there are (say) 10<sup>6</sup> possible paths through it, vs. only 10<sup>3</sup> through the other room. Even if a perfectly-uniform distribution over all these paths is possible given our other constraints, the robot will end up entering the larger room almost all of the time. But from the point of view of an adversary that wishes to avoid being seen by the robot, the exact path is not relevant: what matters is *which room* the robot will enter, and that is highly predictable. For this application, we need a randomness requirement that enforces diversity not over the output of the improviser, but over some *attribute* of the output.

To enable such applications, in this paper we introduce the concept of *Labelled Quantitative Control Improvisation* (LQCI). This problem extends CI with a soft constraint bounding the expected cost of generated traces, and a randomness constraint requiring near-uniformity of the *label* of a trace, given by an arbitrary label function. We study the theory of LQCI, establishing precise conditions for when an LQCI problem is solvable and a general construction for solving it. We use our construction to develop efficient improvisation algorithms for a broad class of specifications given by finite automata, including common cost functions such as mission time or path length. For specifications not easily encoded to (reasonably-sized) automata, we provide an approximate improvisation algorithm based on constraint solving that handles symbolic specifications encoded as Boolean formulas. We also explore an extension of the LQCI problem for finding the *maximum-entropy* distribution satisfying the other constraints (as in [30]), and develop an algorithm for solving it using convex optimization. Finally, we conduct a case study demonstrating that our approach allows us to formalize and solve realistic robotic planning problems.

In summary, the main contributions of this paper are:


We conclude in Sect. 8 with a summary of results and directions for future work. For brevity, we defer full proofs of all results to the Appendix [21].

#### **2 Overview and Problem Definition**

In this section we formally define the LQCI problem, first using applications to robotic planning and fuzz testing to motivate various aspects of our definitions. We will return to the robotic planning example for our experiments in Sect. 7.

#### **2.1 Motivating Examples**

**Robotic Planning.** Consider the problem of generating a path for a package delivery robot, where the robot should efficiently visit various drop-off points, visiting charging stations as necessary along the way. Discretizing the world into a grid, we can represent a path as a finite sequence of north, south, east, and west moves. We might have various requirements for such paths, falling into the three types of constraints of a control improvisation problem described above: hard constraints such as completing mission objectives and not navigating into impassable terrain, soft constraints such as preferring shorter paths, and randomness constraints to ensure the chosen path is unpredictable. However, as we saw in Sect. 1, randomness over paths can be less important than randomness over specific features of a path: here, it might be that charging leaves the robot vulnerable for an extended period, so that it is important to limit the extent to which an adversary can predict ahead of time which charging station will be used. If there are 3 charging stations, then all possible paths are divided into 3 classes, and we want the class of a generated path to be unpredictable; we can formalize this as a *label function* which assigns labels to paths, and require that the distribution over labels be close to uniform. Since we do not want to simply pick a single path from each label class, we can also enforce randomness within each class, either by bounding the conditional probabilities of paths (so that no path is too likely relative to others in its class) or by taking the maximumentropy distribution that satisfies our randomness-over-labels condition (we will return to this approach in Sect. 6).

For efficiency, we want our robot to use routes which are as fast as possible, taking into account varying terrain. We could model this using a *cost function* assigning numerical costs to each path: here, the total time needed to traverse it. However, as mentioned in Sect. 1, prior work on CI can only encode Boolean soft constraints, such as requiring the cost of a path to be at most 5 with probability at least 0.9. While this does allow for some control over the cost, it requires setting an arbitrary threshold, and otherwise ignores the actual values of the cost; thus, a path of cost 6 is treated no differently than a path of cost 10<sup>5</sup>. Instead, we want to bound the *expected* cost of a path, so that both the probabilities of individual paths and their absolute costs are taken into account.

Putting all this together, we define our example planning problem as generating paths through the grid worlds in Fig. 1, subject to the following constraints:

#### **Hard Constraint:**


#### **Cost Constraint:**

The expected time to complete the mission must be at most a constant c. **Randomness over Labels:**

For each choice of charging station, the chance that the robot uses that station must be at least λ and at most ρ.

**Fig. 1.** Grid worlds for our robotic planning example. Darker background indicates higher cost and letters indicate: start and end points (**S**, **E**), impassable locations (**X**), delivery locations (**O**), charging stations (**C**).

#### **Randomness over Words:**

Conditioned on selecting a certain charging station, the probability of picking any path must be at least α and at most β.

Here, we assume that each grid cell has a cost representing how long it takes to traverse, with the cost of a path (the total mission time) being the sum of the costs of its cells. In Fig. 1, we show higher-cost cells as being darker, with the costs ranging from 0–3 for the small world and 0–10 for the large world. The layout of the map was chosen to admit a variety of different paths, motivated as follows: we envision an impassible river dividing the top and bottom halves of the map, with one low-cost bridge and two high-cost fords. The top-left charging station is a windmill and requires climbing a hill to access; there is also a hydroelectric station next to the river, and an easily-accessible substation near the main northsouth road.

**Fuzz Testing.** Prior work has shown that a variety of programs and protocols can be comprehensively tested by randomly sampling from automata encoding constraints on acceptable tests [11]. LQCI allows us to preserve such guarantees while exercising additional control over which tests are generated.

As an example, consider the problem of generating randomized network activity for a set of devices communicating over TCP; this could be useful to test robustness of a network monitoring application or network stack. There are a variety of different constraints we might wish to impose on the sequences of packets we generate: each connection should conform to the TCP protocol, so that the tests are meaningful<sup>1</sup>; tests should exhibit a variety of different behaviors

<sup>1</sup> We might also want to generate tests that *deviate* from the protocol. This could be done in a variety of ways, e.g. modifying our constraints to allow certain types of deviations, or first generating tests that conform to the protocol and subsequently mutating them.

such as successful/failed connections, interleaving of packets between different connections, etc.; and tests should be as short as possible while still exhibiting these different behaviors, so that we can maximize the number of tests we can perform in a given time. These constraints have trade-offs: for example, tests with failed connections that must be retried will necessarily be longer. As in the robotic planning example, we formulate these requirements as cost and label constraints, which allow us to balance our randomness and control needs.

For concreteness, consider the specific example of generating packet traces for 5 systems communicating over TCP. Our hard constraint can enforce that each connection follows the TCP protocol, using an encoding of the operation of the protocol as a finite automaton [24] (we will present efficient algorithms for LQCI with automata specifications below). Our cost function can assign a cost equal to the length of the trace, so that we prefer shorter sequences (whereas if we simply sampled uniformly from the language of the TCP automaton up to some length, longer sequences would be generated more frequently as there are exponentially more of them). Our label function could use two labels, distinguishing traces with connections that terminate cleanly from those that involve system failures and timeouts (we could also further subdivide into several types of failures). There are many more ways for a connection to fail than to terminate cleanly, and these two classes of traces might have significantly different lengths on average, but we want to ensure that our tests cover both cases adequately. By imposing constraints on the expected cost of a trace, as well as randomness constraints over the label and within each label class, we can control test length while enforcing sufficient diversity among the tests. In fact, we will see below that our LQCI algorithms can find the *minimum-cost* distribution consistent with the randomness constraints, thereby allowing us to test as efficiently as possible given coverage requirements.

#### **2.2 Problem Definition**

To formalize synthesis problems like those described above, we define the LQCI problem. Following the definition of CI [16,17], we frame the problem as sampling words over a finite alphabet Σ subject to several constraints. We use the general term *specification* to refer to an encoding of a property of words (a language): for example, a deterministic finite automaton (DFA) is a specification, where the DFA accepts a word if and only if it satisfies the specification; a Boolean formula is another kind of specification. The complexity of the LQCI problem will vary depending on the type of specifications used, as we will see later.

**Definition 1.** *A* Labelled Quantitative Control Improvisation (LQCI) instance *over an alphabet* <sup>Σ</sup> *is a tuple* <sup>C</sup> = (H, <sup>K</sup>, L, m, n, c, λ, ρ, α, <sup>ˆ</sup> <sup>β</sup>ˆ) *which contains:*


We note that the specifications and functions above are abstract, and our definition does not make any assumptions about how they will be encoded in a particular problem. For example, the hard constraint H over words might be instantiated as the language of a DFA, context-free grammar, etc. Later in the paper we will develop algorithms for solving classes of LQCI instances with specification formalisms that satisfy certain properties.

The restriction to finite traces (via the length bounds m and n) is consistent with prior work on using CI for robotic planning [19]: we frequently want plans that complete within a time limit. Likewise in fuzz testing we want tests of bounded length. Furthermore, as we will see, finite-trace LQCI is still a highly nontrivial problem, so we leave its extension to infinite traces as future work.

Given an LQCI instance, we define several convenient notations:


**Definition 2.** *Given an LQCI instance* C*, a distribution* D *over* Σ<sup>∗</sup> *is an* improvising distribution *for that instance if it satisfies the following constraints:*


*We say that an LQCI instance is* feasible *if there exists an improvising distribution for it (and* infeasible *otherwise). An* improviser *for an LQCI instance* *is a probabilistic algorithm which takes no input, has finite expected runtime, and whose output distribution is an improvising distribution. Given an LQCI instance* C*, the* LQCI problem *is then to determine if* C *is feasible, and, if so, to generate an improviser for* C*. Finally, an* improvisation scheme *for a class of LQCI instances is a probabilistic algorithm with finite expected runtime that solves the LQCI problem for instances in that class.*

As described in the preceding sections, the goal of our problem definition is to provide formal guarantees about the randomness of improvisations while respecting the various constraints. In some applications, we may simply wish to maximize randomness: then precise control over the randomness parameters for each label class is not needed, and in fact finding values of ˆαi, βˆ<sup>i</sup> which maximize randomness while remaining feasible is nontrivial. Building on our analysis of the basic LQCI problem in the next several sections, in Sect. 6 we will introduce a *maximum-entropy* version of LQCI which directly maximizes randomness without requiring ˆα<sup>i</sup> and βˆ<sup>i</sup> to be explicitly specified.

#### **3 Feasibility Conditions and the Greedy Construction**

In this section, we introduce a greedy construction which will be used to provide necessary and sufficient conditions for an LQCI instance to be feasible. This construction will also form the basis of the improvisation schemes presented later in the paper. For now, we will present the construction without assuming any particular specification formalism and ignoring algorithmic concerns: the description presented here will consider traces one by one and thus be inefficient. The next section will develop efficient implementations of these ideas.

The *greedy LQCI construction* is separated into two phases. In the first phase, the *greedy cost construction*, we define a distribution over each label class individually, greedily optimizing cost by giving as much weight as we can to the cheapest elements while respecting the randomness over words condition. In the second phase, the *greedy label construction*, we define a distribution over labels, greedily assigning maximum marginal probability to the label classes with the cheapest expected costs under the distributions from the first phase while respecting the randomness over labels condition. The intuition is that we want to first make sampling within each label class as cheap as possible, and then sample from the cheapest classes as often as possible, while satisfying the randomness requirements. We will prove below that this greedy approach in fact yields an improvising distribution whenever one exists.

*Toy Example.* We will begin with a toy example which illustrates the idea and correctness of the greedy construction. Suppose we want to sample from words of length 3 (m = n = 3) over the binary alphabet Σ = {0, 1}, subject to the hard constraint that each word must contain at least one 1. We will have two label classes: words with an odd number of 1s will be in label 1, and those with an even number in label 2. The cost of each word will be its integer value in binary. The label parameters will be λ = 0.2 and ρ = 1.0, so that each label must be sampled from with a probability at least 0.2 and at most 1.0. The word randomness parameters will be ˆα<sup>1</sup> = ˆα<sup>2</sup> = 0.1 and βˆ<sup>1</sup> = βˆ<sup>2</sup> = 0.5, so that when sampling from a particular label class, each word in the class must be selected with probability at least 0.1 and at most 0.5.

Figure 2 shows the greedy construction applied to this LQCI instance. Beginning with label 1, we need to construct a probability distribution over the words 001, 010, 100, and 111. We start by assigning 0.1 to each word, since ˆα<sup>1</sup> = 0.1. Then we assign as much additional probability as we can (up to βˆ<sup>1</sup> = 0.5) to the cheapest words first until a total of 1 is reached, as shown in the bottom left of Fig. 2. The result is that there are 3 distinct probabilities within the label class: the minimum ˆα<sup>1</sup> = 0.1, the maximum βˆ<sup>1</sup> = 0.5, and the overflow probability 0.3 on the word 010. This process results in a distribution over label 1 with expected cost 2.2, the minimum achievable while satisfying the randomness over words constraint. A similar process yields a distribution of expected cost 4.1 on label 2. Now that we know the minimum expected cost for each label, we should sample from the cheaper label as frequently as possible. Since λ = 0.2 and ρ = 1.0, we sample from label 2 with probability 0.2 (the minimum allowed) and from label 1 with probability 0.8, yielding a distribution over improvisations with expected cost 2.58. Our analysis will show that this is in fact the minimum possible expected cost over all distributions satisfying conditions (1) (3), and (4) in Definition 2. So if the cost bound c in the LQCI instance is at least this large, then we have an improvising distribution, and otherwise the instance is infeasible.

We now describe the two phases of our construction formally.

*The Greedy Cost Construction.* For a particular label class i ∈ {1,..., |Ω|}, we proceed as follows. Let δ<sup>i</sup> = (δ<sup>i</sup> 1,...,δ<sup>i</sup> |Θ| ) be a list of all the cost classes Ii,k with label i, sorted in increasing order of cost. Then fix o<sup>i</sup> = <sup>1</sup>−αˆ*i*|I*i*<sup>|</sup> <sup>β</sup>ˆ*i*−αˆ*<sup>i</sup>* , whose floor is the maximum number of words that can be assigned βˆ<sup>i</sup> probability (the maximum allowed) while still leaving at least ˆα<sup>i</sup> probability (the minimum allowed) for each remaining word. Then, moving through the cost classes in the order given by δ<sup>i</sup> , we assign βˆ<sup>i</sup> probability to each word in the class, until we get to a class δ<sup>i</sup> <sup>r</sup> where the cumulative number of words so far (including the new class) would exceed <sup>o</sup>i. To this class we assign <sup>β</sup>ˆi(o<sup>i</sup> <sup>−</sup> <sup>r</sup>−<sup>1</sup> <sup>k</sup>=1 <sup>|</sup>δ<sup>i</sup> <sup>k</sup>|)+ˆαi( <sup>r</sup> <sup>k</sup>=1 <sup>|</sup>δ<sup>i</sup> <sup>k</sup>| − oi) probability (spread uniformly over words in the class), the maximum allowed while leaving exactly ˆα<sup>i</sup> for each remaining word. Assigning ˆα<sup>i</sup> to the remaining words, we obtain a distribution D<sup>i</sup> over the whole label class Ii.

We note that this process is not well-defined when ˆα<sup>i</sup> = βˆ<sup>i</sup> (in which case we simply assign probability ˆα<sup>i</sup> to every word in Ii) or when ˆαi|Ii| > 1 (in which case the instance is infeasible due to ˆα<sup>i</sup> being too large); also, the process does not result in a probability distribution if <sup>β</sup>ˆi|Ii<sup>|</sup> <sup>&</sup>lt; 1 (in which case the instance is infeasible due to βˆ<sup>i</sup> being too small). Except in these cases, we get a well-defined distribution D<sup>i</sup> over I<sup>i</sup> which satisfies conditions (1) and (4) of Definition 2. Moreover, the expected cost of D<sup>i</sup> is minimal among all such distributions, since it assigns as much weight as possible to the words with lowest cost.

**Fig. 2.** Applying the greedy LQCI construction to our toy example. Counter-clockwise from upper left: table of improvisations, the greedy cost construction, the greedy label construction, and the final improvising distribution.

*The Greedy Label Construction.* Given the distributions D<sup>i</sup> for each label class I<sup>i</sup> from the first stage, we now choose a distribution over labels. Following a similar pattern as before, let δ be a list of the distributions D<sup>i</sup> sorted in order of increasing expected cost. Then fix <sup>u</sup> <sup>=</sup> <sup>1</sup>−|Ω|<sup>λ</sup> <sup>ρ</sup>−<sup>λ</sup> , which is the number of label classes that can be assigned probability ρ (the maximum allowed) while still leaving at least λ (the minimum allowed) for each remaining class. We assign ρ probability to the first u label classes in δ. To the next label class we assign probability 1−ρu−λ(|Ω|−u−1), the maximum allowed while leaving exactly λ for each remaining label class. Finally, we assign λ to all remaining label classes, and call the resulting distribution over labels Dˆ. Similar to before, this process will be well-defined and result in a distribution when <sup>1</sup> <sup>ρ</sup> ≤ |Ω| ≤ <sup>1</sup> <sup>λ</sup> ; otherwise, ρ is too small or λ is too large for condition (3) of Definition 2 to be satisfied.

To complete the construction, we obtain a final distribution D over words by first sampling a label i from Dˆ and then sampling from Di. The greedy cost construction ensured that D<sup>i</sup> is defined over the class I<sup>i</sup> ⊆ I and assigns probability between ˆα<sup>i</sup> and βˆ<sup>i</sup> to each word, so D will satisfy the hard and randomness over words constraints in Definition 2. The greedy label construction ensures that Dˆ assigns probability between λ and ρ to each label, so D will also satisfy the randomness over labels constraint. Finally, since each phase selects a distribution of minimal cost amongst those satisfying the corresponding constraints, if *any* improvising distribution exists then D will have no greater cost, thereby satisfying the cost constraint and being an improvising distribution. Formalizing this argument yields the following theorem (see the Appendix [21] for details):

**Theorem 1.** *An LQCI instance is feasible if and only if all of the following conditions are true:*


We conclude this section with a reminder that the greedy LQCI construction is a *construction* and not a practical algorithm: it defines a distribution but not a practical way to compute it for a specified LQCI instance. With common specification formalisms such as DFAs and Boolean formulas, the number of possible improvisations can easily be exponential in the size of the problem instance. In this case, assigning probabilities to words one at a time as described above in the abstract construction would be highly impractical. Instead, the algorithms we present in the following sections are able to avoid enumerating exponentiallylarge sets by working with implicit representations to create distributions equal to or approximating the one produced by the greedy LQCI construction.

#### **4 Exact LQCI for Automata Specifications**

The greedy LQCI construction from Sect. 3 gives us a way to determine if an LQCI instance is feasible and, if so, to build an improvising distribution. Implementing the construction requires several operations—such as computing the size of the label/cost classes—which may or may not be tractable depending on the types of specification used in the instance. In this section, we will identify a sufficient list of operations which yield an efficient generic improvisation scheme for any class of LQCI instances with specifications supporting these operations. Then we will instantiate the scheme for two natural classes of specifications given by deterministic finite automata, obtaining efficient improvisation algorithms.

Following the description of the preceding section, we can see that for a given LQCI instance, the operations listed below are sufficient to complete the greedy LQCI construction and sample from the resulting distribution:

**Definition 3.** *(Sufficient Operations) Given an LQCI instance* C*:*


If we can implement these operations in polynomial time, we can build a polynomial-time improvisation scheme in the sense of [16,17], i.e., an algorithm which solves the LQCI problem in polynomial time, and whose generated improvisers themselves run in polynomial (expected) time. To do this we first compute the list of possible costs and the size of each Ii,k. We then perform a modified version of the greedy construction which assigns probabilities to entire cost classes instead of individual words. As each word in a class has the same label and cost, we can satisfy our cost and randomness requirements with a distribution that assigns the same probability to every word within a class. Then to implement placing probability p on each word of Ii,k without enumerating this potentially exponentially-large set, we simply choose the set with probability p|Ii,k| and then sample uniformly from it (see the Appendix [21] for a detailed argument).

**Theorem 2.** *Suppose for a class of LQCI instances the operations in Definition 3 can be performed in polynomial time (in the size of the instance). Then there is a polynomial-time improvisation scheme for that class.*

One broad class of specifications to which this scheme can apply is deterministic finite automata (DFAs): for example, we can encode the specifications from our robotic planning example as DFAs. While a DFA can encode the hard specification H directly, encoding cost and label functions is not as clear. We consider two natural encodings: most simply, we can label each state of the DFA with an integer, assigning the associated label/cost to words ending at that state.

**Theorem 3.** *Consider the class of LQCI instances where* H *is a DFA,* K *and* L *are given by DFAs which output an integer cost/label associated with the state they end on, the length bounds are given in unary and all other numerical parameters in binary. This class has a polynomial-time improvisation scheme.*

*Proof (Sketch).* Operation (1) is trivial. For (2) and (3), we can easily construct DFAs accepting all improvisations with a given label and cost, then apply classical techniques for counting/sampling from the language of a DFA [23]. 

To capture cost functions like path length or mission time (as in our planning example), we consider a second encoding using weighted DFAs: states are again labeled with integers, but the cost is now given by *accumulating* costs from every state passed through. Here, the number of possible costs can grow linearly with the largest cost of a single state, and so be exponential in the size of the (binary) encoding; as a result we only obtain a *pseudopolynomial* improvisation scheme by applying Theorem 2. The algorithm can still be feasible, however, when the magnitude of possible costs is not too large, as we will see in Sect. 7.

**Theorem 4.** *Consider the class of LQCI instances as in Theorem 3 but where* K *is given by a weighted DFA, i.e. summing the integer costs associated with each state of a DFA accepting path (with multiplicity). This class has a pseudopolynomial improvisation scheme.*

*Proof (Sketch).* We can perform operation (1) by dynamic programming over the states and word lengths up to the length bound n. If the maximum cost of a state in the DFA for K is M, then the cost of an improvisation is at most M(n + 1); so for (2) and (3) we can build DFAs of size *poly*(M,n) recognizing Ii,k and then apply counting/sampling as above. If state costs were encoded in unary, the operations above would take polynomial time and Theorem 2 would apply. Converting from binary to unary yields a pseudopolynomial scheme. 

#### **5 Approximate LQCI for Symbolic Specifications**

The LQCI algorithms for DFAs that we developed in the previous section cover many useful specifications; however, as we will see in Sect. 7, even fairly simple specifications can require very large automata when represented explicitly. In this section we propose an algorithm that avoids such blowup by working with *symbolic* specifications given by Boolean formulas. We cannot use our scheme of Theorem 2 directly, because counting the number of solutions of a Boolean formula is #P-hard. Nevertheless, we will show that by leveraging recent advances in SAT solving, we can *approximately* solve LQCI to any desired accuracy.

We consider LQCI instances with specifications given by Boolean formulas, whose variables encode traces and costs; for modeling convenience, we also allow a vector of auxiliary variables z. Specifically, we assume we are given:


We further assume that the instance has only a polynomial number of labels, although there can be exponentially-many costs.

Given such an instance, we can readily build a CNF formula φi(x, y, z) which is satisfiable iff x encodes a word which has length between m and n, satisfies the hard constraint, belongs to label i, and has cost y. The solutions x for a particular choice of i and y comprise the associated cost class, so that the operations we need for the greedy construction are instances of the *model counting* and *uniform generation* problems for SAT.<sup>2</sup> Recent work has yielded practical algorithms based on SAT solvers which solve these problems approximately [7,27] 3:

<sup>2</sup> Since we do not want to count over the auxiliary variables z, we actually require *projected* counting/sampling, which the algorithms we use can also perform [7,17].

<sup>3</sup> We note that UniGen [6,7] is not strictly speaking an almost-uniform generator as in Definition 4 since it only supports sufficiently-large tolerances; for theoretical results, one can substitute the algorithm of [4] to do *exact* (projected) uniform sampling.

**Definition 4.** *([7]) An* approximate counter *is a probabilistic algorithm* C *which given a CNF formula* F *with set of solutions* R<sup>F</sup> *, a tolerance* τ > 0*, and a confidence* 1 − δ ∈ [0, 1) *guarantees that*

$$\Pr\left[|R\_F|/(1+\tau) \le \mathcal{C}(F,\tau,1-\delta) \le (1+\tau)|R\_F|\right] \ge 1-\delta.$$

*An* almost-uniform generator G *is a probabilistic algorithm that, given* F *as above and a tolerance* > 0*, guarantees that for every* y ∈ R<sup>F</sup> *, we have*

$$1/((1+\epsilon)|R\_F|) \le \Pr[\mathcal{G}(F,\epsilon) = y] \le (1+\epsilon)/|R\_F|.$$

We can modify our greedy construction to work with only approximate counting/sampling as follows. If the cost bitvector has |y| bits, the cost of a word is between 1 and 2|y<sup>|</sup> . To avoid enumerating exponentially-many cost classes for label i, we group words into "cost buckets" by subdividing this interval into powers of r for some r > 1, i.e. [1, r), [r, r<sup>2</sup>),..., [r<sup>b</sup>−<sup>1</sup>, r<sup>b</sup>). We will have b = O(logr(2|y<sup>|</sup> )) = O(|y|/ log r) buckets, and we can estimate the size of bucket <sup>j</sup> by approximately counting solutions to <sup>∃</sup>z.[φi(x, y, z) <sup>∧</sup> (r<sup>j</sup> <sup>≤</sup> y<r<sup>j</sup>+1)]. We will then use these estimates to choose a distribution over buckets, following the intuition of the greedy cost construction that we should assign the most probability to buckets with lowest estimated cost, but with some adjustments to bound the error that approximate sampling introduces.

For each label class i with randomness parameters α and β, we apply a modified form of the greedy cost construction, shown in Algorithm 1. We start in lines 1–3 by using model counting as above (with a tolerance τ and confidence 1 − δ to be specified later) to find estimates c<sup>k</sup> of the size of each bucket k, and corresponding lower bounds p<sup>k</sup> on how much probability the bucket would have received in the exact greedy construction (the extra 1 + τ factor accounting for possibly overestimating the size of the bucket). If these lower bounds total more than 1, then we know there are too many improvisations for the instance to be feasible (assuming the model counts are within their tolerance) and we return false on line 4. Otherwise, on lines 5–7 we proceed as in the greedy construction, starting from the cheapest bucket, increasing the assigned probability per word to (1 + τ )β until a probability of 1 is reached. The factor of 1 + τ ensures that, even if the model counts have underestimated the size of the cheaper buckets, we still assign them at least as much probability as the exact greedy construction would. Next, line 8 checks if there are too few improvisations, similarly to line 4. Finally, we return our distribution over buckets, as well as a lower bound on its expected cost that we will use next.

If Algorithm 1 does not return false for any label class, then we complete our approximate LQCI algorithm by running the greedy label construction from Sect. 3, using the lower bounds from Algorithm 1 as the expected cost of each label class. As before, we declare the instance infeasible if the construction fails or if its expected cost exceeds the cost bound c. Otherwise, we obtain a distribution over all the cost buckets; our improviser then simply chooses a bucket from this distribution and applies almost-uniform sampling to sample a word from it.

Choosing the bucket count and counting/sampling tolerances appropriately, our algorithm can approximate an improvising distribution to within arbitrarilysmall multiplicative error, using polynomially-many calls to a SAT solver:

#### **Algorithm 1.** ApproximateGreedyCost(i, α, β, r, b, τ, δ)

1: **for** k = 1 to b **do** 2: <sup>c</sup>*<sup>k</sup>* := #SAT(∃z.φ*i*(x, y, z) <sup>∧</sup> (r*<sup>k</sup>*−<sup>1</sup> <sup>≤</sup> y<r*<sup>k</sup>*), τ, <sup>1</sup> <sup>−</sup> <sup>δ</sup>) 3: p*<sup>k</sup>* := αc*k*/(1 + τ ) 4: **if** *b <sup>j</sup>*=1 p*<sup>j</sup>* > 1 **then return** False 5: **for** k = 1 to b **do** 6: p*<sup>k</sup>* := min((1 + τ )βc*k*, 1 − - *j*-<sup>=</sup>*<sup>k</sup>* p*<sup>j</sup>* ) 7: **if** *b <sup>j</sup>*=1 p*<sup>j</sup>* = 1 **then break** 8: **if** *b <sup>j</sup>*=1 p*<sup>j</sup>* < 1 **then return** False 9: Lo := *b <sup>j</sup>*=1 p*<sup>j</sup>* r*<sup>j</sup>*−<sup>1</sup> 10: **return** {p*j*}*<sup>b</sup> <sup>j</sup>*=1, Lo

**Theorem 5.** *There is an algorithm which, given a Boolean LQCI instance* C*, a cost tolerance* ζ > 0*, a randomness tolerance* γ > 0*, and a confidence* 1 − δ ∈ [0, 1)*, runs in poly*(|C|, 1/ζ, 1/γ, log(1/δ)) *time relative to an NP oracle and either returns* <sup>⊥</sup> *or an algorithm sampling from a distribution* <sup>D</sup> *over words. With probability at least* 1 − δ*, if* ⊥ *is returned then* C *is infeasible, and otherwise:*


## <sup>α</sup>ˆi/(1 + <sup>γ</sup>) <sup>≤</sup> Pr[<sup>y</sup> <sup>=</sup> <sup>w</sup> <sup>|</sup> <sup>w</sup> <sup>∈</sup> <sup>I</sup>i, w <sup>←</sup> <sup>D</sup>] <sup>≤</sup> (1 + <sup>γ</sup>)βˆ<sup>i</sup>

#### **6 Maximum-Entropy LQCI**

Our LQCI definition requires providing conditional probability bounds for every label, which while allowing maximal control of the distribution, can be unwieldy to use. However, if we drop conditional bounds entirely, trivial solutions with unnecessarily-poor randomness can appear. For example, consider an LQCI instance with parameters λ = 0.5, ρ = 0.5, ˆα = (0,..., 0), βˆ = (1,..., 1). With this choice, any distribution will satisfy the randomness over words constraint, and all labels have the same marginal probability of being selected. Then assume that we have two labels, costs Θ = (1, 2), and cost bound c = 1.5, along with the following cost class sizes: |I<sup>1</sup>,<sup>1</sup>| = 1, |I<sup>2</sup>,<sup>1</sup>| = 1, |I<sup>1</sup>,<sup>2</sup>| = 1000, |I<sup>2</sup>,<sup>2</sup>| = 1000. Now simply assigning 50% probability to I<sup>1</sup>,<sup>1</sup> and 50% probability to I<sup>2</sup>,<sup>1</sup> is an improvising distribution. Assigning 25% probability to all 4 classes is also an improvising distribution, and clearly preferable from the perspective of randomness. Unfortunately, without a nontrivial randomness over words constraint, we have no way to push the improviser to select the second distribution. To enforce this, we introduce the concept of entropy from information theory.

**Definition 5.** *Given a discrete random variable* X *with a set of outcomes* Ω *and probabilities* p : Ω → [0, 1]*, the* entropy *of* X *is* H(X) = − <sup>x</sup>∈<sup>Ω</sup> <sup>p</sup>(x) lg <sup>p</sup>(x)*.*

To obtain a problem formulation that maximizes randomness without requiring probability bounds for each class, we invoke the Principle of Maximum Entropy: amongst all improvising distributions (without a randomness over words constraint), we should select the one with the highest entropy (as first proposed for reactive CI in [30]). This yields a notion of Maximum-Entropy LQCI:

**Definition 6.** *A* Maximum-Entropy LQCI (MELQCI) instance *is an LQCI instance where* αˆ = (0,..., 0) *and* βˆ = (1,..., 1)*. A* τ -improviser *for a MELQCI instance* C *is an improviser (as in LQCI) whose output distribution has entropy at most* τ *less than the maximum-entropy improvising distribution for* C*. We define the* MELQCI problem *as, given an instance* C *and* τ > 0*, determining if* C *is feasible, and, if so, generating a* τ *-improviser for* C*.*

We can solve MELQCI efficiently in the same cases as LQCI:

**Theorem 6.** *Given a class of MELQCI instances for which one can perform the operations in Definition 3 in polynomial time, there is a polynomial-time algorithm which given an instance from the class and a* τ > 0*, computes a* τ  *improviser.*

*Proof.* (Sketch). Once cost class sizes have been computed as in Theorem 2, the search for the desired distribution over cost classes can be formulated as an optimization problem with a separable convex objective (the entropy of the distribution) and linear constraints (improviser constraints). This problem can be solved in time polynomial in the size of the instance and log(1/τ ) [10].

As in Sect. 4, we can transform this algorithm into a *pseudopolynomial* scheme for accumulated-cost DFA specifications.

#### **7 Experiments**

We ran several experiments on the robotic planning problems from Sect. 2 (code available at [20]). These experiments aim to demonstrate that we can encode practical problems as LQCI instances solvable using our algorithms, highlight the relative advantages/disadvantages of our exact/approximate algorithms, and show the necessity of the label function in ensuring meaningful randomness.

As a minimal experiment, we used a 6 × 6 grid world with a small range of costs (0–3 per cell, 8–39 for paths); we compared against a 7 × 7 grid world with a much larger range of costs (0–9 per cell, 38–137 for paths).<sup>4</sup> We encoded the specifications in Sect. 2 both as DFAs for our exact LQCI and MELQCI algorithms, and as Boolean formulas for our approximate LQCI algorithm. The Boolean encodings were obtained by formulating the specifications in the SMT theory of bitvectors, and bit-blasting them with Z3 [26]; the resulting formulas had several thousand variables and tens of thousands of clauses. We used UniGen3 [7,27] for uniform generation with its default tolerance<sup>5</sup> of 17, and an

<sup>4</sup> A larger 8 <sup>×</sup> 8 map exceeded our 24-hour wallclock timeout for all exact and approximate experiments.

<sup>5</sup> UniGen3 cannot guarantee a multiplicative error of less than 7.48 [6]; see footnote 3.


**Table 1.** Experiment parameters and improviser construction times (in minutes).

<sup>a</sup> The LQCI/MELQCI runtimes were nearly identical, since MELQCI reuses the LQCI computations and adds a convex optimization step, which took negligible time.

in-development version of ApproxMC [8,27,28] for approximate model counting with tolerances of 1.4, 6.7, and 23.25, so that the overall γ values were 10<sup>2</sup>, 10<sup>3</sup>, and 10<sup>4</sup>. To put these values into context, the small/large maps had on the order of 10<sup>7</sup>/10<sup>9</sup> improvisations, and we required that no word have > ρ = 10−<sup>5</sup> probability of being selected. Therefore, with our tightest/loosest γ we are guaranteed that no word will be more than 0.1%/10% of the distribution respectively. The confidence was set to 0.8 (δ = 0.2), ApproxMC's default confidence. Each model counting call however required a much higher confidence to achieve an overall δ of 0.2.

For the small/large maps respectively we used length bounds of (1,25)/(1,30) and cost bounds of 30/50. We used label probability bounds of (0.3, 0.4) throughout, except for unlabeled "QCI" experiments. The experiments were run on a 64-core machine with 188 GB of RAM; we used 62 parallel threads, unless this exhausted RAM, in which case we used 16 threads. The experiments are summarized in Table 1; due to significant runtime variability for the approximate experiments, we report means and standard deviations over 10 repetitions. For all exact experiments which completed within the 24-h wallclock timeout, RAM usage was ≤ 6 GB per thread, and the average time to sample an improvisation was ≤ 1 ms; all approximate experiments required ≤ 250 MB RAM per thread and took ∼ 20 s to sample an improvisation.

**Fig. 3.** Randomly-selected traces generated by the QCI/MELQCI improvisers for the 6 × 6 map. Note that all the QCI traces use the same charging station.

We can draw several conclusions from these results. Improviser construction with the exact algorithm is significantly more expensive than with the approximate algorithm, in both CPU time and RAM. This is not surprising, as the exact encodings resulted in enormous DFAs which, for the large map, approached 10<sup>10</sup> states. Conversely, sampling is much faster for the exact algorithm, with no SAT queries required. We can also see that the approximate algorithm can be used to practically solve problems that are infeasible to solve exactly, such as the large-map problem. We expect new developments in the relatively young field of approximate model counting/sampling will further speed up our algorithm.

Visualizing several randomly-chosen traces from our exact QCI and MELQCI experiments in Fig. 3, we can see the importance of labels. In unlabeled QCI, the robot always charged at the substation near the main road due to the lower expected cost of such paths. In contrast, MELQCI yielded a near-uniform distribution over the charging stations. This increase in diversity was not free, with the average cost rising to 21.4 for MELQCI from 8.7 for QCI. This tradeoff demonstrates how LQCI allows us to balance the need for control over our improvisations with the need for meaningful diversity (not merely randomness) by choosing appropriate label functions.

#### **8 Conclusion**

In this paper, we introduced *labelled quantitative control improvisation* as a framework allowing correct-by-construction synthesis of randomized systems whose behavior must be diverse with respect to a label function and near-optimal with respect to a cost function. We studied the theory of LQCI problems and developed algorithms for solving them for broad classes of specifications encoded as finite automata or Boolean formulas. Our experiments demonstrated how our framework can be used to formalize and solve realistic robotic planning problems.

There are a number of clear directions for future work. Scalability is an evident concern: our experiments show that our algorithms can require substantial resources to solve even relatively small LQCI problems. While LQCI with Boolean formulas is a difficult #P-hard problem, our algorithms will directly benefit from future progress in model counting; our DFA algorithms could also be improved through the use of abstraction to reduce state-space explosion. We also plan to explore generalizations of our algorithms, such as extending our approximate scheme to MELQCI and to problems with exponentially-many labels, as well as potentially infinite traces. Finally, we are investigating extensions of the LQCI problem to *reactive* settings with adversarial environments, and to *black-box* settings for design-space exploration and other problems where we do not have complete models for the cost function and other constraints.

**Acknowledgements.** The authors thank Skyler Stewart for designing Fig. 2, and several anonymous reviewers for their helpful comments. This work was supported in part by DARPA contract FA8750-20-C-0156 (SDCPS).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

## **Author Index**

Agarwal, Chaitanya II-3 Albert, Elvira I-430 Alt, Leonardo I-325 Alur, Rajeev II-343 Amir, Guy I-219 Appel, Andrew W. II-272 Arvind II-317 Baccelli, Emmanuel II-293 Badings, Thom S. II-26 Bae, Kyungmin I-524 Bak, Stanley I-490 Bang, Seongwon II-386 Banno, Ryotaro I-447 Bansal, Suguman II-343 Bao, Jialu I-33 Barbosa, Haniel II-205 Barrett, Clark II-92, II-205 Bastani, Osbert II-343 Becchi, Anna I-469 Bellés-Muñoz, Marta I-430 Besson, Frédéric II-293 Beutner, Raven I-341 Bian, Song I-447 Blicha, Martin I-325 Blondin, Michael II-468 Bogomolov, Sergiy I-490 Bouajjani, Ahmed I-282 Boutglay, Wael-Amine I-282 Cai, Shaowei II-227 Casadio, Marco I-219 Castro, Pablo F. II-48 Chatterjee, Krishnendu I-55 Chatterjee, Prantik I-304 Cheang, Kevin I-538 Chen, Haibo I-257 Chen, Mingshuai I-79 Chen, Taolue I-385 Chlipala, Adam II-317 Choi, Joonwon II-317 Chun, Inwhan II-386 Cimatti, Alessandro I-469 Coenen, Norine I-407

Cohen, Joshua M. II-272 Colange, Maximilien II-174 D'Argenio, Pedro R. II-48 Dachselt, Raimund I-407 Daggitt, Matthew L. I-219 Dardinier, Thibault II-130 de Oliveira Oliveira, Mateus II-447 Demasi, Ramiro II-48 Dimitrov, Dimitar Iliev I-127 Doveri, Kyveli II-109 Dubois, Jérôme II-174 Duret-Lutz, Alexandre II-174 Faella, Marco II-249 Fan, Yuxin I-385 Fang, Bin I-257 Fang, Wang II-408 Feng, Weizhi II-152 Finkbeiner, Bernd I-341, I-407, II-505 Fischer, Marc I-127 Fremont, Daniel J. II-526 Frenkel, Hadar I-407 Fu, Hongfei I-257 Gaddamadugu, Pranav I-538 Ganty, Pierre II-109 Gbaguidi Aisse, Alexandre II-174 Giannakopoulou, Dimitra II-490 Gillard, Clément II-174 Gittis, Andreas II-526 Godbole, Adwait I-538 Goharshady, Amir Kafshdar I-55 Golia, Priyanka I-363 Goubault, Eric I-511 Gros, Timo P. II-430 Guan, Ji II-408 Guha, Shibashis II-3 Gurfinkel, Arie I-19

Habermehl, Peter I-282 Hahn, Christopher I-407 Hasuo, Ichiro I-235 Havlena, Vojtˇech II-188

Hencey, Brandon I-490 Hermanns, Holger II-430 Hoffmann, Jörg II-430 Horak, Tom I-407 Hsu, Justin I-33 Hym, Samuel II-293 Hyvärinen, Antti E. J. I-325 Isabel, Miguel I-430 Jansen, Nils II-26 Jhoo, Ho Young II-386 Ji, Yucheng I-257 Jin, Peng I-193 Jothimurugan, Kishor II-343 Juba, Brendan I-363 Junges, Sebastian I-102, II-26 Katis, Andreas II-490 Katoen, Joost-Pieter I-79 Katsumata, Shin-ya I-235 Katz, Guy I-219 Klauck, Michaela II-430 Klinkenberg, Lutz I-79 Kochdumper, Niklas I-490 Köhl, Maximilian A. II-430 Kokke, Wen I-219 Komendantskaya, Ekaterina I-219 Kori, Mayuko I-235 Kˇretínský, Jan II-3 Laeufer, Kevin I-538 Lal, Akash I-304 Lauko, Henrich II-174 Lee, Jia I-524 Lee, Juneyoung II-386 Lengál, Ondˇrej II-188 Leutgeb, Lorenz II-70 Lew, Ethan I-490 Li, Bohan II-227 Li, Yannan II-364 Li, Yong II-152 Lin, Shaokai I-538 Liu, Wanwei I-385 Manerkar, Yatin A. I-538 Martin, Antoine II-174 Matsumoto, Naoki I-447 Matsuoka, Kotaro I-447 Mavridou, Anastasia II-490

Mazowiecki, Filip II-468 Mazzocchi, Nicolas II-109 Meda, Jaydeepsinh I-304 Medioni, Thomas II-174 Meel, Kuldeep S. I-363 Meggendorfer, Tobias I-55 Metzger, Niklas I-407, II-505 Mora, Federico I-538 Moser, Georg II-70 Moses, Yoram II-505 Müller, Peter II-130 Muruganandham, Pazhamalai II-3

Nam, Seunghyeon II-386 Niemetz, Aina II-92 Nötzli, Andres II-205

#### Offtermatt, Philip II-468

Parlato, Gennaro II-249 Parthasarathy, Gaurav II-130 Pathak, Drashti I-33 Paulsen, Brandon I-149 Pham, Long H. I-171 Polgreen, Elizabeth I-538 Potomkin, Kostiantyn I-490 Preiner, Mathias II-92 Pressburger, Thomas II-490 Putot, Sylvie I-511 Putruele, Luciano II-48

Refaeli, Idan I-219 Renault, Etienne II-174 Renkin, Florian II-174 Reynolds, Andrew II-205 Rodríguez-Núñez, Clara I-430 Roy, Subhajit I-33, I-304 Rubio, Albert I-430 Rungta, Neha I-3

Schlehuber-Caissier, Philipp II-174 Schumann, Johann II-490 Seshia, Sanjit A. I-538 Sharygina, Natasha I-325 Siber, Julian I-407 Singh, Gagandeep I-127 Šmahlíková, Barbora II-188 Song, Fu I-385 Spaan, Matthijs T. J. I-102 Sprecher, Christian I-127

Stoelinga, Marielle II-26 Suenaga, Kohei I-235, I-447 Summers, Alexander J. II-130 Sun, Jun I-171

Talpin, Jean-Pierre II-293 Tian, Jiaxu I-193 Tinelli, Cesare II-205 Trivedi, Nitesh I-33 Turrini, Andrea II-152

Urabe, Natsuki I-235

Vardi, Moshe Y. II-152 Vechev, Martin I-127 Vin, Eric II-526 Volk, Matthias II-26

Waga, Masaki I-447 Wang, Chao I-149, II-364 Wang, Jingbo II-364 Wang, Qinshi II-272 Weeks, Noé II-130 Wen, Xuejun I-193 Winkler, Tobias I-79 Wolf, Verena II-430

Ying, Mingsheng II-408 Yu, Geunyeol I-524 Yuan, Shenghao II-293

Zandberg, Koen II-293 Zhang, Liangfeng I-385 Zhang, Lijun II-152 Zhang, Min I-193 Zhang, Xindi II-227 Zhi, Dapeng I-193 Žikeli´c, Ðor de I-55 Zuleger, Florian II-70