LNCS 13125

Marinos Ioannides Petros Patias (Eds.)

# **3D Research Challenges in Cultural Heritage III**

### **Complexity and Quality in Digitisation**

### **Lecture Notes in Computer Science 13125**

Founding Editors

Gerhard Goos Juris Hartmanis

### Editorial Board Members

Elisa Bertino, *Purdue University, West Lafayette, IN, USA* Wen Gao, *Peking University, Beijing, China* Bernhard Steffen , *TU Dortmund University, Dortmund, Germany* Moti Yung , *Columbia University, New York, NY, USA*

The series Lecture Notes in Computer Science (LNCS), including its subseries Lecture Notes in Artificial Intelligence (LNAI) and Lecture Notes in Bioinformatics (LNBI), has established itself as a medium for the publication of new developments in computer science and information technology research, teaching, and education.

LNCS enjoys close cooperation with the computer science R & D community, the series counts many renowned academics among its volume editors and paper authors, and collaborates with prestigious societies. Its mission is to serve this international community by providing an invaluable service, mainly focused on the publication of conference and workshop proceedings and postproceedings. LNCS commenced publication in 1973.

Marinos Ioannides · Petros Patias Editors

## 3D Research Challenges in Cultural Heritage III

Complexity and Quality in Digitisation

*Editors* Marinos Ioannides UNESCO Chair on Digital Cultural Heritage Cyprus University of Technology Limassol, Cyprus

Petros Patias Aristotle University of Thessaloniki Thessaloniki, Greece

ISSN 0302-9743 ISSN 1611-3349 (electronic) Lecture Notes in Computer Science ISBN 978-3-031-35592-9 ISBN 978-3-031-35593-6 (eBook) https://doi.org/10.1007/978-3-031-35593-6

© The Editor(s) (if applicable) and The Author(s) 2023. This book is an open access publication.

**Open Access** This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

UNESCO Chair on Digital Cultural Heritage at the Cyprus University of Technology, The Department of Antiquities, Cyprus and The Holy Bishopric of Morfou, Cyprus.

This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

### **Preface**

Tangible cultural heritage digitised in 3D can be a source of new knowledge, including with respect to climate-related impact and adaptation. It has also great re-use potential in many sectors, including the creative and cultural sectors, but also education and tourism. The innovative re-use of digitised cultural heritage can be a significant contribution to a European sense of belonging and to European integration. It can also generate jobs and growth, in the creative and cultural sectors, but also for tourism in areas that are not among the most known or visited sites. By redistributing tourism flows more broadly, such processes will also contribute to making the most visited tourist sites more sustainable, by relieving some of the pressure.

The latest developments in the domain of 2D/3D technologies have led to an expansion and rapid progress with far-reaching impact in numerous applications spanning very diverse areas both for highly-skilled professionals – such as architecture and engineering for digital factories of the future and advanced simulators for flight and surgical training – as well as for the general public now using Virtual Realities, Metaverse and enjoying cloud computing for 3D computer games. In general, it has turned out that the handling of 3D data poses different challenges but also provides growth for new, exciting and innovative opportunities compared to more established multimedia. Some 3D technologies are already being used successfully also in the cultural heritage (CH) domain, especially in the area of data acquisition, modelling, archiving in local repositories and harvesting in digital libraries like Europeana (www.europeana.eu) and very soon at the UNESCO Dive into Heritage platform.

The European Commission launched a unique study (VIGIE2020/654) on 3D digitisation of tangible cultural heritage to enable cultural heritage professionals, institutions, content-developers and academics to define and produce high-quality digitisation standards for tangible heritage (https://digital-strategy.ec.europa.eu/en/library/ study-quality-3d-digitisation-tangible-cultural-heritage).

The UNESCO Chair on Digital Cultural Heritage at the Cyprus University of Technology coordinated the consortium of nine key actors from across Europe conducting this unique study, which aimed to identify all the relevant elements for the 3D data acquisition of tangible cultural heritage, classifying them by degree of complexity and purpose or use. It had also covered the specific types of equipment used throughout the different stages of the 3D digitisation process, and all the types of relevant data, including geometry, colour, texture and materials. In addition, the study, catalogued the technical parameters that determine the level of quality of 3D digitisation; existing digital formats, standards, benchmarks, methodologies and guidelines for 3D digitisation; and past or ongoing 3D digitisation projects and existing 3D models and data sets that can serve as benchmarks.

3D digitisation has significant potential and long-term value in the area of cultural heritage. By signing the 2019 Declaration of Cooperation on advancing the digitisation of cultural heritage in Europe, 28 Member States and Norway have acknowledged the importance of 3D digitisation technologies for cultural heritage and the urgent need to make full use of them. The declaration also endorses a call for common standards, methodologies and guidelines for comprehensive, holistic 3D documentation of European 3D cultural heritage assets.

During the implementation of this study, the idea for this volume originated, with the intent of gathering 3D research challenges for the digital cultural heritage domain. Contributions from renowned researchers in this specific area, study partners and supporters were selected.

The aim of this book is to provide an insight into ongoing research and future directions in this novel, continuously evolving field, which lies at the intersection of different multidisciplinary areas such as digital heritage, engineering, computer science, material science, architecture, civil engineering, and archaeology. Overall, in our opinion, the chapters in the book reflect the following 3D challenges in the CH domain:


The acquisition of virtual 3D computer models, typically using laser scanning and/or photogrammetry technology but also using computer-aided design, raises various challenges related to the huge number of objects that should be dealt with, as well as their complexity such as size, material, accessibility, etc., as well as the quality of the final data sets:


of animated 3D scenes and 3D authoring tools for interactive experiences, usable by specialists for the CH content who should not have to be very familiar with IT issues.

• 3D technologies should also help CH professionals in their daily research work, for example, assisting – literally – in the search for missing pieces in archaeological settings.

New 3D printing technologies allow on the one hand, the detailed reconstruction of 3D CH objects in previously unknown fidelity in shape and material and on the other hand they are finally giving people viable options to turn available virtual CH models into tangible real entities. This can support real CH object exhibitions of previously unseen objects and assist in the repatriation of some of these replicas back to their origin. Naturally some of the above-mentioned challenges are very well-known from other 3D application areas but may have a different twist in the CH domain. The different issues are obviously interwoven which becomes apparent when studying the chapters of this book.

The contributions in this volume show that substantial activities have been carried out in CH-related projects funded by the European Commission. The EU Project Eureka3D (https://eureka3d.eu/) is intended to deliver to Europeana more than 4,000 CH objects based on the results of this unique study. Moreover, the EU projects ERA CHAIR MNEMOSYNE (https://erachair-dch.eu/), TRIQUETRA (https://triquetra-project.eu/), ENIGMA (https://eu-enigma.eu/) and ANCHISE (https://www.anchise.eu/) are using the results of the study in combination with breakthrough technologies for 3D reverse engineering, reconstruction and recovery of memory in cultural heritage.

> Marinos Ioannides Petros Patias

### **Contents**


### **The Complexity and Quality in 3D Digitisation of the Past: Challenges and Risks**

Marinos Ioannides1(B) and Petros Patias2(B)

<sup>1</sup> Cyprus University of Technology, Limassol, Cyprus marinos.ioannides@cut.ac.cy <sup>2</sup> Aristotle University of Thessaloniki, Thessaloniki, Greece patias@auth.gr

**Abstract.** This paper is focusing on the exceptional results of the EU Study (VIGIE2020/654) to map the parameters, formats, standards, benchmarks, and methodologies relating to 3D digitisation of tangible cultural heritage (CH). The overall objective of the paper is to further the quality of 3D digitisation process by enabling cultural heritage professionals, institutions, content-developers, stakeholders, and academics to define and produce high-quality digitisation standards for tangible cultural heritage assets. This study identified for the first time in this domain, key parameters of the digitisation process, estimated the relative complexity and how it is linked to technology, its impact on quality and its various factors. It will also present standards and formats used for 3D digitisation, including data types, data formats and metadata schemas for 3D structures.

**Keywords:** Complexity · Quality · 3D Digitisation · Cultural Heritage

### **1 Introduction**

This paper summarises the results of the Study on Quality in 3D Digitisation of Tangible Cultural Heritage (VIGIE 2020/654), represented in full in the extensive Final Report. The work was based on the combined efforts of the in-house study team at Cyprus University of Technology (CUT) and a group of nine (9) sub-contracted collaborators, together with individual external experts whose research inputs are included in the results.

The study was organised according to a structure of five (5) main process-oriented tasks together with separate project management and dissemination tasks. A mid-term workshop was organised to provide expert validation of the interim findings. The Final Report combines the conclusions from the nineteen (19) planned Outputs (OU) of the study.

### **2 The Process of Digitising Movable and Immovable Tangible Cultural Heritage**

According to UNESCO1, the term Tangible Cultural Heritage (CH) can be classified in the following two categories:


Both categories contain objects often heterogeneous (made of different materials by using diverse techniques) with an inherently complex geometry and surface texture. For example, structures such as monuments with sculptural or pictorial decoration, or one of the oldest types of archaeological artefact, the jewelery, have a complex form.

The digital recording of CH is an essential step in understanding and conserving the values of the memory of the past, creating an exact digital record for the future, providing a means to educate, skill, and communicating the knowledge and value of the tangible objects to the society. Therefore, the primary goal of recording is to know and understand the values and significance of the CH object - historical, scientific, aesthetic, social, and economic. In addittion, the digital representation of CH objects, structures, and environments is essential for practical analysis, conservation, and interpretation. Selecting the ideal technology and workflow for the 3D digitisation of tangible CH objects is a complicated, very challenging procedure and one that requires careful consideration.

However, there is no internationally accepted framework or methodology for specifying the quality of detail and accuracy in CH digitisation. Documentation projects are typically determined on a case-by-case basis using the many available methods and often require significant multi- and interdisciplinary cooperation. An object needs to be carefully examined and inspected in order to define the best available digitisation options.

Therefore, the recording of tangible CH requires a thorough understanding of the stakeholder requirements, the necessary technical specifications, the existing environmental conditions, and the intended use of the final 3D model. Selection of the optimal human resources and digitisation technology are usually related to the technical specifications, size, complexity, material, texture, location, accessibility, IPR and accuracy required. For large surface areas, such as monument sites or architectural mapping, a

<sup>1</sup> UNESCO: https://bit.ly/3oEEdRB.

combination of regular aerial and topographic surveys, laser scanning and photogrammetric techniques is often used. In addition to the cost of hardware and associated software, a considerable investment in knowledgeable staff and time dedicated to specialised training has to be taken into account.

#### **Accuracy and Precision**

Accuracy refers to how close a measurement is to the true or correct value, whereas precision is how close the repeated measurements are to each other. Measurements can be both accurate and precise, accurate but not precise, precise but not accurate, or neither. A reliable survey instrument is consistent; a valid one is accurate.

### **Planning the Process of Digitisation**

The 3D digitisation of tangible CH is an inherently complex multi-stage process. Project planning should accurately address and coherently develop a documentation dataset, while keeping in mind project constraints including, but not limited to, environmental and safety conditions, available equipment, budget, and timescale.

#### **Documentation Methods**

Measurement methods for geometric recording range from conventional simple topometric methods for partially or uncontrolled surveys to elaborated contemporary surveying and photogrammetric methods for completely controlled methods. However, there is no generally accepted standard for specifying the detail and accuracy requirements for the different geometric recordings of tangible objects. Most answers to such challenges are primarily based –at the moment- on cost specifications and time limitations defined by stakeholders.

#### **Active and Passive Recording**

For the 3D geometric documentation of movable and immovable assets, the range of object sizes could start from few "mm" and goes up to a couple of thousand metres, while the number of acquired points should practically have no limit. These documentation methods may be grouped in several ways. Firstly, according to those involving light recording and those which do not. Secondly, a form of radiating energy is always used for gathering geometrical and visual information, therefore a first distinction can be done between *penetrating* and *non-penetrating* radiation systems.

In the penetrating category systems based on X-Rays devices allow the capture of inaccessible internal structures and surfaces of small objects. Similar X-Rays devices are used in areas such as medicine, mechanical engineering, on the airport security and on detail investigations by the police. On a larger dimension the use of cosmic rays are being experimented for attempting the 3D scanning of the interiors of the Maya monuments, or the Egyptian pyramids. For the non-penetrating 3D, the electromagnetic energy covers the visible and the InfraRed spectrum. The latter actually may allow a little penetration under the illuminated surface depending on the actual wavelength used, ranging from fractions of a millimetre for Near InfraRed (NIR), to several millimetres for the Far InfraRed (FIR), used in the so-called TeraHertz imaging. However, for 3D applications possible little penetrations inside the material are usually neglected, and this is the reason why light sources for 3D never go beyond NIR. Within non-penetrating devices a further distinction has to be done between *active* and *passive* systems. The main distinction between documentation methods is whether they are active or passive. Active recording methods use directed radiant energy to mark a point in space, whereas passive methods record the reflected radiation from a surface.

### **Indoor and Uncontrolled Acquisition**

Indoor image acquisition is more often used for objects or artefacts in museums or collections (which are not allowed to be moved), typically of small or medium size, and presents several challenges. The size, weight, special illumination conditions, materials, properties of the artefacts, and their interior structure directly influence the documentation complexity.

Uncontrolled acquisition is typically used for outdoor scenes or any other environment where the conditions (shadows, weather, etc.) are not under complete control. Large scale objects such as buildings, excavations, or archaeological sites still with high accuracy demands (mm-cm) are classified in this category.

The report describes other factors and approaches concerning outdoor acquisition in greater depth, which require consideration when planning a documentation project. These include video frames extraction, object/site recording, and limitations on accessibility due to skills requirements, weather, ambient conditions, operating hours and lighting conditions, site modification and distance optics.

Moreover, it is essential to consider the state of condition and remedy options. This includes geometrical symmetry, together with factors affecting surface reflectivity, such as resolution, distance from the sensor to the object/image scale, angle of incidence, safety regulations, focal point/spot size, field of view and precision.

### **3 Defining Complexity**

The term complexity can be described as "*the state or quality of being intricate or complicated*" or "*the state of having many parts and being difficult to understand*". Consequently, complexity characterises a system's behaviour or an object whose components or elements interact in multiple ways and sometimes follow local rules, meaning there is no reasonable higher instruction to define the various possible interactions. The report explores in greater depth the meanings of complexity and their application in CH.

Defining complexity is essential because the level of complexity:


### **Determining Complexity**

Complexity is inherent in all different tangible CH objects and it is a critical consideration when planning a geometric documentation project. It refers to the geometric, surface/texture, material composition, and scale/application variants. In addittion, a key dimension of complexity resides in the stakeholder requirements also, which may include the location, state of condition, the set-up of the data acquisition project, the experience of the multidisciplinary operators on site, and the fusion of multiple datasets from different devices and their users/specialists (equipment and data pre-processing) into one archive that can be visualised in an easily accessible and searchable way.

The main challenges in defining a workable end-user definition of complexity relate to establishing equipment needs and acquisition methodologies – and in client comprehension of additional costs in complex processes and post-processing.

An online survey aiming at establishing perceptions of the 3D digitisation community (944 responses received from 420 survey respondents) identified the following parameters as the top three factors for increasing complexity: a) Surface conditions; b) Site access; and c) Quality of scanned data.

Moreover, interviews during our work with 49 key stakeholders and skilled professionals in 3D digitisation showed that among the factors most frequently mentioned were the importance of the stakeholders' requirements for 3D digital documentation frameworks, object conditions before/during the recording process, location and environmental conditions during digitisation, and the levels of expertise of the people involved.

The challenge is to manage all related activities that run simultaneously during the data acquisition phase to produce high-quality results without losing information. Optimal digitisation technologies are usually related to the desired technical specifications, size, state of condition, material, texture, location, accessibility, and required accuracy. These considerations are incorporated into the operational schema developed during the study. An object's material or materials adds a further dimension of information and complexity. Representing these traits in 3D can pose significant challenges (Fig. 1).

**Fig. 1.** Information example indicating the degree of complexity – detailed breakdown (Complexity Radar Pie).

#### **How is Complexity Connected to Technology?**

It is a fact that some data capture technologies and recording methods are more suitable

for specific applications than others (i.e., for special object investigations in the lab such as computer tomography). Selection of the data acquisition technology such as hardware, and software are usually related to the desired stakeholder requirements, technical specifications, size, complexity, material, texture, accessibility, and required accuracy. Therefore, the report describes mainstream technologies used for the 3D documentation of CH tangible assets, in terms of the degree of complexity.

### **Impact of Complexity on Quality**

A comprehensive understanding of object complexity is crucial as it has a high impact on various aspects of 3D digitisation. The use of the term in this field has remained vague with no clear definition, subjective methodology of calculating or apparent connection to quality, purpose-of-use, or other imposed restrictions. In other words, there was a gap in the collective understanding of the data acquisition project and object complexity as a decision support tool.

However, the complexity of 3D digitisation in CH as a value of its own cannot be a matter of subjective estimation. Still, it can be defined after the stakeholder requirements are determined, the project specifications are set, the object's location and environmental conditions are known, and the object is defined. Any definition of object complexity should have the following characteristics:


The process requires a focus on the complexity of the object and the set-up of the 3D digitisation project. Well above the object complexity, the digitisation process emphasises the complexity of the process itself.

The study's online survey analysed the perception of complexity that experts have concerning the use of technologies. In their opinion, complexity is related to the degree and the kind of information they want to obtain, issues with software and budget, the challenges that the surface of a specific object presents, and the location of a monument.

Any definition of the complexity in 3D digitisation of CH assets should consider the following parameters:


• The knowledge for hardware and software to be used and are available for the preprocessing of the scanned 2D (images) and 3D data (3D point clouds).

#### **Parameters of Complexity**

The report illustrates the study's research into the complexity and the potential issues that would affect the complexity of a digitisation process, including stakeholder's requirements, object, project, team, environment, hardware/software, and pre-processing. Each item is subcategorised into five levels (Fig. 1).

### **4 Exemplification of Complexity**

During the study, the set of parameters determining levels of complexity and related more broadly to quality were considered in the context of 43 cases (25 Immovable and 18 Movable). Annex 1 of the final report provides outline descriptions of each selected case.

This work has supported the development of a catalogue of data acquisition technologies and their output formats. Other contextual taxonomies have been developed for Movable and Immovable heritage, based on UNESCO's World Heritage conventions and recommendations.

When building a taxonomy of complexity for Movable Heritage, we must consider each unique element (object-specific) of the question to refine and define the material. An essential requirement for the holistic digitisation of an asset refers to collecting data regarding these factors and accurately representing them. Key reference examples presented in the Final Report include:

#### **Movable heritage:**


#### **Immovable heritage:**


### **5 Parameters that Determine Quality**

As the terms 'complexity' and 'quality' are used without a precise definition, this presents a significant challenge since tangible CH is exceptionally diverse.

Quality may comprise of different parameters such as the degree of detail, the geometric accuracy of the 2D and 3D shape, the spectral, scale and texture, material properties and chemical composition, and structural health monitoring status. These parameters can be combined in the following categories: a) Geometry; b) Image; c) Material; and d) Structural Health Monitoring.

Quality parameters refer to different stages of the 3D digitisation process and vary depending on the type of tangible CH and the equipment and methodology used. The possible purposes or uses of the resulting 3D material also determine different combinations and levels of those parameters to identify the minimum level of quality that fits the definition.

From the study's survey responses, the top three parameters of quality categorised as the most important by respondents to ensure quality in the digitisation process were: a) surface conditions, b) quality of images, and c) environmental conditions.

Quality is a fundamental component of the 3D digitisation in CH, and it is an essential challenge since tangible CH hand- or natural-made structures are remarkably different.

The possible uses for the resulting 3D material also determine different combinations and levels of those parameters to achieve the minimum level of quality that fits the definition. It is also essential to distinguish the differences between data accuracy (as an acceptable margin of error), precision and resolution regarding the geometry. Accuracy refers to the closeness of a measured value to a standard or known value. Dimensional precision is a measurement of the repeatability, or consistency, of that measurement (Fig. 2). Quality parameters may also comprise the degree of detail, the geometric accuracy of the 3D shape, or the fidelity of the capture of colour/texture. The report puts all relevant parameters into three main categories: Geometry, Radiometry, Completeness and connects these parameters to Complexity, as discussed above.

**Fig. 2.** Information example indicating the degree of quality – detailed breakdown (Quality Radar Pie Chart).

### **6 Standards and Benchmarks**

One conclusion of the study's 49 interviews with key professionals in this domain is that there are no standards for planning, organising, setting up and implementing a 3D data acquisition project. Some experts mentioned the need to distinguish between the standards available for the management, administration of projects, safety, health and accessing the object/site for the personnel, the movement of the objects and the standards available for the data.

The report analyses most usually employed formats, along with a full discussion of the two focus area formats, terrestrial laser scanning/3D modelling and photogrammetry/digital photography. It also elaborates on the distinctions to be made between proprietary and open-format data limitations (minimum or maximum), and on judging data correctness in the absence of international protocols for data quality assurance. An important observation is that formats evolve as users and developers identify and incorporate new functionalities.

### **7 Identification of Gaps, Additional Formats, Standards, Benchmarks, Methodologies, and Guidelines**

There are no guidelines on ways and minimum amounts of data to be collected or the quality to be achieved during data acquisition, which entirely depends on the stakeholder requirements. There appears to be little common understanding among the international multidisciplinary teams regarding what 2D/3D digital data acquisition standards means, as well as the obsolescence when new software does not provide backwards compatibility with older file formats. Also, open-source software communities may withdraw support for older formats, if these are no longer generally needed by the community. Obsolescence can also be accidental: both businesses and open-source communities can be led into erroneous practices for different reasons. Digitisation can generate a considerable amount of original and post-production data. When defining a project, it is crucial to understand the stakeholder requirements about the various production file formats to avoid inconsistent deliverables and inoperable proprietary data sets. There are hundreds of different file formats, noting that terrestrial laser scanners, for example, produce raw data in a variety of formats. Proprietary formats, such as TIFF or JPG, are seen as robust; however, these formats will ultimately be susceptible to upgrade issues and obsolescence. Furthermore, open-source formats can be seen as being neutral, non-reliant on business models for their development; however, they can also be seen as vulnerable to the susceptibilities of the communities that support them.

At the novice level, or for those with limited expertise, it cannot be ignored that different and more basic forms of guidance may be required to promote skills that enable widespread 3D digitisation in Europe. Some of the key questions have been beyond that and around the conceptual framework needed to address the use cases for digital dioramas, including by adding depth to the current 2D images, and by embedding one or more canvases within a 3D scene (e.g., multiple paintings or texts, or music/liturgy associated with a cathedral, temple, amphitheatre, or interior of a suitable model). However, with growing user and institutional demands, technical developments, and examples of advanced research collection and integration of virtual resources (e.g., Sketchfab, Smithsonian3D, 3DHOP, Potree, ScanTheWorld, Clara.io, morphosource.org, exhibit.so, hubs.mozilla.com, sayduck.com, Europeana.eu, etc.), there is a pressing and urgent need for a technical specification to ensure interoperability and longer term sustainability. Therefore, the plan for different multi- and interdisciplinary expert groups such as the IIIF 3D Technical Specification Group, CEN and ISO Technical Committees are to continue a collaborative approach to clarifying and specifying interoperable frameworks for 3D data, including common ways to:


### **8 Uncertainty**

Recognising the challenges and lack of consensus on the expression of uncertainty in measurement, different organisations worldwide have collaborated with the world's highest authority in metrology, the Comité International des Poids et Mesures (CIPM), to develop a more workable definition. Uncertainty concerning the complexity and quality in data acquisition is discussed in further detail in the context of the expression of quality 3D digitisation in the report.

### **9 Forecast Impact of Future Technological Advances**

Expected advancements in 2D/3D data acquisition software combined with artificial intelligence algorithms in different devices will make 3D digitisation easier, faster, more accurate, and more informative. The automatic compilation of different data types from various devices and manufacturers, the extraction and recognition of geometrical features, materials and environmental issues will create new challenges and impose greater demands. Development in this area will likely require new competences, specialised expertise and training. New standards, regulations and international accepted methodologies for data acquisition will be required.

Moreover, automatic compression and data transfer through 5G, 6G and strong Internet connections with many gigabytes of bandwidth from the field to the cloud will soon be in place to enhance archiving, real time global use and long-term availability and preservation. Guidelines for the CH domain will be needed on future formats for data, metadata and paradata, ensuring interoperability and data longevity. For analytics, blockchain, cloud and mobile computing, ontologies, Internet of Things (IoT), aerial and terrestrial LiDAR, and machine learning are just a few technologies that have transformed the construction industry and will undoubtedly impact the CH sector very soon. The increased interest in A/V/MX Reality, UAV s, Artificial Intelligence/Machine Learning, cloud and mobile computing will enable these new systems to play an indispensable role in the management, documentation, modelling, conservation, interpretation and protection of CH. Consequently, the development of these systems will have a direct impact on the CH industry (i.e., Virtual Museum, Virtual Sites, Smart Cities, 3D- digital libraries, fabrication and eArchiving). The report explores the potential of these technologies in more detail alongside that of open data, Heritage Building Information Modelling (HBIM) and the digital twin.

### **10 3D Digitisation Process Complexity**

The complexity cannot be estimated subjectively; it can be defined only after all measurements of the object are conducted, which means that object complexity is not useful for 3D digitisation planning and decision-making. Likewise, its neutrality to the intended use re renders it impractical for choosing the best technology or setting up the technical specifications for 3D digitisation. Regardless of the definition applied, an indication of complexity can only occur after the 3D digitisation of a CH object has been completed. This includes obtaining all surface detail, texture characteristics and accuracy metrics and making a subjective guess at the object complexity. In practice, this would be a fruitless exercise. Therefore, it makes sense to reverse this thinking and start from the technical specifications which are dictated by the purpose of the 3D digitisation activity in question. In accordance with this argumentation, there is a need to shift attention from "Object Complexity" to "Model Complexity". This means that the focus is not on the complexity of the actual object (which is connected only to the data capture phase) but on the complexity of the produced model, which is connected to the entire process of data acquisition and processing. This may look like a conceptual compromise, but the alternatives are worse. In effect, one would have to chose between ignoring this factor or making subjective guesses (Fig. 3).

**Fig. 3.** Moving from Object Complexity to Process Complexity.

We therefore define process complexity as the degree to which a process is difficult to analyse, understand or explain. One way to analyse it is to use a process control-flow complexity measure which examines the control-flow of consecutive processes and can be applied to data acquisition processes and workflows; then to evaluate the control-flow complexity measure to ensure that a high quality of results can be achieved on time. In this study, a 'process' is defined as a sustained series of events or actions that effects change through a series of stages. It resembles an interactive algorithm where elements such as the stakeholder requirements, the 3D object properties (such as professional expertise, equipment) interplay with environmental parameters (see) and reorganise, or rearrange entities such as activities, decisions, or contexts.

Therefore, the proposed Data Acquisition Process Management System (DAPMS) provides for the first time, a fundamental infrastructure to define and manage different processes in the area of 3D digitalisation of tangible CH objects. The proposed approach and the steps to be followed are illustrated in the sequence of graphics within:


Figure 12 represents a logical dynamic graph for a 3D digitisation project and summarises visually the relation of complexity to quality. Figure 13 then shows the Radial Pie Chart tool developed by this study to represent the complexity of a digitisation project. This tool lies at the heart of our efforts to obtain a concrete measurement of complexity in 3D projects that can be used for practical purposes. Outside the direct remit of the study, a DAPMS Application (App) has been developed and at the time of writing this report is in the final stage of revision and testing in a series of 3D digitisation CH case studies (Fig. 14).

### **The Radial Charts for Complexity**

Every complexity factor resulting from stakeholder requests including the assigned time

**Fig. 4.** Overview of the VIGIE2020/654 proposed DAPMS

horizon, total budget availability/priority and overall vision indirectly controls time allotment and all resources allocation in general, based on the digitisation purpose, desired level of detail, location, type, etc. (Fig. 15).

Digital CH Data Preprocessing requirements tend to vary significantly, depending on the scalability associated with the data to be acquired (often recorded from different sensors, as part of disparate sources, in different formats etc.). This in turn imposes changes in demand for Data Consolidation/Registration (collection, selection, merging or integration), Cleaning (missing value imputation, noise control), Transformation (normalisation, aggregation/discretisation) and Reduction (decreasing number of variables/cases, balancing skewness); each coming with additional hardware and software implications (Fig. 16).

DCH software requirements tend to differ in terms of reliability, operability, compatibility, maintainability, quality of results, security etc.

The demands in computational power, bandwidth, memory, time, and cost are, however, always determined in relation to the corresponding hardware constraints. DCH data acquisition alone (multi-sensing), can call for considerable variance in hardware selection (cameras, scanners, drones etc.). Additionally, demands in storage capacity (cache/physical memory, disk drive and partitions, cloud capacity, etc.), processing power (operation/network configurations) and representation (monitors, printers etc.) are often determined ad hoc. The constraints in computational power, bandwidth, memory, time, and cost restrictions are always decided in relation to the corresponding software demands (Fig. 17).

Environmental conditions (controlled or not) that may be perceived as contributing factors of complexity are included here. Both long-term (climate) conditions known to interfere with 3D data acquisition in general, such as rain, snow, wind, frost, fog, and sunshine, as well as physical measurements that become critical in reporting, such as

**Fig. 5.** The Tangible Object as a parameter of complexity

temperature, humidity, barometric pressure, wind speed/direction, air pollution, etc. are taken into consideration (Fig. 18).

The Team/expert parameter incorporates all complexity factors associated with personnel grouping (team formation, communication, interaction and collaboration) including HR responsibility and accountability (Fig. 30). This ranges from user qualifications and corresponding worldwide recognised certification, licenses, and equipment/infrastructure distribution, to interpersonal coordination together with quality assurance implications in the field (Fig. 19).

The Project parameter includes all complexity factors pertaining to digital CH project planning, performance monitoring and management (Fig. 29). This includes setting up an integrated management framework to effectively share resources, experience, knowledge

**Fig. 6.** The Stakeholder Object's description as part of the complexity

and expertise in pursuit of collective intelligence, subjected to any physical, operational, technical or financial constraints and logistics (Fig. 20).

Complexity factors stemming from object attributes or specifications (Fig. 28), including states of conditions, physical, chemical, and functional properties as well as dimensions, classifications, permissions for transportation and any other object-specific concerns (health and safety, legal, ethical etc.) regulate the digitisation process. An increasing requirement of the CH community and corresponding research institutions (such as at the of writing this report running H2020 MSCA ITN CHANGE52 project) relates to the fidelity of colours, ranging from the usual colour calibration within an image-based modelling pipeline, to more demanding reflectance measurements such as light-material interactions. Such a requirement is a novel complexity gap for the 3D modelling pipeline, including the visualisation step. At the moment of writing this report, there is still no universal consensus on the best format for rich colourimetry measurements (Fig. 21).

#### **The Radial Charts for Quality**

Image quality in DCH is often defined by spectroscopic features achieved via theoretical, experimental, and numerical techniques that strive to meet multi-objective photometric criteria (spectral regions). These include Absorbance, Transmittance, and Reflectance levels mapping to particular source, range, wavelength and frequency configurations (Fig. 22).

Quality in digital CH is often perceived as an indication of potential detail in an image, referring to the relative difference in size (or distance) between the image and the (radiometric) features represented on the ground. The quality of the calculated scale

**Fig. 7.** The Project as a parameter of Complexity.

depends on the accuracy of the measured distance, as well as the spatial resolution (pixel ratio), affecting color range and (bit) depth (Fig. 23).

Image quality in DCH digitisation often comes down to realistic 3D visualisations, employing sufficiently detailed rendering techniques, to support object representation in multi-dimensional space. That is, calculating and adjusting textures based on recorded physical material characteristics such as opacity, contrast, and granularity, to a point where external structure approximations reflect the desired shape accuracy and color depth (Fig. 24).

Similarly, with 2D attributes, those same quality factors could concern the 3D aspects of geometry, for when generating high-resolution point clouds via specialised equipment (multi-view cameras, depth sensors, TOF, etc.), often calling for advanced signal processing tools and (semi-automated) modeling practices. In cases of complex background or textures, 3D moving objects, and severe occlusions, relative measures might dictate computationally intensive self-calibration/registration and synchronisation methods (Fig. 25).

A substantial subset of quality factors relating to computational geometry, such as accuracy and precision, may coincide with 2D attributes that could be efficiently represented on a coordinate plane. Relative measures are often estimated with respect

**Fig. 8.** The Human Resources as a parameter of complexity

to requirements in point density and corresponding (lack of) completion, with enhanced capturing resolution in mind (Fig. 26).

Another important DCH quality factor is the extent to which the digitisation process responds to adverse structural changes, looking for structural reliability and life-cycle management. This implies a meticulous condition assessment that goes beyond common compositional analyses to appropriately cover states of conservation, connectivity, foundation strength/integrity and material quality for large-scale built objects, monuments, and sites (Fig. 27).

All quality aspects in answer to the complexity imposed by the characteristics of the material(s) involved such as individual strength attributes like yield, fatigue, tensile or toughness could be directly or indirectly, individually, or jointly interacting with the overall quality of the digitisation process. To mention a few, these include chemical composition, moisture, corrosion, carbonation, resistance and porosity.

**Fig. 9.** The Environmental Conditions as a parameter of complexity.

**Fig. 10.** Data Acquisition Techniques (equipment) for indoor and outdoor 2D/3D digitisation as a parameter of complexity.

**Fig. 11.** The Location of the tangible object as a parameter of complexity.

**Fig. 12.** Overview diagram illustrating the relation of complexity to quality.

**Fig. 13.** Radar chart depicting the parameters for complexity.

**Fig. 14.** Layers of the Stakeholder's Requirements complexity parameter

**Fig. 15.** Layers of the Pre-processing complexity parameter

**Fig. 16.** Layers of the Software and Hardware Equipment complexity parameter

**Fig. 17.** Layers of the Environment complexity parameter

**Fig. 18.** Layers of the Team complexity parameter.

**Fig. 19.** Layers of the Project complexity parameter.

**Fig. 20.** Layers of the Object complexity parameter.

**Fig. 21.** Layers of the Spectral image quality parameter.

**Fig. 22.** Layers of the Scale image quality parameter.

**Fig. 23.** Layers of the Texture image quality parameter.

**Fig. 24.** Layers of the 3D geometry quality parameter.

**Fig. 25.** Layers of the 2D geometry quality parameter.

**Fig. 26.** Layers of the Structural Health Monitoring quality parameter.

**Fig. 27.** Layers of the Material quality parameter.

### **11 Conclusion**

This unique study paid special attention to the fact that the 3D digitisation of movable and immovable CH can be an exceptionally complex process with numerous factors limiting the object's eventual quality. Parameters such as budget, time available, location and object's conditions, accuracy and precision are significant in setting the production effort and output standard. This issue is crucial for CH as there are no internationally recognized standards or guidelines for planning, organising, setting up and implementing a 3D data acquisition project. As acquisition technologies and software systems become increasingly accessible, with photorealistic renderings now commonplace, it is even more crucial to understand the physics behind the hardware, the fundamentals of data capture and processing methodologies.

The complexity of a 2-/3D digitisation project can be determined after factors such as the stakeholder requirements, project specifications, personnel qualifications, the type and object's location and corresponding environmental conditions, the equipment to be used, the real object and the pre-processing software are defined. These issues will inform the level of production effort. The determination of quality can comprise the degree of detail, the precision and resolution of the geometric accuracy of the 3D shape, or the fidelity of capturing colour/texture. This EU study -for the first time- demonstrates that complexity and quality are fundamental considerations in determining the necessary and required effort for a digitisation project, including the needed high grade or value of the output.

Moreover, highly skilled and competent personnel in place, future technological advancements, such as the integration of more efficient artificial intelligence algorithms for the automatic merge of different point clouds from different sensors, including recognition and extraction of geometrical features in faster and more accurate sensors, together with greater computing power directly linked with high bandwidth internet connections such as 5G on cloud computing infrastructures, will surely result in improvements in the process of capturing and processing 2–/3D data. This will make it easier to increase the quality of data acquisition results in the fastest possible period, work with larger data volumes and bigger 3D models of higher grade and finally to contribute in the implementation of the EU DigitalDay2019 declaration.

**Acknowledgments.** Study at the request of and financed by the European Commission, Directorate-General of Communications Networks, Content & Technology VIGIE 2020/654.

### **References**


### **Data Acquisition – Technologies and Tools**


microscopy. Trends Anal. Chem. **72**, 193–201 (2015). https://doi.org/10.1016/j.trac.2015. 03.026


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Towards Sustainable Digitization**

### **Technology Solutions for Complex and Challenging Survey Projects**

Daniel Blersch(B) , Christoph Held, Markus Mettenleiter, and Christoph Fröhlich

Zoller + Fröhlich GmbH, Simoniusstrasse 22, 88239 Wangen i.A., Germany info@zofre.de

**Abstract.** Complexity and data quality of a project are strongly linked together. The higher the need for better data quality is, the more complex projects become. In digitization, various complexity factors have an impact on the quality, such as the object properties or environmental factors. However, it also largely depends on the employed technology. Digitization instruments are designed for specific purposes and applications and have defined specifications and limitations. Some devices perform better with one material or another, and can capture invisible detail or geometry at large distance.

The aim of a holistic documentation often implies to take into account significant amount of additional data and of different nature. Equipping devices with multiple sensors, allows capturing more information and detail at the same time, which means a massive increase of data - intensifying the challenge. Thus, it is mandatory to be aware of the strengths and limitations to decide which technology, or a combination thereof, fits best for each case.

Smart technological ideas and sensor design can mitigate the complexity. At the example of terrestrial 3D laserscanners by Zoller + Fröhlich, this article provides insight to manufacturer's strategies about ensuring maximum survey quality and proposing efficient solutions for complexity reduction in survey projects. The technological answers range from sensorial adaptions to sophisticated workflow approaches and the incremental use of intelligent algorithms to an outlook on the necessary focus for the future. The approaches are illustrated with examples from emblematic use cases of outstanding tangible Heritage digitization in Europe.

**Keywords:** Digitization · Complexity · Quality · 3D Laserscanner

### **1 Introduction**

Cultural heritage projects are manifold and each object or site is unique. Hence, each digitization project is challenging in its own way. The approaches to document moveable and non-movable objects differ and there is no single technology that fits best for all documentation projects. The decision about the instruments to use is primarily based on the required level of detail. It directly affects the choice of the appropriate deployed capturing technology. Thus, it also influences the resulting data quality and the complexity of the documentation process. The higher the required detail, the more the capturing data and the more time it will take, which increases the project costs.

© The Author(s) 2023

The example in Fig. 1 shows an excavated mosaic pavement. The digitization costs would vary significantly between documenting just the rough structure and color and recording each individual tile in detail.

**Fig. 1.** Digitization of a mosaic pavement with an example of the detail provided by the point cloud (right). House of Eustolios at Kourion (Cyprus).

The smallest item size of the documentation defines the level of detail to be adapted which includes also an indication about the targeted level of data quality. Hence, the appropriate 3D survey technology can be chosen.

In general, there are three different categories for contactless 3D documentation technologies:


### **2 What Makes Heritage Documentation so Complex? Key Limiting Factors in Surveys with TLS**

### **2.1 Technological Aspects**

Each technology has its strengths but also limitations and it is crucial that the user is aware of these to choose the best solution for each project. No single technology may fit perfectly to all needs. Often, the solution to a holistic documentation is a combination. This chapter gives an overview on some of the main parameters, which decide about data quality with active optical systems.

**Resolution and Distance to the Object.** No matter which optical measurement technology (active, passive or hybrid systems) the principle is based on a receiving sensor for signal analysis and optical components. Fundamentally, digitization is a discrete method, which results in a digital approximation of a surface. The resolution is a fixed sampling raster without automatic adaption to the needs of the current object. The higher the sampling amount – thus, the resolution - the more accurately its digital twin will represent the object. As a rule of thumb, the minimum resolution needs to be at least twice as fine as the smallest required detail. Otherwise, the detail may not be readable from the spatial survey data, as can be seen in Fig. 2.

**Fig. 2.** Principle of minimum resolution for sensing small surface features.

Resolution is defined by an angular increment and changes with range. As depicted in Fig. 3, the point spacing increases linearly with the distance from the sensor. For instance, a sampling resolution of 5 mm at 10 m distance means a point spacing on a frontal surface of 50 mm at 100 m and of 0.5 mm at 1 m.

**Fig. 3.** Principle of linear increase of the point spacing by distance from the sensor.

**Focal Point.** From photography, it is known that the focus needs to be adjusted for to capture a sharp image of object across various distances. Usually, with laserscanning devices the focal point is static. It is optimized for the entire range of the instrument and cannot be adjusted by the user (Fig. 4).

**Fig. 4.** Relation between spot size of the laser beam and point spacing

It is an important fact to be aware of though, since it is not beneficial to increase the resolution of the sampling raster much beyond the actual footprint size of the laser beam on the object. The maximum factor depends on the system used, however, as orientation, it should not exceed 1.5x. Therefore, details of high importance should be captured from closer setup positions rather than being surveyed across longer distances. However, the instrument should not be too close either, as the focus is usually not closely in front of the instrument and often noise increases on short ranges as well.

**Angle of Incidence.** As shown in Fig. 5, the resolution also changes with the angle of incidence of the active optical signal onto the object's surface. The flatter the angle of incidence, the coarser the resolution becomes. It is preferable to have a frontal view onto the object - and thus a steep angle - in order to achieve higher resolution of details. This might be easy for a wall, but cultural heritage objects – except paintings - are rarely flat. It is also worthwhile noting, that with a flat angle of incidence, less light is being reflected back to the instrument, and thus noise in the data can increase.

**Fig. 5.** Principle of incidence angle variation of the laser beam

**Field of View.** Each optical survey sensor has a defined field of view. All single recordings need alignment in a common reference system in order to describe the 3D shape of an object correctly.

**Fig. 6.** Color mapped scan (extract) and example of single imagery tiles from the integrated camera sensor with overlaps (right). Example: Panagia Church of Asinou (Cyprus)1.

To be able to align all recordings to each other – a process that is referred to as registration - the individual parts need to be captured in a way that they share common

<sup>1</sup> Courtesy by Cyprus University of Technology.

areas with another part (Fig. 6). An ideal overlap area duplicates the amount of recorded area by 20–40%, depending on the situation and the technology employed. Thus, the complexity increases with a smaller field of view.

In addition, the accuracy of the registration process influences the data quality. After each pairwise alignment of images or scans a residual error remains, which accumulates with the amount of alignments and thus, can influence the quality of the result. Hence, a larger field of view of the capturing sensor with more geometrical information in one shot allows reducing the errors of the registration process.

**Range.** As discussed above, the data quality usually decreases with further distance to the instrument, especially regarding the resolution and focal point. In addition, over longer ranges, less emitted light returns to the instrument, which increases the range noise (error in depth) in the data in an almost linear relation, similar to angular errors. While decision makers often focus on the maximum range an instrument can cover, the importance of the minimum range is often underestimated. In narrow and tiny spaces, such as corridors and technical rooms, a larger minimum range can leave larger blank areas in the data, see Fig. 7. These holes require filling with data from additional setups.

**Fig. 7.** Data loss (magenta) by min. Range at 30 cm (left) and 60 cm (right) inside a tight space.

**Accuracies.** Each instrument is subject to certain accuracies, which manufacturers of professional equipment should specify in an appropriate data sheet. High-level devices allow verifying at any time in their life cycle, whether the accuracies stated in the data sheet are valid and offer the possibility of calibration by the manufacturer, or alternative countermeasures.

**Measuring Time.** Another fundamental factor for data quality is the measuring time, or data rate as it strongly affects the range noise in the data.

For passive optical sensors, such as cameras, mainly the exposure time and sensor sensitivity determines how dark and noisy the image is.

For active laser scanners, the data rate (amount of measurements per second) has a similar impact. A higher rate leaves less measuring time per pixel, which increasingly causes more range noise. Therefore, a lower data rate leads to less range noise and subsequently to higher quality of the data (Fig. 8).

In this context, it is important to be aware that the resolution setting of laser scanners entail a higher data rate setting in the background, which will increase the range noise, typically by factor 1.4x. Only a few instruments allow adjusting the data rate via a quality setting to compensate that effect. However, this of course implies a longer scanning time. Therefore, with respect to an optimum of accuracy, data volume and scanning time, the resolution should not exceed the real needs.

**Fig. 8.** Noise at data rates: fast (left) - slow (right). Panagia Church of Asinou (Cyprus)1.

### **2.2 Challenging Object Properties**

The complexity and data quality depend also on the object properties.

**Shape.** The shape of the object is a key factor for the setup configuration. Comparing to a flat shape captured frontally from very few positions, a staggered shape with concavities and convexities requires multiple setups from different perspective and vantage points in order to cover the shaded sides. Each scan or photo contributes and provides data to fill in gap-areas, shadowed from previous viewpoints (Fig. 9).

**Fig. 9.** Shadow effect on the survey data by staggered surfaces (tympanum and jamb figures at Cologne Cathedral, west portal) and with filled gaps2.

<sup>2</sup> Courtesy by 3DOM: Fresenius Univ. Köln, Heriot-Watt Univ. Edinburgh, Dombauh. Köln.

Depending on the accessibility and the movability of the object, as well as the limitations of the used technology, some cavities might not be documented at all or require the usage of different technology.

**Dimensions.** The physical dimensions of the object have a direct impact on the complexity due to the need of more scan data, like additional small pieces in a big puzzle (Fig. 10). The position and orientation of all these tiles need to be established and each one needs to share sufficient data with others in order to fit together. Each alignment introduces a residual error, which then may accumulate over the entire project.

**Fig. 10.** Cologne Cathedral: point cloud of the north elevation with the urban context. Scanner setup positions on the roof (right). Geometrical identities due to large-scale symmetries require a sophisticated project structure for the correct assembling of scans.<sup>2</sup>

Whilst the multiplication of capturing positions at ground level mostly faces organizational questions, the multiplication on vertical levels quickly leads to physical limitations and cutbacks in coverage and data quality.

**Fig. 11.** Alignment of three 3D scans. Result (left) and single scans (right).

**Symmetry.** Especially on a large scale - geometric symmetry challenges algorithms and users, when trying to align the pieces together (see Fig. 11) and the likelihood of a false alignment increases. Thus, more time is needed for the alignment and makes a thorough check of the data even more important.

**Material.** The data quality largely depends on object materials. Thus, it is important to be aware of its properties and effects on technology when choosing the appropriate digitization system. In this context, the reflectivity of an object is the most crucial factor. The more reflective, the more light returns to the instrument, and within limits, the data will be less noisy. On the other side, less light is returned by dark or distant surfaces, which leads to an increase in noise.

Further, the surface finishing is important, whether matt or glossy. The latter can cause strong reflections, which may even blind the sensor. This effect is well visible on polished metal. Fully reflecting materials, such as mirrors, lead to range measurements of reflected objects falsely allocated on the vector of the sensing laser beam.

**Fig. 12.** Laser light diffusion on marble [7]

Laser measurements on translucent objects, such as marble, often suffer of higher range noise and incorrect measurements, as the light penetrates the surface to a certain extent and is reflected from various depth levels of the object (Fig. 12). Translucent materials, such as glass, lead to wrong range measurements, as the refraction properties of the material are unknown and cannot automatically be compensated for. In addition, the state of aggregation matters. Water, rain or fog might have further limiting effects on the data capture. Especially water or liquids in general cannot be digitized with common technology and may cause artefacts.

**Object Conditions.** When looking at the material it is important to look also at the condition of the object surface. Vegetation, such as moss, or vandalism may also affect the quality of the survey data (Fig. 13).

**Fig. 13.** Factors of survey data alteration: vandalism and surface flora at Cologne Cathedral2.

### **2.3 Challenging Circumstances and Environment**

**Accessibility.** The ideal vantage points for the survey are sometimes out of reach or may lead to challenging logistics (Fig. 14). Often, the practical aspects of the survey equipment - such as weight, size and autonomy – become key factors for overcoming these limitations. The accessibility is directly linked to the physical properties of the object and its direct surroundings. Vertical extensions and narrow passages lead to steep angles of incidence by the sensor signal while digitizing the surfaces.

**Fig. 14.** Exposed logistics on the exterior of a tall gothic spire (Cologne Cathedral)2.

**Ambient Conditions: Moving Objects, Illumination.** Moving Objects potentially cause complications regarding the processing steps of the survey data (alignment, coloring) and temporarily obstruct the sensor's view towards the object, which results in more or less large data gaps. Especially on touristic sites, it can become a relevant limitation. However, often these sites cannot be closed to a large extend to public visitors during the survey. Then, other strategies become necessary, for example collecting additional data from similar vantage points, which may increase the project time and complexity. In addition, animals may obstruct the view, especially when present in large numbers, such as pigeons (Fig. 15).

**Fig. 15.** Hostile climatic conditions and wildlife presence during survey of historic British bases in Antarctica3.

<sup>3</sup> Courtesy by British Antarctic Survey and the United Kingdom Antarctic Heritage Trust.

Adverse environmental conditions can be hostile and limit the access to the site or part thereof, such as temperatures, radioactivity or explosive dust and gases, or just weather factors, such as wind, fog, rain, snow, or ice.

Finally, the illumination has a significant impact on the data quality. While high-end laser scanners are not limited by sun or ambient light, it is still critical for additional sensors of the system, such as RGB-photography (shadows, color temperature etc.). For instance, light conditions may vary during the fieldwork time and induce color variations in the final data.

### **3 Technological Strategies for Managing Complexity and Data Quality in High Profile Projects**

### **3.1 Software and Workflow Strategies**

The strategies for managing complex projects largely depend on the employed technology. A general survey of the topic would be beyond the scope of this paper. In the following, the authors will focus on strategies for 3D laserscanner devices and data.

Complex projects usually imply more data volume to be managed. In terms of 3D laser scans, the data needs to be puzzled together with sufficient overlap. The alignment process of the 3D puzzle tiles is commonly known as registration. For this purpose, dedicated field solutions have appeared on the market, which automatically download the scanner's data and register all scans in parallel to scanning [3, 4]. It allows verifying the completeness and data quality of the project immediately in the field that no data is missing and sufficient detail is captured. A field software helps managing the project complexity by providing tools:


**Field Registration.** Similar to solving a puzzle, the alignment of scans is not a trivial task even for powerful computers. In order to simplify the registration process, additional markers or physical objects can be placed in the scene to be recorded by the laserscanner and located in the data by software. Although markers, i.e. targets, are largely considered highly accurate for the alignment of survey data, in the reality of cultural heritage survey the usage of such targets is often restricted. In this context, the physical size of targets is relevant and in some cases, the placement may cause damage to the surface, e.g. by using adhesive tape to attach these to walls or objects. Also, the use of survey targets at large scale requires an additional planning step in order to achieve an ideal distribution. Thus, it is a significant time factor.

The so-called Cloud-to-Cloud registration is an established alternative to substitute or integrate with targets. The Iterative Closest Point (ICP) algorithm [2, 5] identifies closest point neighbors between scans and minimizes the distance between these. It can be considered a statistical fine-tuning of the alignment and delivers reliable results especially when the scans are already pre-orientated, typically within meters.

Another method is to identify certain features in the 3D scans. The so-called Planeto-Plane registration [8] offers an alternative target-less approach by detecting abstracted planes that allows initial alignment and an accurate reliable registration.

Even though automatic processes exist, a registration process is not fail proof and often requires manual intervention for completion. In complex projects, the manual alignment can be cumbersome and time-intensive. The scan alignment methods without targets require 30–50% overlap between scans and it is difficult for an operator to keep track of all covered surfaces. Thus, depending on the size and complexity of a site, surveyors often tend to scan more than necessary for avoiding registration issues back in the office. Addressing these issues, Z + F presented a field registration workflow in 2015 [4], which, has been widely accepted as a standard principle for 3D laserscanning by today. The position and orientation of the single scans is estimated by additional sensor information, such as video, inertial systems (IMU) or GNSS. The system combines this information with target-less ICP registration on site. In this manner, the operator monitors the registration quality continuously and the redundancy is lowered towards a minimum level without risk.

The current systems are often subject to technical limitations and thus, for example, are used only outdoors (GNSS based) or indoors (videometry based). Others allow a combination thereof. Zoller + Fröhlich proposes the so-called "Blue Workflow" [4]: the scan positioning system (a combination of GNSS, compass, barometer and IMU sensors), which tracks the displacement of the device between consecutive scan positions, works in- and outdoors. The software Z + F LaserControl® Scout runs in parallel on a WiFi connected tablet PC, automatically downloading and registering (cloud-to-cloud) the incoming pre-orientated data. Feedback about sufficient overlap and the registration outcome is provided immediately. The registration can also be combined with targets, if required. Hence, the operator proceeds with a minimum number of scans without inadequate high redundancy of covered area or potential registration issues. Since the tablet PC allows parallel processing, the operator efficiently uses idle time until the completion of the next scan. The algorithm may not be able to register all scans automatically due to limited overlap, symmetry or noise by moving objects. Therefore, the solution integrates a toolset for immediate manual adjustment. In addition, the software allows to examine and to assess the data directly on site. The user is able to verify and ensure the required point coverage on the focused objects, as well as the data quality of the measured points.

Such on-site processing tools allow the user to decrease the complexity of a project and ensure the best data quality possible, as well as the overall project success.

**Annotations, Metadata and Project Planning.** Annotation capabilities of the employed software are of great benefit, as starting with the first visit on site by the decision-making staff, these tools can log field notes and meta data (as text, photography, audio, video or any format) to create detailed tasks for the field team and include referenced photographs about key setups [6]. Further, these tools allow planning scan positions in advance in order to prevent unexpected and significant deviations of the main project objectives, which result in increasing costs.

**Data Management.** Opposed to industrial applications - where scan data typically is replaced with best-fit primitives - every data point may matter for documenting cultural heritage where irregular, non-planar surfaces prevail. Surfaces of ancient structures hardly fit to geometric primitives without significant loss of information. Filling larger gaps in the data is not trivial but always work intensive. Therefore, it is good practice to multiply scan positions and capture with lower resolution from various different angles than to scan with higher resolution from less positions. The coverage increases and an overall high point density is achieved by overlapping multiple scans. Generally, depending on the complexity and the required data quality, the aim to achieve a maximum coverage tends to increase the size of the dataset and the complexity for the registration. Further, difficult scan positions require an immediate visual validation option of the registration result before moving to the next position.

The software Z + F LaserControl® works on a scan-by-scan logic for processing or analysis [4]. Thus, the solution is scalable to process extensive projects on a tablet PC.

#### **3.2 Hardware Approaches**

Today there are laserscanner hardware solutions available, which combine the spatial data with data from additional sensors in order to mitigate the complexity for the use of different documentation technologies. These data are complementary and the processing is highly automated.

**Handheld Integration.** A static 3D laserscanner is typically able to capture panoramic full-dome scans, repeated from various vantage points in order to reach a high coverage. At a certain ratio between redundant and uncovered surfaces, the efficiency of additional static full-dome scans may decrease. Then the complimentary use of handheld scanning devices may be a method to quickly fill-in small remaining gaps.

**Fig. 16.** The Z + F scanner can serve as a data hub for handheld devices (here: DotProduct) sending the data to the field software Z + F LaserControl® Scout for registration [3]

The handheld data integration requires sufficient overlap with the panoramic scans. A dedicated workflow from Z + F synchronizes via WiFi F between both devices (Fig. 16). The data of the handheld scanner is directly registered and verified its completeness.

**Infrared Camera.** Infrared cameras allow reading the surface temperature. For conservational analysis, it can provide indirect information about the material condition, such as thermal bridges, structural discontinuities, electrical faults or voids. In some cases, the infrared data can even unveil information from below the object's surface.

Combining spatial and infrared data manually requires the identification of common points (homologies) in both data sets. This had been a quite challenging task before the advent of automatic solutions due to the typically great difference in resolution between both systems.

**Fig. 17.** Mapped point cloud with infrared image unveiling hidden technical installations. Add-on IR-camera on a 3D laserscanner (right) [3]

Zoller + Fröhlich developed a dedicated calibrated infrared system, which senses thermal information and spatially allocates each value automatically (Fig. 17). Hence, thermal analysis is carried out directly on the 3D geometry, as opposed to 2D images [3].

**High Dynamic Range (HDR) Technology.** In cultural heritage applications, the interest beyond the pure geometry includes capturing high quality RGB information in order to map scan data with photographic information. The combination with image data is used for surface analysis of materials and assessment of degradation [3]. Further, museum and immersive experiences require photorealistic color of the data.

In this regard, it is challenging to ensure a constant color quality as the lighting conditions change in the course of a project. In addition, photos can differ in appearance, depending on where the internal sensors focused on. The image can result under- or overexposed, if the focus or light measuring area differs within the same image frame. HDR photography is a standard technique for panoramic photography. Outdoors, the scene is usually well illuminated but shaded objects are often too dark. Standard photography can hardly find one suitable exposure setting for all areas in an image without over- or underexposed parts. Instead of just one image at one exposure setting, HDR

**Fig. 18.** HDR photography. Mapped result (left) and raw images with different exposures2.

merges a set of images with different exposure times into one single image (Fig. 18). Thus, this technology is often integrated in laserscanning devices.

**SmartLight.** Color scanning in dark environments is a challenge. Due to massive presence of visitors at daytime, some applications require to execute the capturing work at night hours. Then, illumination of the environments most likely is required, which can be a challenge due to cables, shadows and even multi-shadows different light sources. Therefore, Z + F sensors operate with a HDR enabled camera system and provide integrated spotlights to illuminate the scene [3, 4]. These lights are mounted radially around the camera optics allowing shadow-free imagery even in near proximity, which is then mapped onto the point cloud.

**Photogrammetry.** Although deriving from different technologies, the output of active (e.g. laserscanners) and passive (e.g. separate, non-integrated cameras) systems can be successfully combined. For this purpose, the camera photos are used to create a 3D model with algorithms of photogrammetry, using the laserscanning data as a reference or skeleton model. Photos from an external sensor are usually taken at the same instance with similar lighting conditions and ensure best RGB color results. On the opposite site, laserscans provide highly accurate geometric detail. It should be noted that highly accurate data can be acquired locally on close-range with photogrammetry alone. However, for large sites it is much more complex to achieve the same accuracies as with laserscanners. Combining both technologies, the advantages of a combination ensure highest data quality in terms of color and geometry [1].

**Special Equipment.** When the physical size of the object exceeds common dimensions, then the vantage points for data capturing need to be chosen also in elevation.

Special fixtures and support structures may be necessary to reach these positions. There are different solutions available from technology suppliers, such as lever arms or inverted tripods (Fig. 19). The latter can be used to orient a scanner upright or upside down. Especially when lowered down to privileged positions, such as below the key stones of tall cathedral vaults, it allows to enrich the data significantly at a high quality level.

**Fig. 19.** Shaft tripod with laserscanner lowered through a vault keystone and special support bracket for scans exterior to the spire (right) at Cologne Cathedral2.

### **4 Conclusions and Outlook**

As shown above, technology is key for an efficient documentation of cultural heritage and can minimize a project's complexity while ensuring highest data quality. Also, technological supplements or sensor fusion, as shown above, can significantly improve data quality.

Although different solutions are available, 3D digitization is still a time intensive process. Thus, there is a trend towards much faster mobile systems, which acquire 3D data while the sensors are moved through the scene. However, the systems cannot yet provide the same data quality as of static panoramic laserscans. However, mobile systems will play a significant role in the future. They also collect a much higher data volume to be processed in less time in combination with higher resolution. Thus, it is mandatory to explore new processing and storage possibilities. Computer clouds can help in this regards, but a dedicated cultural heritage solution is still missing.

Apart from data capturing, there is still intensive research and development necessary to segment and classify the data automatically. At present, the data is visualized as a large collection of 3D points or surfaces consisting of triangles. However, the elements do not provide information about the object they belong to. Manual augmentation of this information is a tedious process and thus automation, maybe with the help of artificial intelligence, is certainly one of the future key research topics.

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Evolving Standards in Digital Cultural Heritage – Developing a IIIF 3D Technical Specification**

Ronald Haynes(B)

University of Cambridge, Cambridge, UK rsh27@cam.ac.uk

**Abstract.** IIIF (International Image Interoperability Framework) is a collection of open web standards, broadly used for presenting and annotating high resolution images and audiovisual content, adopted by hundreds of GLAM (galleries, libraries, archives, museums) organisations worldwide, and permit the reuse and recombination of digital assets across traditional digital institutional boundaries. While these standards have been successful for 2D images and AV (Audio/Video) data, there is a growing interest in how they might be applied to 3D content. An example of this would be to combine shards of pottery from disparate collections into a single whole within a viewing experience. The IIIF 3D Community Group has been collaboratively considering common challenges and potential solutions to key areas, with major 3D developers and researchers, and ongoing dialogue with VIGIE2020/654 study colleagues, forming the IIIF 3D Technical Specification Group, engaging even more widely with other specialists and representatives across user communities and standards bodies.

**Keywords:** International Image Interoperability Framework · IIIF 3D Community · Standards · Cultural Heritage · Memory Institutions · GLAM · Virtual collections · 2D/3D interoperability

### **1 Overview**

IIIF (International Image Interoperability Framework) is a collection of open web standards broadly used for presenting and annotating high resolution images and audiovisual content [1]. These standards are adopted by hundreds of GLAM (galleries, libraries, archives, museums) organisations worldwide [2], and permit the reuse and recombination of digital assets across traditional digital institutional boundaries [3]. While these standards have been successful for 2D images and Audio/Video (AV) data, there is a growing interest in how they might be applied to 3D content. An example of this would be to combine shards of pottery from disparate collections into a single whole within a viewing experience.

The IIIF 3D Community Group [4] has been cataloguing user stories and user needs [5], exploring various 3D workflows [6], and initiated a viewer comparison project [7] with major 3D developers and researchers, and ongoing dialogue with VIGIE2020/654 [8] study colleagues. The viewer project was collaboratively considering common challenges and potential solutions to key areas, as part of forming the IIIF 3D Technical Specification Group [9], for which the 3D group is engaging even more widely with other specialists and representatives across user communities and standards bodies.

Some key questions have been around the conceptual framework needed to address the use cases for 3D scenes or digital dioramas, including by adding depth to the current 2D IIIF canvas model, and by embedding one or more canvases within a 3D scene (e.g. multiple paintings or texts or music associated with a cathedral, temple or palace, held in separate collections, reunited with their original walls and interiors of a suitable building model). With growing user and institutional demands, technical developments, and examples of advanced research collection and integration of virtual resources (e.g. Sketchfab [10], MorphoSource [11], Exhibit [12], Mozilla Hubs [13]), there is pressing need for a technical specification to ensure interoperability and longer term sustainability. The VIGIE2020/654 study has more information about these needs.

The plan for the IIIF 3D Technical Specification Group is to continue a collaborative approach to clarifying and specifying interoperable frameworks for 3D data, including common ways to:


The group will continue its work with other standards bodies and 3D image viewer developers, to collaboratively address the many challenges around this developing area. The combined and widespread expertise from the many 3D specialists will continue to guide the work of the IIIF 3D Technical Specification Group, as it outlines sustainable options for the interworking of existing open standards (e.g. VRML/X3D [14], WebXR [15]), established foundations (e.g. WebGL [16], Three.js [17], react-three-fiber [18]), and emerging proposals (e.g. *<*model*>* tag [19]), to provide recommendations for expansions to and modifications of IIIF APIs to better interoperate with the evolving digital ecosystem of online 3D content.

These 3D developments will complement the ongoing updates and continued widespread adoption of the IIIF technical specifications for 2D and AV data, which has been enabling greater access to widespread resources – of even greater significance for teaching and learning and research during the global pandemic – as well as providing for richer presentations, close object inspection through deep zoom ranges, and shareable annotation of media using W3C's Web Annotation Data Model [20].

IIIF specifications also enable the recombination of long-separated parts of an original whole (e.g. missing sections or leaves in a digitised medieval manuscript, viewed with missing pieces contained in another digitised collection). IIIF adoption continues to enable effective sharing and support for the preservation of cultural heritage resources, whether as individual items or as combinations of media from one or more collections, locally and globally.

These technical developments for the community rely on regular meetings and input from the community. Planning for the IIIF 3D technical specification includes recorded monthly meetings, complementing the more general 3D Community Group presentations and discussions, and involving group problem-solving with regular input from and interactions with representation from 3D researchers and media and viewer developers, including from University of Cambridge, Duke University, University of Edinburgh, UC San Diego universities, Deutsches Museum, IIIF Consortium, Mnemoscene, MorphoSource, MPEG consortium, Sketchfab, Smithsonian Institution, Visual Computing Lab (CNR-ISTI, Italy), Web3D Consortium. Google has presented and been part of our deliberations, and we plan to follow up with Apple to learn more about the plan by the WebKit team to propose a *<*model*>* HTML element for 3D. We also intend to maintain a close engagement with existing web-based 3D standards such as X3D, with its VRML roots, and WebXR.

We regularly collaborate with 3D-related projects and funding proposals, and ask funding bodies to help engage others in further developing this collaborative community, ensuring that we interconnect with more communities, to further develop 3D standards which will be the most widely usable and adopted across current and future proposals and projects.

### **2 Key Use Cases and Prototypes**

Drawing on more than a year engaging with various individuals, institutions, and professions through regular IIIF and other group meetings and conferences, the IIIF 3D Community Group identified a wide set of user stories [21]. With that input, and as part of the work of the group making preparations for the 3D TSG, the core practical use cases identified are:


To clarify the viability of meeting these cases, some research and demonstrations were able to highlight the availability of existing practical options, as a basis to proceed and to indicate areas for further development in the future. For instance, there was interest in an initial demonstration of the sharing of two or more 3D models, stored in different formats, which could be rendered in the same visual space. This is a basic requirement, similar to the common rendering of differently formatted 2D images together (e.g. JPEGs and PNGs) in the same viewer, which is essential to enable the important IIIF options for interoperability across the necessary formats.

While it has long been possible to manually load 3D models in a web browser, using something like the popular three.js cross-browser JavaScript library and the underlying WebGL application programming interface (API), there are well-developed viewers built on such frameworks which provide additional options useful for interacting with the models. As part of the IIIF 3D group efforts, even more advanced options have been shown in some technical proofs of concept that demonstrate multiple assets in different formats being rendered in the same virtual space, most using IIIF in some capacity, in particular:

• The Infinite Canvas [22]: Combines a 3D asset, an audiovisual clip, and multiple still 2D images in a single navigable 3D space, all using IIIF manifests for each asset (Fig. 1).

**Fig. 1.** Figure from the initial screen of The Infinite Canvas, a proof of concept application which is shown displaying 2D, an audiovisual clip, and a 3D model.


**Fig. 2.** Figure of the Antikythera Mechanism, combining three different types of 3D files – GLTF, CT (computed tomography), and point clouds.

**Fig. 3.** Figure from a Mozilla Hubs gallery, displaying 3D assets from three sources – MorphoSource, Royal Pavilion & Museums Trust, The British Library; used with permission from Julie Winchester, Doug Boyer, Edward Silverton, and respective collection holders.

### **3 Combining Content in 3D Environments**

To focus on the operational use cases for combining media in a 3D environment, and to consider best options to enable the intersection of 3D and non-3D content in the same rendering environment in a web browser, the IIIF group discussions and follow up research identified two main approaches for translating the traditional IIIF canvas into a 3D scene. These are high-level and use case oriented solutions for creating IIIF manifests that address some options for adding a third dimension to extend the existing 2D canvas or otherwise fitting the 2D canvas in a 3D space. While not necessarily mutually exclusive, these options require further consideration and experimentation to help enable and encourage interoperable and sustainable development in the 3D context.

### **Canvases and Scenes**

By default, the IIIF presentation specification makes use of a canvas concept which has some of the following basic properties:

	- Standard assets have parameters to indicate size and position within the canvas
	- Placement of the secondary canvas is determined by parameters indicating size and position within the canvas

We have proposed a kind of straw man concept, to compare and contrast with the canvas, calling it a "scene". Scenes are similar to canvases but have some important differences. The following basic properties apply to scenes:

	- All assets have parameters to indicate scale (3D) or size (2D/AV), position, and rotation within the scene relative to the scene's coordinate origin
	- Placement of the secondary scene is determined by parameters indicating size, position, and rotation within the scene. Objects inside the secondary scene respect the coordinate origin of the secondary scene, but end up modified by the placement of the secondary scene inside the primary scene. Example: An overall "world" scene that has placed a single secondary "room" scene at position (0, 0, 10). The room has a lamp inside of it at the room's origin (0, 0, 0). To observers, the lamp would be located at world coordinates (0, 0, 10).
	- Placement of the secondary canvas is determined by parameters indicating size, position, and rotation within the scene.

Additionally, the scene concept requires one additional basic property to be added to canvases, namely that:

	- Placement of the secondary scene is determined by parameters indicating size and position within the canvas

Therefore, canvases and scenes can nest inside of each other (and canvases can nest inside canvases, and scenes can nest inside scenes, etc.). It could be argued that canvases and scenes are really the same thing, but with different component-style properties (e.g., see the IIIF-ECS extension proposal [27]). These ideas could easily be expanded to treat canvases and scenes as specific manifestations of a more generic entity. This would make some of the operational use case examples even simpler to describe than they are explained here, at the expense of introducing more lower-level complexity.

While there is a question of whether this proposal would entail a commitment to setting a 0,0 origin point in the top left corner of a canvas, with a z-axis extending from there outwards, there is also consideration that this would be unnecessarily complicated to implement compared to a common configuration of an infinite scene setting 0,0,0 in its centre. The work of the IIIF Technical Specification Group has been supported by the understanding that the Shared Canvas Model can potentially be altered to permit "scenelike" functionality. In order for 3D to become a useful part of the IIIF specification it will require the same level of changes and the resulting kind of core status as has been achieved with that addition of the IIIF AV technical specification, as part of the major update of the Presentation API in 2021.

#### **Scenarios for Transforming 2D into 3D**

**Extend 2D Canvas I:** *I want to display a single 3D object on a flat webpage for rotation, zooming, panning of the object.*

This is an operational use case that IIIF 3.0 does currently support for 3D meshes, via the "Model" or "PhysicalObject" annotation types, although the standard parameters for size (height and width) don't translate naturally to a 3D object and there is no way to specify properties such as rotation.

Using the concepts described above, this use case would be achieved by a primary canvas nesting a single secondary scene, with the scene's size dimensions equal to the size of the canvas. The 3D object would be placed within the scene at the origin with scale, rotation, and other properties explicitly specified.

The currently supported IIIF manifest that presents a 3D Model or PhysicalObject annotation as the only annotation on a canvas could easily be considered a simplified convenient way of describing the more complex setup described above. In other words, creating a IIIF manifest that posits a canvas with a single PhysicalObject annotation could be read as a convenience method to describe a Canvas -*>* Scene -*>* Physical Object (Scale 1, Rotation null, Position {0, 0, 0}) hierarchy. In fact, a potential strength of this solution is that it allows for "collapsing" or "expanding" complex circumstances as is needed.

**Extend 2D Canvas II:** *I want to display several 3D objects in separate views on a flat webpage so that viewers can compare and contrast multiple objects at once.*

This could be achieved with a single primary canvas nesting multiple secondary scenes, with the gridding and size of each scene determined at the canvas level. Each secondary scene would place a single 3D object at the origin with standard scale and no rotation.

For reference, this approach is how the Aleph viewer [28] on MorphoSource uses IIIF manifests currently, although it obviously goes beyond the current IIIF specifications in order to display volumes as well as meshes.

**Fit 2D Into 3D Space I:** *I want users to interact freely (in flat, VR, or AR contexts) with a virtual space. This virtual space should have a mesh and a point cloud, bounded by two "walls" displaying 2D content that are already accessible via two standard IIIF 3.0 manifests.*

The primary entity in this operational use case is a scene rather than a canvas (or perhaps a canvas specifying flat screen size that nests the world scene within it). The mesh and point cloud are nested within the scene at the desired positions, scales, rotations, etc. Finally, the two already-existing IIIF manifests - each with its own respective canvas - are nested within the 3D scene with their own position, scale, and rotation. The placement of the assets within each canvas are determined by the original manifests for those canvases, and are just displayed in the world scene.

**Fit 2D Into 3D Space II:** *I want to fill a virtual gallery hall with individual virtual rooms. Each room has been designed by a separate group and they consist of a mix of 3D objects displayed on room floors and 2D content displayed on room walls.*

Each room in this example is equivalent to the operational use case described above this, and so each room is a scene nesting combinations of assets and IIIF manifests describing canvases to serve as "walls." Rooms are then nested as multiple secondary scenes inside a larger world scene, which provides the position/rotation/scale information necessary to connect the rooms together, make them sit alongside each other, etc. Objects in each room are placed using the coordinate systems of the rooms (or in some cases the top-left origin coordinate system of canvases inside rooms!), but the world scene brings these rooms together in a way that makes sense for the overall purpose of the project.

For reference, this is the approach taken in Mozilla Hubs and the Infinite Canvas demonstrations noted above, with the latter also obviously going beyond the current IIIF specifications in order to display the many media types.

### **4 Requirements for Annotation in a 3D Environment**

The operational use cases for annotation in the context of IIIF and in a 3D environment have two fundamental purposes – to add textual and other forms of commentary to a 3D model, and to add multiple models from different sources into a combined scene. The essential and broad use of annotation in a IIIF context needs special considerations for 3D environments, as highlighted in the following details from the IIIF 3D Technical Specification Group.

#### **Annotations for Commentary**

The most basic form of 3D commentary annotation we can identify, specifies a point on the surface of a model by its coordinates (x, y, z).

The point itself has no dimension, and serves as an anchor for a visual marker, often represented as a pin, or circular hotspot. This marker can either be displayed in screen space, overlaid on top of the 3D scene, or in world space, displayed within the 3D scene with its own geometry (such as a sphere).

In the case of screen space annotations, they may always be visible regardless of position given that they are overlaid on top of the 3D scene. However, this is not always desirable, and a content publisher may want them to appear only when on a surface facing the camera.

To achieve this, it is necessary for each annotation to store the surface normal vector of the mesh at the point where the annotation was created. This can be used to determine if the annotation is facing the camera and should be displayed, or if on the reverse side of a model and hidden.

In the case of annotations located within the scene's world space, it is not always possible to ensure that they are visible to the user just by defining their position, as the 3D model may occlude itself. For example, the arm of a statue might occlude an annotation on the torso, or if there are multiple objects within the scene one object might occlude another.

Therefore it is necessary to optionally include a camera position and orientation along with the annotation coordinates in order for a content publisher to ensure that an annotation is visible to the user. If for example the user were using a simple previous/next interface to page through annotations, this would allow the camera view of the scene to be automatically adjusted for each annotation in order for it to be visible, without requiring the user to manually rotate the scene.

The necessity to specify camera position and orientation per annotation is not limited to world space annotations however, and can also provide a satisfying user experience for screen space annotations.

Beyond the concepts of screen space and world space, there is also "object space", i.e. the coordinate system relative to each 3D model's individual origin. This is a useful concept when loading models from various sources which carry their own commentary annotations, in that the annotations are positioned relative to each model. This bypasses the need to translate each model's annotations into world space and means the models can easily be remixed in various ways without losing their associated annotations.

While there are other forms of annotation possible beyond points on a surface, such as regions of a surface, or volumes within an object, we believe that focussing on the simplest case first will provide insights into how to extend into other modalities.

### **Annotations for Scene Building**

Annotations with a "painting" motivation are used to combine images within a "shared canvas" coordinate space in IIIF. This concept could also extend to 3D models, where the body of an annotation is a 3D model URL, and the target is a canvas.

In order to create a 3D scene such as by placing artworks from various sources within a gallery space, or combining disparate shards of pottery into a whole, we will need the ability to specify xyz coordinates for each individual model within the shared canvas (as with commentary annotations).

If for example each pottery shard was produced using a slightly different method, resulting in varying object scales, origins, and rotations, it will be necessary to override/normalise these properties per annotation in order to combine them effectively. This is similar to how the Web Annotation Data Model allows images to be painted onto a canvas using a given xywh parameter.

It is common practice in 3D viewers such as Sketchfab for users to specify an initial camera position and orientation for a 3D model or scene. This can satisfy an aesthetic or informative preference on behalf of the content publisher. This is similar to how commentary annotations require an associated camera position and orientation, and the same concept may apply to "painting" annotations. One could imagine a scene combining multiple 3D models, where clicking on each model animates the camera to a new position and orientation in order to best view that model.

### **5 Authentication and Search APIs for 3D Content**

Given the ongoing developments with the Authentication and Content Search technical specification groups (TSGs), and the related changes introduced in version 3.0 of the IIIF Presentation and Image APIs, as well as developments in the browser community and the evolving web landscape, the 3D Technical Specification Group expect similar challenges, as well as the need for collaboration with the other TSGs in these areas of mutual concern.

The 3D TSG will coordinate efforts with those groups to ensure that any authentication or search solutions pertaining to 3D content are either consistent with any updates from those groups, or can clarify and inform those groups about any changes which may be suggested by particular features of the broad base of 3D content forming part of the work of the 3D TSG.

That said, and given the IIIF bases upon which the related 3D Community efforts are already developing (e.g. with reference to Sketchfab.com, and the open source technologies used by MorphoSource.org), we anticipate no new difficulties with searching the text content of manifests and annotations to be associated with 3D data. Existing examples of annotations used with 3D data include key viewers used with Sketchfab, Smithsonian Institute, MorphoSource, and X3D content. Some annotation models are closer to existing IIIF annotation approaches, and all annotation models will be key features of the 3D TSG's further research and review of search-related concerns.

Search and authentication requirements for 3D data may at some point (perhaps in a subsequent version rather than initial specification) include extra differentiation between component parts of a composite object, or with reference to individual items in a scene comprised of combined data which may require login (e.g. for selected images and audio-visual items). These will be matters for future endeavours, following the initial phase of 3D technical specification.

### **6 IIIF 3D Technical Specification Group Charter**

The group effort to establish the IIIF 3D Technical Specification Group expanded as it involved more committed institutions and volunteers, over the course of more than a year. That group includes research and cultural heritage institutions who are already publishing 3D content and who are willing to experiment with various approaches to help improve the creating, storing, cataloguing, sharing and sustaining of this media.

The initial IIIF 3D Technical Specification Group Charter, formally accepted by IIIF in December 2021, includes a list of those collaborating institutions and their representatives, along with more information about planned scope and roadmaps and other key details. For reference, it is included here.

### **IIIF 3D Technical Specification Group Charter** [29]

### **Introduction**

As IIIF has evolved from an initial focus on 2D images, encompassing new media modalities such as audiovisual content with time-based data, we see a plethora of unmet use cases throughout the cultural heritage field relating to the display of 3D media and related metadata. We recognise these requirements and the need for further developing a conceptual framework, which can complement and extend existing IIIF specifications.

The IIIF 3D Technical Specification Group will collaboratively clarify and specify common interoperable frameworks pertaining to the 3D data domain. This will include ways to:


The group will work with other standards bodies and 3D image viewer developers, and will collaboratively address challenges around this dynamic area, which shows great potential for a IIIF resolution, as practical options for media sharing and interchange, for which there is substantial demand and no demonstrably sustainable alternatives. Guided by widespread expertise from the 3D Community, committed to this purpose, the IIIF 3D TSG will outline sustainable options for the interworking of existing open standards, to provide recommendations for expansions to and modifications of IIIF APIs to better interoperate with the evolving digital ecosystem of online 3D content.

#### **Scope**

It is the intention of this group to explore suitable IIIF extensions and identify any necessary changes to the core IIIF specifications (e.g. adding a third physical dimension) to support display of 3D media using IIIF tools. After a period of time and suitable feedback from implementers, we expect to propose changes to the Presentation API, to accommodate stand alone and combinations of media, whether 2D, A/V or 3D.

This group will focus on the use cases identified, in versioned phases, including options to:


IIIF 3D TSG will look forward to working with other IIIF groups, especially where there are shared areas of interest (e.g. museums, archives, maps), and welcomes contributions to collection of user stories, ongoing community discussions, and specialist app development.

### **Deliverables**

The expected initial deliverables are IIIF API extensions and a specification change to define interoperable methods to enable:


### **Roadmap**


### **Communication Channels**


### 62 R. Haynes

• Calls: complementing the 3D Community Group calls (see Community Calendar)

### **Community Support**


### **Technical Editors**


### Feedback: iiif-discuss@googlegroups.com

**Acknowledgements.** Many thanks go to fellow IIIF 3D group Co-Chairs Edward Silverton *<*ed@mnemoscene.io*>* from Mnemoscene, and Julie Winchester *<*julia.m.winchester@gmail.com*>* from MorphoSource and Duke University, who helped author and shape key material for this article, along with the longstanding input and creative discourse with our many other colleagues in these collaborative 3D developments.

Special thanks go to Marinos Ioannides *<*marinos.ioannides@cut.ac.cy*>* from the Cyprus University of Technology and the colleagues from the VIGIE2020/654 study, for the continuing encouragement, collegiality, and inclusive engagement in building a larger collaborative community, which will help make IIIF 3D and the related practical standards a reality.

Special mention goes to colleagues in the Cambridge Digital Library, Cambridge Museums and related collections, Cambridge Digital Humanities, the Darwin 3D project group, and the emerging Cambridge 4D community.

### **References**


#### 64 R. Haynes


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **The Quality in 3D Acquisition of Cultural Heritage Assets: Challenges and Risks**

Stamatis Chatzistamatis , George E.Tsekouras , and Christos-Nikolaos Anagnostopoulos(B)

Cultural Technology and Communication Department, University of the Aegean, 81100 Mytilene, Greece canag@aegean.gr

**Abstract.** Cultural heritage is an integral part of history and in order to better preserve, manage and highlight our cultural heritage it is necessary to first proceed to digitize and document it. Geometric documentation of a cultural heritage asset is the process of collecting, processing, rendering, and recording data to determine the location and actual form, shape, and size of a monument in three-dimensional space at a given time. This chapter presents the modern methods and technologies for the 3D acquisition of cultural heritage assets and discusses the related challenges and risks. Moreover, we are presenting the possible quality aspects observed in the fieldwork that arise from the equipment, the environmental conditions that prevail during the surveying, the object itself, the personnel expertise and the stakeholder conditions.

**Keywords:** Digital Cultural Heritage · 3D Acquisition · Quality and Complexity Aspects

### **1 Introduction**

Digitization is now a fundamental step in the study and preservation of our cultural heritage (CH). The most common digitization product of tangible cultural assets is a 3D model, while the primary phase is the acquisition of the asset, whether it is a very small one or an entire archaeological site. The need to preserve a CH asset is directly linked to its documentation both for reasons related to its study and for reasons related to its promotion. One of the most important tasks at the stage of analyzing and documenting a monument is its digitization, i.e., the systematic recording and visualization of the elements that define the geometric form and the position in space of the individual parts of the monument at a given moment in time.

In the strictly scientific sense of the term, digitization is a very specific process of the geometric representation of a real object, through observations made by measuring systems, using some suitable computational model which will return, in addition to the representation itself, and the possibility of assessing the accuracy and precision of the deliverable. This representation of the object being captured must end up in a medium of communication (eg a 3D model) to be usable by all the stakeholders.

© The Author(s) 2023 M. Ioannides and P. Patias (Eds.): 3D Research Challenges in Cultural Heritage III, LNCS 13125, pp. 65–76, 2023. https://doi.org/10.1007/978-3-031-35593-6\_4

Digitization, as a process of representing a real object, depends on the accuracy and precision in determining the geometric elements of the object in the single threedimensional space and the density (or spatial resolution) of the points that will be represented to reconstruct its geometry. The combination of these three parameters (accuracy, precision and density) that determine the digitization, determines the quality of the representation. The type of surveying can be determined by the technique to be applied. The choice of the appropriate technology depends on a number of factors, which must be taken into account in choosing the procedure to follow. These factors concern the characteristics of the object to be surveyed, such as its shape, size, geometric complexity, surrounding space, cost and available time, and the possibility of applying new technology to the process [1]. The traditional techniques that are applied for the surveying of objects or large-scale sites are the topographic and the photogrammetric method.

The development of technology in the imaging sciences in recent years has introduced new techniques for measuring space and objects. The introduction of laser technology at the beginning of the 60s and the understanding of the advantages of the characteristics of the beam it uses, such as monochromaticity, good alignment and the ease of shaping the laser beam, led from a very early stage to its application in measurement technologies and imaging. In recent years, the surveying of cultural heritage sites with 3D laser scanners has been added to classic methods. This chapter presents the modern methods and technologies for the 3D acquisition of cultural heritage assets and addresses the factors observed in the fieldwork that affect the quality of the final product.

### **2 Technologies for 3D Acquisition**

Scanning instruments are distinguished, depending on their nature, into contact and noncontact. Contact scanners measure the object through physical contact, giving relative coordinates to an embedded recording system. These systems are not used usually in cultural heritage objects but mainly in industrial applications. The reason they are not used on cultural heritage objects is their main disadvantage, that is, they require physical contact with the object, which can lead to the change or even the destruction of the object. Heritage items are very sensitive and therefore this method is not suggested.

On the contrary, non-contact instruments and methods are suitable for acquiring cultural heritage assets, as long as they do not emit a beam that can harm the asset. Figure 1 depicts the taxonomy of non-contact instruments used today to acquire cultural heritage objects. Choosing the correct method depends on factors, most importantly the size and complexity of the object. The authors in [1] clarify the term complexity of an object and argue that choosing the correct method is based on the complexity of the object and the complexity of the whole process. Such factors that influence the selection of the correct method are the availability of equipment, the expertise of the personnel and the budget. In the literature, and especially in large-scale digitization, we find the fusion or integration (see Fig. 2) of these methods [2–6].

**Fig. 1.** Taxonomy of technologies for 3D acquisition of cultural heritage assets.

#### **2.1 Photogrammetry Methods**

Photogrammetry is the art, science, and technology of extracting 3D information from images. The process involves taking overlapping images of an object, structure, or area and converting them into two- or three-dimensional digital models. The basic principle of photogrammetry is if the directions of the same objects photographed from two extremities of the measured base are known, their position can be located by the intersection of two rays to the same object.

There are two main types of photogrammetry: aerial and terrestrial or close-range. Aerial is the process of using an aircraft to capture aerial images that can be converted into a three-dimensional model or digitally mapped. The most common type of aerial photogrammetry is photography by a suitable camera mounted on an Unmanned Aerial Vehicle (UAV). UAVs have facilitated the safe aerial photography of hard-toreach objects or hard-to-reach areas where traditional surveying could be dangerous or impractical. Terrestrial or close-range photogrammetry is when images are captured using a hand-held or tripod-mounted camera. The result of this method is usually the construction of 3D models of smaller objects.

The main advantage of photogrammetry methods is that they are cheap, but the disadvantage is that require a lot of processing of the images taken to create the 3D model. However, the evolution of computing power along with the evolution of computer vision algorithms have led to the possibility of an accurate 3D model in just a few hours even for large-scale sites [7]. Along with acquiring images, it is necessary to acquire control points that will help to "stitch" the images and create the model to the correct scale.

**Fig. 2.** Point cloud delivered by integration of technologies in "DigiArc" project (https://www. digiarc.eu); UAV photogrammetry (top), Terrestrial Laser Scanning and Handheld Scanning (coat of arms).

### **2.2 Laser Scanners**

3D laser scanners are active imaging instruments that provide real-time coordinates of the object being captured in three dimensions. Their basic operating principle is that a laser beam is emitted and for each point in space its coordinates (XYZ) are recorded. At the same time, the scanner records the reflectivity of the point giving an intensity value (i). While scanning, modern scanners also collect images with their built-in camera to define the color and texture of the imaged surface.

**Fig. 3.** Terrestrial Z <sup>+</sup> <sup>F</sup><sup>1</sup> laser scanning in "DigiArc" project (https://www.digiarc.eu).

A scan can be completed in a few minutes and give us a detailed point cloud that forms and renders the geometry of the object in great detail. All laser scanners work via line of site. This means that on a typical project multiple scans need to be taken from different vantage points to ensure a complete dataset. A change in the position and orientation of multiple scans must be applied so that each point cloud uses a common homogeneous coordinate system of reference. This process is characterized as cloud alignment or registration. This is done by using artificial targets (most common are checkerboards or spheres) in the scans or by allowing enough overlap in the scans to register by recognizing common features (cloud-to-cloud).

<sup>1</sup> https://www.zofre.de/en/laser-scanners/3d-laser-scanner/z-F-imagerr-5016.

3D laser scanners are classified in airborne (known as LiDAR-Light Detection and Ranging), terrestrial (see Fig. 3) and mobile. Airborne scanners are equipped with sensors to measure the position, orientation and altitude of the aircraft during data collection, through inertial sensors [8]. Combining these measurements with distance data collected by the laser scanner produces a 3D point cloud representing the topography of the ground, similar to that generated by a terrestrial laser scanner. Mobile laser scanners are distinguished into those that are vehicle-based and those that are handheld or backpacked [9]. The latters find application in the acquisition of large archaeological sites where, due to the peculiarity of the terrain, the use of a vehicle is impossible.

Based on their operating principle, laser scanner systems are divided into distance measurement scanners and triangulation scanners. Scanners of the first category can measure distance in two ways: a) Time of Flight (Tof) where a laser beam is emitted at the object and the distance between the source and the reflected surface is calculated from the travel time of the signal between emission and reception, and b) Phase Comparison where the emitted beam is converted into a harmonic wave and the distance is calculated from the phase difference between the transmitted and received wave. Triangulation scanners operate under the principle where a transmitter sends the laser beam at a specified variable angle from the edge of the mechanical base to the object and a camera at the other end of the base locates the laser spot on the object. The three-dimensional position of the object is carried out by solving the triangle formed. Triangulation scanners are only applicable to close distances and small objects as the accuracy of the distance between instrument and object depend on the square of the distance.

#### **2.3 Handheld for Close-Range Acquisition**

These are portable, small-sized devices that scan small objects with high precision and use the principle of triangulation or the principle of structured light. At the same time, passive receivers capture color data of the object in order to create a more complete model. Scanning with handheld scanners is usually aided by marking on the object certain points with the help of self-adhesive reflective tapes (as with the use of artificial targets).

Structured light is a technology where an infrared emitter emits structured light, i.e. a pre-designed pixel pattern. This is non-visible light (near-infrared wavelength), which passes through a filter and is scattered into a semi-random but constant pattern of small dots projected into the environment in front of the sensor. The depth sensor receives the reflected pattern and calculates the shape and position of the object. The combination of depth information and color information from the camera is combined to create the point cloud.

#### **2.4 Supplementary Instruments**

Using control points in a survey can serve two purposes. The first is to provide a network of precise points so that point clouds can be successfully identified in a common coordinate system. Registration using cloud-to-cloud techniques can be very successful, but around a larger structure, for example, or where it is difficult to maintain a good overlap, control points are essential. It also provides certainty and, with some redundancy in the network of points, an estimate of overall accuracy. The second possible purpose of control points is to relate the network of points for the object being surveyed to either a wider coordinate system or a known coordinate system and altitude data.

A very accurate coordinate system can provide the basis for long-term tracking or structural analysis. For large-scale sites, a network of points associated with a known coordinate system can also provide tracking service over a wider area. The georeferencing of the surveyed monuments can lead to studies of their spatial relationship and a more extensive archaeological analysis. An important aspect of the data collection process is defining a network of control points to which all other field metric data can be referenced. The specifications set are to require full georeferencing in a known coordinate network throughout the site. Also, control points are very likely to be needed locally to identify scans, for example, of the interior and exterior of a structure.

**Fig. 4.** Measurements with Total Station (left) and GNSS (right) in "DigiArc" project (https:// www.digiarc.eu).

To create a network of high-precision control points, measurements are required mainly with two instruments: a Global Navigation Satellite System (GNSS) that works with the principle of Real Time Kinematic (RTK) method and a geodetic station (Total station) (see Fig. 4). RTK is a kinematic determination, in which two receivers (base rover) are used, provided that there is communication between the receivers, which is carried out either with a UHF modem or a GSM modem. The mobile receiver continuously receives corrections from the base, which has known X,Y,Z coordinates, and uses them to resolve errors from the satellite imagery to achieve greater horizontal and altitude accuracy at the measurement locations. With this method, an accuracy of the centimeter can be achieved almost in real-time. A Total Station is a topographic instrument capable of measuring angles and distances. It combines in a single device, a digital theodolite with electronic distance measurement (EDM) to measure both vertical and horizontal angles and the slope distance from the instrument at a specific point, and an onboard computer for data collection and performing triangulation calculations. It allows its user to collect all the measurements, especially control points in remote areas, necessary for a topographic survey using digital technology.

### **3 Factors Affecting the Quality of 3D Acquisition**

The quality of surveying and their final product depends on a range of parameters that do not only concern the accuracy of the instrument and the measurements. Factors such as the environment, the characteristics of the object to be scanned, and the methodology followed in each surveying process can affect the quality of the final result of the survey [10]. These factors can be distinguished (see Fig. 5) into those related to the operation of the instrument, the form and nature of the object, the environmental effects and the choice of methodology during the measurement process [11, 12].

**Fig. 5.** Taxonomy of factors affecting the quality of the 3D acquisition of cultural heritage assets.

#### **3.1 Instrumental**

These factors depend on the type of instrument and mainly affect the accuracy of the calculation of the measurements of distances and angles of the laser beam. The dimensions of the laser beam spot greatly affect the resolution of the generated point cloud and the determination of the position of the points. The greater its width, the more chances there are to create discrepancies in locating the coordinates of the points to be captured. The shape of the laser beam affects in the same way, which tends to grow at a distance from the instrument [13] and presents a dispersion effect. This factor is known as "beam divergence" and it is a specification provided by the instrument's manufacturer. When surveying, the specification of beam divergence must be considered when positioning the instrument (see Fig. 6). New instruments tend to allow multiple measurements of a point leading to better measurement.

Another factor that can affect the quality of the point cloud and is directly related to the laser beam is the edge effect. This factor is observed on the edges of the captured objects. When the laser beam reaches a point that is at the edge of the desired object, only a part of it is reflected and returned to the scanner. The remainder may be reflected from a surface behind the edge, if present, from an adjacent point, or not at all. Thus,

**Fig. 6.** Beam Divergence.

signals from different areas return to the scanner. Defining the position of the point is done by averaging these measurements, causing the edge of the object to appear in the wrong position. The error, in this case, can range from fractions of a millimeter to a few centimeters. The latest updates of software packages that accompany the instruments can correct some of the edge effect mismeasurements.

Most scanners use mirrors to deflect the laser beam in a specific direction. Any deviations created in the angle created by the emitted beam and the mirror surface can lead to the calculation of incorrect point coordinates. Errors related to the geometric stability of the scanner axes also lead to the definition of the wrong point position. The geometry of the scanner is described by three axes, the vertical, the horizontal and the alignment axis. The vertical axis is the axis of rotation of the scanning head, the horizontal, the axis of rotation of the mirrors and the third is the axis defined by the laser beam. These axes if are not perfectly aligned, errors may occur in the measurements of the angles formed by the alignment axis and the horizontal (alignment error) and by the horizontal and vertical (horizontal axis error). The instrument has a specific time of operation before calibration is needed. Referring to the manufacturer's specifications can indicate the time that the instrument needs calibration.

### **3.2 Object Materials and Geometry**

The different characteristics of the surface of the objects to be acquired can affect the accuracy of the point distance measurements, e.g. from the scanner, which depends on the reflection of the laser beam. Depending on the material, color, roughness, temperature and humidity of the surfaces of the object being acquired, the reflected beam shows variations. For example, on surfaces characterized by high reflectivities, such as white, distance measurements are more reliable than those with less reflectivity, such as black. This happens because white surfaces absorb less of the visible radiation and thus the intensity of the reflected ray is high, in contrast to black surfaces, which absorb more of the radiation, and the intensity is weak.

If the reflectivity of the object is too high or too low, measurements may not be made or show unacceptable accuracy. Corresponding error results also arise in cases of translucent material surfaces, in which the refraction of the laser beam and its reflection on the surface itself is observed (see Fig. 7). Refraction is the change in direction of propagation of a wave when the wave passes from one medium into another and changes its speed. Reflection of light occurs when the waves encounter a surface or other boundary that does not absorb the energy of the radiation and bounces the waves away from the surface.

**Fig. 7.** Reflection and Refraction effect.

Other object characteristics that affect the results of point distance measurements are size, curvature, and orientation. A typical example is when we have surfaces where the effects of refraction and reflection are observed, leading to incorrect measurements [14]. Optical properties of materials such as glass windows or water (e.g. in a fountain) are a common aspect that affects the quality of the distance measurement, due to the refraction effect. The surface reflection on a mirror of a laser beam normally causes reflected beams in many directions. A solution is to cover these surfaces during the surveying process.

#### **3.3 Environmental Conditions**

Environmental factors such as atmospheric temperature, pressure, rain, humidity, and vibrations affect the accuracy of measurements. The presence of wind can cause the presence of dust, which can lead to incorrect measurements whether it is a laser scanner or photogrammetric techniques. For example, temperature changes during the process of a survey affect the speed of the laser beam, which is highly dependent on the density of the air. This results in reducing the quality of its measurement. Also, possible radiation interference from external light sources, such as a projector or sunlight, can alter the measurement results. In the case of aerial photogrammetry, the correct time of day must be chosen so that there is adequate lighting, as shadows greatly affect the photography, and thus the final result. Another factor that affects the measurement results is the movements of the instrument in relation to the object being captured. Throughout the scan, the scanner must be stable to avoid erroneous data collection. Photographing an object with a drone can be affected by the presence of wind. All technologies are accompanied by technical specifications concerning environmental conditions, which must be taken into account and strictly observed during the surveying process.

### **3.4 Human Factor**

Even if the three previous categories are considered, the human factor always comes into the "equation" of quality. The personnel expertise and therefore the methodology they will follow can affect the quality of the deliverables. These factors are related to the methodology chosen either in the phase of defining the specifications during the planning process or the surveying process and in the processing of the generated data. Such factors can arise from a poor choice of settings such as the desired resolution of the sampling and the distance from the object to be acquired or can result from the wrong approach to georeferencing the point cloud. Thus, during the planning of a survey, critical questions must be answered, such as choosing the correct technology or a combination of technologies. For the case of scanning with a scanner, the correct resolution, the use or not and the correct placement of artificial targets, the overlap or not with a percentage of at least 30% must be taken into account for the correct registration of the point clouds with the cloud-to-cloud method and the placement of control points to control the quality of the produced product. In the case of photogrammetry, the correct choice of camera and lens, the desired resolution of the images, the flight plan, the correct orientation of the camera (oblique, nadir), the overlap of the photos at least 80% and the use and the placement of control points to assist further processing and control the quality of the product produced [15].

### **4 Conclusions**

This chapter presents the latest technologies and systems available for the 3D acquisition of tangible cultural heritage assets. Each of these technologies has advantages and disadvantages, while many times in the surveys, especially of large-scale sites, a combination of these technologies is required. Choosing the correct method for a survey depends on factors such as equipment availability, personnel expertise, budget, and item complexity. However, despite the development of technology, many factors can affect the quality of the acquisition and thus of the final product, which must be taken into account during the planning and the process of surveying, so that the metric information is accurate. Along with the evolution of the instruments, the accompanying software for processing the received data is also evolving. These software packages, if updated, can algorithmically reduce many of these factors that affect the quality.

**Acknowledgments.** The project "DigiArc - The Digitization, Preservation, Protection and Presentation of Medieval Cultural heritage in the island region of the Aegean and Cyprus" (https:// www.digiarc.eu) is co-funded up to 85% by the European Regional Development Fund and up to 15% by national resources of Greece and Cyprus, under the framework of Interreg VA Greece Cyprus Programme 2014–2020.

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **How to Measure Quality Models? Digitization into Informative Models Re-use**

R. Brumana(B)

Department of Architecture, Built Environment and Construction Engineering, DABClab-GIcarus, Politecnico di Milano, Via Ponzio 31, 20133 Milan, Italy raffaella.brumana@polimi.it

**Abstract.** 3D models from passive muted subjects, often used in the books and in preservation design reports as powerful images dense of contents, have nowadays the opportunity to become 'live gears' leveraging knowledge, interpretation, and management into preservation objectives till to better-informed fruition. To this aim, we need to build up reliable and re-usable 3D Quality models. How to shift from a 3D model toward a 3D quality model?

This contribution intends to focus on the parameters defining a 3D Quality model catching the heritage complexity with its components in a holistic methodological and practical vision. A radar chart has been used to manage all the parameters. First of all, Geometry describes a quality model: parameters for data acquisition, on-site surveying, and model processing to obtain 2D-3D Geometry quality are defined. The concept of scale associated with measurable parameters defining the Grade of Accuracy is proposed and applied to the surveying and to the 3D models. 3D models can be considered tools to decode the complexity of cultural heritage made by the different transformations across the centuries, anthropicnatural hazards, climate change threats and events (such as earthquakes, fires, wars). Thus, Geometry is not enough to describe such complexity; it represents the first step. Materials and Construction technologies analysis is the second pillar qualifying a quality model. The connection with the indirect data source (i.e., historical reports and archives documents), is the third pillar to be reconnected to the Geometry and Material analysis in the quality definition. HBIM represents a multidisciplinary environment to convey the information related to geometry and models. Furtherly, several parameters are identified to describe the quality of informative models, as in the case of Object Libraries and Building archeology progressively feeding such models. BIM Level of Developments (phases) and Level of Geometry (contents, not scale!) have been adapted to the HBIM, introducing digitization, surveying, and HBIM modeling into the preservation process. Finally, a quality model is defined by the capability to be re-used circulating Information and Models among the end-users as in the case of informed VR/AR through CDE and XR platforms.

**Keywords:** 3D Quality models · Complexity · HBIM · Heritage Building Information Modelling · construction systems · monitoring · Scan-to-HBIM · modeling · generative modeling · Scale · GOA · LOD · LOG · building archaeology · 3D volume stratigraphy · CDE · VR/AR/MR · XR

### **1 Introduction**

The digitization processes need a systemic approach, generating holistic multi-faceted models: 'live' digital twins continuously growing within a complex built environment, under the hazards, natural anthropic pressures as climate change, earthquakes and at the same time inheriting the complexity of the past.

It can be helpful to remember the deep meaning of the term "complex" from Latin "*complector"*, hold tightly, and, metaphorically, gird, embrace, and comprehend.

Edgar Morin "From the Concept of System to the Paradigm of Complexity" [1]: "*In order to make sense of the concept of system, we must postulate a new, nonholistic principle of knowledge. However, this will be possible only if we conceive of systems in general and generic or generative terms- that is, in terms of a paradigm. A paradigm being defined here as the set of fundamental relations of association and/or opposition among a restricted number of master notions-relations that command or control all thoughts, discourses, and theories)".*

*"… Finally, there is the impossibility of understanding a complex mental structure from the wall of a reductive or simplifying mental structure (while the inverse is possible)"*. Morin asserts that to know is to separate, to analyze and reconnect to synthesize or make complex, to '*complexize*'.

What we can derive, among others, from this assertion is that complexity need to be intended as a circular process, where we can separate and filter some aspects to understand some aspects: but they require to be reconnected at a further step; as well as we can't simplify at the beginning but at the end.

### **2 Complexity and Quality: #Parameters into a Multiple Variable Radar Chart**

3D quality models can help in driving complexity: moving from their 'limbo', between media, society of image and big data circulation thanks to the technological potential offered by digitization; moving from a substantial under-use toward their re-use supporting preservation analysis and activities during the extended life cycle maintenance, and data transfer toward a better communication rising awareness through informed contents.

To become subjects of knowledge, they need a qualitative leap addressed to content creation-content use-content transferring-content re-use.

Objectives of 3D quality models can be summarized as follows:


Different parameters (#parameters) describing the quality of surveying till to the data management and re-use of the informative system (HBIM) are hereafter identified. This paper is part of the result of VIGIE expert member's research on the "Study on quality in 3D digitization of tangible cultural heritage" [2]: a radar chart has been set up by the coordinator to manage all the parameters identified by the different contributions to the multiple tasks [3].

The proposed radar quality diagram is here applied in the following paragraphs to one of the selected success story case studies (the Basilica di Santa Maria di Collemaggio), considering the multi-variables crossing surveying, processing, and data transfer, when facing the heritage complexity. Such parameters are hereafter described to measure the contents enriching the quality model: geometry, model scales, grade of accuracy, materials, construction techniques, informative systems, content transfer, building archeology, level of development and level of geometry, Long Life Cycle management through Common Data Environment.

### **2.1 A Success Story into the Radar Chart: The Basilica di Santa Maria di Collemaggio (L'Aquila, Earthquake 2009) Reopened to the Public in 2018, Awarded by Europa Nostra 2020**

The project 'Re-start from Collemaggio' is here considered to derive the lessons learned crossing the digitization process finalized to the preservation after the earthquake, helping to define the concept of quality and complexity when dealing with a challenging context and a stratified architectural document. According to the stakeholder requirement (the Superintendence Office), the generation of an advanced HBIM based on reliable surveys to address decision-making by the different actors involved in the preservation process has been undertaken [4]. This case study has been used to describe the parameters proposed to measure the quality and results obtained. The Italian multinational energy company (EniServizi), after the earthquake occurred at L'Aquila in April 2009 (Fig. 1), launched the project "Re-start from Collemaggio" funding around 16 million Euro to preserve the damaged Basilica. HBIM model of the Basilica of Santa Maria di Collemaggio supported the preservation design project and the restoration after the earthquake (L'Aquila, 2009).

The Basilica of Santa Maria di Collemaggio received the European Heritage Award/Europa Nostra Award (2020, May 07, | Conservation | Italy | L'Aquila), [4]. The Basilica attracts about 30,000 people for the Forgiveness Feast Day (Festa della Perdonanza) on 28-29th August, established by Pope Celestino V (before the Jubilee - 'Giubileum' - set up by Pope Bonifacio VIII recurring in the world every 25 years). The Feast is celebrated with the procession ceremony to the Holy Door of the Basilica: it represents an extraordinary unicum of tangible and intangible values. The design and the preservation have been intended to transfer them to the future. The Basilica successfully reopened to the public in December 2017.

**Fig. 1.** The area struck by the earthquake at L'Aquila and the damaged Basilica di Collemaggio

### **3 How to Define the User Needs? #***Scale* **as a Parameter**

User needs requirements are fragmented depending on many factors and circumstances, including funding availability. The decision to choose a high-medium-low level of precision and accuracy depends on many factors. It requires to be coherent with the state of art. It needs parameters to address the surveying since the starting phases. But sometimes, even if we are in a complex contest needing a high level of precision, due to the state of the art, the decisions are not or can't be coherent and consistent.

Thus the decision among high-medium-low level of details and contents requires to be transferred to the users with transparency to avoid mismatches during the use and reuse. Models, once generated, need to be circulated and re-used, sharing all the knowledge and Information. Information are progressively growing within the multidisciplinary preservation process, supporting sustainable interventions.

Models are expected to support users, experts and non-experts to extract content. Models need to become a live instrument to accompany all efforts to transmit cultural heritage to future generations, throughout the preservation of all its materials together with construction technologies, considered in their physicality, a vehicle of immaterial, tangible and intangible contents.

Given that object models are generated by different professionals for different uses (with different required precisions and details), this paper intends to identify the parameters describing a quality model rising situational awareness among the experts and operators in the re-use of such models.

Thus, the adoption of simplified models needs to be declared in order to avoid faulty inferences when comparing or re-using libraries of objects modeled with different precision and accuracy.

### **3.1 How to Measure Quality Model? Surveying to Digitization: Scale as a Parameter to Check Quality Control and to Re-use Models**

**#Scale parameter.** A feasible model is based on a reliable data acquisition starting from the on-site surveys! Model scale can be a parameter to measure the quality: to this aim the HBIM Grade of Accuracy (GOA) is here inherited and associated to the different scales; the concept of scale can be considered as a switch to guarantee the user needs freedom of choosing the 3D quality model most appropriate to their different objectives; at the same time, to ensure a coherent use of such models being the users aware of the low-medium-high level of accuracy (scale) of the data collected and processed.

A 3D quality content model starts from the proper identification of the instruments and methods to acquire and process the data. The scale concept can be considered a parameter for selecting the proper instruments (precision) and reliable processing to obtain reliable results (accuracy). The concept of scale is commonly adopted by professionals (i.e., architects, engineers), even if qualitatively. The higher is the scale, the higher is the level of detail. But the scale concept has a robust meaning, mostly unknown: it allows to handle measurable parameters under a qualitative and quantitative point of view. Scale requirements have been adopted for many decades in the tenders' specifications at a worldwide level. As it is in the case of the generation of technical maps at the cartographic scale: the scale 1:25.000 has been adopted for territorial maps (i.e., IGM maps by the Istituto Geografico Militare/Geographic Military Institute), the scale 1:10.000 and 1:5000 has been adopted for the technical regional-scale maps, 1:2000– 1:1000–1:500 for the municipality technical maps. The specifications are standardized in the aero-photogrammetric process in the cartographic domain and inherited by the architectural surveying specifications (i.e., 1:100–1:50–1:20–1:10–1:1).

### **3.2 3D Quality Content Models: #scale as a Parameter to Select the Proper Instruments and to Process Reliable Data (Precision and Accuracy): Specifications, Scales of Representation, Minimum Detail, and Tolerance Parameters**

**#Scales parameters: minimum detail and tolerance**. Several specifications have been fixed to guarantee the required accuracy in the map generation since the starting phase of the data acquisition through the data processing. The two parameters conventionally adopted to measure the scale are (i) the minimum level of detail (the so-called Graphic Error, G.E. fixed = 0,2mm for all the scales) and (ii) the related Tolerance (T) = 2 ÷ 3 G.E (Table 1). The choice of the 'scale' depends on the survey's objective and on the use of the final product: such parameters allow to identify the proper scale, the proper instruments, and methods of processing, to validate the output and to re-use the output (Fig. 2).

For example, at 1:50 scale, the G.E. value is 1 cm, and 2,5 cm the tolerance (T). In the case of photogrammetric restitution (as rectified images or orthophoto), the minimum detail of the data acquisition is fixed at half the G.E. value at the given scale, with a restrictive requirement in order to consider the processing resampling.

**# 3D Model Parameter (Grade of Accuracy).** The concept of Grade of Accuracy (GOA) has been related to the scales (GOA10, GOA20, GOA50, GOA100) and proposed to describe also the 3D object models quality [4], thus defining the minimum detail and tolerance values in the modeling generation (Fig. 2).

The Grade of Accuracy of the different scales is associated with G.E and T parameters [5]. The identification of the surveying instruments and restitution scale needs to be coherent with the correspondent parameters to make them reliable for the different uses and user needs (Table 2): 3D Geometry Scale and Image Scale are defined through the given parameters (Fig. 3). The precision of the surveying, as in the case of cloud scans acquired with TLS (i.e., laser scanner FARO Focus 3D), allowed the extraction of vertical and horizontal profiles with high accuracy (2 ÷ 5 mm), thus coherent to the scales GOA20- GOA50. Even if we assume a 1:50 model scale, the precision related to the surveying method, as profiles for geometric analysis or out of plumbs analysis, is

**Fig. 2.** Scale as a parameter of surveying, 2D drawings, 3D models, and HBIM: it is inherited from the technical cartographic maps specifications, here applied to the 3D model objects

relative to 1:20/1:10, and it will joke in favor of the object model that will be generated. Viceversa, Mobile Mapping Systems (as ZEB GEOSLAM), very useful for massive data acquisition, have a precision coherent with a scale range 1:100–1:200.

Using the conventional definition of Graphic Error fixes the smallest detail that can be represented at a given scale (G.E. = 0,2 mm), and of the related Tolerance (T = 2 ÷ 3 G.E. value), we can apply such values to all the different model scales obtaining indicators of precision and of interchange in the usability and data sharing among actors and experts. As an example, the mosaic floor of the Basilica of San Marcus has been realized at a 1:1 scale to support the maintenance intervention on the single 'tessera' maintaining the 3D waved shape [6].

**Table 1.** Scale as a parameter to measure surveying, 2D drawings, 3D models, and HBIM: it is inherited from the technical cartographic maps specifications and here applied to the 3D model objects associating the Grade of Accuracy to the scale concept.


**Fig. 3.** Geometry and image scale are parameters of 3D quality models: their components can be acquired and processed at different scales. Courtesy of ABClab-GIcarus (F. Banfi, R. Brumana)

The proposal is to apply the surveying specification concept, traditionally linked to the representation scales, also to the 3D quality models, and to the HBIM model objects. GOAs are proposed introducing Levels of Accuracy linked to the scale measurability, inherited from the BIM logic and adapted to the heritage preservation purposes, referring to the richness of model details. In the modeling generations, GOA20 model accuracy has been adopted for vaults components in the Basilica di Collemaggio, GOA10 model for the octagonal columns, etc.


**Table 2.** The results obtained by the surveying: instruments precision, methods, and accuracy as parameters for reliable results of the 2D-3D output (restitutions of plans, sections and fronts)

### **4 Beyond Geometry Through Geometry: # Parameters to Manage Informative Enriched 3D Quality Models Contents Transferring**

**# 3D Geometries.** Geometries can be intended as the result of the wealth of details ensured conjugating massive data acquisition with high accuracy; advanced processing methodologies supporting the representation of the complex geometries are capable of shaping the uniqueness of the architectural complex and its components.

As explained in the previous paragraph, it requires an advancement on the specifications to identify the proper model accuracy, scale, and richness of details to decode, analyze and represent heritage complexity as a whole and, at the same time, in its components (i.e., structural elements, vaulted systems, columns, walls, till to the decorations, frescoes and stuccos). 3D models, once generated, start circulating to be re-used and shared by common data environments and platforms: thus, they need to be validated not just on the surveying side but also on the modeling and processing side. AVS available tools can be easily used.

But precision is not enough!

Models can be enriched by different parameters going beyond the geometry but intrinsically connected to the geometry. Hereafter, a list of parameters contributes to 3D quality models till to the informative system enrichment.

**# Generative Modeling, HBIM Enabling Models.** The generative model process is a crucial parameter to get the Object models at the required scale starting from reliable surveys. Proper procedures of NURBS generative modeling contribute to guarantying the interoperability among pure modelers and parametric modelers, avoiding simplifications derived from not fully enabling models, enhancing capacity building, and upskilling among users and modelers (Fig. 4), [7].

The quality of a HBIM model depends on many parameters: the capacity of the model to embed geometry, images, radiometric information. It requires proper processing to get NURBS based Material (and Decay) Mapping as illustrated in the following paragraph, in order to support data management, and related metric computation (i.e., Area, Volume) together with all the information.

### **4.1 Not Just a Matter of Measuring: Geometries Embodying Materials and Construction Techniques Through HBIM**

Not just a matter of measuring: a quality model cannot be defined just by its geometric accuracy. We have to add parameters that come from the material analysis supported by high-resolution photogrammetric data processing: as in the case of rectified images and orthophoto to be associated to the 3D models: geometries embodying materials and construction techniques contribute to defining enriched 3D model. And, vice versa, geometries and interpretation coming from the materials and construction techniques can improve the 3D Quality Model.

**#Informative Quality Models (as HBIM, GIS and DBS).** Model objects can be progressively enriched by the Information, within the informative models; they support different BIM uses (as the preservation design project, the conservation plans, the materials, decay analysis, construction techniques, structural behavior, Finite Element Analysis, energy performances) and decision-making processacross the life cycle management (Fig. 5). Parameters define which Information are required to be detected and described at the HBIM Information level are fixed by the conservation experts.

**#Mesh Based Surfaces.** We have many meshes circulating and coming from automatic processes by the users, architects, and engineers, thanks to common tools (as in the case of photogrammetric textured models) that could be very useful if used to enhance the content interpretation. In general, they are destined to remain at a passive level, used as background in the BIM tools. Enabling 3D meshes to embed all the properties Information and connections to external DB, as for costs computations, means to turn them into Object Models within HBIM.

**#Shifting from Surfaces to HBIM Object Models.** We need a shift from the surfaces to the object models. Surfaces are the first sentinel that can be easily surveyed, but we have to take into account that the surfaces are just a face of solid objects components of the architectural heritage or archeological site. We need to go behind the surfaces, to understand the Object that has been built and transformed across the centuries with all the decay and issues in order to manage quality models and not just the skin surface! 3D Object shifting means recognizing the 'habeas corpus' of each component as an identity card. A wealth of knowledge to valorize the uniqueness of all the architectural components: each Object description is a 'unicum' made by geometries integrated by materials and construction techniques.

**#Mapping Materials, Construction Techniques, Decay.** Knowledge of materials and construction techniques - including the techniques of craftsmen transferred from generation to generation across the centuries – are parameters enriching models quality.Material analysis, as well as decay analysis, can be mapped generating HBIM Objects to be managed with external Databases that can be added to the Information by the preservation experts and professionals [8]. 3D HBIM quality models can conjugate geometry with surface texturing (Fig. 5). Moreover, as demonstrated by many kinds of research, the geometry is influenced by the construction techniques as in the vaulted systems where the shape is influenced by the arrangements [9].

**# NDT.** Non Destructive Techniques (i.e., IRT Infrared thermal cameras, NIR Near-Infrared cameras, Hyperspectral data, geo-radar, sonic analysis). NDT can be useful to better understand - behind the surface plaster - the arrangement of different stratigraphic units as well as chimney channels [10]. Here applied to the vaults with a passive method (a parameter to be considered), thus with fewer results than the rooms' active heating, unfortunately here not possible for the contest.

**#Correlating Material Analysis to the 3D Arrangement.** 3D content models capable of decoding the complexity of behavior that comes from the stratified transformation phases impacted by hazards in a fragile context.

The out plumbs after the earthquake, the anomalies, the irregularities, the reading of the different stratigraphic units, the relations among the different components (as columns-arches-naves) allowed us to better understand the disconnections and the damages preventing disjoining with proper connections in the preservation plan.

**Fig. 4.** The quality of a model depends on the generative model process to get the required scale starting from reliable surveying. Material and Decay Mapping as a parameter of quality for HBIM data management: NURBS model supports the HBIM Material mapping process. Courtesy of ABClab-GIcarus (F. Banfi, C. Stanga, R. Brumana)

**Fig. 5.** Materials, construction techniques, and IRT analysis, together with the decay analysis, embedded within the HBIM, supporting the conservation plan Work Breakdown Structure linked to the Object models. 3D volume stratigraphy was not integrated into the HBIM. Courtesy of ABClab-GIcarus (F. Bafi, D. Oreni, L. Cantini, R. Brumana)

**#Stereotomy.** High-quality models digitization is returning to us the history of constructive wisdom and the art of cutting stones in the space (stereotomy): complex shapes not belonging to the simplified typological classifications cited in the historical manuals, give back us masonry techniques with a multiplicity of results. It is the case of 'Trompe' shaping (Guarino Guarini) or framed vaults across Europe with the art to build in the space complex models saving wood centerings fasting the construction process with a unique variety of solutions [11].

HBIM models of all the columns embody the out of plumbs together with the 3D shaped ashlars elements with the hands-on survey of all the damaged ashlars (Fig.6): it was hypothesized an average of 35% of damaged ashlars (yellow) to be necessarily substituted during the intervention, in the consciousness that this percentage would undoubtedly increase during the works. But it contributed to hypothesize the 'scuci e cuci' to reduce the replacement to the damaged elements.

**#HBIM Object Libraries.** Each single informative object model – considered as a node - can contribute to the generation of libraries of the objects across space and centuries; these nodes can be compared highlighting permanence and mutations of the construction techniques, highlighting the unsuspected richness of the vault systems generating HBIM based inventory [12]. Criteria and tools to catalogue brick-masonry vaults through informative geographic models contribute to describing the specificity and richness of the architectural components [13], fostering the history of historical construction techniques [14].

To correctly share a model, such as a vault HBIM object, one needs to know: which was the commitment purpose and thus the required scale; how it has been surveyed; if the model has been generated in a congruent way, that depends on the object level of complexity and the type of geometrical survey to generate an Object Model with the proper accuracy. Such Information must be included as metadata within the Level of Information, of each object model in order to manage the different Levels of Geometry within the different phases of development of the preservation process and the correct re-use in the data sharing process in the cloud (i.e., throughout Object Libraries).

The 3D object model of the main façade integrated geometry from laser scanning and material documentation by mean of the orthophoto from photogrammetric surveying at the GOA20 scale to read DEM morphology after the earthquake and GOA5 to read the different marble finishing in the different restoration phases in the past connected to different skilled workers capacities (Fig. 7).

**#HBIM LOGs-LODs (Level of Geometry and Level of Development):** These parameters cross the different phases of the preservation actions and plan by mean of the HBIM, as explained in the following paragraph.

**#HBIM WorkBreakdown Structure:** HBIM enabled models allows the professionals to connect the WBS plan of activities to the information and models, supporting the metric and costs computations planning all the activities from the preliminary phases to the executive phases.

### **4.2 Structural Health Monitoring: #Finite Element Analysis Complex Models as a Parameter of Quality (BIM-to-FEA)**

**#Structural Health Monitoring (BIM-to-FEA).** The importance of taking into account all the components within the overall architectural heritage, and their own behaviors, taking in account the materials construction techniques, within Structural HealthMonitoring with their characteristics has been illustrated in many kinds of research: it contributed to understanding failures due to long term behavior of heavy structures [15].

In the case of the Basilica di Collemaggio the importance of field failures analysis of the Basilica S. Maria di Collemaggio has been highlighted [16]. Starting from the surveying and on site-analysis, a Finite Element Analysis has been realized using the HBIM Objects model (Fig. 8).

**Fig. 6.** HBIM of the octagonal columns, the stereotomic analysis of the ashlars embedded in the HBIM, and the restoration with the final result. Courtesy of ABClab-GIcarus (F. Banfi, D. Oreni, R. Brumana)

**Fig. 7.** HBIM object libraries: a growing open catalog made by all the richness and specificity of the single object model: here the façade concept, the decorations, and the richness of the Rose windows. Courtesy of ABClab-GIcarus (F. Banfi, R. Brumana)

The BIM-to-FEA process has been carried out to get the interoperability and model transformation for the Finite Element Analysis [17]; the model has been obtained taking into account all the structural components as in the case of the HBIM façades objects, the vaults, the HBIM octagonal column with the stone ashlar data and arches, together with the Information coming from the HBIM-DB of the material and decay analysis obtaining the simulations models of the structural behavior.

It contributed to addressing the identification of the design solution for the replacement of the crashed dome and roof with the ancient wooden solutions. Different GOAS has been adopted in the processing tools by the structural engineering research team.

**Fig. 8.** Structural Health Monitoring: Finite Element Analysis complex models as a parameter of quality (HBIM-to-FEA). Courtesy of ABClab-GIcarus (F. Banfi, A. Franchi, R. Brumana)

### **4.3 3D Quality Models Enriched by Building Archeology: HBIM 3D Volume Stratigraphic Units and Levels of Uncertainties**

**#Building Archaeology.** Stratigraphic Units detection and 3D volume stratigraphy as a parameter of quality models: 3D volume stratigraphy and building archeology taking into account the relations among the Stratigraphic Units represent a further parameter to describe the quality level of a HBIM, as shown in the Fig. 9 respect the level of Fig. 5. The methodological approach of the so-called building archaeology [18, 19] helps to clarify not just the recognition of the stratigraphic units [20] starting from the material analysis and the decay mapping of the surfaces but also the interdependencies and relations among the different Stratigraphic Units (Fig. 10). As an example 3 different relationships generally symbolized are described as follows:


A shift of the 3D quality model toward volume representation represents a further level of degree of complexity that the Building archaeology informative modeling turned into 3D volume stratigraphy can contribute to decoding [21]. It can be applied to the single components, as in the case of facades or vaulted systems [22] to better investigate the result of 3D geometry influenced by the construction techniques and/or the transformation phases. The Scan-to-BIM process has been extended to Building Archaeology informative model to transfer these 3D models augmented by the Information into VR experiences [23]. The methodology has been applied to the 3D stratigraphy of the St. Francesco Church in Arquata del Tronto components (walls, roof, ceiling, flooring), which could be enriched by NDT analysis. Some components of the church (i.e., the façade) were deeper analyzed thanks to Building Archaeology: the Building Archaeology HBIM of the façade embeds the DB on materials and construction techniques, which shows the provenance of the Information in order to make aware of the reliability of the collected data that could come from direct (i.e., on-site observation) or indirect (i.e., archival document) sources. The material and decay DB related to the HBIM was realized according to the Building Archaeology, and each item has its stratigraphic Information, picture, texture drawing, and on-site and historical documentation.. To this aim, a Building Archeology property model devoted to managing the properties related to the 3D Volume stratigraphy have been defined, including a field where to highlight the level of Uncertainty of the relationships among the different transformation phases that occurred in the past, given that not all of them have been clarified.

Informative models can contribute to enriching and feeding VR/AR transferring contents to citizens. The case of AR project developed for the St. Francesco church represents the potentials of virtual-visual storytelling (VVS) allowing users to convey the informative models, including the construction phases, discover tangible and intangible values of the building through interactive virtual objects (IVO).

**Fig. 9.** Building Archaeology informative modeling: 3D volume stratigraphy (St. Francesco Church, Arquata del Tronto after the 2016 earthquake). Courtesy of ABClabGIcarus (F. Banfi, C.Stanga, R. Brumana)

**Fig. 10.** Toward informative models feeding AR/VR. Courtesy of ABClab-GIcarus (F. Banfi, C.Stanga)

### **5 Toward Quality Model Specification: BIM to HBIM LOGs-LODs, Reversing the Logic Simplex-to-Complex BIM Model (Building Construction) in Favor of Complex Modeling Knowledge-Creation Since the Design Starting Phases**

**#HBIM Level of Developments and Levels of Geometry (LODs and LOGs).** Informative quality models (as BIM, GIS models) can be gained progressively enriching the Information coming from the different BIM uses (materials, decay analysis, construction techniques, decorations, surfaces finishing, structural behavior, energy performances): they can support the preservation design project and decision-making process till to the life cycle management. LOD and LOG applied to HBIM con be considered parameters to drive HBIM quality models.

In the BIM logic of the building construction, the progression of different LOGs (Level of Geometry) within the LODs (Level of Development of the different phases as the pre-design phase, the design phase till to the facility management) is justified by the progressive enrichment of the geometry and Information connected to the new construction process (LOG-LOD100–500). When turning toward the preservation aims, we have to take into account the specificity requiring since from the starting phase a level of richness and complexity in the HBIM modeling to understand the phenomena and to carry on the assessment.

### **5.1 # Level of Development Phases Inherited by the BIM Logic and Adapted to the HBIM**

BIM LOG/LOD100–200-300–400-500–600 are hereafter adapted to the HBIM specific domain of the architectural heritage. The architectural heritage with all its components requires the maximum effort of knowledge and details since the starting phase to support a decision-making process devoted to proposing sustainable and reliable solutions in the design process and to preserve the heritage subject in the intervention and its maintenance across the time.

LODs-LOGs have been turned as follows:

**# LOD100 (Pre Design):** the LOD100 (Pre Design) has been turned to data collection (LOG100 "Conceptual model, historical reports and archives");

**# LOD200 (Digital documentation phase):** it has been added a "Digital documentation phase" (LOD200) to replace the schematic design phase, devoted to the acquisition of the "Appropriate geometry (LOG200)" correspondent to the digital 3D survey and onsite data collection" not present in the current specification nor in the Heritage domain nor in the Infrastructure domain.

**# LOD300 (As-found HBIM model):** an "As-found HBIM model" (LOD300) phase obtained by the "Precise Geometry, SCAN-to-BIM model object" (LOG300) devoted to the HBIM modeling phase has been introduced. As previously described, the adoption of one or more scales to model the objects allows professionals to identify the proper scale of each component, function of the contest, geometry, funding, and purposes.

**# LOD400 (Design development – Conservation Plan):** the Design Phase has been shifted and adapted to the HBIM uses, thus addressed to the "Conservation Plan" (LOG400).

**# LOD500 (Construction Stage):** this phase is simply shifted and tuned to the HBIM construction site, supported by LOG500 "Conservation site".

**# LOD600 (Facility Management):** it is integrated by the LOG600 "As-Built, LLCM, CDE, HUBs": it implies, as in the BIM logic, the management, monitoring, and communication process from the as-built phase.

### **5.2 Levels of Geometry Proposal in the Heritage Domain: HBIM LOG100–200-300–400-500–600**

**# HBIM Levels of Geometry (LOGs).** The proposal to adapt the current LOD-LOG definition – so far adopted for the new building domain - has been introduced to manage the 3D quality model within the conservation process, taking into account each object unicity. The concept of GOAs scales previously defined can be adopted in the function of the different phases and needs in the specific LOGs.

HBIM LODs-LOGs managing the 3Dquality model within the preservation process has been introduced not present in the process as in the following cases (Fig. 11).

The LOG progression is no more related to the scale content defined by the GOA: thus, within each development step, the scale can be defined with respect to the user needs (Fig. 12).

**Fig. 11.** The proposal of targeted LODs-LOGs adapting the current BIM LODs-LOGs to the HBIM domain. GOAs (models scales) cross all the LOGs, being adopted by the different actors involved in the different phases (LOD200–300-400–500-600). Courtesy of ABClabGIcarus (F. Banfi, R. Brumana, C.Stanga)

The HBIM LOGs has been defined as hereafter proposed:

**# LOG100 - Historical report:** it has been introduced to collect the historical drawings, documents from Libraries, Archives, Maps Collection, Open Data Digital Collections (as Europeana), replacing the 'pre-design' concept model;

**# LOG200 - Appropriate geometry:** 3D surveying and on-site data acquisition have been introduced within the HBIM LOGs addressed to the surveying to get 2D/3D drawing and mesh digitization, so far totally missed within the BIM logic;

**# LOG300 - Precise geometry:** Scan-to-HBIM model Objects generation has been specifically introduced to manage the generative HBIM modeling phase to get HBIM enabled objects, not supported by the mesh or mass models;

**Fig. 12.** HBIM turned into preservation aims to support the preservation phases described by the Level of Development (LOD) and the Level of Geometry (LOG): the richness of geometry obtained since from reliable LOD300 (HBIM object model generation – Precise Geometry) supports the LOG400 (HBIM uses) and the Conservation site management (LOG500). Courtesy of ABClabGIcarus (F. Banfi, R. Brumana, C.Stanga)

**# LOG 400 - HBIM uses:** HBIM-Uses data management (i.e., preservation plan, inserting the missed material-decay analysis, Work Breakdown Structures- WBS, BIM-to-FEA, or Energy Efficiency analysis); LOG400 manages different scale models (GOA Scale), complex models with high level of accuracy together with simplified model scales. The choice depends on the preservations experts needs but also on tools, as in the case of Energy Efficiency tools requiring so far simplified 3D models;

**# LOG500 - Conservation site:** HBIM construction site management (CoSiM), on site interventions of preservations;

**# LOG600 - Facility Management;** it has been addressed to manage Long Life Cycle Management (LLCM) starting from the As-Built, within Common Data Environment (CDE).

The richness of reliable quality model geometry obtained since from the starting phase by LOD300 (HBIM object model generation) supports the LOG400 (BIM uses), the preservation site management (LOG500), as shown in the Fig. 13.

### **5.3 Sharing 3D Quality Models, Re-use and Circulation: Common Data Environment Toward XR Platforms and Interoperability**

**# Re-use and models circulations.** In the UK, Level 2 is also known as Common Data Environment (CDE) or a 'federated model' where it is possible to use different models to create a digital HUB, giving a complete holistic view of the building.

Assuming that a LOG400 (BIM-uses) can express the concept of model interoperability, it is also useful to consider how this three-dimensional Information can be shared among users. LOG500 and 600 introduce the idea of sharing models through CDE, increasing the level of collaborative communication: HBIM are evolving into interactivity, immersive fruition and interoperability through digital model re-use [23].

In the Digital Cultural Heritage (DCH), archaeological sites, architectural heritage, and infrastructures require the management of different Information previously described. Consequently, in this specific context, it is necessary to underline that CDE should be fed by many data, formats, and models, oriented towards different uses, to improve the management of the Objects over time, allowing different users and operators to access informative models with additional commitment and limitations in the data modifications.

Therefore, the shared data environment (CDE) should be considered a multi-access to multi-source information used to collect, manage, and share graphical models with non-graphical data in conventional data formats evolving toward XR platform to support AR/VR informative models [24]. According to PAS 1192–2: 2013, a CDE may use a project server, an extranet, a file-based retrieval system, or other suitable toolsets.

**Fig. 13.** 3D quality informative models possible workflow: managing different parameters and progressively enriching information and documents from digitization to XR experiences. Courtesy of ABClabGIcarus (F. Banfi, R. Brumana)

### **6 Conclusions and Remarks**

Quality driven models allow a progressively increasing of knowledge creation, opening the doors of informed communication using co-working spaces, common data environment, and geospatial hubs with linked informative model objects (as HBIM), in the form of Linked Open Data (LOD), enriching and augmenting reality also by using VR/AR/MR. Therefore, it requires CDE and platform capable of digesting models progressively enriched by the information supporting eXtended Reality experiences. The re-use requires to foster common languages for data exchange and vocabularies, taking into account the richness and specificity of the construction techniques and of the information linked to the HBIM. It also requires filling the gap of lack of skills boosting capacity building, especially in the modeling phase growing DG content skills and in the use of HBIM by the experts.

A multi-parameters radar chart application can be implemented to better support the users in the decision-making during the 3D quality model generation helping measure the level gained during the process.

Standardization and ad hoc guidelines addressing the complexity of cultural heritage, as in the case of LOD-LOG-GOA definition in the HBIM, can contribute to avoiding misunderstanding and misuses in the re-use and circulation of the 3D model libraries and sites.

**Acknowledgments.** EU VIGIE2020/654 Study at the request of and financed by the European Commission, Directorate-General of Communications Networks, Content & Technology. Coordinated by the Digital Heritage Research Lab / UNESCO Chair on Digital Cultural Heritage at the Cyprus University of Technology.

The researches leading to the results illustrated by the figures published in this paper have been carried out within the ABClab-GIcarus (Sc. Resp. R. Brumana, F. Banfi, M. Previtali, C.Stanga, F. Roncoroni): particularly under the Consultancy Contract 'Surveying to HBIM of the Basilica di Collemaggio' - ENI-Service (R. Brumana S. Della Torre), my thanks to L. Cantini and D. Oreni. My thanks also to A. Grimoldi, A.M. Landi, F. Banfi, C. Stanga for the HBIM and VR cooperation on the Basilica di San Francesco (Arquata del Tronto).

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **The Holistic Documentation of Movable Cultural Heritage Objects - The Case of the Antikythera Mechanism**

Kyriakos Efstathiou1,2(B) , Marianna Efstathiou2, Alexandros Basiakoulis2, Harriet Cliffen1, Nenad Joncic1, and Rafaella Georgiou1

<sup>1</sup> ERA Chair on Digital Cultural Heritage, Cyprus University of Technology, Limassol, Cyprus kyriakos.efstathiou@cut.ac.cy

<sup>2</sup> Mechanical Engineering Department, Aristotle University of Thessaloniki, Thessaloniki,

Greece

**Abstract.** The Antikythera Mechanism is the oldest extant complex geared device, and an amazing example of an early analogue computer. It was built approximately 2150 years ago. The device was operated manually by a user, who would set a date on a dial. All necessary calculations were made using a set of gears (at least 39), while the results were displayed on several scientific scales. The Mechanism was used to calculate the diurnal and annual motion of the Sun, the Moon and probably the planets among the stars. It implemented the astronomical knowledge of ancient Greeks about the motion of these celestial bodies with astonishing accuracy, taking into account the anomalous orbit of the Moon using a system of eccentric gears. It could also predict eclipses of the Sun and the Moon from the Saros period, which was found in one of its scales. It calculated the dates of the major crown games that took place in ancient Greece (e.g. the Olympic Games). Finally, it was accompanied by an extended User's Manual. More than 20 references to astronomical mechanisms can be found in classical literature from 50 BC to 500 AD. In this study, the first approach for a holistic documentation of the Antikythera Mechanism is presented.

**Keywords:** Cultural Heritage · Holistic documentation · Antikythera Mechanism · Gears · Ancient Astronomy · Ancient Technology

### **1 Introduction**

The Antikythera Mechanism is a highly complex object, and since it was found in 1900 has been the subject of many extensive studies. It is one of the oldest extant geared devices and was an astronomical "computer" used for the calculation of the diurnal and annual motion of the Sun, the Moon and potentially the planets, as well as for calculating the dates of the Olympic Games. From the extensive studies [1–4], we have a huge amount of data and as such this is an excellent case study for proposing an approach for the holistic documentation of cultural heritage objects. This paper is made up of two constituent parts, the first outlines an approach for how the Mechanism can be holistically documented

M. Ioannides and P. Patias (Eds.): 3D Research Challenges in Cultural Heritage III, LNCS 13125, pp. 103–130, 2023. https://doi.org/10.1007/978-3-031-35593-6\_6

© The Author(s) 2023

within a conceptual data model, and the second outlines some of the data that is necessary to be collected and recorded in order to holistically document the asset. Both of these sections have data representational of what is available, from the physical object and the relevant contextual information (modern, ancient, astronomical, geographical). An integral part of this being classed as a success story comes from ensuring that both the digitization process (i.e. the use of x-ray tomography and Polynomial Texture Mapping (PTM) Dome technology) and the documentation process, including how the data is represented and mapped in a digital format, are done to a high quality by engaging with the complexity of all of these factors.

### **2 Identifying Necessary Data for the Holistic Documentation of Movable Archaeological Findings**

Cultural Heritage (CH) objects are distinguished between tangible and intangible as well as movable or immovable. One of the most iconic and complicated museum object is the Antikythera Mechanism. The Mechanism proves that the Greek engineers of the Hellenistic period were far more advanced in the design and manufacture of geared devices than the surviving written sources imply.

The digital holistic documentation of movable objects, like the Antikythera Mechanism must include: all possible data regarding the physical object, including the 3D geometry, the materials of its construction, the ancient context including intangible elements such as, its purpose, utility, handling and operation, its geographical and chronological origin and the modern excavation and analysis data. Based on the information gathered, the identification of the scientific and technological knowledge necessary for the design and construction of such a complex device should be recorded. Simultaneously, a review and documentation of the existing knowledge regarding the mechanism during the period of its construction should be completed. Furthermore, accurate digital and physical representations of the original discovery must be created, as well as, the complete reconstructed functional copies of the object.

In the following paragraphs a representation of all the data necessary for a holistic documentation of the Mechanism are presented. The more detailed explanation of this data and the subsequent discoveries regarding the Mechanism will be presented in Sect. 3. These data are available in digital form and can be easily introduced into an integrated digital environment with corresponding interfaces, so as to create a digital holistic recording of the Mechanism.

A conceptual model of data for the holistic documentation of the Antikythera Mechanism was created, dividing up into the categories of different types of data needed in order to holistically record a CH object. The initial stages of the model, from "*Tangible*" through to "*Mechanism*" follow the general taxonomic system under construction in the *MNEMOSYNE* project [30], and categorises the Mechanism within the classes of "*Movable*", "*Instruments and Manufacture*", "*Astronomy*" and "*Mechanism*" which indicates the type of the object (Fig. 1). From this point we start to represent the data needed to record each tangible facet ("*Elements*") of the physical object and its intangible information ("*Context*") – both historical and modern - which will be explained further in *Sect.* 3.

Within this conceptualization, the data is separated out into contextual information, which includes information relating to the "*Ancient*" and "*Modern*" contexts – including, its production, use, deposition, excavation and post excavation studies, as well as relating information regarding the "*Geographica*l" and "*Astronomical*" data, which provides important explanations and knowledge, that enhance our understanding regarding the need for and the use of such an object. The other main category is "*Fragments*", which refers to the information related to the physical object. It is important to highlight, that the Antikythera Mechanism is a particularly complicated object; many elements of the structure have been proposed but no physical remnants have been found – for example some of the gears, much of the frame etc. Additionally, the mechanism has been found in a calcified fragmentary form with large portions remaining undiscovered, thus, many of the links between elements have been proposed through extensive studies using the application of high level technologies such as x-ray tomography and model building.

**Fig. 1.** Conceptualisation of "holistic" documentation regarding the Antikythera Mechanism.

The collected information was carefully consolidated to support contextual knowledge redistribution and preventive conservation of historic assets through heritage significance awareness. Holistic recording of the context of the mechanism revealed a series of important categories (Fig. 2). The categories of data types are connected by solid lines, whilst values and object related data are represented by dotted lines, to differentiate between the different aspects of the model (the categories versus the value based data). Firstly, the modern context including the excavation, related objects from the wreck, conservation, storage and related research projects related to the mechanism. Stages of data acquisition and processing leading to the production of digital and physical reconstructions of the mechanism are shown in Fig. 7. The figure highlights the sheer quantity and variety of data (and data formats) involved in the digitization process of documenting such a complex object; from cutting-edge technologies of x-ray tomography, through to the necessary manipulation of the multi-layered data sets handled within sufficiently detailed point clouds. Secondly, the ancient context, including the manufacture, use and deposition. Thirdly, the astronomical context, revolving around lunar and solar occurrences and astronomical knowledge that was woven into the manufacture and use of the object. Finally, the geographical context, which is also integral for understanding the ancient context of the object. The mechanism was calibrated to work optimally between 33.3–37.0° latitude, which falls in the mid-Mediterranean range, and is consistent with Rhodes; a possible manufacturing site [16]. Recording and studying related information from a broad range of areas (from physical objects from the same context, to astronomical research), have been integral for piecing together the knowledge regarding and the story of the object, so recording all of this related information within the same system is extremely important. Figure 2 shows the overall taxonomy of the context (i.e., the intangible information). Figures 3, 4, 5 and 6 present the detailed taxonomy of the branches associated with the context, which show the various data and metadata individually related to the discovery, research and history of the asset, as well as its astronomical uses and geographical information.

Figure 8 shows the overall taxonomy of the elements (i.e. the tangible information) that compose the mechanism. Figures 9, 10, 11 and 12 present the detailed taxonomy of the branches associated with these elements, which show the various data and metadata individually related to the frame, dials, pointers, shafts and gears. The elements in blue have been physically found (or identified within tomography images) and the elements in pink have been proposed by the various research teams, but have not been physically found.

The main criteria that are modelled here, or are expected to be modelled within the next period are:


It is important to clarify that each of these criteria, whilst initially modelled independently, will be interlinked where necessary and they will be further enriched with the

**Fig. 2.** Conceptualisation of data regarding the contextual information of the Antikythera Mechanism. The expansion of these branches is shown in Fig. 3, 4, 5 and 6.

contextual information. It should be highlighted that the mechanism consists of many small parts of equal importance working in synchronization.

As can be seen in the example of Antikythera Mechanism, within the category of elements, each constituent part is separated out into form type, i.e. gear, pointer, plates, dials, shaft, pin and slot mechanism and frame and this is then subdivided into the actual or proposed element, i.e. a specific gear. From this stage, the data represented is extracted from the various studies outlined throughout Sect. 3 of this paper, some of

**Fig. 3.** Context taxonomy: "Discovery" branch.

**Fig. 4.** Context taxonomy: "Research" branch.

which is contradictory or at least come to different conclusions, which is often the case among different study groups and technological advancement. A key example of this is the number of teeth on a gear, which have been postulated upon by a number of different researchers. However, this variable research is part of the holistic process and feeds into and supports the holistic documentation of the object.

Empirical evidence suggests that digitization concerns have overgrown technical issues of conversion and often expand to cover the management of collections, mediation and representation of cultural heritage in the digital environment, economics of digital repositories, business models, quality and sustainability of digitization initiatives etc. This type of interrelating categorizations was meant to form an all-inclusive functional data model, consistent with role-based online analytical processing (OLAP)

**Fig. 5.** Context taxonomy: "Ancient" branch.

**Fig. 6.** Context taxonomy: "Astronomy" and "Geographical" branches.

operations. This means they could easily serve as a benchmark for integrated visualizations built in powerful dashboard design software packages that come with visual analytics capabilities; thus, enable specific users to conduct ad hoc queries (and filter on-demand), with little or no programming skill. One very interesting application is the

**Fig. 7.** Data acquisition to reconstruction workflow for the Antikythera Mechanism.

**Fig. 8.** Elements taxonomy branches for the Antikythera Mechanism.

ability to recursively traverse a dimension hierarchy and print-out aggregations at the actual browsed location. For example, providing grounds for an engineering specialist to select individual shafts and inspect the corresponding engaged gears (Figs. 13 and 14). As can be seen in these figures, the items listed in the hierarchy (left-hand side) are taken directly from the elements of the asset's taxonomy, which are shown on the right-hand side of Figs. 13 and 14. The selected individual shafts and their corresponding engaged gears, coloured in blue, are visually distinguishable.

**Fig. 9.** Elements taxonomy: "Frame", "Plates" and "Dials" branches.

**Fig. 10.** Elements taxonomy: "Pointers" branch.

**Fig. 11.** Elements taxonomy: "Shafts" branch.

**Fig. 12.** Elements taxonomy: "Gears" branch.

**Fig. 13.** Antikythera Mechanism. Shaft "i" and gear "i1".

**Fig. 14.** Antikythera Mechanism. Shaft "b" and gears "b01", "b02", "b1", "b2", "b3".

### **3 The Existing Data for the Holistic Documentation of the Antikythera Mechanism**

### **3.1 The Antikythera Shipwreck and the Discovery of the Mechanism**

In 75 BC a large Roman ship sailed between the mainland of Greece and Crete. The boat sank off the shores off the small Greek Island Antikythera. The ship was loaded with works of art and other precious artifacts. Two thousand years later, during Easter in 1900 AD sponge-divers from the Greek Island of Symi, accidentally discovered the ancient shipwreck off the coast of Antikythera. The underwater excavation began at the end of November 1900 and a few months later important findings were recovered, such as the famous Antikythera Ephebe [4] (Fig. 15). Most of these findings are now exhibited at the National Archaeological Museum of Athens [4]. Coins from Pergamon were also retrieved, which gave the wreck a relative date between 86 and 67 BC [5].

**Fig. 15.** The Antikythera Shipwreck and the underwater excavation.

Amongst the findings, a strange bulk of material that was broken, worn and calcified, with signs of the presence of bronze was found (Fig. 16) [4]. In the first publication of the Antikythera shipwreck [6] the existence of the Mechanism was mentioned with the suggestion that it was an astronomical instrument. Through investigation, the Antikythera Mechanism, had the potential to change the knowledge we had so far on the technological skills of ancient Greeks.

**Fig. 16.** The biggest fragments of the Antikythera Mechanism [4].

#### **3.2 The Investigation of the Antikythera Mechanism**

The first scholar, who studied the function of the Mechanism extensively, was Derek de Solla-Price, with the help of Charalambos Karakalos from the Research Centre Demokritos in Athens. He worked for more than 30 years and eventually published an extensive account, "Gears from the Greeks" [7]. He declared that "the Antikythera Mechanism is the oldest proof of scientific technology that survives today and completely changes our view of ancient Greek Technology".

After de Solla-Price, research was undertaken by Michael Wright and Alan Bromley. Unfortunately, Alan Bromley died in 2002. However, Michael Wright published a series of papers, where he postulated that the back dials of the Mechanism were spirals and that the upper dial was built to follow the draconic lunar month. He also elaborated on the pin and slot mechanism (see Sect. 3.2.1.1) and proved its epicyclic function [8]. Additionally, he made strides towards creating a reconstruction of the Mechanism and he produced superb bronze replicas.

In 2001, Mike Edmunds and Tony Freeth (Cardiff University), Xenophon Moussas and Yanis Bitsakis (University of Athens) and John Seiradakis (University of Thessaloniki) created the "Antikythera Mechanism Research Group". They received a grant from the Leverhulme Foundation, U.K. and the permission to undertake a new investigation from the Ministry of Culture of Greece. After the permission was granted, Eleni Magkou and Mary Zafeiropoulou (National Archaeological Museum) and Agamemnon Tselikas (Cultural Foundation of the National Bank of Greece) joined the team, which was soon supported by an international team of astronomers, archaeologists, mathematicians, physicists, chemists, computer engineers, mechanical engineers, epigraphologists and papyrologists.

In September 2005, they undertook a major new investigation of the Antikythera Mechanism, using an innovative and state of the art high power micro-focusing x-ray tomography, specially constructed by X-Tek Systems [9] (Fig. 17, left) and the Hewlett Packard, USA, PTM Dome technique [10] (Fig. 17, right). In November 2006 the results of the investigation were announced during an international conference in Athens and published in the international journal Nature [11]. This technique allowed the acquisition of three-dimensional images of the fragments of the ancient mechanism. The images were examined to reveal internal details of gearing and inscriptions that had hidden due to the preservation state of the fragments which remained underwater for more than 2000 years and the previous lack of the necessary technology to access this information. All inscriptions are written in Greek. A new font (True type fonts) was developed at the Aristotle University of Thessaloniki, in order to reproduce the fine art letters.

Realizing the importance of the Antikythera Mechanism, one of the largest active groups of researchers was formed in the Aristotle University of Thessaloniki (AUTH) of Greece, which they have worked to produce numerous publications on the analysis and reconstruction of the mechanism [12–16].

**Fig. 17** X-ray tomography (left) and PTM Dome technique (right) applied for the study of the Antikythera Mechanism.

### **3.2.1 Investigation Results**

### **3.2.1.1 Description of the Antikythera Mechanism [2–4, 11, 12, 14–16]**

The Antikythera Mechanism is relatively small, approximately 30 cm × 20 cm × 10 cm in size, which was found in poor preservation state, consisting of 82 fragments that have been studied together to reveal the information and data that will be discussed within the following sections (Fig. 18). The largest seven fragments are named with the letters A, to G and the smallest fragments are referred to as numbers 1–75.

### **Construction's Materials of the Mechanism**

The material used to reconstruct the various parts of the Mechanism, except for its wooden mounting box, is bronze, a copper - tin alloy. In the period from 1970 to 1974, chemical analysis was performed to determine the chemical composition of the metal alloy used in the manufacture of the Mechanism. The chemical analysis was performed by a spectroscopic method. Two small fragments of the Mechanism were studied. The chemical analysis showed that the fragments were made of bronze, with a tin content of about 5% [4]. Newer analyzes by Panagiotis Mitropoulos in 2018 revealed three alloys, the main components of which are copper, tin and lead [4]. The shares of copper tin and lead varied. It can be assumed that the individual parts of the mechanism consist of copper alloys of different composition [25].

#### **Structural Elements**

### **Gears**

The gears found in the Antikythera Mechanism are the earliest known to resemble the shape and design of modern gears. Their triangular teeth were designed to transmit angular motion, not power. Detailed studies of the fragments of the Mechanism revealed that it had at least 39 gears, 29 of which have been identified in the largest of the calcified

**Fig. 18.** Fragments of the Antikythera Mechanism from the National Archaeological Museum of Athens.

fragments - 28 in Fragment A and 1 in Fragment C. The existence of ten more gears has been determined taking into account astronomical calculations [11, 12, 20, 21]. A functional diagram of the gear trains is presented in Fig. 19. The gear pairs are displayed with the driving gear first (to the left), except for the triple where gear b2 drives the two other gears, c1 and l1.

The Antikythera Mechanism incorporated also 2 crown gears as well as 19 shafts and axles with complex geometry. Two pairs of them are concentric, which means that one of the two shafts passes through the other. The two shafts are rotating independently. Moreover, there is an axis, which has two eccentric cylindrical bearing points for two gears. At various locations, mainly inside the Mechanism, there were also various components that supported the shafts and other parts of the device to make the construction robust and functional.

### **Pin-and-Slot Mechanism**

In the pin-and-slot mechanism (Fig. 20), the axial distance of the rotation's axis of the two gears is to approximately 1.1 mm. The lower gear has a pin that engages with a slot on the upper gear, forcing it to rotate. The epicyclical movement of the upper gear tracked the motion of the Moon in the sky with great accuracy [18, 19].

#### **Dials and Inscriptions**

The Antikythera Mechanism was a complicated instrument. By using x-ray microfocusing tomography, inscriptions that have not been read for more than 2000 years were revealed. Approximately 3500 letters and symbols have been deciphered so far.

**Fig. 19.** Gear trains of the Antikythera Mechanism.

**Fig. 20.** The pin-and-slot mechanism [20].

There meaning fall into three broad categories: astronomical inscriptions, geographical inscriptions and technical inscriptions. Several astronomical terms have been read referring to the Sun, the Moon, the ecliptic, the Metonic and Saros cycles and other astronomical phenomena. The word "-THPIMO-" (stationary point) is mentioned several times, likely referring to planetary stationary points.

The Mechanism had 7 pointers, which provided 8 indications in the scales of the Mechanism (the pointer of the Moon gives two indications). It had three main dials, one at the front side with two concentric scales, and two at the back in the form of spirals. On the front side (Fig. 21, left), there were two concentric circular scales. The outer scale had 365 subdivisions the days of the year. The inner scale had 360 subdivisions and the names of the 12 zodiac constellations. The date scale had the shape of a ring (Fig. 21) which was removable. Behind this date ring, there were 365 holes, and the ring was secured at one of the holes using a pin. Every four years, the operator was able to remove the ring and move it by one hole, to take leap years into account.

A very careful analysis of the gears' co-action revealed their use in calculating to a high level of accuracy, the position of the Sun and the Moon on the zodiac (the zone that contains the zodiac constellations). This position was shown by the pointers of the front dials. Similarly, a crown gear drove a black and white coloured spherule, showing the current phase of the Moon.

On the back side of the Mechanism (Fig. 21, right) there were two spiral scales. The upper spiral consists of 5 windings, with its total length divided in 235 sections. Using this dial, the user could read the position of the Moon within the Metonic cycle of 19 tropical years, which is almost equal to 235 synodic (lunar) months. The Metonic cycle is almost equal to the least common multiple of the tropical year (365.2422 days) and the synodic month (29.5306 days). The difference between the two periods (of 19 tropical years and 235 synodic months) is only 2 h. This knowledge allowed the calculation of the exact day a full Moon would occur, which is a very useful knowledge for agricultural or nautical activities 2000 years ago, when no electricity was available. The accuracy of the position of the Moon was achieved by a pin-and-slot mechanism [8, 17] that reconstructed Hipparchus' first anomaly of the Moon's motion (due to its elliptical orbit around the Earth).

This anomaly is, in fact, Kepler's 2nd law.

**Fig. 21.** The front side with the removable date scale ring and the back side.

Calippos 100 years later corrected Methons calendric system. Every 4 periods of Meton, i.e., every 76 years, one day needed to be removed. The Callippic pointer of the subsidiary dial within the upper back spiral of the Antikythera Mechanism indicated when the correction must take place.

A subsidiary dial within the upper back spiral of the Antikythera Mechanism displayed the celebration date of the ancient Panhellenic crown games. On the circumference of the dial the words Olympia, Pythia, Isthmia, Nemea and Naa have been deciphered. In each quadrant of the interior, the four years Olympic cycle are indicated. All these games were crown games, with winners being rewarded with olive branch crowns.

The lower back dial is a Saros eclipse-prediction dial, arranged as a four-turn spiral. This dial contained the 223-month eclipse Saros cycle (of approximately 6585.3213 days, or nearly 18 years and 11 1/3 days). 223 lunar months (one Saros cycle) after an eclipse, the Sun, Earth, and Moon return to approximately the same relative geometry, and a new, nearly identical, eclipse cycle begins. The Saros cycle was marked with the dates (month, day, and hour) when a possible solar or lunar eclipse would occur. The markings were engraved with symbols ("H" – HIO- – Sun and "-" – -EHNH – Moon, etc.), The fact that both letters, "H" and "-", appear simultaneously in some glyphs, most probably means that the glyphs represent predictions of future eclipses and not records of past eclipses.

A subsidiary Exeligmos dial, within the Saros dial, extended the eclipse prediction capabilities to three Saros cycles, indicating that 8 and 16 h should be added respectively in the second and third Saros cycles to the eclipse times indicated by the inscriptions.

The most impressive part of the Mechanism is related to the moon phases and the moon's movement anomaly. In the seventeenth century, Johannes Kepler, in relation to the sun and moon movements, claimed that the "holy circular motion was not circular" and suggested that it was an ellipse orbit (first and second law of Kepler). Inside the Mechanism, a gear system is identified, that simulates this motion with a high-level of accuracy.

The display of this movement was achieved by the use of two eccentric gears (Fig. 20), taking into account the moon's movement anomaly caused by its eccentric orbit around the Earth, was achieved by the use of two eccentric gears (Fig. 20).

#### **The Fragment D of the Mechanism**

The latest outcomes of the AUTH team are related to the investigation of fragment D. In all modern studies of the Mechanism, fragment D is considered as a lone fragment and does not exist in any constructed model of the Mechanism. After a long-term study, it was determined that Fragment D is indeed part of the mechanical arrangement that calculates the equation of time [2].

#### **3.2.1.2 The Antikythera Mechanism, from Physical to Digital 3D Representations**

Starting in 1998 by M. Roumeliotis [26], to date many animations and simulations of the Mechanism have been created, which can be searched online.

Regarding physical models, from 1900 until today, many scientists have involved with a reconstruction of the Mechanism. In 1928 Admiral I. Theofanidis with the contribution of E. Zinner, R.T. Gunther and W. Hartner, listed the visible gears and circles from the back side of the Mechanism and they characterized the Mechanism as an astrolabe. Admiral Theofanidis achieved to read 350 letters of the inscriptions and was the first who tried to make a replica model of the Mechanism [4, 20].

De Solla Price studied the Mechanism from 1950 until 1974. First, he analyzed the constructed materials. Under the supervision of De Solla Price, Robert J. Deroski was constructed in 1975 two replicas of the Mechanism as a proposal for the possible function of the device [4, 20]. Afterwards various manufacturers reproduced Price's models with few functional deviations, such as the models of J. Gleave in 2000 and D. Kriaris in 1999 and 2007 [4, 20].

In 1985, A. Bromley has continued the research of Price including his remarks. At his first attempt he cooperated with F. Percival and a replica of the Mechanism was constructed Five years later in 1990, he cooperated with M. Wright, and they used xray Linear Motion Tomography in order to define the array of the gears into space. This method didn't achieve the desired results. M. Wright claimed that the Mechanism could predict the movements of the Sun, the Moon and of five planets [19]. In 2005, he constructed a model using approximately 40 gears [8].

In 2008, the Aristotle's University of Thessaloniki began to develop models of the ancient Mechanism [11–13]. In 2011 five accurate and functional models were constructed (Fig. 22). From then until now many updated accurate and functional replicas in scale 1:1 as well as in scale 3:1 are constructed and are exhibit in various museums and other institutions worldwide [20, 28, 29].

The AUTH used complicated digitization processes, such as, using three - dimensional images obtained from the x-ray tomography, the PTM Dome techniques and using the Volume Graphics specific software system VGSTUDIO MAX [22], to develop 3D digital models of the four basic fragments (Fig. 16). Using the same data, the whole Mechanism was digitally recreated. In Fig. 23 and Fig. 24 illustrates the 3D digital reconstructions of the gear trains and the whole functional Mechanism is presented.

**Fig. 22.** Accurate and functional model of the Mechanism scale 1:1 (AUTH 2011).

**Fig. 23.** 3D digital reconstruction of the Mechanism [20].

Figure 24 illustrates how the front side of the Mechanism (left) and the gear system (right) was designed based on components revealed from the x-ray tomography images.

**Fig. 24.** Mechanical parts localized in the tomographies and corresponding designed elements.

### **3.2.1.3 Importance, Purpose, Utility, Operation, and Handling of the Mechanism**

### **3.2.1.3.1 Importance and Purpose of the Mechanism**

Taking into account the theoretical and technological knowledge required for the construction of the Mechanism, it can easily be ranked amongst the Wonders of Ancient World [2].

The Antikythera Mechanism was the first surviving geared analogue computer in history. The next extant geared device is a Byzantine clock calendar, which was built in the 5th or sixth century, and the next mechanical calculators were built more than 800 years later. Other examples of a similar complexity to the Antikythera Mechanism came much later, with at the beginning of the thirteenth century the astronomical indicator of Wallingford, 50 years later (1348–1364) the astronomical clock of Dondi, and in 1410 the Prague astronomical clock were developed. Later in the seventeenth century, the calculator of Schickard (Kepler's collaborator) and the Pascaline of the great French scientist Pascal were built. The Antikythera Mechanism inspires manufacturers to date as for example the design of the years dial of the Atmos millennium clock [23] and the mechanical computer of the spacecraft "Automaton Rover" [24].

### **3.2.1.3.2 Utility, Operation, Handling and prediction's Accuracy**

The main use of the Antikythera Mechanism was to calculate the exact position of the Sun, the Moon, the phases of the Moon and the lunar or solar eclipses and possibly the position of the planets in the sky. Additionally, besides the predictions of astronomical events, the Mechanism could determine dates related to religious, social and agricultural rituals and events.

As previously discussed the subsidiary dial on the upper back spiral of the Antikythera Mechanism displayed the dates of the Olympic Games, which were held during the first full Moon after the summer solstice. The Mechanism could accurately calculate this date as well as the date for the Panhellenic crown games of Isthmia (Corinth), Nemea (Nemea), Pythia (Delphi), Naa (Dodona) (Fig. 25) [21].

**Fig. 25.** The subsidiary dial displayed the dates of the Panhellenic crown games.

In Fig. 26 the Parapegma of the Mechanism is shown [16]. The Parapegmas were essentially calendars of astronomical and meteorological events, which were widely used in ancient Greece. The astronomical events referred in the Parapegmas are events that associate the sunrise and sunset of stars or constellations with the sunrise or sunset. For many years, seasons, with their different climatic conditions, were an important unit of time as they played a crucial role in people's lives. Over time, however, it was found that the start date of each season, could not be determined with the usage of classic calendars, based on lunar months. Therefore, people turned to stable natural phenomena, to define the seasons. Some of these phenomena were the rising or sinking of some stars. These phenomena appear every year at a fixed date in the sky. The occurrence of these events once during a solar year has contributed to use them in order to organize practical-social activities such as agriculture and navigation [16]. Hesiod mentions that the harvesting period is the season when the constellation of Pleiades appears for the first time in the sky, and the time of plowing is the time after the temporary disappearance of the Pleiades. The grape harvest must take place when Arcturus appears in the sky for the first time. Such phenomena and the date on which they will occur, are predicted using the Parapegma on the Mechanism [16].

**Fig. 26.** The Parapegma of the Mechanism.

The rotation of any of the gears or any of the pointers gives movement simultaneously to all other gears and subsequently cases the seven pointers to move, which shows the various astronomical phenomena in the related mathematical scales. Based on an analysis of the applied torque of the operator when handling the Mechanism, and the necessity of a precise positioning of the pointers, e.g. the Sun/Date pointer on the front of the Mechanism, the most probable scenario for the handling of the Antikythera Mechanism is the rotation of the Moon pointer [2]. The operator, by rotating the moon pointer and choosing a date, can determine the astronomical phenomena that may occur on that day. Respectively, by choosing an astronomical phenomenon it can be observed on which date it will happen.

Regarding the accuracy of the predition, it is mentioned that, from the construction of the Mechanism to date, the position of the constellations in the sky shifted approximately 30°, which corresponds to one zodiac constellation. To find out the accuracy of the Mechanism's predictions for the present era, in the model shown in Fig. 27 we have captured on the front of the Mechanism, the current positions on the scales of dates and zodiacs. On the front side of the Mechanism, the indications of the pointers of the sun and the moon during the operation of the Mechanism, for the 15 June 2011 are shown. Based on current estimates for June 15, 2011, the predictions were: Sun in the Taurus Constellation, Moon in the Sagittarius Constellation, Full Moon, and a Lunar eclipse at 23:00. The corresponding displays on the front side of the Mechanism for this date are exactly the same (see Fig. 27): Sun in Taurus, Moon in Sagittarius Constellation and Full Moon. Thus, a moon eclipse was possible.

Observing the pointer of the eclipses on the back side of the Mechanism for this date (Fig. 28), a lunar eclipse would likely occur.

Regarding the moon eclipse, theMechanism showed: a) Eclipses pointer:- (εληνη ´ = Moon), = 9 and b) Exeligmos pointer: H = 8.

**Fig. 27.** Displays on the front site of the Mechanism for June 15, 2011.

**Fig. 28.** Predictions on the lower part of the back side for June 15, 2011.

Taking into consideration, that the first hour in antiquity was the 6th hour the day, the full coverage of the moon from the shadow of the Earth will occur at 23:00 (9 + 8 + 6), exactly at the same time of the prediction of the device.

### **3.2.1.4 Place and Date of the Construction of the Antikythera Mechanism**

The Mechanism was possibly built in Rhodes, between 150 and 200 BC. Early evidence of similar machines, are references reported to Archimedes (287–212 BC) as a constructor of devices which depicted the celestial bodies [2, 3]. The Mechanism cannot have been built later than the shipwreck dated by the Pergamon coins within a few years of 60 BC. The inscriptions letter style suggested that it was constructed in the period of 50–200 BC. An important date could be set if we knew when the parameters required for the pin- and-slot lunar anomaly Mechanism were first deduced. It is known through Ptolemy that Hipparchos did characterize and quantify the anomaly by epicyclic and eccentric models of the lunar orbit. Therefore, it would be necessary for Hipparchos's values to be involved the manufacture so it must have taken place after 170 BC [2].

The optimum latitude for fitting the astronomical phenomena listed in the parapegma of the Mechanism is consistent with the mid-Mediterranean around 35° [16]. Rhodes (36°) remains as the most likely candidate. The Antikythera ship may have called there before it wrecked. Rhodes was known as a highly technological naval port with a thriving bronze industry. It was home to Hipparcho and is the place for which we have a record of the Mechanism being sighted. It may also explain the presence of the Halieia Games on the Games' dial as they were held in Rhodes [2].

Another connection to Rhodes is the construction of similar Mechanisms, which are referenced by Cicero, from when he visited the laboratory of Poseidonios (135–51 BC) in Rhodes, where he admired a celestial sphere made by Poseidonios [2].

The Metonic cycle, found in the upper back dial, contained a full calendar, which is repeated 19 times. Comparing this calendar with the calendars of the ancient Greek cities, it was found that it coincided with the cities of Kerkyra, Vouthrotos and Dodona (in NW Greece) and Tavromenion (in Sicily) [16]. This could indicate that the Antikythera Mechanism was used, but not, necessarily, constructed in NW Greece.

### **3.2.2 Documentation of the Necessary Knowledge for the Mechanism Design, Construction and Manufacturing at the Time and Place of Its Construction**

### **3.2.2.1 Scientific and Technological Knowledge Necessary for the Design of the Mechanism**

A question, which arises, is whether the Greek astronomers 2200 years ago, had the necessary knowledge to calculate astronomical phenomena, using the Mechanism. Various sources, discussed below, show that they did.

The only accurate clock that the Ancients had was the moon months. Meton in the fifth century BC, connected the moon calendar to the annual. He calculated that 19 years include 235 lunar months [2]. Callipos (370–300 BC) calculated that Meton was making a mistake one day every 76 years [2]. These calculations are performed by the Mechanism and appear on two scales on the back side of the Mechanism. The Callippic pointer of the subsidiary dial within the Meton scale (upper back spiral of the Mechanism) indicated when the correction must take place.

Aristarchus (310–230 BC) was the first astronomer in history, who discussed the heliocentric system [20], he calculated the size of Earth, Sun and Moon. Hipparchus [20] (190–120 BC) is considered as one of the greatest astronomers of all time. He lived contemporaneously to the time of the construction of the Mechanism in Rhodes. He calculated, the transient motion of the Earth which lasts 25,800 years. Another calculation developed was the determination of the distance between the Earth and the Sun & the Earth and the Moon.

### **3.2.2.2 Technological Knowledge Necessary for the Manufacturing of the Mechanism**

Another question, which arises, is whether the ancient Greek mechanicians had the necessary technological knowledge for the manufacturing of the Mechanism. Various sources show that they had them.

Until the discovery of the Antikythera Mechanism the construction of the first real gears was dated centuries later. Aristotle describes the rotation of cylindrical objects by friction, due to the roughness of their cylindrical surfaces. The creation of surfaces with higher roughness slowly led to the development of teeth and gears. The gears of the Antikythera Mechanism (second century BC) are the first known example. Two references related to calculations and constructions of gears are from Heron and Pappus. Heron of Alexandria was an engineer and geometry, who lived in Alexandria, Egypt. In his description of the construction of an odometer he mentions and describes gears. Pappus of Alexandria describes several machines that were described by earlier mathematicians and engineers, such as Archimedes, Heron, etc. and included gears [26]. For two gears to work together, they must have the same ratios of diameter to the number of teeth. This relationship is called today "module". From Papus's writings, it is clear, without any doubt that the Greeks knew the module in antiquity.

Very likely, the gears of the Mechanism were made of cold forged thin bronze plates by sawing, removing redundant material and leveling with a hammer [25].

In order to manufacture the mechanism particular tools were necessary. The text of the inscription from the fourth century BC shown in Fig. 29 concerns the construction of bronze axes "o´λoς" for the Filonian gallery in Eleusis, using lathe. On this marble inscription is written among others *"… a copper alloy from Marion (Cyprus) must be used, consisting of 11 parts copper and one part tin …*" This alloy is called bronze today. The parts of Antikythera Mechanism are made of bronze (see Sect. 3.2.1.1). Subsequently is written *"… Turn the axes according to the example …*" This inscription shows that many years before the creation of the Mechanism, the Greeks had and used lathes. This is also apparent from other sources [25].

For the machining of bronze pieces, steel cutting tools are necessary. It follows from several sources that the Greeks at that time had such machine tools and cutting tools [25].

**Fig. 29.** Marble inscription fourth century BC. Archaeological Museum of Eleusis.

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Digital Documentation of Reflective Objects: A Cross-Polarised Photogrammetry Workflow for Complex Materials**

Adam Frost1, Sophia Mirashrafi1, Clara Molina Sánchez2, David Vacas-Madrid3, Elina Rodriguez Millan2, and Lyn Wilson1(B)

<sup>1</sup> Historic Environment Scotland, The Engine Shed, Forthside Way, Stirling FK8 1QZ, Scotland lyn.wilson@hes.scot

<sup>2</sup> Historic Environment Scotland, South Gyle Conservation Centre, Edinburgh EH12 9EB,

Scotland <sup>3</sup> CyberHawk, EICA Ratho, Edinburgh EH28 8AA, Scotland

**Abstract.** The quality of digital documentation of cultural heritage sites and objects is influenced significantly by the complexity of the subject. Complexity is an encompassing concept comprised of a range of factors including material properties and surface characteristics. Highly reflective items pose a challenge to traditional workflows, which has been previously addressed with a range of mitigating techniques and strategies, often including cross-polarised photogrammetry lighting setups to omit specular type reflections. However, in the removal of reflections to achieve more robust surface geometry, important information about the specular albedo of the surface can be lost. In this paper we demonstrate an accessible single camera cross-polarised photogrammetry workflow to retain the diffuse and specular albedo information. The results have enabled qualitative assessments about the individual objects and their materials through the workflow. The items discussed include ceramic, metal and wood material types, and through the separation of specular information offer unique improvements to their visualisation and insights to their physical condition for conservation purposes.

### **1 Introduction**

There are many ways in the field of digital documentation that artefacts can add layers of complexity to their capture. Each artefact is unique and will therefore offer unique challenges. One such challenge that spans a great many objects is that of shiny and reflective surfaces. Glossy finishes that return specular (or mirror-like) reflections negatively affect 3D data capture techniques that rely on light, such as laser scanning and photogrammetry. The adverse impact on the quality of the data capture results in lower geometric accuracy and 'bakes' specular reflections into image textures. This in turn reduces the usefulness of the output models for cultural heritage purposes, where detailed condition monitoring and inspection are essential conservation objectives. This paper explores the use of a cross-polarised photogrammetry workflow for complex surfaces in a cultural heritage context, with a predominantly qualitative discussion of results with a conservation focus.

© The Author(s) 2023 M. Ioannides and P. Patias (Eds.): 3D Research Challenges in Cultural Heritage III, LNCS 13125, pp. 131–155, 2023. https://doi.org/10.1007/978-3-031-35593-6\_7

The items were digitally documented as part of Historic Environment Scotland's (HES) Rae Project (Historic Environment Scotland 2020), a commitment to digitally document our 336 Properties in Care and the 40,000 + collections objects they house - many of which have ceramic glazes, polished metal varnishes, and other complex reflective or glossy surfaces. A general introduction to digital documentation and the Rae project, including some of the key technologies, guidelines, methodologies and HES case studies are outlined in the HES Short Guide 13: Applied Digital Documentation in the Historic Environment (Historic Environment Scotland 2018).

The approach to the complexity of digitisation in this paper links with the objectives of the European Commission funded VIGIE-2020–654 study regarding the quality and complexity of 3D digitisation of tangible cultural heritage. The study identified factors governing both the complexity of the digital documentation process, and quality of the outputs. Figure 1 is a diagram produced by the VIGIE-2020–654 study showing some key parameters as relevant to this paper. Of these factors, material and condition are both central to the investigation and documentation of items with complex or highly reflective surfaces (Fig. 1A). For quality parameters in Fig. 1B, 3D geometry, image texture and material properties are the most important factors for assessing the quality of outputs.

**Fig. 1.** Diagrams from the VIGIE-2020–654 study (A) 'Object' segment of complexity (B) 'Image texture' segment as a factor of quality.

#### **1.1 Background**

Previous work addressing this problem has shown that the use of a cross-polarised light approach for photogrammetry can improve the results of data capture for reflective objects and surfaces in this context (Menna et al. 2016; Noya et al. 2015; Nicolae et al. 2014). Furthermore, the photogrammetric capture of materials that exhibit even more complex properties, such as subsurface scattering, has also been explored using this controlled methodology (Anghelu¸t˘a and Radvan, 2020). Cultural artefacts are constructed from a range of materials, and items or environments that have heterogeneous surfaces represent one of the greatest challenges both in terms of data capture and visualisation of the results. Metals paired with detailed painted surfaces are commonly encountered, and whilst both can be highly reflective, behave differently under a cross-polarised configuration (Hallot and Gil 2019). For physical records ('reflective originals'), often documents, maps etc., the ISO standard 19264-1:2021 sets out key quality control parameters for imaging systems (International Organization for Standardization 2021). While it does not address the use of polarised light explicitly, it makes clear that the use of imaging systems is specific to different use-cases where different outputs may be required.

Previous approaches use similar photographic equipment and cross-polarised configurations to achieve their results, with the primary objective of improving both the geometric accuracy of the 3D model, as well as the diffuse albedo texture map (including colour accuracy). This is essential to improve the quality of the data capture processes and outputs. However, to improve visualisation of most of these complex items and materials requires the isolation of both diffuse albedo and specular albedo to more closely represent the combined reflectance characteristics through the Bidirectional Reflectance Distribution Function (BRDF) (Bhandari et al. 2022 p. 313). Developments in the field of computer graphics provide examples of cross-polarised photographic systems and workflows to separate these reflection maps and improve visualisation of complex and spatially varied surfaces, including static objects, and more complex subjects such as human faces (Alexander et al. 2009; Ghosh et al. 2010).

#### **1.2 Our Approach**

As shown in this paper, we have bridged these approaches by adapting a relatively common single camera cross-polarised photogrammetry workflow to isolate the specular albedo map from the diffuse albedo map. This has the dual benefit of improvements to the data capture and visualisation of these cultural artefacts, some of which exhibit complex heterogeneous surfaces. For many items, the specular reflection characteristics can contain significant information about the condition or material of the artefacts, such as the presence and quality of ceramic glazing, metal oxidation or tarnishing and polished areas. This information directly informs subsequent inspection and condition monitoring activities, which can then inform decision making for conservation strategies.

Working in partnership with Spectrum Heritage (a commercial digital documentation team), HES explored a cross-polarised photogrammetry workflow that would enable the removal of specular reflections from the model diffuse albedo map and facilitate the calculation of a map approximating specular albedo to improve visualisation. The subjects of this workflow are collections items within the care of HES, selected for their varied materials and relative complexity of geometry. Figure 2 illustrates an example of one of the digitally documented items, with a visual breakdown of the maps used to produce the image rendered using a physically based rendering shader.

**Fig. 2.** Composite image showing different workflow outputs overlaid. From left to right: Final render; specular albedo; diffuse albedo; optimised geometry wireframe. Not pictured: tangent space normal map. Item is Greyhound Desk Weight, accession number DC351.

### **2 Methodology**

'Cross-polarised' photography is a workflow used to achieve reflectionless photography using 'polarised' light (Bhandari et al. 2022). Some light is already strongly polarised, including natural sky light as the sun's light scatters in the sky (Hanlon 2018). By utilising polarising filters in front of the lens to strip out this polarised light, photographers can remove strong sky reflections from certain surfaces, like water or glass. In a controlled lighting environment such as a studio, or otherwise under artificial lighting, these unpolarised light sources must first be polarised to enable the separation of diffuse and specular reflections.

When polarised light bounces off a rough surface that diffusely reflects light, much of that light will become unpolarised. However, smooth microstructure surfaces can reflect light (often referred to as 'specular' reflections) in a way that maintains the light's polarisation, such as glass, glazed ceramics, and many plastics (Dorsey et al. 2008, p. 62). It is worth noting that polarised light, and cross-polarised inspection methodologies already see use in the field of conservation with microscopy to assist with the identification of mineralogical composition in historic mortars and archaeological ceramics (Riederer 2004; Blaeuer and Kueng 2007).

The key aspects of the methodology can be broken down into two parts: 1) data capture and 2) data processing.

### **2.1 Data Capture**

The setup is not dissimilar to a typical photogrammetric approach to digitally documenting artefacts. Figure 3 illustrates this set up and team at Spectrum Heritage capturing images of a collections item in HES's care. The hardware utilised included a remotely triggered DSLR camera controlled by a laptop, relatively diffused two-point lighting, and a turntable to place the item on. In addition, colour calibration charts were used with metric distance scales to ensure the final product was colour-accurate and correctly scaled.

**Fig. 3.** Physical photographic setup enabling remote triggering and image quality control in realtime to ensure optimum removal of polarised light. Pictured: Clara Molina Sánchez and David Vacas-Madrid.

The cross-polarised workflow was achieved using 3D printed bespoke filters to hold linearly polarised film, which were rotated to dial in the angle of the incident light. Linear polariser film was placed over two LED flash lights, and a polariser filter was also fixed on the camera lens. Then, the polariser filter was rotated until it was perpendicular to the polariser film, therefore achieving a "cross-polarisation". In order to calibrate the polarity of the incident light, a glossy black ball was initially placed within the frame, allowing the team to align the two light sources relative to the lens filter.

This capture process was time extremely intensive, each object taking around a day to capture properly. Every photograph was taken twice: one photograph with the full specular reflections, and its twin with reflections removed by rotating the filter 90 degrees (non-polarised and polarised respectively), as highlighed in Fig. 4.

### **2.2 Data Processing**

#### **Reality Capture**

In the office, the photographs were processed into 3D models using photogrammetric software Reality Capture (2021). Both the polarised and non-polarised images were imported and aligned in one project, whilst only using the polarised images to create the mesh to reduce the influence of the specular reflections on geometric reconstruction. Several levels of detail of model were produced in line with the requirements of the project, with the highest level of details enabling possible outputs such as 3D reproduction, and to facilitate the baking of high-resolution details to lower resolution model iterations. Optimised models were produced using geometric 'simplification' to reduce the triangle/vertex count, whilst ensuring manifold geometry and UV mapping to enable the baking of tangent-space normal maps.

To texture the mesh, both the non-polarised and polarised images were successively used to create two diffuse texture maps with identical alignment. For the final diffuse texture, the team used the albedo map without reflections. The team then calculated the 'reflection' map by overlaying and subtracting the resulting two textures, described further in the following section.

#### **Texture Maps**

The application of different texture maps is what brings the 3D representation of artefacts as close to reality as possible. Physically based rendering (PBR) refers to this idea of using realistic shading and lighting models alongside measured surface values to represent realworld materials as accurately as possible. This rendering approach is currently used by most modern 3D software and 3D web viewers, such as Sketchfab (2021).

As mentioned above, the team made the specular or 'reflection' map from the polarised and cross-polarised diffuse maps by calculating the difference between the two using Substance 3D Designer (2021) and Adobe Photoshop. Figure 5 illustrates the diffuse and specular albedo maps alongside the rendered resulting 3D model for the Wassail Bowl from the Duff House collection.

As material shaders on different platforms handle input values uniquely, the difference maps often needed further processing in the image editor to tailor them for the final output. The workflow below illustrates these stages.

**Fig. 4.** The photogrammetry set up (top) including the two flash lights, and camera with polarised filter in order to achieve first full specular reflection, or non-polarised (bottom left), and one polarised image without reflection (bottom right). Item is Flower Holder, accession number DC475.

#### *Specular*

This map contains the specular albedo reflectance information from surfaces. In order to allow the renderer to convey the model's spatially varying reflectance values, black areas are shown as completely matt, while white areas are completely reflective.

To achieve this, the team overlaid the two diffuse albedo textures derived from the polarised and non-polarised images and calculated the difference and converted that

**Fig. 5.** Image illustrating the different texture maps applied to an artefact. From left to right: Diffuse albedo; specular albedo; final render. Not pictured: tangent space normal map. Item is Wassail Bowl, accession number DC334.

to greyscale in Substance 3D Designer (Fig. 6)**.** This greyscale map produced through that image-difference process represents the intensity of specular reflections (specular albedo) on an object's surface.

In order to ensure the reflection output will be rendered correctly when used in the material shader, this raw 'reflection' map will need further image processing. The extents of this image processing will vary depending on a range of factors, including but not limited to the initial capture conditions, the material shader, and the final output platform.

### *Roughness/Glossiness*

These maps describe the microsurface of an object and dictate the sharpness of any reflections. White areas represent roughness, and black indicates smoothness. Different platforms, rendering systems and material shaders vary in their interpretation of these values. For this project, as Sketchfab (2021) was one of the intended dissemination platforms for the optimised models, the levels were adjusted by normalising the histogram to evenly represent the full range of the grayscale values. This obtains a greater contrast, helping the web viewer to recognise the roughness of the surface more effectively.

#### *Metallic*

Generated when there are instances of spatially varied metallic areas on an object, this map indicates to the shader aspects of the object that are metal. Metallic maps may be represented either in greyscale or colour across different rendering systems. For this project the maps were produced in grayscale, with white depicting areas of high metallic

**Fig. 6.** Illustration of Specular map creation in Substance 3D Designer. Left to right: Diffuse Map with cross-polarisation and without; the calculated difference; converted to greyscale; inverted; levels adjusted for final product.

albedo, and non-metal/dielectric areas, denoted with black. There can be transitional grey values that indicate something covering the raw metal, such as dirt or tarnish.

### *Normal and Ambient Occlusion*

Finally, tangent space surface normals and ambient occlusion (AO) maps are integral to completing the models for visualisation. The baked surface normals enable the geometrically optimised model to render with finer details found in the surface topology. The AO map simulates the soft shadows that occur in the crevices of an object, adding an optional depth to the re-topologised model.

### **3 Results and Discussion**

This section will explore the results for selected objects from the study by material type. It was assumed by the authors at the outset that different materials would respond differently throughout this workflow to the effect of cross-polarised light and produce varied results. The following qualitative comparison between objects and the results of the workflow will reflect on whether this technique improves the level of information captured and enhances inspection. This includes the clarity of visual features apparent in the diffuse and specular albedo image texture maps, and whether the approach better facilitates inspection of the object surface properties relating to its condition from a conservation perspective.

The items are fully listed in the Appendix, with images showing input photographs alongside rendered outputs for visual comparison.

#### **3.1 Ceramic**

Cultural artefacts made of, or including, ceramic material makes up 3% of HES' collections. Ceramics often represent a challenging and mixed material to digitally document with photogrammetry, due primarily to the variation of surface finishes that can range from completely matt (such as an unglazed earthenware vessel) through to a buff polished or highly reflective glazed finish. Often, these surfaces may contain imperfections or characteristics created during the manufacture or subsequent use that can reveal important information about the item's usage or condition. Ceramics are considered dielectric materials, though some may contain metallic/conductive components which may require the 3D model to use separate 'metallic' texture maps to render these areas appropriately (Westin et al. 2004).

### **St Andrews Glazed Tile (SAC422)**

Item description: "*This unusually decorated medieval floor tile is believed to be from Blackfriars church in St Andrews, the remains of which survive just off of South Street in town. Its Low Countries redware fabric indicates a trade connection. It may have been imported from Low Countries such as the Netherlands or Belgium*". Item dimensions: 67 × 50 × 25 mm (HES, Vernon Collections Management System, 'Object - SAC422' accessed 25/10/21).

**Fig. 7.** St Andrews Glazed Tile: Detailed area excerpt of UV texture sheet maps in the following order left-to-right: (A) diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. Assession number SAC422.

### *Discussion and Interpretation*

This ceramic tile includes a single glazed face (visible in Fig. 7), with an otherwise exposed fabric on all other sides. The fabric is a light red with dark grey inclusions, a rough matt surface and shows accreted remains of a light-coloured mortar. Crazing within the glaze was observed in both the cross-polarised and unpolarised texture maps. The cross-polarised results show the differences between the diffuse and specular albedo maps, the former of which is clear and does not suffer from the 'baked in' veiling reflection often visible in image textures for highly reflective objects. In the specular albedo map, the glazed surface shows detailed and dense surface scratches, gouges, marks, and damage to the glaze and underlying decoration revealing areas of fabric. This pattern of damage is consistent in distribution across the glazed area, and the linear scratches mostly appear random in direction and length, with a few strong exceptions that are parallel and may be attributable to the same source.

#### **Japanese Kutani Vase (TRH135)**

Item description: "*This is one of a pair of similar but not identical Japanese Kutani vases at Trinity House. The vases are painted in reds, blues, and gilts, with panels of flower spray. Kutani ware is a style of Japanese porcelain traditionally from Kutani, now a part of Kaga, Ishikawa, in the former Kaga Province.*" Item dimensions: 690 × 310 mm (HES, Vernon Collections Management System, 'Object - TRH135' accessed 25/10/21).

**Fig. 8.** Japanese Kutani Vase - Detailed area excerpt of UV texture sheet maps in the following order left-to-right: (A) diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. Assession number TRH135.

#### *Discussion and Interpretation*

With multiple painted, gilt and glazed surfaces, the finish on this vase presents a challenge to capture photogrammetrically. It is also an opportunity to more accurately visualise the mixed materials of the surface decoration. Figure 8(B) shows the relatively smooth surface of the normal map for the body of the vase. Figure 8(C) immediately shows the higher specular albedo associated with the gilt areas, and the relatively high average specular value across the body of the vase should be noted due to the reflective glaze. An unpolarised photogrammetry approach would not differentiate between the gilt areas and those with similar albedo levels or colour values.

As seen in Fig. 9, under normal 'unpolarised' lighting conditions the painted and gilt areas are occluded where the specular highlight falls on the vase body, with a light veiling sheen across the surrounding area. The combination of these effects reduces the contrast of the diffuse albedo map, obfuscating detail.

**Fig. 9.** An excerpt of a cropped photograph from the 'unpolarised' photogrammetry imagery for item TRH135. Note the specular highlight from the studio light visible in the top centre area.

The specular albedo map in Fig. 8(C) enables spatially varied specular reflection through the material shader. However, the metallic map is used in addition to denote the spatially varied metallic response from the object surface. The difference map produced from the unpolarised and cross-polarised diffuse albedo maps is compared to the unpolarised diffuse map in Fig. 10. Figure 10(B) shows the metallic gilt areas, with a strong gold response highlighting the areas on the vase containing decorative gilt finish, in contrast to the painted and unpainted areas. The influence of the glaze in reducing contrast through veiling light is negligible. For material shaders that require a grayscale raster image for the metallic map, such as Sketchfab's PBR shader (2021), this 'difference map' can be further processed as discussed in Sect. 2.2.

### **3.2 Metal**

Metallic items, or artefacts and surfaces with metal components, can exhibit a range of reflective properties largely dependent on the condition and finish of the surface. For example, heavily oxidised metal surfaces may show lower specular albedo values, such as rusted iron or a bronze verdigris are present. Highly polished metals, often associated with jewellery can reach a mirror-like reflective level. Brushed metal finishes can

**Fig. 10.** Comparison between (A) the unpolarised diffuse albedo map, and (B) the difference map, calculated from unpolarised vs. cross-polarised diffuse maps. Minor histogram levels adjustments were made to increase contrast for the higher intensity values.

often exhibit anisotropic reflections, which can be represented by more complex physically based material shaders. As conductors, bare (i.e., unpainted, un-oxidised) metals behave differently to dielectric materials in physically based rendering systems (Westin et al. 2004). Usually these have a dedicated parameter for material shaders representing spatially homogenous surfaces, or a texture map for spatially varied surfaces.

#### **Greyhound Desk Weight (DC351)**

Item description: "*This is one of a pair of greyhound desk weights, dating to the late 18th or early 19th-Century. Although very French in their making, cast gilt-finished metal (ormolu) and ornament of laurel-leaf swags, these little dogs have had a long classical tradition. An example of a similar sculpture are the 'Townley Greyhounds', which date from the 2nd century BC and were an inspiration to a generation of artist animaliers (animal painters) in the 19th century*." Item dimensions: w172 mm (HES, Vernon Collections Management System, 'Object - DC351' accessed 25/10/21).

*Discussion and Interpretation*

The cross-polarised results for the metal Greyhound Desk Weight show a striking difference to the other items. The metallic surfaces are finished to a high sheen, which shows in the diffuse albedo when compared side by side with the cross-polarised and unpolarised workflows (as visible in Fig. 12). The specular albedo contains high values, with surface scratches, dents and marks highly visible.

The metallic composition and condition of item DC351 has resulted in a surface of varied colour. This can be seen on the body of the Greyhound figure, where a visible

**Fig. 11.** Greyhound DeskWeight - Detailed area excerpt of UV texture sheet maps in the following order left-to-right: (A) diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. Assession number DC351.

gradient appears to show a transition from lighter bronze/gold colour to a deeper red. This can be seen partly in Fig. 11(A), and more closely in Fig. 12. These differences in colour between swatches A and B appear visible to the naked eye between the techniques. To better define the differences between the unpolarised and cross-polarised approaches we have opted to use the CIEDE2000 method (Sharma et al. 2005) to illustrate:


Unpolarised workflow CIEDE2000

### Cross-Polarised workflow CIEDE2000


These results indicate that the CIEDE2000 shows greater difference between the swatches from the cross-polarised workflow. The extent of the colour difference is in the perceptible range (>1) to the observer, which suggests an incremental improvement in the ability to distinguish between the colour of the surface characteristics of the metal item using the cross-polarised workflow.

It may be that this variation in colour is due to the deterioration or change to the gilt-finished ormolu surface. If this is the case, this comparison illustrates the ability to better differentiate between the areas where the finish is intact, document its present condition and better record any further changes.

**Fig. 12.** Comparison between unpolarised (left) and cross-polarised (right) diffuse albedo texture maps for item DC351. Swatches A and B (for both images) are sampled from identical locations and represent averaged colour values for identical areas on both maps.

A further observation is the presence of fine particulate matter on the item surface, e.g., dust, lint, etc. Typically, this matter can collect on an item depending on storage, or more likely, display conditions. As shown in Fig. 13, the presence of this is immediately obvious in the cross-polarised image texture and the derived specular map. This is due to the highly diffuse characteristics of the particulate matter. This is nearly indistinguishable in the unpolarised (i.e., typical) approach due to the occluding glare of the reflective surface from the lighting conditions. The ability to detect this could be valuable for fine-level assessment of condition, for example, of items before and after public display (where it may be exposed to uncontrolled environmental conditions), or before and after the item returns from a loan.

#### **3.3 Wood**

As a natural, organic material, wood can vary hugely in its visual characteristics. Its use for cultural artefacts is usually accompanied by different decorative and/or protective finishes that can include the use of pigments in paint application, varnishes and waxes, that can all alter its reflective properties. Fundamental differences such as softwoods versus hardwoods can govern many aspects of the visual appearance. Whilst items made from bare or natural wood might have a visually consistent presentation for the wood substrate, as an assembly an item might include metallic fixings or other inorganic materials. Like ceramics, wood is a dielectric material. Cross-polarised light has also been used for specific research applications into wood related to enhancing tree-ring inspection (Gärtner et al. 2015).

**Fig. 13.** Comparison between excerpts of item DC351 image texture maps showing diffuse albedo (unpolarised, left), diffuse albedo (cross-polarised, centre) and specular albedo (right).

### **Stirling Head (STC022)**

Item description: "*Oak roundel, part of a series of carvings that decorated the ceiling of the King's Presence Chamber at Stirling Castle. The pose and costume in this Head appears in contemporary paintings of flamboyant noblemen. He is shown wearing an elaborate and stylish quilted doublet. The Stirling Heads are constructed of 3 panels, originally only secured by glue. The great majority of the Heads, however, are in part constructed in 2 ply, an additional board or block having been placed over the middle board to enable the carver to model the central portion of the medallion in high relief. Nail holes are apparent in the borders which indicate the way in which the Heads were fixed to the ceiling.*" Item dimensions: diameter 762mm. (HES, Vernon Collections Management System, 'Object - STC022' accessed 25/10/21).

*Discussion and Interpretation*

Initially the specular map results are apparent across the entire surface of the Oak roundel. A subtle, consistent level of reflectance can be observed from the specular map in Fig. 14(C). Areas of obvious occlusion and a reduction in specular response from surface geometry can be found for carved recessed details, such as the figure's mouth, fingers and details of the clothing. This can be compared with the normal map in Fig. 14(B), showing a geometric correlation to these details.

Figure 15 compares an unpolarised photograph with a cross-polarised photograph, showing the veiling effect of the specular response for the underlying wood tone. Whilst the specular response does not appear to affect the legibility of wood grain, it does occlude the surface colour of the wood, particularly around the specular highlight in the image centre.

**Fig. 14.** Stirling Head - Detailed area excerpt of UV texture sheet maps in the following order left-to-right: (A) Diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. Assession number STC022.

**Fig. 15.** Split image showing side-by-side results for photographs of item STC022 from (A) the cross-polarised setup (top half) and (B) unpolarised photograph (bottom half).

The 'figure' of wood (often simplified, or referred to, as 'grain') is a key feature of the material's appearance. It can provide key diagnostic information for inspection purposes, both in terms of identifying wood species and the orientation of the surface from cutting (Hoadley 2000, p. 25). Figure 16 illustrates an example wherein the characteristics of this figure can be seen. Initially there appear to be few differences between the crosspolarised and unpolarised results, however the specular albedo map and surface normal map reveal that the undulations of the surface geometry caused by the carved wood surface interacting with the figure of the wood to affect the reflectance characteristics. The specular map clearly captures the recesses and raised areas in detail, allowing more accurate visualisation and interrogation of the surfaces in the captured data.

**Fig. 16.** Comparison of item STC022 between excerpts from the diffuse albedo (cross-polarised, left), diffuse albedo (unpolarised, centre-left), specular albedo (difference product, centre-right) and tangent-space surface normal (right).

It was unclear from the outset of the project whether there would be an appreciable benefit from using the cross-polarised workflow for a wooden item. The possible benefits demonstrated above should be weighed against the increased fieldwork/data capture and processing time. This is particularly true for sensitive organic items that may normally be stored in a protective environment.

### **4 Conclusion**

Structure from Motion photogrammetry has become widely embraced as a powerful and flexible 3D data capture technique. However, as indicated in this paper more complex materials, surfaces and items can hinder robust and consistent results. The technique has seen development within the realm of computer imaging and rendering, which has driven more sophisticated modes of data capture and visualisation. However, best practices and guidance for geospatial application of photogrammetry is still primarily focussed on recording 'diffuse' surfaces within a measurable survey tolerance. This often neglects the rich and spatially varied material characteristics seen across cultural heritage that are able to be represented through sophisticated material shaders.

While not unlike a typical photogrammetry workflow, this cross-polarised method of capture provides information about cultural heritage artefacts that another technique could miss. Indeed, in some cases imperfections on the surface that are only revealed with these methods change the appearance of an object entirely.

More than improving the visualisation of artefacts, there are potential conservation applications in using richer material shaders and a greater depth of surface information, including:


The workflow described here is far more involved that the standard approach to single camera photogrammetry, extending the time required during both data capture and processing stages. That said, these results are promising when applied to documenting complex objects that are a challenge to capture using unpolarised light workflows. The improvements to the quality of image textures and the 3D data captured for these objects has been identified and explored for these cultural heritage artefacts. These items are exemplars of complex surfaces with varied materials, and spatially varied reflective properties. The work described here has established a foundation for further research in creating a standardised cross-polarised workflow in a heritage context in the future.

### **Appendix**



### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

### **Author Index**

**A** Anagnostopoulos, Christos-Nikolaos 65

### **B**

Basiakoulis, Alexandros 103 Blersch, Daniel 34 Brumana, R. 77

**C** Chatzistamatis, Stamatis 65 Cliffen, Harriet 103

**E** E.Tsekouras, George 65 Efstathiou, Kyriakos 103 Efstathiou, Marianna 103

**F** Fröhlich, Christoph 34 Frost, Adam 131

**G** Georgiou, Rafaella 103 **H**

Haynes, Ronald 50 Held, Christoph 34

**I** Ioannides, Marinos 1

**J** Joncic, Nenad 103

**M** Mettenleiter, Markus 34 Millan, Elina Rodriguez 131 Mirashrafi, Sophia 131

**P** Patias, Petros 1

**S** Sánchez, Clara Molina 131

**V** Vacas-Madrid, David 131

**W**

Wilson, Lyn 131