# Electronic Imaging & the Visual Arts EVA 2016 Florence

PROCEEDINGS Editors: Vito Cappellini and Enrico Del Re

#### Proceedings e report

**Electronic Imaging** R **the Visual Arts**

**EVA 2014 Florence**

7 – 8 May 2014

*edited by*  Vito Cappellini

Firenze University Press 2014

# **Electronic Imaging** R **the Visual Arts**

# **EVA 2016 Florence**

11-12 May 2016

*edited by*  Vito Cappellini and Enrico Del Re

Firenze University Press 2016

Electronic Imaging & the Visual Arts : Eva 2016 Florence / edited by Vito Cappellini and Enrico Del Re. - Firenze : Firenze University Press, 2016. (Proceedings e report ; 111)

http://digital.casalini.it/9788866559740

ISBN 978-88-6655-973-3 (print) ISBN 978-88-6655-974-0 (online)

#### Peer Review Process

All publications are submitted to an external refereeing process under the responsibility of the FUP Editorial Board and the Scientific Committees of the individual series. The works published in the FUP catalogue are evaluated and approved by the Editorial Board of the publishing house. For a more detailed description of the refereeing process we refer to the official documents published on the website and in the online catalogue of the FUP (www.fupress.com).

#### Firenze University Press Editorial Board

G. Nigro (Co-ordinator), M.T. Bartoli, M. Boddi, R. Casalbuoni, C. Ciappei, R. Del Punta, A. Dolfi, V. Fargion, S. Ferrone, M. Garzaniti, P. Guarnieri, A. Mariani, M. Marini, A. Novelli, M.C. Torricelli, M. Verga, A. Zorzi.

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0: https://creativecommons.org/licenses/by/4.0/legalcode)

CC 2016 Firenze University Press Università degli Studi di Firenze Firenze University Press via Cittadella 7, 50144 Firenze, Italy www.fupress.com Printed in Italy

# **PROGRAM**

# **Electronic Imaging** ! **the Visual Arts**

### **'The Foremost European Electronic Imaging Events in the Visual Arts'**

#### **Forum for Users, Suppliers & Researchers**

The key aim of this Event is to provide a forum for the user, supplier and scientific research communities to meet and exchange experiences, ideas and plans in the wide area of Culture & Technology. Participants receive up to date news on new EC and international arts computing & telecommunications initiatives as well as on Projects in the visual arts field, in archaeology and history. Working Groups and new Projects are promoted. Scientific and technical demonstrations are presented.

## Main Topics Main Topics


#### **WHO SHOULD ATTEND**

*THE CULTURAL SECTOR:* The Visual Arts Community including Museums, Libraries, Archaeological Sites, Educational Institutions, Commercial Galleries and Dealers, Auction Houses, Artists & Collectors

*THE HI-TECH INDUSTRY SECTOR:* Multimedia Systems, Image Acquisition & Analysis, Data-bases, Display & Printing, ICT Industry, Telematics & Systems Manufacturing, On-line Information Services

*MEDIA & RELATED SECTORS:* Publishing, Press, Film, Television, Photography, Printing, Advertising, Graphics Design, Consumer Media

*IMAGING SYSTEMS RESEARCHERS:* Imaging Systems, 3-D Acquisition, Reconstruction & Representation Systems, Information Sciences

*TOURISM & TRAVEL SECTOR:* Tourism Agencies & Operators, Travel Agencies

*THE GOVERNMENT SECTOR:* Ministries of Culture and other Institutions involved in Cultural Heritage, Ministries of Industry, Education, Research and Science, Regional Governments

#### **SPONSORS & SUPPORTERS INCLUDE:**

ALINARI, ICESP, and CASSA DI RISPARMIO DI FIRENZE

EUROPEAN COMMISSION, THE UNIVERSITY OF FLORENCE*,* 

REGIONE TOSCANA, COMUNE DI FIRENZE,

DIPARTIMENTO DI INGEGNERIA DELL'INFORMAZIONE - UNIVERSITA' DI FIRENZE,

CENTRO PER LA COMUNICAZIONE E L'INTEGRAZIONE DEI MEDIA,

ISTITUTO DI FISICA APPLICATA "NELLO CARRARA" – C.N.R.

CONFINDUSTRIA FIRENZE,

GTTI, CNIT,

HITACHI LTD.,

GRUPPO SESA,

CABEL,

T.T. TECNOSISTEMI,

FRATELLI ALINARI IDEA,

CENTRICA,

NEUMUS,

VIDITRUST,

VIRTUITALY,

INN-3D,

ICESP,

CONSERVATORIO DI MUSICA LUIGI CHERUBINI,

**Chairman: Vito Cappellini – Florence University Co-Chairman: James Hemsley – EVA Conferences International**  ENTE CASSA DI RISPARMIO DI FIRENZE.

**vito.cappellini@unifi.it jrhemsley@hotmail.com**

#### **Co-Chairmen: Vito Cappellini, Enrico del Re – Florence University vito.cappellini@unifi.it enrico.delre@unifi.it**

#### **ADVISORY BOARD**

President: James Hemsley, EVA Conferences International jrhemsley6@gmail.com

Cristina Acidini, Accademia delle Arti del Disegno, Florence, Italy Paolo Blasi, University of Florence, Florence, Italy Enrico Bocci, Progetto Ricerca e Innovazione, Confindustria Florence, Florence, Italy Nikolay Borisov, ITMO University, Saint Petersburg, Russia Edoardo Calia, Istituto Superiore Mario Boella, Turin, Italy Andrea De Polo, Fratelli Alinari IDEA, Florence, Italy Takashi Hamazaki, DIS Project, Hitachi Ltd., Yokohama, Japan

EVA Conferences Organiser: Monica Kaayk, acinom\_5@hotmail.com

#### **EVA 2016 Florence Technical Organising Committee includes:**

Antonella Castaldi, Roberto Caldelli, Alessandro Nozzoli, Alessandro Piva, Francesca Uccheddu, Luca Belloni, Riccardo Saldarelli, Giuliana Signorini, Silvia Capecchi, Paola Imposimato, Claudia Riva di Sanseverino

EVA Organiser: Dipartimento di Ingegneria dell'Informazione (DINFO) - University of Florence Via di Santa Marta, 3 – 50139 Firenze, Italy Tel.: (+39) 055 2758575 - Fax: (+39) 055 2758570 Web pages ~ http://iapp.det.unifi.it/uploads/documents/highlights/announcement.pdf

For general information: Dipartimento di Ingegneria dell'Informazione (DINFO) Università degli Studi di Firenze Via di Santa Marta, 3 – 50139 Firenze, Italy Tel.: (+39) 055 2758575 – Fax: (+39) 055 2758570 E-mail: vito.cappellini@unifi.it

*For information on the Exhibition: CENTRICA S.r.l. Piazza della Madonna della Neve, 5 - 50122 Florence - Italy Tel.: (+39) 055 24 66 802 – Fax: (+39) 055 20 09 785 E-mail: info@centrica.it Web pages ~ http://www.evaflorence.it*

## **PROGRAM**

## **1 - CONFERENCE**


## **2 - WORKSHOP**

## **3 - SPECIAL EVENT**

Wednesday, 11 May: 19,30 – 21,30

## **4 - TECHNICAL EXHIBITION**

Thursday, 12 May: 10,00 – 17,00

Venue: **Hotel Pierre**  Via De' Lamberti, 5 50123 Firenze Tel. +39 055 216218 Fax +39 055 2396573 E-mail: pierre@remarhotel.com www.remarhotels.com

**ROOM A** 

## **1 - CONFERENCE**

## **Wednesday, 11 May**


14,15 *Opening: Luigi Dei, Rector of University of Florence Enrico Del Re, Director of Department of Information Engineering – University of Florence James Hemsley, EVA Conferences International Pier Luigi Rossi Ferrini, Vice-President Ente Cassa di Risparmio di Firenze Paolo Castellacci, President GRUPPO SESA* 

15,15 Coffee Break

#### **ROOM A**

15,30 **SESSION 1 – STRATEGIC ISSUES**  *Chairman: Paolo Blasi, University of Florence, Florence, Italy* 


"Museo Nazionale del Bargello" Ilaria Ciseri

Museo Nazionale del Bargello, Florence, Italy

*"*Uffizi Virtual Experience. Da Giotto a Caravaggio"

Marco Cappellini1 , Paolo De Rocco<sup>1</sup> , Paolo Romoli<sup>1</sup> , Marxiano Melotti<sup>2</sup> , Vito Cappellini<sup>3</sup> , 1 VirtuItaly, Florence, Italy 2 Bicocca University, Milan, Italy 3 University of Florence, Florence, Italy

#### **ROOM A**

#### 17,30 **SESSION 2 – NEW SCIENCE AND CULTURE DEVELOPMENTS & APPLICATIONS**  *Chairman: Konstantine Karczmarski, University Innovation Office, ITMO University, St.Petersburg, Russia*

"Beauty as a Bridge Between Cultures" Giuseppe O. Longo

"Recent advances in spectral imaging for cultural heritage documentation and analysis"

"Application of Big Data and Content Curation to Exploitation of Cultural Heritage"

"Piezomusicolor. A natural Form of Technological Art"

Jon Yngve Hardeberg, Sony George, The Norwegian Colour and Visual Computing

Laboratory, NTNU, Gjøvik, Norway

Konstantine Karczmarski Department of Innovations, ITMO University, Saint Petersburg, Russia

Information Theory, University of Trieste, Trieste, Italy

Giuseppe Caglioti<sup>1</sup> , Marco Marcon<sup>1</sup> , Tatiana Tchouvileva<sup>1</sup> , Riccardo Della Ragione<sup>2</sup> 1 Politecnico di Milano, Milan, Italy 2 Magic Music s.a.s, Livorno, Italy

"The Use of Patient Specific Instrumentation during Total Knee Replacement Surgery"

Lawrence Camarda, Antonio D'Arienzo, Salvatore Morello, Michele D'Arienzo Orthopaedic and Traumatology Department, University of Palermo, Palermo, Italy

## **Thursday, 12 May**

#### **ROOM A**

#### 9,00 **INTERNATIONAL FORUM ON "CULTURE & TECHNOLOGY**  *Chairman: Vito Cappellini, University of Florence, Florence, Italy*

The structure of the FORUM is presented. Actual developments and perspectives are outlined:


*Speakers Include*:

*- Cristina Acidini, President Accademia delle Arti del Disegno, Florence, Italy* 

*- Nikolay Borisov, President of Center of Design and Multimedia, ITMO University, Saint Petersburg, Russia* 

*- Edoardo Calia, Research Director, Istituto Superiore Mario Boella, Torino, Italy* 

*- Alberto Del Bimbo, Director Centro per la Comunicazione e l'Integrazione dei Media, Florence, Italy* 

*- Monica Carfagni, President Promo Design, Calenzano, Florence, Italy* 

*- Fabio Donato, University of Ferrara, Italian Representative in HORIZON 2020-SC6 Committee* 

*- Paolo Zampini, Director of Conservatorio di Musica Luigi Cherubini, Florence, Italy* 

11,00 Coffee Break

11.15 **SESSION 3 – NEW TECHNICAL DEVELOPMENTS & APPLICATIONS**  *Chairman: Jon Yngve Hardeberg, The Norwegian Colour and Visual Computing Laboratory, NTNU, Gjøvik, Norway*

"Getting Past the Fear of Virtual Innovation" Eugenia Romanelli

Italian Journalist Rome, Italy


13,15 Lunch Break

#### **ROOM A**

#### 14,40 **SESSION 4 – MUSEUMS – VIRTUAL GALLERIES AND RELATED INITIATIVES**

*Chairman: Andrea De Polo, Fratelli Alinari IDEA, Florence, Italy* 

"Terahertz Advanced Research Techniques for Non-Invasive Analysis in Art Conservation (THz-ARTE)"

M. Picollo<sup>1</sup> , A. Aldrovandi1,2, G. Bartolozzi1 , A. Casini<sup>1</sup> , C. Cucci<sup>1</sup> , A. Doria<sup>3</sup> , K. Fukunaga<sup>4</sup> , G.P. Gallerano<sup>3</sup> , E. Giovenale<sup>3</sup> , R. Olmi<sup>1</sup> , M. Poggesi<sup>1</sup> , L. Stefani<sup>1</sup> 1 "Nello Carrara" Institute for Applied Physics of the Italian National Research Council (IFAC-CNR), Sesto Fiorentino, Florence, Italy 2 Opificio delle Pietre Dure (OPD), Florence, Italy 3 National Institute of Information and Communications Technology (NICT), Tokyo, Japan 4

ENEA-Frascati, Frascati, Rome, Italy 14

"The Secrets of the Work of Art. Mysteries concealed behind old Restorations and the Truth Revealed "

Sara Penco, Restorer ad Creator for the "SMARTICON Project", Rome, Italy

"Cultural Assets, 'Extended' Museums and the Historic City: the Case Study of an App for Genoa"

Valentina Fiore, Lauro Magnani, Sara Rulli Dept. D.I.R.A.A.S. Università degli Studi di Genova Genoa, Italy

"3D Services at Museo Galileo" Luisa Barattin, Marco Berni, Elena Fani Museo Galileo - Institute and Museum of the History of Science, Florence, Italy

16,00 Coffee Break

**ROOM A** 


## **2 - WORKSHOP**

#### **ROOM A**

#### **WORKSHOP INNOVATION AND ENTERPRISE – INNOVAZIONE E IMPRESA**

*(Italian Language)*  9,00 – 13,00

*Chairman: Enrico Bocci, Progetto Ricerca e Innovazione, Confindustria Firenze e Presidente Opera Medicea Laurenziana, Florence, Italy* 

Technological requirements in the Cultural Heritage field are outlined and opportunities for Italian Enterprises and SME's working in the field, using new technologies, are presented.

Regional and national applied research Programs in Italy are described.

Activities by National Organizations and Firms working in the area of Telecommunications, Informatics, Environment and Infomobility are presented.

Funding by European Commission is considered, with particular reference to multimedia and telematics for Cultural Heritage. Special consideration is given to the new EC Plan HORIZON 2020.

Initiatives regarding the "know-how" transfer from Research Organizations to the Industrial Sector are described.

Organizations and Companies present their activities and experiences.

*Invited Speakers: - Andrea Arnone, Pro-Rettore al Trasferimento Tecnologico e Presidente di CsaVRI, Università degli Studi di Firenze*

*- Laura Castellani, Responsabile del Settore Infrastrutture e Tecnologie per lo Sviluppo della Società dell'Informazione, Regione Toscana*

*Speakers include: - Renzo Zampini, INFOCAMERE* 


## **3 - SPECIAL EVENT**

### **Wednesday, 11 May 19,30 – 21,30**

*Visit to Opera Medicea Laurenziana Piazza San Lorenzo, 9, Florence* 

 *(in cooperation with Antica Compagnia del Paiolo)* 

## **4 - TECHNICAL EXHIBITION**

#### **Thursday, 12 May: 10,00 – 17,00**

#### *ROOM B*

*For information on the Exhibition: CENTRICA S.r.l. Piazza della Madonna della Neve, 5 - 50122 Firenze - Italy Tel.: (+39) 055 24 66 802 – Fax: (+39) 055 20 09 785 E-mail: info@evaflorence.it Web pages ~ http://www.evaflorence.it/home.php*

# PROCEEDINGS

# "#\$%#&'()!(""\*&"!

Vito Cappellini and Enrico Del Re (edited by), *Electronic Imaging & the Visual Arts. Eva 201* 21 *6 Florence.* ISBN 978-88-6655-973-3 (print) ISBN 978-88-6655-974-0 (online) © 2016 Firenze University Press

## INNOVATIVE TOOLS FOR THE CREATION, REUSE AND RETRIEVAL OF DIGITAL CULTURAL CONTENT

Rossella Caffo

President of Michael - Culture Association

<rosa.caffo@beniculturali.it>

The Michael Culture Association will present 4 open-source tools developed within the AthenaPlus project, targeted mailnly to GLAMS curators who want to valorize their collections and cultural content in innovative ways. MOVIO: An open source and user-friendly tool for creating digital exhibitions in an easy, creative and fun way. It can be used by curators, students, schools, and whoever wants to create a digital exhibition. Urban Explore: a tool to create and manage mobile apps as touristic and cultural digital guides for collective or solo visits. As a complement to a spoken story (by a heritage expert, historian, city planner, or art professional), the mobile app offers an access to the audio-visual history or memory of a place, in an original and emotional way. CityQuest: a tool that allows cultural organisations to easily create a quest online, and publish it to a mobile app. SchoolTrip: a tool that allows students to create their own school journey. Through an online interface the teacher can set a couple of parameters defining the skeleton of the trip.

## IP STRATEGIC ROLE

#### Daniela Mainini

#### President of Anti-Counterfeiting Center

<presidenza@anticontraffazione.org>

#### ARSTRACT

IPR are crucial for the development of Industry and its ability to compete in a global economy.

The establishment of IP Rigths is one of the principal means by which companies, creators and inventors generate returns on their investment in knowledge.

In practical terms, through the granting of temporary exclusive rights, IP is directly linked to the production and distribution of new and authentic goods and services, from which all citizens benefit.

The March 2015 European Council reaffirmed the importance of IP as a Key driver for growth and innovation and highlighted the need to fight against counterfeiting to enhance the EU's industrial competitive scenario.

To this end, the key to achieving these goals is an optimal and economically efficient IP "infrastructure " which spans the legal recognition, utilization and balanced enforcement of all forms of IP rights.

#### Price and not value:

The Consumers often see only the price and not the value of what they have bought

As Warren Buffett says: Price is what you pay. Value is what you get.

The value of an authentic product is not just the price. Value includes legal employment, production practices, R&D, proprietary designs, marketing and product associated services.

To this end, we are working to educate the consumer but one of the major tasks is to grow awareness among entrepreneurs.

Italy should therefore dominate the ranking of the IPRI (International Patent Registration Index) and emerge as one of the countries that patents the vast majority of new products. It should be taking greater measures to protect intellectual property and manufacturing processes. This is clearly not the case. As the evidence from the European Patent Office (EPO) shows, Italy is behind Germany and France in the number of patents registered.

#### ICT

Information Communication Technology plays an essential role in protecting retail distribution and logistics systems and helps detect and exclude IP infringing products.

In order to ensure product distribution chains are protected and not infiltrated by IP infringing products, technologies are being applied - such as the combination of BAR codes and the application of Radio frequency Identification (RFID)

ICT has, of course, also allowed for an increase in direct delivery to the final consumer opening the way to the web and its huge supermarket, which is open 24/7. Equally, ICT solutions are also empowering consumers to fight against counterfeiting and piracy.

Of course: the basis for consumer empowerment is to be found in education e.g. Ethical traceability which tracks a product from farm to plate, is now widely used in the food sector, and aims at giving consumers an informed choice.

The Italian Parliament is working to incentivise enterprises that are using ICT devices in their Supply Chains and finally the patent box has entered into force.

However it is impossible to ignore the challenges presented by advancing technology . In the next twenty years we will be able to download a copy of art works which will be identical to the original. We only have to look 3D printing to understand that IP rights protection must keep step.

## HIGH QUALITY ARCHIVE PROJECT FOR POLO MUSEALE FIORENTINO: EXPLOITATION ACTIVITIES

#### Demonstrations and Exhibitions in Japan

#### Takashi Hamazaki

Director DIS Project, Hitachi, Ltd., Yokohama, Japan takashi.hamazaki@hitachi.com

#### SUMMARY

The "High Quality Archive Project for Polo Museale Fiorentino", developed by MICC -University of Florence, Hitachi Ltd. and Centrica S.r.l., with supervision by Polo Museale Fiorentino, has been completed the first stage of digitization. Major activities in Japan under this project since the last presentation at EVA2015 so far are as follows.

#### 1. UVM2016 in Tokyo

The "Uffizi Virtual Museum (UVM) 2016" was opened to the public from 13 February to 13 March 2016 at Italian Cultural Institute in Tokyo as one of events for "150" Anniversary of diplomatic relationship between Italy and Japan". UVM2016 was promoted by Italian Embassy in Japan, Italian Cultural Institute and Department of Information Engineering at the University of Florence.

We have fully upgraded contents of the previous UVM, which was developed in 2011 and presented at EVA2012. First of all, the number of artworks covered this time was increased from 10 to 23. As for the quality of digital image presentation, we have utilized 4K devices to show them in better quality. Our unique high definition digital technology could make the best use of the latest devices, in a sense.

The exhibition was composed of following 4 parts, i.e. "Life-Size Replica", "Digital Theater", "Masterpiece Navigation" and "Feel Uffizi". The combination of these various kinds of expressions could deeply move audiences more effectively.

#### 2. Permanent Exhibition (Hitachi DIS Showroom)

Hitachi DIS Digital Museum, permanent Exhibition site at Yokohama, continues (since its opening in October 2012) to show UVM contents and other replica & digital data newly digitized to general public for free of charge. Its main purpose is to introduce Japanese and Western Culture including Italian Culture through our technology.

Approximately 50 people visit the Museum on monthly basis with favorable reputations. Since this site is a real contact point with various type of audience, it should be very useful for us to develop a better plan.

#### 3. Miscellaneous Demonstrations

To take advantage of the outcome from the project, we demonstrated our high-definition digital data at any opportunities, which include following events.

The Lecture session for art-works of Renaissance, which is promoted by Italian Consulate General in Osaka, by Prof. Okada, Kyoto University, has taken place at the Museum of Kyoto in Nov. 2015. Masterpiece Navigation with digital data has been utilized to visualize his session effectively.

The Exhibition of Leonardo Da Vinci by Kansai Telecasting Corporation has been taken place from Dec. 2015 to Feb. 2016 in Osaka. Masterpiece Navigation with digital data of Leonard Da Vince has been presented there.

## eFoto Hamburg Lab

Christoph Wienberg1, Horst Scholz2

1TT and Project Management Specialist at the Office of Hamburg's CIO. Hamburg, Germany Christoph.wienberg(@fb.hamburg.de

2 Art Historian and Head of Information Technology and Digital Projects at the Hamburg Ministry of Culture, Hamburg, Germany horst.scholz@kb.hamburg.de

#### Abstract

Like many metropoles, Hamburg has vast collections of historic photos. Creating access for everyone to these collections is a task assigned to the Ministry of Culture by the Hamburg government as part of comprehensive programs covering digital / smart city strategies and the eCulture Agenda 20201.

To achieve digital access to the collections, several issues have to be addressed. Images must be digitalized and indexed. Legal issues have to be solved. A central digital platform to manage, store and publish images and metadata has to be built. But most important of all, applications to present those images in an entertaining and interactive, yet reputable way have to be created, to achieve public attention and participation.

Hamburg has set up the eFoto2 project to address this complex challenge. In eFoto, we will include image collections from several stakeholders, ranging from museum collections of photographic art (scanned and scientifically indexed) to local historic societies. Concepts and use cases have been developed in cooperation with Professor Jan-Christoph Meister and his team at the University of Hamburg, Faculty of the Humanities, using concepts of Niklas Luhmann.

For a start, the topics of industrial history, national socialism and urban development / gentrification have been used to create historic city walks in the city quarter of Ottensen, using historical fotos from various sources and re-recorded historic statements. All Content has been researched and developed especially for eFoto by "Stadtteilarchiv Ottensen", a local historical society. Users will soon be guided through Ottensen by our mobile application. They will also be able add their own recordings of contemporary witness statements. A field test with pupils is scheduled for June 2016. A rollout to more partner organisations in

<sup>4</sup> http://www.hamburg.de/kulturbehoerde/eculture

<sup>2</sup> http://www.efoto-hamburg.de/

<sup>3</sup> http://jcmeister.de/

http://stadtteilarchiv-ottensen.de/

Hamburg, more features and a browser interface are planned for a second release scheduled for the end of 2016.

Our implementation partner is HITeC e.V. , a spinoff of the IT faculty of Hamburg University. A central multi-client repository is in the process of implementation, as well as mobile apps for iOS and Android for the city walks (fed from a central content management system). Current software modules include Liferay (a CMS capable of community management, free content object definitions and multi-language pages), Neo4J (a graph database used to store object relations) and RIAK, an object store used to store media data and compatible to Amazon S3. The metadata model is based on CIDOC and LIDO.

We would like to invite interested parties to join our workshop. Have a look at the brand new beta release of the eFoto App and platform; discuss technical matters and (future) ideas of presentation and participation. Cooperation would be welcome in IT development (it's all open source), but also in the use of the eFoto platform, filling in your own local content.

<sup>5</sup> http://www.hitec-hh.de/

#### MUSEO NAZIONALE DEL BARGELLO

Ilaria Ciseri

Museo Nazionale del Bargello Florence, Italy <ilaria.ciseri(a)beniculturali.it>

The Museo Nazionale del Bargello is among the most important museums in Florence. Here are displayed collections of Renaissance sculpture, with marble, wood and bronze masterworks by Donatello, Verrocchio, Desiderio da Settignano, Della Robbia, Michelangelo, Cellini, Giambologna and many others great artists. The museum includes also famous collections of decorative arts, coins and medals, weapons, textiles, paintings and one of the main collection of maiolicas in the world. The Bargello celebrated its 150th anniversary in 2015, since it is the first National Museum of Italy, estabilished in 1865 in the ancient Palazzo del Podestà: among the founding collections was the historic Grand Ducal Armoury, that includes the Medici and Della Rovere weapons and comprises over 2000 items. Every year the Museum organizes temporary exhibitions concerning sculpture and decorative arts, hosting also innovative projects that open towards advanced technologies. In 2015 has been presented "Mnemosyne - Smart Museum Project", a sperimental project funded by the European Commission and Regione Toscana and realized by the MICC of the University of Florence: the project developed a system wich analyse visitor's behaviour through a multimedia touchscreeen collocated in the Donatello Room.

## UFFIZI VIRTUAL EXPERIENCE-DA GIOTTO A CARAVAGGIO

Marco Cappellini¹, Paolo De Rocco¹, Paolo Romoli¹, Marxiano Melotti², Vito Cappellini3.

1VirtuItaly, Florence, Italy, m.cappellini@virtuitaly.com , p.derocco@virtuitaly.com, p.romoli@virtuitaly.com 2Bicocca University, Milan, Italy marxiano.melotti@gmail.com 3University of Florence, Florence, Italy vito.cappellini(@unifi.it

ABSTRACT

This paper presents "Uffizi Virtual Experience - da Giotto a Caravaggio", the first immersive and interactive virtual exhibition created in Italy, which brought to Milan the masterpieces of the Florence Uffizi Gallery with high-resolution digitized images.

The exhibition, which was produced by VirtuItaly, a Centrica startup spin off company, conveys a new approach to edutainment, where cultural recreation is achieved through an immersive educational and entertaining experience.

Thanks to digital technologies and interactive systems, educational content and cultural enjoyment are integrated with emotional aspects to create a satisfying formative activity.

The paper dwells upon the technological, sociological and cultural aspects of this new way of presenting and promoting cultural heritage in our society.

# i&Z!")(&i)&!%i]!)\*F#\*\$&! ]&J&FI17&i#"!! \!%11F()%#(Ii"!

## BEAUTY AS A BRIDGE BETWEEN CULTURES

#### Giuseppe O. Longo

Professor Emeritus of Information Theory University of Trieste, Italy giuseppe.longo41(@gmail.com

> The perception of beauty is produced by an emotional catastrophe René Thom

#### Abstract

A naturalistic definition of beauty is proposed based on our immersion in a systemic context and in a vast ongoing co-evolutionary process: beauty exhibits itself and is recognized by us through the interaction and the reciprocal dynamical adaptation between ourselves and the environmental context. This system evolves in time and so does that adaptation, hence beauty has a historical character and depends on the physical, chemical and biological properties of the environment, on the psycho-biological characteristics of human beings, on their individual histories, and on the specific cultural and geographical contexts.

#### INTRODUCTION

The debate on the "two cultures" started in 1959 with the famous lecture of Charles P. Snow entitled precisely The Two Cultures. The opposition between scientific culture and humanistic culture has sometimes assumed overheated and almost grotesque tones. Actually the relationships between the so-called two cultures are very tight and their scopes and aims are closer than one can suppose. First, they meet in, and are produced by, human beings; and, second, they share the goal of seeking adequate and meaningful representations and models of the world and of man. Although they speak different languages and are driven the first one by tendentially objective and communicative goals and the other by subjective and expressive goals, finally both turn out to be intersubjective. The recognition of beauty is subjective, but the common nature and shared experiences of human beings make it intersubjective. One can therefore wonder whether it is justified any attempt to evaluate beauty in a tendentially objective way, going beyond the particular codes of interpretation and representation to reach the fundamental level of reality that mathematics and physics try to investigate. In this contribution I would like to put forward some considerations on beauty, in the hope that it turns out to be recognized as a bridge between the two cultures.

#### AESTHETICS AND ETHICS

I now come to the core of my contribution. I propose the following systemic and evolutionary definition of aesthetics, considered as a combination of sensitive abilities and activities, in particular in front of beautiful objects:

*! Aesthetics is the subjective (but shared) perception of our bond with the environment in which we are immersed and of which we are part; such a bond is characterized by a deep and balanced dynamical harmony and displays itself in the recognition of beautiful objects through senses and emotions.* 

i-L6!(!D/N-!/!,6-=!0+2</24!ME!/,,-2635C!69/6B!35!DE!5/6;2/.3,638!=-2,=-863@-B!/-,69-638,!3,! 8.+,-.E!.35N-4!<369!-6938,B!0+2!<9389!(!=2+=+,-!69-!0+..+<35C!4-035363+5K!

#### *! Ethics is the subjective and intersubjective capability of conceiving and carrying out actions that keep our bond with the environment healthy and balanced. Such actions maintain and increase beauty.*

&6938,!/54!/-,69-638,!/2-!69-!6<+!,34-,!+0!69-!,/D-!8+35!,358-!69-E!/23,-!02+D!69-!,62+5C! ,E,6-D38!/54!-@+.;63+5/2E!8+?3D=.38/63+5!M-6<--5!,=-83-,!/54!-5@32+5D-56!/54!M+69!/2-! oD322+2!3D/C-,o!+0!69/6!8+?-@+.;63+5!<36935!+;2,-.@-,H!

#9;,!/-,69-638,!3,!69-! O356-2P,;MA-863@-!=-28-=63+5!+0!69-! M/./58-4! O@36/.P!3DD-2,3+5!35! 69-! -5@32+5D-56! /54! -6938,!3,!69-! O356-2P,;MA-863@-! =-28-=63+5! +0!69-! 5-8-,,/2E! 2-,=-86! 0+2B!/54!,;36/M.-!/863+5!<36935B!69-!-5@32+5D-56H!%,!/!8+5,-R;-58-B!-6938,!=-2D36,!;,!6+! N--=!/-,69-638,!/54!/-,69-638,! 0;5863+5,!/,!/!C;34-!35!/8635C!-6938/..EH![+69!-6938,!/54! /-,69-638,!/2-!2++6-4!35!+;2!-@+.;63+5/2E!93,6+2EB!35!0/86!35!69-!8+?-@+.;63+5!M-6<--5!69-! -5@32+5D-56! /54! +;2,-.@-,H! ";89! /! =23D+243/.! -@+.;63+5/2E! 8+55-863+5! 8+58-25,! M-/;630;.!*objects*B!<9-2-/,!M-/;630;.!*concepts*B!-H!CH!69+,-!69/6!,+D-!0354!35!D/69-D/638,B! /2-!D;89!D+2-!2-8-56!=93.+C-5-638/..EB!9-58-!2-8+C53G35C!69-32!=+,,3M.-!M-/;6E!3,!D+2-! 430038;.6B! 35@+.@-,! 2/63+5/.36EB! 3,! .-,,! ,=+56/5-+;,! /54! 2-R;32-,! /! ,=-83038! 3543@34;/.! 62/3535CH!(5!69-!8/,-!+0!5/6;2/.!+MA-86,B!/!C2-/6!=/26!+0!,;89!/!62/3535C!</,!8/223-4!+;6!35! +;2!=./8-!ME!69-!=/,6!C-5-2/63+5,H!pUq!

%,! 8/5! M-! 5+638-4B! 35! 69-! =2+=+,-4! 4-035363+5,! 69-2-! 3,! /! ,9306! 02+D! 69-! *subjective* =+,363+5!6+! 9-!*intersubjective*! +5-H! ";89! /! ,9306!3,!A;,6303-4! ME!69-! 8+DD+5! 5/6;2-! /54! 93,6+2E! +0! 9;D/5! M-35C,B! 02+D! 69-! 0;54/D-56/.! =9E,38/.! /54! 89-D38/.! 8+D=+5-56,B! 692+;C9! 69-! ,9/2-4! C-5-638! /54! M3+.+C38/.! 2++6,B! ;=! 6+! 69-! 8+DD+5! -L3,6-563/.! -L=-23-58-,H!#9-!-.-862+5!89/2C-B!69-!C2/@36/63+5/.!0+28-B!69-!=2+6+5!D/,,!/54!,+!+5K!/..! +0!69-,-!89/2/86-23,638,! 9/@-!8+54363+5-4!69-! 23,-!/54! 4-@-.+=D-56! +0!.30-! +5!69-!-/269! /54!9/@-!M--5!3D=2-,,-4!;=+5!;,H!#93,!A;,6303-,!<9/6!J362;@3;,!OlT?Sc![H!)HP!/54!./6-2! 69-! /.89-D3,6,! /,,-26-4B! 69/6! 69-2-! 3,! /! ,62386! 8+22-,=+54-58-! M-6<--5! 69-! D/82+8+,D! O69-!<+2.4P!/54!69-!D382+8+,D!OD/5PH!

#### **\&-3.G!()!9-.1&9-.(%+!-)'!"1G+(%+!**

!!!!!#9-! /66-D=6! 6+! 2-=2-,-56! 2-/.36E! 8+58-25,! 35! =/2638;./2! D/69-D/638,H! [;6! /26! /54! D/69-D/638,!-L93M36!/!4--=!4300-2-58-K!/86;/..E!<-!.3@-!35!/!<+2.4!+0!0+2D,!/54!8+.+;2,! 69/6!<-!=-28-3@-!<369!+;2!,-5,-,B!/54!+5.E!,-8+54/23.E!<-!4<-..!35!/!<+2.4!+0!8+58-=6,! 69/6! <-! ;54-2,6/54! <369! +;2! D354,H! #+! ;54-2,6/54! D/69-D/638,! <-! 5--4! 6+! D/,6-2! 430038;.6! 6-8953R;-,! /54! 6++.,! 69/6! <-! 8/5! +5.E! /8R;32-! 692+;C9! /! .+5C! /54! 430038;.6! 62/3535CH!&@-2EM+4E!8/5!.3,6-5!6+!/!,ED=9+5E!+2!C/G-!/6!/!=/35635CB!-5A+E35C!36!/6!,+D-! .-@-.B!<9-2-/,!D/69-D/638,!8/55+6!M-!.3,6-5-4!6+!+2!C/G-4!/6!35!69-!,/D-!</EH!#9-!035/.! 6-,6!+0!/5!/26!<+2N!2-,34-,!35!69-!,;MA-863@-!O+2!M-66-2!356-2,;MA-863@-P!/-,69-638!=.-/,;2-! 69/6! 36! C3@-,B! <93.-B! 8+58-2535C! D/69-D/638,B! D/69-D/6383/5! 7+223,! W.35-! ,/E,K! Y#9-! R;-,63+5!<9-69-2!D/69-D/638,!=+,,-,,-,!+2!5+6!/!N354!+0!M-/;6E!8/5!+5.E!M-!/5,<-2-4! ME!69+,-!<9+!9/@-!/!8;.6;2-!35!693,!4+D/35H!\*50+26;5/6-.EB!6+!D/,6-2!D/69-D/638/.!34-/,! one needs years of study and there is no via regia or king's highway that can shorten the process."

For instance, very few laymen would be able to appreciate the fascination of the following amazingly simple expression (due to Euler)

e™+1=()

that connects five of the most important number in mathematics, i. e. 1, 0, i, e, and n. Many mathematicians consider this to be the most beautiful mathematical formula. But to appreciate its beauty one needs to be a mathematician, because we did not co-evolve with mathematics, but with the world, and our interaction with the world has been mediated more by the body than by the mind. For sure, mind, too, has its evolutionary history, but the abstraction process that has led to mathematics is very recent. The coevolution between us and the world is much more ancient, and it has impressed in us the coordinates (images) of sunsets and trees and animals and human beings, not of mathematical expressions.

The great mathematician Geoffrey H. Hardy was also a supporter of beauty. In his book A Mathematician's Apology he wrote: "The mathematician's patterns, like the painter's or the poet's must be beautiful: the ideas, like the colours or the words must fit together in a harmonious way. Beauty is the first test: there is no permanent place in this world for ugly mathematics." [2]

The prominent physicist Paul A. M. Dirac, Nobel laureate (1933), would take beauty both as a guide in finding meaningful results and as a criterion to test the validity of the formulae. He went as far as to write: "It is more important to have beauty in one's equations than to have hem fit experiment. It seems that if one is working from the point of view of getting beauty in one's equations, and if one has really a sound insight, one is on a sure line of progress. If there is not complete agreement between the results of one's work and experiment one should not allow oneself to be too discouraged, because the discrepancy may swell be due to minor features that are not properly taken into account and that will get cleared up with further developments of the theory." [3]

#### CONCLUSION

It is almost obvious to everybody that beauty holds sway in the realm of humanities. Although we are probably not able to define beauty, we recognize it when we meet it, both in nature and in art, literature, music. So it seems that beauty is of paramount importance in both the so-called two cultures, and can well be considered as a bridge between them, so reaffirming the unity of culture and, more deeply, of man.

#### References

[1] G. O. Longo, "The Dynamics of Beauty", in The Two Cultures: Shared Problems, Springer, E. Carafoli, G. A. Danieli, G. O. Longo Eds., p. 207-237, 2009. [2] G. H. Hardy, A Mathematician's Apology, Cambridge University Press, 1940. [3] P. M. A. Dirac, "The Evolution of the Physicist's Picture of Nature", Scientific American, 208, 47, May 1963.

## **RECENT ADVANCES IN SPECTRAL IMAGING FOR CULTURAL HERITAGE DOCUMENTATION AND ANALYSIS**

**J. Y. Hardeberg, S. George**  The Norwegian Colour and Visual Computing Laboratory NTNU, P.O.Box 191, N-2802 Gjøvik, Norway E-mail(s): jon.hardeberg@ntnu.no, sony.george@ntnu.no

#### **EXTENDED ABSTRACT**

Cultural heritage documentation and analysis with imaging techniques has changed tremendously over the past few decades. After the transition from film cameras to digital technology, the next objective was to record scientifically valuable image datasets of the artifact. Different imaging devices have been used to fulfill this drive including spectral cameras, 3D imaging, reflectance transformation imaging, etc.

 There has been several efforts to capture accurate colour, shape, and spectral / material properties of cultural heritage objects. In this paper, we discuss the progress in research and development at The Norwegian Colour and Visual Computing Laboratory (Colourlab) devoted to acquisition and analysis of paintings using spectral imaging. Originated from the field of remote sensing, hyperspectral imaging technology has been explored in various sectors, including computer vision, medical imaging, and material classification. In the cultural heritage domain, spectral imaging has been used for precise scientific documentation, material identification, enhanced visualisation, virtual restoration etc. Many of the used spectral imaging systems were not initially developed for cultural heritage imaging, but taken from other application domains, there is several pre and post-processing steps need to be investigated and implemented to acquire quality image dataset [1, 2], which can be used to achieve the above-mentioned objectives.

The Colourlab has engaged in several spectral image acquisition and analysis campaigns of cultural heritage paintings at museum environments and at imaging laboratories [3]. For practical reasons, the same image acquisition setup do not work the same way in all these conditions. This causes spectral and spatial distortions of several forms which affects the acquired images. In order to include appropriate correction methods in the processing workflow, we investigate the kind of distortions from imaging systems in different configurations and propose new correction techniques [4].

Pigment mapping or material identification is an interesting topic in the cultural heritage sector. The potential of hyperspectral imaging technology to capture reflectance spectra at every spatial point of the scanned region make this a promising non-invasive tool for the identification of pigments and its spatial distribution across the painting. Investigations has been made in this direction to classify pigments based on their spectral characteristics [5]. Different spectral classification and unmixing algorithms has been evaluated on a hyperspectral image dataset of Edvard Munch's masterpiece 'The Scream' and the results are validated with an analytical study carried out before [3].

 Crack detection for cultural heritage objects is another interesting area still to be explored by the spectral imaging community. The potential of hyperspectral datasets to process the data metrologically can be employed for crack detection in paintings. Although crack removal may be considered a way to improve the perceived image quality of the painting, detection and documentation of cracks in paintings are motivated by several reasons such as understanding the degradation/aging process, as an indication of the painting's authenticity. The colourlab has taken a step further in this research to develop mathematical tools for crack detection of from hyperspectral images [6]. Distance based spectral morphological methods are developed to detect cracks in paintings. The performance of the proposed methods were validated with a set of artificially constructed images and real hyperspectral images.

 Many of the spectral imaging systems are not just limited to the visible range, but extended over the spectral region - near IR (NIR) and short wave IR (SWIR). This wide span of spectral data at very closely spaced wavelengths allows visualizing the acquired images in several ways - the true colour image of the artifact under any illuminants and the false colour images. For false colour visualisation of the artifact, we employ different mathematical operators and dimensionality reduction algorithms to choose the images with maximum distinct information. This brings our many barely visible details in different layers of the painting and serves as a tool for museum experts and visitors [7].

#### **References**


### APPLICATION OF BIG DATA AND CONTENT CURATION TO EXPLORATION OF CULTURAL HERITAGE

#### Konstantine Karczmarski

Department of innovations, ITMO University, Russia, <konstantine.karczmarczyk(@gmail.com>

#### ABSTRACT

Introduction. Different social groups have desire to consume cultural content, but their perception in many cases is so different that to meet multiple audiences' expectations we need to provide a variety of technologies, content layers and interfaces.

1. The pain. Like in most of the media industry, the problem of finding the right audience for a particular piece of knowledge has not been solved without tremendous amounts of money poured into advertising. Which, in case of cultural heritage, can create only "blockbuster culture" leaving most of the heritage without slightest attention. At the same time most of non-professional viewers are also left without access to culture because channels of communication between them and the cultural heritage are either not established or jammed by "blockbusters".

2. Current attempts of solving this problem. Even though there are certain attempts of connecting diverse audience, using big data, for example to drive people from the web to multimedia exhibitions (so far the most successful example of technology-driven cultural experience) , the results are far from desired. Reasons are: not enough relevant data cross-sections, and only analysis without attempts of synthesis of content. And content is not only "raw", digitized artifacts, it is essentially stories, narratives. And each audience needs it's own story.

3. Difficulties in production of digital content without sufficient data about audiences. Content that really draws attention is expensive to production represents very high risks associated with the lack of knowledge about potential audiences and it's locations. Available "traditional" metrics does not provide enough data. Hence content is not produced.

4. V-Memes, or "cultural codes" of Don Beck's Spiral Dynamics Integral. Building another layer of data about audience and artifacts and unlocking its potential to establish communication between cultural heritage and diverse audiences.

5. Application of Big Data in synthesis of narratives, storytelling and content distribution.

## PIE ZOMUSICOLOR A NATURAL FORM OF TECHNOLOGICAL ART

Giuseppe Caglioti¹, Marco Marcon¹, Tatiana Tchouvileva¹, Riccardo Della Ragione²

+Politecnico di Milano, P.zza Leonardo da Vinci, 32, 20133 Milano (ITALY) giuseppe.caglioti/marco.marcon(@polimi.it 2Magic Music s.a.s, Via della Cinta Esterna,44 - Livorno (ITALY) info@magicmusicsas.it

Abstract - PiezoMusiColor (PMC) is a vibro-chromo-phonoscope, allowing to disclose the visual and vibro-tactile dimensions of sounds, music and voice. Its applications range from: education to entertainment and social inclusion for impaired people. In education it is fruitfully applied in Learning by playing where some relevant physical laws can be explained in a interactive approach, while in the entertainment context PMC offers a full immersive multisensorial (acoustic, visual and tactile) approach able to hypnotically capture the user's attention. Concerning social inclusion, PMC allows people affected by hearing impairment or cognitive deficiencies, to correlate sound with touch and vision increasing their fruition capabilities.

### INTRODUCTION

In the history of culture very few phenomena have occurred capable to attract and to intrigue not only artists and musicians but also humanists, scientists and laypersons. Listening to music while perceiving its patterns and colours in an immersive experience is one of these phenomena.

This inspiration goes back to Isaac Newton, who conceived a parallelism between sound waves and light waves " ...May not the harmony and the discord of colors arise from the proportions of the vibrations propagated through the fibers of the optic nerve into the brain, as the harmony and discord of sounds arise from the proportion of the vibrations in the air?" (Isaac Newton, 1666) [1]. Nowadays the proposed solutions usually focus on computer graphics programs that are commonly based on algorithms able to convert sound into colours and random patterns "following" the music itself. The systems usually adopted for this application are categorized as "Visual Music"|2]. Historically the cathode ray tube made possible the oscilloscope, an early electronic device that can produce images that are easily associated with sounds from microphones.

Figure 1 Some Lissajous figures obtained on cathode ray tubes "dancing" with a properly processed audio signal

The modern Laser lighting display displays wave patterns produced by similar circuitry. The imagery used to represent audio in digital audio workstations is largely based on familiar oscilloscope patterns.

The Animusic company has repeatedly demonstrated the use of computers to convert music - principally pop-rock based and composed as MIDI events - to animations. Graphic artistdesigned virtual instruments which either play themselves or are played by virtual objects are all, along with the sounds, controlled by MIDI instructions[3].

Figure 2 Laser patterns obtained from audio inputs.

However the results often exhibit repetitive, cold and predictable structures that are criticized by musicians, painters and art critics for the underlying synthetic approach.

The Musicolor technology is not based on any software or handcrafted algorithm, it is a natural form of multimedia graphic-art delegating to the music of all genres - or in general, to any kind of sound, like the voice itself - the function to trigger and control a process producing in a real-time an analog self-portrait of sound. Activating a natural process the music manifests its second nature: an endless stream of patterns, with rainbow colours, rhythm, timbre and intensity of sounds.

Unlike Painting and Sculpting - static arts defined in two and three space dimensions respectively - Music is an art defined in the time domain. In the early years of the last century, futurist painters disclosed the time dimension to the static canvas of the pictorial space[4][5]. In a manner complementary to Futurism, Musicolor discloses the chromatic and space dimensions to the dynamics of the times of sounds.

#### THE PIEZOMUSICOLOR PARADIGM

Musicolor technology was initially developed in collaboration with Göran Rämme from the University of Uppsala (Sweden) [6], then, in recent years, vibrating components has been implemented revealing the third nature of music - its tactile dimension -. These components convey the vibrational self-portrait of sounds to the hands (by vibrating actuators) and/or to other constituents of the body, depending on the final user. The audio frequency spectrum has been properly reshaped and shrink in order to fit to the tactile spectrum (which is almost an order of magnitude smaller in the frequency range with respect to the audio one).

Among the vibrational (or piezo-) components of the resulting analog PiezoMusiColor vibrochromo-phonoscope we also included "AcousticA", a sonic armchair designed by Magic Music s.a.s. for the multi-sensory stimulation. AcousticA has been created combining the traditional techniques of violinmaking with the most innovative technology in sound research.

Figure 3 The AcusticA vibrational armchair invented by Riccardo Della Ragione

It is not a lounger that uses the normal speakers or sub-woofers: it is a tool capable of bringing the sound frequencies into resonance with different parts of the human body. The result is a deep massage acting on all muscular bands and the nerve endings of the body. The AcousticA armchair is activated by music composed by Riccardo Della Ragione specifically for music therapy, using physical and medical criteria, rhythms and sound frequencies that give optimal stimulation on the human body. It is currently adopted in hospitals and rehabilitation facilities for the elderly and children with disabilities, and it is also adopted in wellness areas and protocols of complementary medicine with excellent results. As an alternative to the AcousticA armchair a vibro acoustic pillow, the "CuscinosonorO", realized by Magic Music s.a.s., can be adopted. it is syncronous with Musicolor and it can be activated by all music genres.

#### A possible PiezoMusiColor Setup

In Figure 4 it is possible to see a possible implementation of the PMC project where a properly processed audio signal is mechanically transduced into the vibration of a thin film whose density is locally changing accordingly to global and local material properties. Micrometrical variations in the film thickness act as a classical Fabry-Perot interferometer [7] selecting different wavelengths (different colors) for the reflected light.Figure 1

Figure 4 An implementation step of the MusiColor system: the audio signal is transformed into a PMC video and recorded in real time.

In particular it is possible to identify in the generated video sequences some components that are more related to deterministic and predictable behavior and some more chaotic components related to complex fluid dynamics. The overall effect is an harmonious mixture of these two components as can be seen in Figure 5.

Figure 5 Two frames of a PMC realization, well defined periodic patterns superimposed on chaotic movements are well visible.

### APPLICATIVE SCENARIOS

During the last years many different applicative scenarios were proposed and adopted for the PMC system, in particular we mention the following ones:

#### Education

The PMC makes possible an interactive teaching of fundamental concepts and principles of natural sciences - learning by playing. An analog transcoding of music and voice gives rise to a faithful self-portrait of music in the audio-visual and vibro-tactile dimensions of sound: a succession of visual forms, sometimes chaotic and unpredictable, as might be the eddies of a river or the color of the flame, dance synchronously with music and with vibrating vibro transducers. For teaching purposes, the joyful sensations produced by the synaesthetic perception of sound trigger curiosity about the wide variety of concepts and laws of nature upon which the operation of the PMC is based.

#### Entertainment

The colorful symphony of visual forms, obtainable by Musicolor without any artificial processing, enhances the emotional impact of music stimulating a synaesthetic tension aiming to correlate the sensory tactile channel with hearing and sight: a novel, hypnotic and immersive music experience.

Furthermore PMC can be fruitfully adopted in an endless set of possible applications; e.g. people can "wear their voice" just printing a frame of the PMC on cloth (an example is visible in Figure 6).

#### Social Inclusion

With the increase of the lifetime expectancy the problem of social integration of persons with physical, cognitive, mental, sensory or emotional disability becomes more and more serious. According to our experience with persons affected by hearing loss or cognitive deficiency, the

PMC is a promising and efficient tool. In particular we deployed a specific protocol at the Don Gnocchi clinic on patients with a large mental impairment verifying, after a couple of weeks, an increase in their cognitive capabilities concerning multi-sensorial stimuli correlation.

Figure 6 "Wear your voice" an example of a possible entertainment application.

#### CONCLUSIONS

In this paper we presented the PiezoMusiColor, a multifaceted and versatile system able to convert whichever kind of audio source into an immersive, multi-sensorial and synaesthetic experience. In a few years from its first realization we have been able to fruitfully integrate it into many applicative scenarios, ranging from entertainment, to a clinical aid for music therapy, from education to an aid for deaf impaired people and we think that, in the close future, it will find further challenging applications.

#### References


## THE USE OF PATIENT SPECIFIC INSTRUMENTATION DURING TOTAL KNEE REPLACEMENT SURGERY

Lawrence Camarda, MD, PhD1 Antonio D'Arienzo, MD Salvatore Morello, MD Michele D'Arienzo. MD < michele.darienzo(@unipa.it>

1 Orthopaedic and Traumatology Department, University of Palermo, Italy

#### ABSTRACT

During the past decade, total knee arthroplasty (TKA) has been markedly increased. In order to improve the position of implants during TKA and to reduce the risk of early failure of the procedure, new technology has been introduced and used by several surgeons. Specifically, threedimensional imaging and custom manufacturing have enabled the development of patient-specific instrumentation (PSI) that are used as cutting jigs. These cutting blocks are specific to a patient's knee anatomy and should help the surgeons to perform bone cuts, reducing the complexity of conventional alignment and sizing tools.

With this technology, a pre-operative MRI or CT is performed on the patient's knee. Hip and ankle images are also obtained for the evaluation of the overall alignment of the limb. With a specific software program (Mimics, Materialise, Leuven, Belgium), manufacturing engineers turn two-dimensional CT or MRI images into three-dimensional representations of the knee and lower limb of the patient's anatomy. Using a specific software, the surgeon is then able to evaluate the 3D planning of the knee implant with the proposed bony resections, modifying as necessary bony resection of the tibia and femur. After the surgeon's approval, custom cutting guide that fit on the patient's native anatomy is manufactured and then sent to the surgeon. During surgery, the PSI guides are used as slotted cutting guides or for accurate pin position for the use of standard manufacturer resection instrumentation.

Several advantages have been advocated using PSI technology. At first, the surgical time is reduced because many steps of the surgery have been already performed such as pre-operative planning. Other advantage includes a decrease in the instrumentation trays optimizing the operative room time and reduction of the operating time. Despite several potential surgical advantages of using PSI technology, there are no long-term implant survival data to support its use. In addition, few studies evaluated the effectiveness of this technology over conventional instrumentation, and results of post-operative alignment accuracy are conflicting.

From September 2012 to December 2014, 22 patients were treated with TKR using PSI. CT images of the lower limb were used to obtain DICOM images. PSI were produced by Medacta (Castel San Pietro, Switzerland), and used in all patients for femoral and tibial bone resection. A post-operative CT performed in all patients treated, showed a suboptimal positioning of the tibial component and an optimal positioning of the femoral component.

# i&Z!#&)bi()%F!]&J&FI17&i#"!!! \!%11F()%#(Ii"!

<sup>45</sup> Vito Cappellini and Enrico Del Re (edited by), *Electronic Imaging & the Visual Arts. Eva 2016 Florence.* ISBN 978-88-6655-973-3 (print) ISBN 978-88-6655-974-0 (online) © 2016 Firenze University Press

#### **GETTING PAST THE FEAR OF VIRTUAL INNOVATION**

#### *by Eugenia Romanelli*

In 2010, a research by Finnish tech company Nokia revealed something astonishing: people check their phones at least 150 times a day. This means once every 6.5 minutes, counting up to 22 calls and 23 text messages only during the day. While this was probably perceived as a huge business opportunity for Nokia, it made many columnists and intellectuals worried, raising concerns about our relationship with technology and virtual forms of communication.

 If a similar research were to be done today, it would reveal much higher rates. As more and more people become more prone to accepting the digitalization of our lives, some still argue that in today's world human existence is nothing but an extension of technology. Starting in 2014, a platform called "Open Forum", as well as American managing consultant Brian Moran, have lead the way in trying to detox people from the digital world, to let help them enjoy what Moran called "here and now". However, if one were to explore our understanding of what living in symbiosis with technology really entails, it would be easy to see that it is by no means anything new: it is the same theme recurring in the story of Ulysses facing the "Pillars of Hercules" — in other words, people's fear of technology is merely a new interpretation of the age old dilemma of human endeavor versus fear of pursuing knowledge.

 Many statistics show that by 2020 almost 75% of the world economy will be dependent on the internet. According to Cisco (one of the world leaders in networking, active in promoting the "Internet of Things" theory) at least 10 billion devices will be connected to the internet by 2018. Considering the incredible amount of business that is being conducted online, Sheryl Sandberg, COO of Facebook, has expressed interest on how the world will look like when 100% of the population will have access to the internet (as opposed to today's 39%).

 Not only is there factual evidence to say that most communities in the world now are structured around digital networks, but it is also generally safe to say that such networks allow us to develop more creativity, can improve our cognitive processes and facilitate exchange of information. Our digital reality is also a great platform for social change, as well as promotion of democratic values.

A paper called "Growing a Digital Social Innovation Ecosystem for Europe", commissioned by the European Commission, was presented in February 2015 at a panel called "Shaping the Future of Digital Innovation in Europe". Written on behalf of the National Endowment for Science, Technology and the Arts (NESTA), the paper argued that one of the largest digital communities is focused on awareness networks and innovation and can provide important tools to promote integration and increase people's well being. It is evident, therefore, that it is essential for politicians, institutions and service providers to be well acquainted with the opportunities that the digital world can be a source of, in order to ultimately strengthen services and inter-personal relationships.

 What is left to deal with is the difficult task of fighting common misbelieves about the digital world. This is yet another example of the "fear of the new", or what Freud called a "conservation struggle", linked with the idea of a "death wish" — in other words, the instinct that pushes us to protect whatever we have at the price of sacrificing our future and hopes. How can we then reassure those who fear that digital networks are replacing real life relationships? How can one argue that new technologies are opportunities instead of burdens? How can skeptics be convinced that digital connections get us closer together, instead of tearing us apart? Generally speaking, how is it possible to humanize technology?

 It is perhaps possible, through a global collective shared discussion, to outline that the digital world is a crucial element to sustain the needs of our communities allowing us to rethink education and social services, in terms of more inclusive and flexible policies. Overall it appears that in order to translate the digital opportunities available into successful solutions for our communities, it is necessary to look at the issue in a different way.

 If we don't help citizens envision the possibilities that their realities offer; if we don't create a new system of values people can look up to; if we don't promote change; then, going beyond the "fear of the unknown", or the "Pillars of Hercules", will never be possible, and we will never cross the new horizons that lie ahead.

Prof. Eugenia Romanelli, italian journalist. romanelli@bazarweb.info

## PERCEPTION-BASED HISTOGRAM EQUALIZATION FOR TONE MAPPING APPLICATIONS

S. Ploumis | Ronan Boitard | Mahsa T. Pourazad23 | Panos Nasiopoulos13

4 Elect. and Computer Eng. Department, University of British Columbia, Vancouver, Canada 2 TELUS Communications Inc., Vancouver, Canada

3 The Inst. for Computing, Information and Cognitive Systems (ICICS), Vancouver, Canada

#### INTRODUCTION

High Dynamic Range (HDR) images represent larger dynamic range and wider color gamut compared to Standard Dynamic Range (SDR) images. To reproduce HDR images on displays with limited dynamic range, a process known as tone mapping is required. Tone Mapping Operators (TMO) aim at mapping the larger dynamic range to smaller one by preserving some of tonal levels and dismissing the rest, such that the visual quality of tonemapped image is comparable with that of the original one. Several TMOs rely on histogram equalization [1]. Indeed, histogram equalization efficiently redistributes tonal levels to preserve information where the pixels' density is the highest in the intensity domain.

#### PERCEPTION-BASED TONE MAPPING

Tone mapping using histogram equalization associates a luminance range in the SDR domain (assuming the general specifications of SDR displays) to each luminance range (bin) in the HDR domain. However, such an approach cannot differentiate relevant information from noise in a uniform background or in dark areas. To address this issue, our proposed scheme first compares the number of distinguishable tonal levels in the HDR luminance range with those in the SDR luminance range. This is achieved using the Perceptual Quantizer (PQ) [2], which is derived based on the peak values of Contrast Sensitivity Functions (CSF). The CSF defines the human vision contrast detection threshold for different spatial frequencies and background luminance. Wherever the number of tonal levels is higher in the SDR range compared to the HDR one, we modify the mapping curve to reduce them. Our technique then redistributes those tonal levels to increase details in highlights and/or to achieve SDR images with higher overall contrast.

Our experimental results show that such an approach can greatly improve the tone mapping of night scenes as well as scenes with uniform background such as a large sky area. Additionally, artistic effects such as fading are also preserved using our proposed scheme.

#### CONCLUSION

We proposed a method to address the main issue of histogram equalization-based tone mapping operators, which is their inability to differentiate visual information from noise. Our technique relies on a model derived from psychophysical experiments to prevent non-visible information in HDR images to become visible in the tone mapped ones. Our method is especially efficient for dark scenes and preserving the artistic effects such as fading.

#### References


#### **FROM** *ENCYCLOPÉDIE* **PLATES TO VISUAL KNOWLEDGE FOR CRAFTSMEN: THE PROJECT** *VISUOPLANCHES*

Marcantonio Catelani<sup>1</sup> , Maria Teresa Zanola<sup>2</sup> , Clara Vecchio<sup>2</sup> 

> 1 Università degli Studi di Firenze, 2 Università Cattolica del Sacro Cuore, Milano

> > *marcantonio.catelani@unifi.it mariateresa.zanola@unifi.it clara.vecchio@uniparthenope.it*

*Encyclopédie* plates have shown the importance of visualisation: Diderot and d'Alembert commissioned several hundred engraved images depicting artistic crafts and common trades in preindustrial France, specifically aimed at illustrating their entries. Since pictorial representations of workmen and shops are not so frequent, the descriptions and explanatory plates in the *Encyclopédie* remain highly important both to scholars of decorative arts and manufacture and to researchers of social and economic history, so as to the current understanding of historical techniques and technologies.

The plates provide a space of non-verbal representation which is profoundly effective for communication among producers, craftsmen and the general public: they describe the concept and fix the relationship between the idea, the corresponding picture and the final term.

Many of the images are not technical drawings but genre scenes portraying a small group of workers in their activities. The plates from the *Encyclopédie* often represent stages in complex technical processes by juxtaposing images of different types. Vignettes representing human figures engaged in various activities are supported by large-scale renderings depicting tools and their proper manipulation. *Encyclopédie* plates have a typical iconic structure: the work is anticipated by the title, which is divided into two distinguished sections. At the top - almost on the third of the plate – the place of production is represented, where men and women act inside the workshop; in the lower part, all the tools presented in the workshop are illustrated, are seen in details, and numbered with the corresponding term. The actions are analysed one after the other, as if a zoom gradually passes through the sequences, focusing on the most important details.

The project *Visuoplanches* aims to rebuild the history of the relationship between the image and the classification of terminology to transfer arts and crafts knowledge, starting from the models established during XVIIth and XVIIIth centuries in Europe. Visualising the tools, in combination with legendas, establishes a reference model for tool classifications and for their effective description: the text accompanying the engravings introduced to the descriptive method which was proposed to beginners destined to become experts in the future. Visual representation confirms the encoding effort and dissemination of new findings. From plates to modern digital image, tools and technical items continue their dialogue between users and craftsmen, between producers and sellers.

Visual representations show arts and crafts which will transfer technical and specialised knowledge: the analysis of some cases will assess the dynamics arising from the dialogue between definition, narratives and visualisation, which reveals to be a sustainable method for transferring technical knowledge and presenting terminology effectively. Various forms of painting and engraving – from artistic drawings to computer-generated images – show craftsmen's knowledge and their need to spread it out.

## CULTURE AND COMPUTER SCIENCE TO CHANGE SOCIETY'S BEHAVIOR

Wilhelmina Ndapewa-Onyothi Nekoto12 Prof. Dr. Dr. h.c. Jürgen Sieck12 Prof. Dr. Hippolyte N'Sung-Nza Muyingi2 1 University of Applied Sciences Berlin, Germany 2Namibia University of Science and Technology wilhelmina0711@gmail.com J.Sieck@htw-berlin.de hmuyingi@nust.na

ABSTRACT- Being Namibia, it is common that both urbanization, the use and influence of Over-The-Top (OTT) services and applications has changed society's behavior towards culture. Many have little cultural knowledge or neither to pass on to their young ones. Advantages of urbanization have influenced a tradition of domestic travelling amongst the youth; however know little of natural surroundings and the underlying intriguing history. In the paper, we would like to discuss and demonstrate our mobile-web application "rock art, fauna flora and landscape game" which we developed to influence and preserve indigenous knowledge and traditions. We will also discuss the server, client and crowd sourcing components of the application.

KEYWORDS: Flora, Rock art, mobile-web application, serious games, indigenous knowledge.

#### 1. INTRODUCTION

African Music, Arts, Literature and Cultural practiced, have always played a significant role in the daily lives and practices of Africans. Even so, it has gained more interested and respect from all over the world, as the world becomes more familiar with the rich tradition of the continent [1]. The arrival of western philosophy of development, haltered, suppressed, outlawed, neglected the indigenous knowledge and traditions of Africans [2]. It is unfortunate that the African norms are no longer frequently practices and applied as common as they were before. This could be influenced by the colonial past experienced amongst developing African countries, which has changed a country's Governance, its people and country. The colonial struggle had many go into exile where they furthered both their studies at tertiary level and joined the armed forces. In finding its own identity or rebuilding its identity. skills and workforce is often needed in achieving the goal. This process attracted an increase in urbanization to the cities: which were once originally created for its natural resources, as most jobs were created in these cities [3].

'Namibia has a rich bio-diversity, which has remained untapped. Coupled with extensive indigenous knowledge' [2] relating to the use of flora, fauna products etc. The push versus pull factors of urbanization play a critical role in the destruction of [3,9] indigenous fauna and flora in the parts of the country, as land is increased for the demand of housing, or for infrastructure that would damage the land, to provide for the majority population in the city. It is thus very important in preserving the Indigenous Knowledge and Traditions of the country and in parallel influence the

behavior of the youth, in understanding and appreciating the cultural and historical value. 'Knowledge still remains amongst the elderly, hence the need to collect, analyze, preserve and appreciated' [2]. It is by these studies, we designed the application faflora 'Rock art, Fauna, Flora and landscape', to bridge the gap and orientate society's behavior towards indigenous knowledge.

#### 1.2 Concept

Based on the results from a survey we conducted amongst a small group of Namibians, we observed that the process of urbanization plays a role on an individual one especially in adapting to the new environment. One often finds they having to apply both their cultural norms and those developed to best shape and fit their current environment as well the community in which they find themselves. A trend that has had an impact on the youth of the country as they battle with 'intercultural identity, an outcome of adaptation. This poses a risk, whereby they are neither part of, nor a part of the host culture. As a result finding it difficult to acquire much of the cultural knowledge, practices, and other indigenous factors' [5], see.fig1. It is due to this fact that the youth have very little knowledge of their cultural norms and practices, as a result of early urbanization see.fig2. On a case study conducted by Suzanne LaFont a staggering 80% of those from a rural background revealed the need to preserve cultural history, whereas those from the city felt it as 'somewhat important' [6], see fig.3. This shows the role of one's geographical location on the cultural knowledge. Urbanization advantages such as employment opportunities have seen travelling trends amongst the youth increase, a luxury for those who can afford it. Information remains unknown due to limited knowledge about the particular surroundings or the rich cultural heritage as depicted in rock-art found in popular tourist destinations.

Fig.1: Count on the cultural knowledge Fig.2: Influence of the birthplace on culture


#### Fig.3: Importance of preserving Indigenous

#### 2. SYSTEM ARCHITECTURE

The application comprises of a game and crowdsourcing section for uploads of indigenous content to be added to the application. For validation purposes, a user is obligated to create an account, for which they can upload and view content. The user can either upload a quiz or application content (fauna, flora, rock-art & landscape), of which all are used in the 'faflora' (flora, fauna, rock art and landscape) application, and added to the gallery. Binary Large Objects (BLOB) such as images is stored in a file system for performance reasons, whereby a request is made by the client to the server to retrieve. If a view or quiz is requested, the server returns the .hbs files from the handlebars template system, to client.

#### 3. IMPLEMENTATION

Fig.4: Schematic System Architecture

#### Client and User Interface: Express handlebars.js, HTML5+CSS

Handlebars are a JavaScript compiler and an extension of the mustache JavaScript template language, known for its 'logic-less' template engine, which keep the pages clean and decoupled from logic-based files; this makes the app much more responsive and interactive. CSS styling and HTML is added for the User Interface.

Database: MySQL was chosen for due to its strong consistency and robust database design. It will store user profile and application content. Binary Large Objects (BLOB) such as images is stored in a file system for performance reasons.

#### Server: Node.js using Express.js & Handlebars

Express is a minimal and flexible Node.js web application, with its core features in handling the client communication. This code in fig. 5 handles request to randomly load different quizzes and draw quiz images from the database.

```
function loadRandomQuiz() {
```

```
return query('SELECT * FROM quiz')
```
.then(({ rows }) => {

let quiz = randomElem(rows)

let img ids = [quiz.hint image id, quiz.correct image id,

quiz.answer1 image id, quiz.answer1 image id, quiz.answer2\_image\_id, quiz.answer3\_image\_id, quiz.answer4\_image\_id, quiz.answer5\_image\_id]

}

Fig.5: Server code to load random quiz

#### 3.1 Solutions and Demonstration

Upon loading of the application, the user is directed to the home page where they can select from either 4 elements, Rock art, Fauna, Flora or Landscape. Once either is selected, they can either choose to view the gallery. Amongst other tabs, the user can click on the play button, where they are redirected to the game page. Each Quiz has a title, question and related question and hint images, which are retrieved from the database. Once the question is read the user can navigate between hint images, for a possible answer of which each has a description to it. A confirmation is given as to whether or not a selected answer is correct, where they can use navigation tabs to proceed to the next or previous quiz. Should a user want to contribute a quiz or related content, registration as a contributor is required, where they contribute indigenous

findings to the application. The contributed content with their details are displayed in the gallery, and or the quizzes upon validation, see fig.6 and 7.

Fig.6: Application homepage

Fig.7: Application game page

#### 4. SUMMARY

Based on our results compared to those of Suzanne LaFont results, a much greater percentage of the participants who live in the city are unaware of the cultural and traditional indigenous knowledge. This could be due to a number of factors previously discussed in the paper such as the role of Urbanization. Those who live in the city are also much more reluctant to preserving Namibian traditions as opposed to those from the rural regions of the country. In conclusion we can deduce that, more of the Namibian youth are likely to make use of the application, as nearly half rarely travel. This unaware of the Rock art or Indigenous fauna or flora found in parts of Namibia, as a mere majority prefer to travel either alone or in a group of friends when travelling. This could be seen as 'a potential for domestic or regional tourism growth. in expanding the countries tourism sector substantially'. With the use of a mobile

phone, this would make it much easier for the user to become increasingly aware and curious as to the surrounding parts of the country. Enticing them to travel and explore the different areas. The use of the application could also be used as a reference tool for appreciating the cultural heritage and pre historical use of the applications elements, Fauna, Flora and Rock art. Which is a step towards changing society's behavior in preserving the land and communities in which these indigenous elements are found.

#### 5. DRAWBACKS AND FURTHER WORK

The current application demonstrates the contribution of indigenous knowledge from an ICT literate person and the Namibian indigenous and traditional elements. The contributors range from academics, professionals and contributors with access to ICT services. However, almost two-thirds of Africa's population [7], and the majority of the elderly in Namibia still live in rural areas. This shows that a rich traditional knowledge can still be found in rural areas. "The democratizing, empowering promise of a mere presence of new media technology is far overstated, as many people are still without access to the web, [7] hence the need to draw from, and make information accessible to low literacy users in underdeveloped regions of Namibia and that of Africa. The next step of the application will see how we can acquire knowledge from the elderly in underdeveloped regions of Namibia with hopes to extend that to other African countries, for games centered on broader African traditions, landscapes and practices. 'Knowledge of the users' culture, values, natural experiences within the environment and perception improve users interaction with ICT services [7]. Thus designing an application, taking into account, a graphical user interface with interaction elements that depict the socio-cultural environment of the targeted users, which has proven to yield a significant success [5,6]. Alternating in a crowd-sourcing application for the elderly in rural areas, as contributors to the application and quizzes on different African countries!

#### REFENCE

[1] African Cultural Center website http://www.africanculturalcenter.org/

[2] NCRST website http://www.ncrst.na/news/

[3] J. Vernon Henderson, Adam Storeygard, and Uwe Deichmann,"50Years of Urbanization in Africa: the role of climate change", (May 29, 2014)

[4] Judith N Martin/Thomas K Nakayama, "Intercultural Communication in Context", 5 Edition, Chapter 8 / Understanding Intercultural Transitions pp327 ISBN: 978-0-07-338512-9

[5] Suzanne Lafont, Beliefs and Attitudes towards Gender, Sexuality and Traditions amongst Namibian Youth.pp.16(2010) ISBN 978-99945-61-19-3

[6] Edim Azom Emanuel, Muyingi Hippolyte, "SPEECH USER INTERFACE FOR LOW LITERAACY USERS OF ICT SERVICES," European Journal of Computer Science and Information Technology Vol.2, No.1, pp.18-29, (March 2014)

[7] EDIM,A.EMMANUEL, MUYINGI,H. NSUNGA-NSA, SIBANDA K, "User Interface for Low-Literacy Users in a Rural Community", International Journal of Computing and ICT Research, Vol.7, Issue 1, (June 2013).

[8] Machel McCatty, The Process of Rural-Urban Migration in Developing Countries

[9] Iain T. Christie, Doreen E.Crompron, "Tourism in Africa, African Region",

Working Paper Series No.12, pp.3 (February 2001)

## NEW TRENDS OF 3D TECHNOLOGIES


ABSTRACT

In the last years, in the manufacturing sector, there has been an increasing diffusion of digital technologies, so that the term "Digital Manufacturing" emerged. These technologies have entailed an impact at the corporate level, through the necessary reengineering process, and socially, making the customer the protagonist in the production of their purchases and enhancing their creativity. This change occurred in response to the reduction of the life cycle of products and to the need to meet the needs of its customers in the best way. #

3D printing is an innovation which, together with digital technologies, has enabled the diffusion of digital manufacturing, facilitating the transition from the paradigm of mass standardization to that of mass customization. The expiration of some patents related to print has encouraged the diffusion of additive manufacturing; now it is increasingly used by consumers and enterprises. Since its birth, it has been applied for prototyping, while in recent years also for the production of products and spare parts. The sectors where it is most widespread are the automotive, the aerospace, the medical and the artistic with relevant feedback in the Cultural Heritage field.

3D printing is considered a disruptive innovation because, encouraging the sale of files instead of a sale of items, it can optimally solve economic and social problems that have been emerging in recent years.#t is considered a technology with strong growth, but it may encounter obstacles in its expansion due to the problems concerning the protection of intellectual property: file sharing can encourage the spread of non checked 3D files and then it leads to the free duplication of digital objects and the consequent violation of intellectual property. A natural consequence may be the failure to control the authenticity of the files, and then the risk is reflected in the spread of counterfeit and uncontrolled goods that endanger public health.#The lack of specific legislation to protect file properties further aggravates the threat of the illegal use of the same #Currently the use of 3D watermarking and encryption of data is a solution to this problem, although an action at national and European level which protects the dissemination of unauthorized files is still desirable.

For the future, if the issue on intellectual protection is effectively resolved, a further increase in the deployment of such printers is expected, especially if it is accompanied by a decrease in the price of 3D printers, which is expected as a result of patent expiries and the development of technological progress

#### New 3D Scene Representation for FTV

Masayuki Tanimoto Nagoya Industrial Science Research Institute Nagoya, Japan. tanimoto@nagoya-u.jp

Abstract - FTV (Free-viewpoint Television) is an innovative visual media that allows users to view 3D scenes by freely changing their viewpoints. MPEG has been developing FTV standards since 2001. The first phase of FTV was MVC (Multi-view Video Coding) and the second phase of FTV was 3DV (3D Video) for multiview display application. The current third phase of FTV targets super multiview, free navigation and 360 3D applications. In this paper, new 3D scene representation is proposed for free navigation application of FTV.

#### INTRODUCTION

FTV (Free-viewpoint Television) [1]-[7] is an innovative visual media that allows users to view 3D scenes by freely changing their viewpoints. MPEG has been developing FTV standards since 2001 [8]. The first phase of FTV was MVC (Multi-view Video Coding) [9] and the second phase of FTV was 3DV (3D Video) for multiview display application [10]. The current third phase of FTV [1] started in August 2013. This MPEG-FTV standardization targets super multiview (SMV), free navigation (FN) and 360 3D applications [12]. 3D scene representation is one of the key technologies of FTV. Multi-view plus depth (MVD) representation was developed at the second phase of FTV. When the MVD is applied to the FN application with large baseline of the third phase of FTV, degradation is observed at the object boundary of the synthesized views. It is caused by inaccurate depth data of the MVD. In this paper, new 3D scene representation is proposed to suppress such degradation.

#### PRINCIPLE OF FTV

FTV was developed based on ray-space representation [13]-[16]. In the ray-space representation, one ray in the 3D real space is represented by one point in the ray-space. The ray space is a virtual space. However, it is directly connected to the real space. The ray space is generated easily by collecting multi-view images while giving consideration to the camera parameters.

Let (x,y,z) be three space coordinates, and (0,0), where -- 1/2 (0 (0 (0 (0 (0 (0 (0 ) 2 (0 (2 ) 2 ) 2 (0 ) 2 (0 ) 2 ) 2 , 0 2 , 0 2 , 0 2 , parameters of direction as shown in Fig. 1 (left-top) for the orthogonal ray-space. These ray parameters construct a 5D ray-space (x,y,z,0,0) of multiview images. We define a function f(x,y,z,0,0) whose value corresponds to an intensity of a ray. This ray-parameter space is the "ray-space." It is clear that ray-space is 6D if time is included as a parameter.

Although the 5D ray-space mentioned above includes all information viewed from any viewpoint, it is highly redundant due to the straight traveling paths of the rays. Thus, when we treat rays that arrive at a reference plane, we can reduce the dimension of the parameter space to 4D. Although the 4D ray-space gives both horizontal parallax and vertical parallax, the horizontal parallax is more important than the vertical parallax. If the vertical parallax is neglected, ray-space becomes 3D with parameters (x,y,0).

Thought the parameterization is simple in orthogonal ray-space, this parameterization cannot represent all rays; for example, rays parallel to plane z=0 (@=π/2). Thus, spherical rayspace shall be defined as shown in Fig. 1 (left-bottom). Given a reference plane with the normal vector in the direction of (0,0) , where -n \_(0,0) , and coordinate of (5,n),

the ray emitting from the surface of an object perpendicular to the reference plane at intersection of (5,n) is defined by f( 5,n,0,0).

Fig. 1 Ray-space representation and view synthesis [4].

The following explains how ray-space is captured, constructed, and free-viewpoint images are synthesized. For ø=0, the orthogonal and the spherical ray-spaces can be captured by using linear and circular camera arrangements, respectively. Ray-space is constructed by transforming captured images by cameras in real world into ray-space domain, in which transformed images are aligned in parallel. The image planes in orthogonal ray-space are flat planes whereas sine shaped plane in spherical ray-space. Fig. 1 (middle) depicts examples of the constructed ray-space for orthogonal ray-space and spherical ray-space when φ=0. Constructed ray-space can be used for free-viewpoint image generation. Given the location of virtual viewpoint, a free-viewpoint image can be generated by cutting the ray-space vertically. As illustrated in Fig. 1 (right), in orthogonal ray-space the vertical cross-section is a flat plane whereas in spherical ray-space is a sine shaped plane. If the captured images are not dense enough for free-viewpoint image generation, ray-space interpolation should be performed to obtain a large number of views. In this process, virtual views are placed between real camera views as if the camera arrangement is very dense.

#### CONVENTIONAL 3D SCENE REPRESENTATIONS

Figure 2 shows major 3D scene representations in the view-geometry domain. In this figure, horizontal axis denotes the number of views "N", where views are images of a 3D scene captured by multi-camera system. Vertical axis denotes the number of depths "M", where depth means depth map. A depth map is an image, in which the intensity of each pixel represents distance between the optical center of a camera and the surface of an object in the 3D scene. Depth corresponds to the geometrical information. Many depths can be represented

in a different form, e.g. a 3D model. A camera position has view and/or depth. It means number of views and depths can be different in a multi-camera setting. When M-N, all camera positions have view and depth. However, if N>M, there are some camera positions that have only view, but no depth. On the other hand, for M>N, there are some camera positions that have only depth, but no view. A general form of view-geometry representation is multiview plus multi-depth (N View + M Depth) that covers all other presentations. Details about different representations will be given as follows.

Fig. 2 Conventional 3D scene representations in the view-geometry domain [4].

Representations with no geometrical information are as follows. The conventional 2D-TV system is achieved by a single view image. Representation with more view than one view is multiview representation. If number of views is very large, multiview representation is equivalent to ray-space, which is explained in the previous chapter.

Representations in the following are for the case of having only one view with variable geometrical information. Using one view plus one depth representation we can achieve a freeviewpoint image generation in a narrow range. By adding more geometrical information or having multi-depth, we can represent the free-viewpoint system as one view plus multi-depth, such as LDI (Layered-Depth Images) [17]. LDI contains more geometrical information, so that it can provide wider range of free-viewpoint. With all geometrical information or 3D model of the scene, we have 3D model plus texture representation that can provide a wide range of free-viewpoint. Point clouds can be converted to 3D models [18].

Surface lightfield [19] contains less amount of geometrical information in comparison with 3D model while several views are required for this representation. This representation can provide wide range of free-viewpoint images. (N View + N Depth) is MVD (Multi-view plus depth) representation adopted at the second phase of FTV by MPEG. GVD (Global View and Depth) [20] that consists of base view, base depth, residual views and residual depths is a compact data format of MVD. Redundancy of MVD is removed in GVD.

In [21], various 3D scene representations are surveyed in one dimension, in which the more geometrical information corresponds to the less number of view and vice versa. As shown in Fig. 1, it is along the axis connecting the most left-top point, i.e. 3D model/texture, to the most right-bottom point, i.e. ray-space.

#### NEW 3D SCENE REPRESENTATION

Concept of the proposed method is shown in Fig. 3. Most of the rays of a 3D scene are captured as rays crossing a reference plane 1. However, there are other reference planes for objects that can't be represented well by the reference plane 1. It is denoted by a reference plane 2 in Fig. 3. If the reference plane 2 is set to be the surface of the object, the reference plane 2 collects rays emitted from the object. However, it can be set to enclose the object as shown in Fig. 3. In this case, the reference plane 2 collects not only rays emitted from the object but also rays from background near the boundary.

Fig. 3 Concept of proposed method.

Figure 4 shows data of the proposed method. The left of Fig. 4 denotes data collected by the reference plane 1. For example, view 1 is a view image at camera position 1 in Fig.3. There are gray portions in the view images. The gray portions correspond to rays from the reference plane 2 and those data are not necessarily needed because they are collected by the reference plane 2.

The right of Fig. 4 denotes data collected by the reference plane 2. There are 2 representations for them. One is view images on reference plane 2 and another is orthographic projection images. The latter is the same as the spherical ray-space representation. However, its range is limited to the rays from the reference plane 2.

Views are synthesized as follows. First, rays are reproduced by using data on the reference plane 1. Then, rays reproduced by using data on the reference plane 2 are added.

Fig. 4 Data of the proposed method.

#### CONCLUSION

New 3D scene representation is proposed for free navigation of FTV with large baseline. It is effective to suppress degradation observed at object boundaries of synthesized views.

#### ACKNOWLEDGMENTS

This research was partially supported by JSPS KAKENHI Grant Number 25289117 and Grant of Support Center for Advanced Telecommunications Technology Research.

#### References


# 7\*"&\*7"!?!J(\$#\*%F!'%FF&\$(&"!!! %i]!\$&F%#&]!(i(#(%#(J&"!

Vito Cappellini 65 and Enrico Del Re (edited by), *Electronic Imaging & the Visual Arts. Eva 2016 Florence.* ISBN 978-88-6655-973-3 (print) ISBN 978-88-6655-974-0 (online) © 2016 Firenze University Press

## TERAHERTZ ADVANCED RESEARCH TECHNIQUES FOR NON-INVASIVE ANALYSIS IN ART CONSERVATION (THz-ARTE)

M. Picollo2, A. Aldrovandi42, G. Bartolozzi4, A. Casini1, C. Cucci1, A. Doria3, K. Fukunaga , G.P. Gallerano , E. Giovenales , R. Olmi+, M. Poggesi1, L. Stefani1 1 "Nello Carrara" Institute for Applied Physics of the Italian National Research Council (IFAC-CNR), Via Madonna del Piano 10, 50019 Sesto Fiorentino, Italy 2 Opificio delle Pietre Dure (OPD), Via degli Alfani 78, 50121 Firenze, Italy 3 National Institute of Information and Communications Technology (NICT), Nukui-Kita, Koganei 4-5-1, 184-8795 Tokyo, Japan 4 ENEA-Frascati 00044 Frascati, Roma, Italy m.picollo@ifac.cnr.it; alfredo.aldrovandi(@beniculturali.it; g.bartolozzi(@jfac.cnr.it; a.casini@jfac.cnr.it; c.cucci(a)ifac.cnr.it; andrea.doria(@enea.it; kaori(@nict.go.jp; gianpiero.gallerano@enea.it; giovenale@frascati.enea.it; r.olmi@ifac.cnr.it; m.poggesi(@ifac.cnr.it; 1.stefani(@)ifac.cnr.it

Abstract - The THz-ARTE proposed to combine together visible (Vis) and near infrared (NIR) hyper-spectral imaging technique with data acquired in the Gigahertz (GHz) and Terahertz (THz) frequencies for the study and analysis of 2-D polychrome artworks.

#### INTRODICTION

The non-invasive diagnostic and analytical approach to the study of works of art is an important topic within the priority research area of "Technologies applied to Cultural Heritage" [1]. Several techniques have been developed in the past to achieve a characterization of paintings as complete as possible and to extract data on the provenience and age of artists' materials [2]. Among the imaging techniques a very important role is played by Infrared Reflectography (IRR) and Vis and NIR hyper-spectral imaging (HSI) techniques [3-7]. The former enables the visualization of details underneath the visible surface of a painting, exploiting the partial transparency of the painted layer when subjected to IR radiation. HSI, in addition to the IRR information, provides a spectral and colorimetric characterization of the entire painted surface.

Within the "Terahertz Advanced Research Techniques for Non-Invasive Analysis in Art Conservation" (THz-ARTE) project the Italian (ENEA-Frascati & CNR-IFAC, Firenze) and the Japanese (NICT-Tokyo & NNRICP-Nara) teams proposed to extend the imaging techniques mentioned above to Gigahertz (GHz) and Terahertz (THz) frequencies, more specifically in the range from 20 GHz to 0.6 THz, which is situated between the microwave and infrared regions of the electromagnetic spectrum.

Radiation in the THz and GHz spectral range is reflected by metals and aqueous materials, while it is transmitted with low attenuation through most dielectric materials, such as paper, several pigments and dyes, binding media [8]. Regarding the characterization of materials used in works of art, GHz and THz imaging systems will provide useful data on revealing the internal physical structure of non-metallic objects without touching the artworks investigated. This non-invasive cross-section image of the object is acquired by extracting the reflected pulse from a particular interface of two media with different refractive indexes [9-15]. This type of information will complemented the data obtained with Vis and NIR HIS thus producing a more complete set of records to be used by other colleagues, conservators, and curators.

In the present communication, some of the first results obtained within the THz-ARTE project as well as the application of the IFAC-CNR Vis-NIR HIS (Fig. 1) and the ENEA

TGHz/Hz 3D (Fig. 2) systems on the fresco-painting on "tavella" depicting St. John Baptist (approx. 1700) by Alessandro Gherardini will be reported.

Fig. 1 Vis and NIR hyper-spectral imaging scanner developed at IFAC-CNR

Fig. 2 GHz/THz 3D scanner developed at ENEA

#### ACKNOWLEDGEMENTS

The authors thank the Italian Ministry of Foreign Affairs and International Cooperation for having financially supported the THz-ARTE project.

They also acknowledge the Museum of San Marco, Florence, for having allowed the investigation on Gherardini's painting.

#### References


## THE SECRETS OF THE WORK OF ART. MYSTERIES CONCEALED BEHIND OLD RESTORATIONS AND THE TRUTH REVEALED

#### Sara Penco

Restorer and creator of the SMARTICON project Rome, Italy < info@sistemapenco.com>

#### Abstract

The passage of time scans some historic steps that, to this day, can represent real secrets that lie behind the image of a work of art.

Our usual way of observing, studying and examining the Cultural Heritage, does not always provides an adequate tool to understand the hidden "truth" and, for this reason, the secrets that can be revealed make the world of art one of the sectors with unrivaled charm.

It is to the figure of the restorer who, not infrequently, the fate entrusts the most extraordinary discoveries: behind the overlaying of dirty patina, yellowed varnish and old repainting, you can uncover the truth camouflaged for centuries behind the hidden veil of time.

#### INTRODUCTION

Although, since the beginning of the eighteenth century, the figure of the restorer already represented an autonomous profession, history witnesses how it was customary to privilege his own creative flair compared to the one of the artist whose work he was going to preserve.

The restorer of the time felt entitled to "reprocess" the work of art by relying on his own interpretation, rather than conserve and preserve with rigor the integrity of the original artifact.

So many works of art, over time, were even "modified"; to the point that, today, it is legitimate to speak of actual "tampering".

In the nineteenth century, thanks to the Italian and French experience, the difference between "mechanical restoration", strictly limited to the restoration of the painted surface, and the "artistic restoration": a real scientific methodology dedicated to the study of techniques for cleaning and reintegration of artifacts, started defining.

This awareness leads us to scan the necessity to adjust and subordinate the conservation interventions to the "constitutive materiality" of the artifacts, revealing the fundamental need to know, examine and evaluate in advance all those restoration processes that, over time, could undergo inevitable changes that would eventually cause what we can define as extensive damage to the Cultural Heritage.

The need to establish an "ethical code" designed to regulate the professional training of the restorer, definitely matured in 1931 when the ICOM (International Conference of Museums), reaffirmed the importance of this issue by signing a document that saw historians of art and museum directors, which confirmed the need to define the training of the professional figure of the restorer.

Over the last decades, the skills of some great experts, as Cesare Brandi (Director of the Central Institute for Restoration in Rome) and other important realities, have provided a valuable contribution to define the role of the person to whom was entrusted the task of preserving an artwork; by raising their awareness to subordinate their work to very specific rules which would establish techniques, technologies and methods: according to a logic oriented to the strict protection of the original material.

#### Investigating a work of art

The preservation of Cultural Heritage, its conservation, enhancement and continuous rediscovery, provide a precious contribution and the essential foundation for the affirmation, development and diffusion of the identity of all people.

This awareness can and should lead the professional figures of this sector to never limit themselves to the mere appearance of what they are admiring, but to aspire to investigate the search for a truth that may have been concealed over time.

While examining a work of art we need to linger to also observe the most negligible details and ask ourselves questions that, although apparently obvious, they can reveal the suitable path for the encoding of a lost language and / or forgotten through the centuries.

The fact that among the best known prerogatives of authoritative teachers of every era there is to conceal, behind the image of a work of art, hidden meanings not easily understood, which often remained secret for centuries is not negligible.

Think of the Sistine Chapel.

The most majestic Chapel of the world represents just one of the best known symbolic art icons and most studied in the world; but it is also one of the most irrefutable testimonies of the ability of a great artist, who was Michelangelo, to express ideas and thoughts steeped with transgression; enunciated through symbols skilfully camouflaged in an absolute masterpiece, that even the client (exactly the latter) would have been able to code.

What is most fascinating is that the deep expertise of these artists made them so erudite to succeed in wisely eluding as well as offhandedly the strict rules imposed by the client. To the point that, five hundred years from their representation, to this day, they still provide the cue for publications and revelations made by leading experts ... dedicated to unravel mysteries still disguised today behind the veil of that time which is waiting patiently to preselect the person to whom fate will entrust the privilege to perform the most daring revelations!

This "refine" as well as complex and acute ability to express thoughts and concepts that only the most acquainted could understand, went far beyond the doctrinal canons established by the usual iconographic "codes" and privileged the artist with an absolute freedom of expression, forged in the knowledge and also the courage to even challenge the papal commission revealing, without its knowledge, the most transgressive concepts and allegorical meanings.

But the passage of time has seen even less competent figures alternating, even though unaware, who have concealed and seriously penalized the preservation of the asset, both in its physical consistency and in the preservation of its historical and iconographic meanings.

Not infrequently the interventions of clumsy "restorers" have ended up in tampering the original appearance of many works of art and changing, inevitably, also the iconographic code represented in them.

This dynamic has been, and is still today, a harbinger of unconfessed secrets hidden behind patinas of dirt and yellowed varnish; this makes the work of art a treasure chest of mysteries, since hundreds of years, waiting to be revealed.

During the restoration of a painting that seems apparently intact, the removal of the layer of dirt has made resurface the original composition, just as the artist had designed and manufactured five centuries ago.

Is it possible to sense the presence of secrets that lead to the research of truth to be revealed? Can the Virgin Mary, be portrayed without the veil?

#### Extravagant tampering and the research for the truth revealed.

The iconographic tradition that, through the ages, has seen the figure of the "Virgin and Child", represented is very complex and varied and develops iconographic types that will undergo from the simplest to the most significant mutations.

Often these different types of depictions which represent a fundamental precondition for the rediscovery of the artifact; Since they allow to locate fundamental historical passages to identify the various customs and artistic traditions to which they relate: both from a doctrinal aspect, rather than in purely artistic and creative terms.

The Western Cultural Heritage, in fact, is often characterized by its complex and mutable iconographic typology: understood not only as an interpretation of a variety of themes and narratives; as well as mutations, often also substantial, characterized by multiform as discontinuous historical and iconographic representation of the same subject.

Although these mutations, occurring in time and in space (namely in different periods and geographical areas), objectively coexist in a single reality, they themselves determine an eclectic and very complex scenery that, not infrequently, precisely interposes the correct interpretation the work of art.

The usual iconographic tradition sees the Virgin Mary portrayed with a red vest and wearing a blue cloak that leads back to the colour of the sky and that associates the image to the "Regna Coeli".

For centuries the Virgin has been represented with the crown that, until the thirteenth century, was laying over a coif usually coloured of red and beneath it the hair was collected: the "maphorium".

It is necessary to wait for the representation of the Stoclet Madonna, an opera of Duccio from Buoninsegna (Siena, 1255-1318 / 19) in Brussels today, in order to see that the coif is removed from the tradition and replaced with a white veil interposed between the hair and a blue veil.

Duccio from Buoninsegna ca. 1290-1300

There are many reflections that can be detected in the complex and significant development of the iconographic tradition related to the representation of the "Madonna and Child": as the interesting comparisons and references regarding the pagan representation of the Egyptian goddess Iside while accommodating her son Horus on her laps.

The testimonies that we receive from the Marian texts coming from Egypt, in fact, represent a source of knowledge that significantly supplies the "recovery" of the iconographic path that allows us to understand the origins of the Marian representation from the III-IV century and on.

Consider that the depictions of the Virgin dating from the early centuries will lead, later, also to the doctrinal connotations placed in connection with the heresies.

With the Council of Ephesus, in 431, the figure of the "Madonna and Child" was placed permanently in connection with the image of "human motherhood", marking the beginning of an iconographic tradition that remained, in centuries to come, extremely flourishing.

The icon of "Platytera", the solemn Byzantine Madonna often seated on a throne with the Child on her knees, depicts the two figures leaving broad interpretation to the tenderness of the maternal relationship. Mother and son are facing each other while Mary, with her right hand, gently pats the shoulder of the Child; they mark an iconographic tradition that remains constant for centuries. The figure of Jesus, although over centuries of representations, interprets the regal and solemn aspect of the Savior: symbol of divine salvation.

The "monumentality" of the figures represented in these early centuries, will characterize a Virgin absorbed in meditation and will mark a long period in which Jesus will be tacitly put beside the Virgin; but always in a loving relationship punctuated by detachment and where the glances between Mother and Son never will meet.

The imperturbability "of this representation eloquently communicates the doctrinal content of the iconographic message, which interprets the Virgin Mary as the one who bears witness to the mystery of the Incarnation of God's Word and His intercession for mankind; but, simultaneously, it focuses and also directs attention to the figure of the Child.

Only at the beginning of the eighth and ninth centuries, the iconographic typology of the "Virgin of Tenderness" will appear for the first time, which will reunite the two figures in a more affectionate and involving attitude, where the faces will approach until they touch and the attitudes will mark the apex of loving mutual encounter.

The representation of the sentimental emotionality between Mother and Son will provide inspiration, manifested in the artist as well as in the client, to subordinate the rigid canons of classical iconography (up to this moment so well marked by the councils as well as the iconographic tradition), to a more passionate and confident scheme, but also more refined and a harbinger of greater creativity.

"Virgin and Child" will indulge in those tender and accomplices gestures dictated by the love that unites Mother and Son; in which mankind naturally recognizes itself, creating an interpretative freedom, that from the monumental rigor of the early centuries, gives way to a representation permeated with the tenderness of maternal feeling, which will become the actual protagonist of the composition.

Leonardo da Vinci 1478-1482 Hermitage Museum Saint Petersburg

#### Mysteries hidden behind old restorations and a revealed truth

Before restoration (fig. 1)

After restoration (Fig. 2)

Can the Virgin Marv, be portraved without the veil?

The experience of the restorer teaches that, over the centuries, a large amount of works of art, even though with the intent to be preserved, have ended up with the most varied types of interventions of "restoration."

Clumsy artists, declined into the role of improvised conservatives of products made by their predecessors, have often interpreted the Cultural Heritage as an asset to intervene on with no will and, what is worse, feeling also entitled to "improve" through renovations, additions and personal interpretations that have tampered and compromised the integrity and authenticity of the products.

For a combination of various reasons and circumstances, a good restorer must be as aware as possible of what he going to preserve.

An example for everyone is the choice of the solvent, which is one of the most delicate decisions because its task is to remove the harmful layers. However, a wrong choice, is likely to irreparably compromise the work of art.

Each step must represent the result of a careful prior evaluation, to be weighted with diligence and to be carried out after an accurate preliminary investigation, which has allowed to investigate in depth the characteristics of the original artifact and subsequent contributions.

It will be appropriate to define the nature of the original materials, their characteristics, types of harmful layers to remove and many other factors.

Observing at the work of art depicted in image 1 (Before restoration) has given the possibility to establish that it is a painting done in tempera on wood, dated from the XVI century.

The transparency of the colours, due primarily to the fact that the pigments have been mixed with a water-binding agent, allows to appreciate a clear and accurate preparatory drawing, by means of which the artist has outlined the entire compositional setting.

The spread of the colours is characterized by a palette with gaudy and striking chromatic colour range, made using pure colours; kneaded with each other on a basis of a criteria of "essentiality."

A considerable layer of dirt and a yellowed varnish camouflaged a previous restoration, dating from several years ago.

But my attention has been "captured" by a striking particular: the incongruity of the representation of the image of the Virgin from the usual iconographic norm.

The flowing hair of the Madonna didn't show any trace of the veil!

Although it is true that the iconographic typology of the Virgin over the centuries underwent various iconographic interpretations, over the centuries, it is also known that the attribute of the veil always remains an indiscriminate constant.

The absolute incongruity of this element has induced me to also observe the fact that the drafting of the pigments with which the hairstyles were made, including that of the Child, St. Joseph and St. John the Baptist, had a paste and a "fullness" that, noting it better, was very different from the chemical characteristics and the material consistency of the rest of the painting.

Impatient to unravel the mystery concealed behind the image of this representation of the "Virgin and Child", and after having previously consolidated the limited areas at risk of "loss of colour", I started the intervention of removal of the harmful layers.

Although I had already sensed that the painting had been subjected to substantial "repainting", their removal has proved to be extremely exciting!

The cleaning, little by little, has made the delicate veil resurface, almost transparent and tinted of a light blue pastel pigment that perfectly harmonized with the colour of the cloth lying on the lap of the Virgin Mary and on which the Virgin lovingly housed the Child, standing next to Her.

Traces of pure gold, which originally finely decorated both the Madonna and the crowns of the Saints, have re-emerged under the coarse repainting.

Once the cleaning has been finished and the compositional integrity of the work has been restored, it has been necessary to intervene with the pictorial reintegration of damaged parts, which made us re-establish the gaps, paying particular attention not to "overflow" to the original colour.

The completion of the restoration has allowed to appreciate again the iconographic coherence of the composition, recovering the original iconographic canons.

They testify obvious references to the passage between the most ancient iconographic doctrine, where the Virgin is intent to caress the Child with her right hand, and the traditional iconography of the "Virgin of Tenderness", that combines the figures of the attitude of a loving mother.

The painting has been subjected to the study by Professor Mina Gregori who returned the authorship to the tuscan artist Francesco Brina (Florence 1540-1586).

Vasari reports his activity, also witnessing that, in 1565, he worked on the construction of the apparatus for the wedding of Francesco de 'Medici and Giovanna of Austria.

The works of Francesco highlight a rather archaic culture, with frequent references to Andrea del Sarto; but also influenced in many ways by the complex structure of the Florentine altarpieces of the time, made by contemporary Giorgio Vasari.

Francesco, although a student of Michele di Ridolfo along with his brother Giovanni, is not to be confused with the latter because of a pictorial quality of different level. In fact, his style is characterized by a much more refined quality; ; so that many works of art are still of controversial attribution today between the hand of Francesco and that of his master Ridolfo.

### CULTURAL ASSETS, "EXTENDED" MUSEUMS AND THE HISTORIC CITY: THE CASE STUDY OF AN APP FOR GENOA

#### Valentina Fiore, Lauro Magnani, Sara Rulli Dept. D.I.R.A.A.S. Università degli Studi di Genova Genoa, Italia valentinafiore1982@gmail.com; magnani@unige.it; sararulli1@gmail.com

Abstract - Getting tourists in normally unaccessible spaces, showing sounds and events to the citizens, all this while causing the same emotions the assignor had felt, increasing the value of spaces and corners of the city that still haven't been properly inserted in a touristic route, reaching a high divulgation to a large public: these are the goals set to the research project for an App aiming to spread information about the city.

The App's software and user interface, developed by Xenia Production and Smartland Project s.r.l., are made to contain different levels of information, from the whole urban area to the single artistic artifact, inviting the visitor to experience a real route in the city and a virtual one in the in-depth analysis of its cultural aspects.

The App is focused on the Strada Balbi and Piazza dell'Annunziata area, where a big part of the University's buildings are located: these buildings - which together form a public "extended museum" lived, inhabited and studied directly by the University - allow the academic environment, its researches, and the new discoveries made by the continuous studies on the artifacts it lives in to speak for themselves. In this regard the App, working with the large patrimony property of the Athenaeum, is meant not only for the external public, but also for the internal one, meaning the students themselves, for which it will become a learning tool and an instrument for a more direct experience. The researches for the study projects, final papers, in-depths analyses and topicalizations will be immediately and directly communicated to the tourists and, why not, the city's inhabitants.

#### INTRODUCTION

The use of multimedia technology, and particularly of virtual reconstruction, allows art historians to not only study the dynamics of monuments, examining in depth different, well defined points of view and to contextualize them in their original environment, but also to share the results of the research with as many people as possible, providing categories and complex interpretations for artistic artifacts, and more importantly, for the city in its whole. Multimedia technology can thus help researches, becoming a testing ground for hypotheses, a way to verify interpretations and a valuable medium for the divulgation and representation of researches and the achieved results all at the same time. Throughout the years studies have been conducted in these fields by the D.I.R.A.A.S. department in collaboration with research groups afferent to the genoese Polytechnic School and young Ligurian companies (see: Acknowledgements), creating a relationship based on interchange and collaboration that aims to entwine liberal arts and scientific and technological know-how. The focus has been put on promoting the genoese Atheneum's assets and the city in general, which is full of palaces (declared UNESCO World Heritage in 2006), frescoes and art collections to be studied in their complexity and with a systematic overall view. Getting tourists in normally unaccessible spaces, showing the citizens sounds and events, all this while causing the same emotions the patron had felt, increasing the value of spaces and corners of the city that still haven't been properly inserted in a touristic route, reaching a high level of divulgation to a large public: these are the goals set to the research project for an App aiming to spread information about the city. The App's software and user interface are made to be as flexible as possible, containing different levels of information, from the whole urban area to the single artistic artifact, inviting the visitor to experience a real route in the city and a virtual one in the in-depth analysis of its cultural aspects.

#### Designing the App: the graphic interface and communication with the user

The needed requirements for an easy, immediate use and the aspects entailing the interface with which the App presents itself to the user, menus, accessibility and choice of itinerary, maps, content visualization formats, social network accessibility and online experience sharing have all been fine tuned.

The graphic interface (see: Acknowledgements) is divided into three main parts: the left side menu (for the main actions), the right side menu (for the contents), and a central part (for visualizing the contents themselves).

This way the user can hold its tablet or smartphone horizontally like a steering wheel, without needing to turn it each time; the positioning of the commands on side menus allows the use of the two hands without moving them on the screen.

The choice of white characters on a black background helps defining a clear interface, with a high contrast ratio even under intense lights.

The left menu, the main one, contains all the login and social network connection commands (the App is constantly connected to the internet and to social networks), but also those for sharing with other users the experiences made in the city ("Share" button) and information on the position of the user ("Where am I?" button).

The "Itineraries" button grants access to a screen that contains a matrix made of more buttons whose number is equal to the routes the user can choose to get to know the city. After choosing one (eg. the Aquarium-Nunziata-Balbi-Aquarium ring) a preview screen appears showing the itinerary and the position of the user on an aerial map.

The "map" button shows the position of the user, a step-by-step view of the route and the succession of the various points of interest, each independently clickable.

The right menu grants access to each point of interest's in-depth information (by selecting them independently) and allows the user to zoom the map in and out, change the point of view, eg. from above (satellite view) or a three quarter view from the four cardinal points (bird's-eye view, using the button on the bottom right).

A following implementation phase of the App will allow the user to mark points of interest other than the default ones, using a remote or local database with remote synchronization and the possibility to edit the map with filters, specific visualizations, etc.

The central area, which can be viewed in full-screen, is used to visualize videos, pictures and text.

### Itineraries of valorization of the historic city and the university center: the Aquarium-Nunziata-Balbi-Aquarium ring

The App invites the user to physically - or whenever this isn't possible, virtually - enter and discover significant and peculiar points of view that, even in usually open buildings, aren't easily accessible, but can be proof of the assignor's particular approach, the layering of the urban structure, views, and relationships between interiors and exteriors, such as the various nymphaea, atriums and hallways behind the front doors, often corners of gardens, openings in the high density medieval urban structure. This App is also made to overcome the existing architectural barriers: the audio, video and textual supports can also help people with hearing, visual or physical disabilities to thoroughly enjoy these otherwise unaccessible artifacts.

To obtain an effective design of the App there was the need to analyze the historical sources and the images of the city in order to consider the "city artifact" as a complex and multi-layered structure where, in the different phases of the its development, historical events and topographical realities have been shaped by the actions of the Republic's government, the aristocratic families and the religious orders, following the needs of the economic, productive and port activities.

To draw the guidelines for the App's development (for the realization of the software, see: Acknowledgements) two different methods of approaching the urban artifact were adopted: on one hand to discover the city following topographical routes in which fundamental points of interest are marked, on the other hand to a thematic in depth analysis of some aspects or events of the city regarding specific artists that have worked there (eg: Galeazzo Alessi) or cultural and artistic monuments that strongly characterized the city's appearance, a UNESCO World Heritage since the organization recognized the peculiar social and economic identity of its manneristic and baroque palaces, full of internal decorations.

The results of the feasibility studies regarding the App and its resulting realization allowed the conception of an innovative way of divulging historical and artistic information to a public that has no specific academic education. The App's innovative character, with complex facts explained in an easy and quick way, makes the product absolutely competitive with the potential to become, with further development, a real business for the involved companies.

The repetitive format chosen for presenting the artifacts, divided in three categories for explaining each cultural asset ("Architecture" - "Decoration" - "Thematic Itineraries"), further divided into sub-layers, together with the easily usable interface of the App, which allows to easily access and choose the itineraries, the maps, and to share one's experiences online thanks to the possibility to connect to social networks, makes the product a real and concrete model that can be exported to other territorial realities other than Genoa, both in Italy and abroad.

The App focuses on the area of Strada Balbi and Piazza dell' Annunziata, containing descriptive and explicative text regarding the Chiesa della Santissima Annunziata, Palazzo Chiavari Belimbau, Palazzo Balbi Cattaneo, the Ex Collegio dei Gesuiti and Palazzo Balbi Senarega. Inside them are sited many different and extraordinary realities of the genoese Athenaeum. These buildings - which together form a public "extended museum" lived, inhabited directly by the University allow the academic environment, its researches, and the new discoveries made by the continuous studies on the artifacts it lives in to speak for themselves. In this regard the App, working with the large patrimony property of the Athenaeum, is meant not only for the external public, but also for the internal one, meaning the students themselves, for which it will become a learning tool and an instrument for a more direct experience. The researches for the study projects, final papers, in-depths analyses and topicalizations will be immediately and directly communicated to the tourists and, why not, the city's inhabitants.

The contents are structured following four different types of text: a general, introductive one briefly but efficiently explaining the historical artifact as a whole, a descriptive one that analyzes the architectural aspects of the asset, an explicative one about the decorations, and a specific one illustrating some peculiar data about the church or palace.

### The three dimensional virtual reconstruction as a way to immediately comprehend the city's and the artifacts' historical phases

The possibility of offering the users virtual, tridimensional models of the different phases of the historical, residential and urban realities of the city during their visits is equally fruitful. This allows to show them in real time the beauty of the architects', artists' and the assignors' works.

As a consequence of choosing Piazza della Nunziata and Via Balbi as the focal point of the itinerary, the attention focused mainly on the base-line study of the different building phases of Piazza dell'Annunziata, an area that has undergone lots of transformations, continuously altering its appearance between the XV and the XIX centuries, and of its buildings, particularly Palazzo De Ferrari Chiavari Belimbau. With the goal of analyzing the different historical phases of the square and the changes brought to the buildings needing renovation, some of the most important iconographic and cartographic sources have been selected and verified to be utilized as guidelines for explaining the user an historically complex urban situation.

A good example would be Palazzo Balbi Senarega, which allowed providing the user with the three dimensional virtual reconstruction of the architectural situation at the beginning of the XVII century, then modified and expanded, since the middle of the same century, with the addition of the garden and the nymphaeum that characterize the palace. The initial phase of the project - testified by Rubens' drawings from I Palazzi Moderni di Genova, published, in a second edition, in Antwerp in 1652 - is thus presented to the visitor and compared with the new spatial values of the midcentury renovations, allowing him to immediately understand the importance and dimension they have meant for the building and its relationship with the medieval city. The architectural extensions and the new decorations have shifted the residence's style to purely baroque. In this case, the potential of three dimensional reconstruction will allow the visitor to experience the genoese architectural dynamics and the different choices of the assignors. In this phase the focus was particularly on the choice of the layout for the final rendering of the reconstructions: avoiding the use of a "mimetic" approach that can be confused with reality, the choice fell on a more "abstract" version excluding explanations regarding the use of materials and that can easily appear as a virtual "maquette", a study model of a reality that doesn't exist anymore.

#### Further development: the Renaissance city of Galeazzo Alessi

For the fifth centennial of the birth of architect Galeazzo Alessi and in preparation for an international conference on this outstanding architect who had worked in Genoa in the second half of the XVI century some thematic routes - which will be developed with the collaboration of Dep. DSA of the genoese Polytechnic School - have been hypothesized to connect and bring visibility to his works, not only for tourists, but first and foremost for the Genoese. This kind of itinerary can serve an autonomous or functional and of in depth analysis purpose to the topographic ones. It can be a starting point to develop a method for museums (eg. the possibility to develop an App for museum itineraries) and contemporary galleries.

#### ACKNOWLEDGEMENTS

This work is born from the collaboration between the D.I.R.A.A.S. Department and young Ligurian companies, coordinated by Prof. Lauro Magnani. The App's graphic interface and audio and video takes were developed by Xenia Production, while the software by Smartland Project s.r.l.. The original research on these aspects of the Ligurian territory as a model for relating the artistic artifacts and the monuments with the purpose of producing scientific content for a multimedia guide was conducted by dr. Valentina Fiore; the historical analysis of the building phases and the structural characters of the buildings and monuments of the genoese urban fabric in need of renovation and the realization and use of 3D digital reconstruction for the App were conducted by dr. Sara Rulli.

Both researchers have received two biennial research checks on the "PO CRO Fondo Sociale Europeo Regione Liguria 2007-2013 Asse IV Capitale Umano ob. specifico I/6; Programmi nn. 55 e 56, area scientifico disciplinare: Scienze dell'antichità, filologico-letterarie e storicoartistiche, settore scientifico disciplinare: L-ART/02 Storia dell'Arte Moderna" for the project "Valorization on the artistic and cultural assets and models of multimedia communication: mediation and accessibility of the cultural contents of the research and study of a specific application for smartphones and tablets."

#### **0(\*3#&+!-)'!.-\/&+**

^3CH!UK!#9-!Y%R;/23;D?i;5G3/6/?[/.M3?%R;/23;D!\$35Cg!

^3CH!SK!%!@3-<!02+D!69-!6+=!+0!69-!D-43-@/.!6+<-2!+0!1+26/!4-3!J/88/B!+5-!+0!69-!Y1+356,!+0!(56-2-,6g! +0!69-!Y%R;/23;D?i;5G3/6/?[/.M3?%R;/23;D!\$35Cg!

^3CH!cK!12-,-56/63+5!+0!69-!Y%R;/23;D?i;5G3/6/?[/.M3?%R;/23;D!\$35CgK!69-!Y1+356,!+0!(56-2-,6g!

Fig. 4: Palazzo Balbi Senarega: 3D reconstruction of the architectural situation at the beginning of the XVII century

Fig. 5-6: Palazzo De Ferrari Chiavari Balimbau: virtual tour

### 3D SERVICES AT MUSEO GALILEO

Luisa Barattin, Marco Berni, Elena Fani

Museo Galileo - Institute and Museum of the History of Science, Florence, Italy l.barattin@museogalileo.it, m.berni@museogalileo.it, e.fani(@museogalileo.it

Abstract - 3D has been widespread for several years now, but for various reasons has not yet found its "everyday use" in the field of cultural heritage. There are, of course, applications specifically designed for museums, but they usually lack an advanced educational use. Most of the examples consists of a mere catalogue of manipulable objects, but the 3D format has the potential to create much more complex learning activities, which at the moment are unfortunately not yet widespread and certainly not standardized. Museo Galileo is testing a series of possibilities to develop new 3D services, keeping in mind that, for educational and dissemination activities, they have to be deeply integrated with the more classic ones.

#### 1 STATE OF THE ART OF 3D USE IN CULTURAL HERITAGE

In the last twenty years the tools for the production of 3D models have evolved to a point that place them outside the pioneering phase and introduce a new stage of daily use.

The most popular software houses offer online services both for the creation of models from photos and for printing and handling shared models. In addition, 3D printers have reached affordable prices, although their spread is still limited. Interesting examples of 3D usage in the cultural heritage domain are shown in a list compiled by Archeolab [01]. Among these are the 3D Virtual Museum [02] with a collection of geotagged artifacts that can be manipulated [03] and consulted by clicking on numbers that return textual information superimposed on the model; the Google Art project [04] that allows you to combine text, images and video to 3D handleable elements; the Smithsonian [05] and the British Museum. This technology has also been pursued by several European projects, such as 3D-COFORM [06], which contributed to the 3D model metadata adopted by Europeana. The examples described in the above list are certainly valid, but when we consider the user interface and interaction, the current models do not go much beyond the simple manipulation of objects, thus not benefiting from the full potential of the 3D interface. The situation is certainly destined to evolve in the near future, thanks to a more technically advanced offer and the achievement of a usage standard recognized by all users.

#### 2 HTML5 AND THE NEW TECHNOLOGICAL OPPORTUNITIES FOR 3D IN WEB AND MOBILE ENVIRONMENT

The efficiency of the fruition method for 3D models is obviously the crucial point for the spread of their use and has always been the weakest link in the whole chain. To achieve this there are two possible solutions. The first relies on specialized hardware, such as stereoscopic visual simulators and systems for augmented reality, in which the content provider has the whole production chain under his control, while the second expects from the producer only the 3D model that will be shown on the user's system. While the first is an experience reserved for the few, the second mode could potentially involve all users of the web, so it is worth investigating this aspect further. In this case both the hardware and the software are beyond the author's control and the risk is that the user experience is not the one imagined or, worse, the 3D model cannot be displayed at all. Obviously for a successful visualization of the model, an agreement on standard and its spread among a critical mass of users become mandatory. In the past, after a period of proliferation of ad hoc systems, two formats have spread, VRML and Flash, the latter of which was later banned for safety and efficiency reasons.

In the meantime the WebGL and HTML5 libraries were adopted by the most popular browsers. WebGL library provides all the basic functions for 3D on the web: other high-level graphic

libraries such as X3DOM, Skecthfab, and recently Unity, are based on it. Since Skecthfab is a proprietary software and Unity is not yet mature in its web version, the most viable solution is the X3DOM standard [07]. X3DOM is the integration in browsers of X3D [08], which is, in turn, the evolution of VRML. The X3DOM framework is maintained by a nonprofit consortium and is a member of the W3C. At present this standard is implemented in almost all popular browsers and can be used on both PC and mobile environment, so it has all the characteristics to be successful in the long term and to allow an even wider uptake of 3D models. To allow an easier spreading of 3D even among implementers not too accustomed to this technology, additional software-layers, such as 3DHOP [09], were built in order to make available a standardized interface and mode of interaction, with some limitations, but without the need to resort to a massive use of programming.

Museo Galileo opted for the use of native X3DOM for maximum freedom in the implementation of the various features and the widest possible circulation.

It is also worth mentioning that the metadata standardization adopted by Europeana, is beneficial for a greater distribution of 3D models.

#### 3 THE DIDACTIC-COMMUNICATIVE PERSPECTIVE AND THE INTEGRATION WITH EXISTING PRODUCTS

Sharing its own collections is, for a museum, a mission, as well as a possibility to increase the number of visitors and to engage a younger audience. Beyond the technical issues, the biggest deficiency in this field is the poor number of these models. The limitations are mainly of two kinds: on the one hand there are few models available since, in addition to the technical difficulties of acquiring certain materials, the creation of a realistic 3D model requires scanner acquisitions or photographic campaigns that need substantial resources; on the other hand the possible means of interaction are usually quite limited, even in relation to the fact that the available examples are often pertinent to archaeological or historical artifacts only. There is therefore considerable room for the development of user interfaces that allow much more advanced uses, especially in the presence of special collections such as those preserved in the Museo Galileo.

The Museum has been working on 3D models since the early 90s, but initially the usable products have been limited to the reconstruction of the physical models and the realization of movies that showed their operating principles [10]. Over the next decade a few interactive models were developed in Flash [11]: these provided the user a more active approach to information. The current need to implement systems that do not use Flash and also reach out to mobile platforms, must be seen as an opportunity to rethink from scratch the possible ways of user interaction with 3D models and to start a debate on the possible uses of 3D for historical and scientific museums.

The new technological opportunities are the necessary basis for the development of new applications, but for a successful result they must be part of an educational and communicative process that allows the user an integrated fruition with the services currently offered by the Museum. The road ahead therefore contemplates both the integration of the technology into the educational activities already proposed to individuals and schools by the Museum staff, and the evolution of the cultural offer in the guise of a support for the onsite visit and the autonomous learning, activities that have been pursued for many years gradually updating the technology. focusing first on the Web and then also on mobile platforms. From this perspective the 3D format can be considered the highest point in a series of integrated tools such as the Multimedia Catalogue of the Museum [12] and the Scientific Itineraries in Tuscany [13] and the upcoming videoguide App for the onsite visit. A close examination carried out inside the Museo Galileo, from a didactic point of view, has proved that 3D can be used in various ways inside the Museum, also considering that, due to the complexity and variety of its collections, it covers a large series of situations that can usually be found in different kind of institutions. The result is a series of possibilities on which we started to work as a basis for a development of new services.

· The manipulation of the object works well only with simple items, consisting of a single element without moving parts, such as the statues. These kind of objects are represented, even if to a small extent, within Museo Galileo. The benefits of this mode are a chance to look at the object from all points of view and the possibility to see or study it remotely. The production of these models in some cases presents objective difficulties related to the accessibility of all parts of the object and to the difficulties of displacement in the case of heavy objects.

· The downloading and printing of the models are instead well suited both for remote usage by students, and as stand-alone continuation (for individuals and groups) of the experiences made in the educational workshops at the Museum. The models can comprise a single piece or several movable parts. Depending on the case the model can span from a mere artistic reproduction (e.g. a statue) to a faithful reconstruction of interactive items used to simulate the operation of a real-world object. In the latter case, the utilization of the model requires not only the ability to download the 3D file, but also the operating instructions that must be provided contextually. For practical reasons it is not possible to offer the printing service inside the Museum.

· Some of the most significant potentials of the 3D can be fully appreciated while developing the interaction with the model. In the case of simple tools, interactivity may be achieved by highting several significant parts showing predefined views with self-explanatory contextual pop-ups that include various media. Most of the instruments of the Museum are composed of many moving parts and pieces that cannot be touched by the public. It is therefore essential that the 3D objects also allow the user to interact with the relative movements of their parts. Interactivity can be developed at various levels: from the simple split of the model which outlines and describes its various components to the simulation of the movement of these parts, or even further to a learning sequence that, with the help of a list of instructions, guides the user toward a result aimed to teach the use of the instrument.

In the case of incomplete, destroyed or never actually built instruments, the 3D is the best way to show them in their original form in order to facilitate their understanding by the visitors. However, for the end user the system cannot be limited to the presentation of objects, but must also give the necessary information to activate and use them and to understand their operation principles. This technology can allow curators to verify the feasibility and functionality of tools that are known only through their description.

· An understanding of some instruments, even complex ones, is greatly facilitated if these are shown in motion and eventually put in their original context. The possibility to interactively choose different points of view makes it an even more effective system than a simple video.

· Finally, a further use of 3D is the exploration of environments. The ability to recreate virtual environments offers many opportunities that will become even more significant as the quality of the representation will increase. This mode is particularly fruitful in the case of areas that cannot be opened to the public or no longer exist, and in the reconstruction of virtual environments in which to place interactive objects that can be connected to multimedia descriptions.

All these possibilities are the basis of 3D application to museum services: however it is important to keep in mind that, for educational and dissemination activities, 3D must not remain just a showcase designed to exhibit the ability to manage an innovative technology, but becomes a new service integrated with the more classic ones and an additional way to achieve the communication objectives of the Museum. These technologies are used to build an interactive environment that enables easy and pleasant user experience in order to stimulates the learning of subjects traditionally considered boring and difficult while maintaining the scientific accuracy.

#### 4 EXAMPLES

The main possibilities that are being considered and will be presented both in the individual applications and in a dedicated section are:

#### Object manipulation

The bust of Galileo [Fig. 1], exhibited in the room dedicated to him, here is not contextualized and is explorable from all angles with a simple drag and drop. The 3D model was made from photographic image processing with freeware software. The service is primarily addressed to the remote user who has the possibility to explore the statue from various points of view.

#### Download and print of the 3D model

The Museum's bookshop already sells kits of cardboard or wooden instruments to be assembled, such as the Mechanical paradox. A printable 3D model could be offered to download. At the same time the Educational Laboratories already teach how to perform experiments with instruments that can be self-built in the premises of the Museum or at school. In some cases they may also use 3D printed parts.

#### Interactivity

In the case of this globe [Fig. 2], in addition to its manipulation, there are predefined viewpoints and hot spots that allow to delve into important topics, thanks to pop-ups integrated into the page using HTML5.

When we manage complex instruments such as the astrolabe [Fig. 3], it is possible to highlight the individual components, providing descriptions for each of them, and to show their relative movements.

#### Incomplete or never built instruments

In the case of Leonardo's world map [Fig. 4], that was transmitted to us only in a manuscript copied by a pupil which is hardly readable, we reconstructed the globe as a browsable 3D [Fig. 5] model to make it

Fig. 4-5- Leonardo's globe

easily accessible to the public, while checking at the same time the consistency of the sketch.

#### Animations

Machines and mechanisms, such as those reconstructed from Leonardo's manuscripts, combine the reciprocal animation of the parts, eventually operable, with the possibility of manipulation already seen in the previous cases.

#### Virtual environments exploration

Fig. 6 - Virtual Museum

3D makes it possible to reconstruct environments, even not real ones, that may be interesting metaphors to access and understand content. In [Fig. 6] the Galilean instruments have been organized in different rooms depending on the time and place in which they were developed, thus creating a proper virtual museum that can be explored interactively. Each environment tool provides in-depth multimedia cards.

#### 5 CONCLUSIONS

The launch of services to the public based on 3D entails a series of costs, because it requires the presence and integration of different skills related to content, dissemination, programming, web sites production, in addition to those relating to the creation and management of 3D models. In the case of the Museo Galileo the presence of an internal Multimedia Laboratory makes it easier and more natural this fusion of skills, following a long-standing practice. The institutions that deal with conservation, study and dissemination of Cultural Heritage have an interest in providing 3D services not because they represent a fashionable innovation, but because the integration of all these possibilities with existing multimedia technologies, allows to improve the efficiency of the means of study and the quality of services to the public. In our personal experience 3D makes it possible both to the visitors inside the Museum and the ones connected remotely to benefit from a useful and personalized service, more effective than many classic approach at disseminating the history of science, the primary mission of the Museo Galileo.

#### References


uary 2016).


# ACCESS TO THE CULTURE INFORMATION

Vito Cappellini and Enrico Del Re (edited by), Electronic Imaging & the Visual Arts. Eva 2016 Florence. ISBN 978-88-6655-973-3 (print) ISBN 978-88-6655-974-0 (online) © 2016 Firenze University Press

## ART IN THE NEUROSCIENCE ERA. HOW THE BRAIN UNDERSTANDS AND CREATES ART.

Raffaella Folgieri1, Ludovico Dei Cas2, Francesco Soave3, Claudio Lucchiari1 1Dipartimento di Filosofia, Università degli Studi di Milano, via Festa del Perdono, 7 20122 Milan - Italy 2 CdL Scienze Biologiche, Università degli Studi di Milano, via Celoria, 26 20123 Milano, Italy 3 Msc Interactive Digital Media, Ravensbourne College of Design and Communication 6 Penrose Way, Greenwich Peninsula, London SE10 0EW raffaella.folgieri(@unimi.it

Abstract - Neurosciences, and particularly Cognitive Sciences, study creativity for the implications in understanding human and machine approach to problem solving. In fact, subjects such as Cognitive Psychology, Brain Neurobiology and Artificial Intelligence, investigate creativity mechanisms related to the human brain and cognition. We can state that creativity is a part of every human daily activity, so that, to understand individuals behaviour and abilities and the mechanisms of mind, we must start from the creative process, as, in general, the search of a different (innovative) solution to a problem. In this paper, we will treat the cognitive aspect of the creative process, particularly starting from cognitive technologies, devices helping researchers to analyse the process by the point of view of the artificial and natural intelligences.

#### INTRODUCTION: ART, CREATIVITY AND COGNITIVE SCIENCES.

"We paint with our brain and not with our hands" ("Si dipinge col cervello e non con le mani"), Michelangelo Buonarroti.

The artistic expressions are universally recognized as creative. This is the reason why Cognitive Sciences are interested in Art. In fact, they study artificial and natural intelligences, and creativity is one of the most investigated field. Among the questions to which Cognitive Sciences try to answer, we wish to recall the following:





From a theoretical point of view, creativity is a complex cognitive process resulting from the search of a balance between conscious and unconscious processes. Indeed, creativity may be considered a borderline state of mind, in which the thought seems to fluid cognitive state. When a new idea arises to the consciousness, and then a balance is achieved, the mind turn back to a "creative-off" state and divergent thinking is replaced by canonical thinking.

Through creative, our brain/mind system is able to find original solutions, new perspectives, further ways to organize the incoming data so as to give rise to an enriched reality. The experience we have of the world is made up of details and information, but is also rich in complex forms that interact each other not only to shape concepts and meanings, but also to evoke emotions, memories, thoughts, which are not directly matched with some physical features of a given stimulus detected by the senses. Consequently, creativity is a basic cognitive process which mechanisms are wired in our brains in order to give to a whole experience of the world, even when we see it for the first time. Hence, these processes allow the human brain to enrich our experience, shaping the deeper motivations that guide our cognition well beyond contextual needs. This is the cognitive core of art and creativity.

However, the creative cognition gives not rise to an illusionary world: simply, it extends mind possibilities, opening new frontiers and unblocking landscapes and options, thanks to an "as if" experience. Furthermore, this view of the brain working through "as if" processes may be integrated with the theoretical perspective by Changeux [1]. This author argues that we should always consider the perception as a creative process. For instance, when someone observes something (e.g. an artwork) in his/her brain a sort of re-creation is "in march", so to find a meaning but also to attribute mental states, emotions to the "source" of that object, thus mentally tracing the path from the creator to the observer.

In Cognitive Sciences, creativity is often studied investigating brain functions such as memory, language, attention, neuro-motor abilities, through the cognitive evaluation of the external stimuli given to a brain, as in Lindsay e Norman [2]. This cognitive evaluation is, obviously, subjective, because it is influenced also by social factors and cultural context of every individual, as suggested by Vygotskij [3, 4]. Also Piaget [5, 6], as Vygotskij, underlines as the interaction between subjects and objects stimulates the constitution of superior psychic processes. Nevertheless, Vygotskij expresses an interesting point of view, strongly related to the consideration of the links between creativity and means (also technological ones) at disposition. In fact, he underlines how the human ontogenesis is determined also by the contribution of the cultural instruments at disposition in the historical and social context.

The links among Art and Technology, creativity and the study of its mechanisms are several: the opportunity for the preservation of the artistic heritage; the potentiality provided by new instruments for the artistic expression; the possibility to study the links among Art, brain and Technology and, jointly, the creative processes, allowed by the progress of Artificial Intelligence, Brain Imaging and technological devices.

In this introductory paper, we will focus on the last point of the list. In the next paragraph we will introduce why and how scholars perform scientific investigations on creativity. In the following, we present the use of technology to express creativity and an experimental example of it, concluding with our considerations.

#### SCIENTIFIC INVESTIGATION ON CREATIVITY

Scientific investigation through new technological tools, such as Brain Imaging and Artificial Intelligence (A.I.), opens a wide scenario on the opportunity in studying relationships among Art, Technology and brain, and, consequently, creativity. Brain Imaging techniques allow notable improvement in the analysis of the brain "in action", that is in the study of the individuals' reactions and brain mechanisms involved in motor, cognitive or perceptive tasks. Thanks to devices and invasive or not methods, such as fMRI (functional Magnetic Resonance), MEG (Magnetic Electroencephalography), PET (Positron Emission Tomography), EEG (Electroencephalography), it is possible, in fact, to verify in real time the response of an individual to specific stimuli. Because of its low invasivity and high time resolution, EEG is the most used technique to investigate mechanisms such as creativity. Intuitively and experimentally, it is possible to state that Art is conceived by artists and observers thank to the neural activities of the brain, creating the aesthetic experience [7]. The study of this mechanism represents the base of the experimental discipline named Neuroaesthetic, officially born in 2001, studying the neural bases of the contemplation or of the creation of Art. Founder and pioneer of Neuroaesthetic is Semir Zeki, known neurobiologist who, in the '90s, begun to study to the links between Art and brain, using psychological tests and electroencephalography. Zeki compares the artist to a neuroscientist [8, 9, 10], "exploring potentiality and the ability of the brain, even if with different tools" and, even with a different language, transmits a knowledge discovered or intuited. Art, so, is not a prerogative of the artist, but resides also in who look at a masterpiece. Think to Duchamp, who has ever underlined that the meaning of a work art is given not only by the artist, but also by who look at the work, through auxiliary interpretations. The aesthetic experience of an artwork integrates and transforms the individual perception of reality in a lived experience with regard to the subject: the artwork disturbs, excites, soothes the individual and the like. Finally, Aage Brandt [11] proposes to consider neuroaesthetics as the study of neuronal process of perception and mental organization of cognitive activity stimulated by the artwork, following both a cognitive and evolutionary approach.

Particularly suited for the described research, the B.C.I. (Brain Computer Interface) headsets are simplification of the medical equipment for EEG [12], and allow to record cerebral rhythms and the direct brain-computer interaction. BCI devices are widely used in research, for the registration completely comparable to the medical EEG, but also for their low cost and high portability. They present, moreover, the advantage to keep in comfort the individual wearing them as they allow a wide movement freedom in the experimental environment. BCI devices collect several brain frequencies, grouped in rhythms: the alpha rhythm (7 Hz - 14 Hz), relaxation, meditation, contemplation; the beta band (14 Hz - 30 Hz), associated with active thinking, attention, problem solving; the delta rhythm (3 Hz - 7 Hz), registered in children and associated with continuous attention activity, as in Leeb et Al. [13]; the theta rhythm (4 Hz - 7 Hz), generally related to emotional engagement; the gamma signal (30 Hz - 80 Hz), usually related to the cognitive interpretation of multi-sensory signals. BCIs allow to investigate the mechanisms of creativity both from the point of view of the artist, while creating a work, and from the point of view of the public, while observing the final result.

In this framework, BCI devices are particularly useful in research, either to register the response to visual and musical stimuli and recognize the emotions valence [14, 15], and to reveal the mechanisms of the visual creativity [16]. The objective of the researches, past and in fieri, is to evaluate the emotive and the cognitive response to stimuli, with the aim of understanding what are the mechanisms triggering creativity or characterizing the creative process (the insight). In some experiments the objectives is to evaluate the emotive and cognitive response to visual-perceptive stimuli [17, 18] based on the concept of priming [19]. Other studies, investigate the mechanisms of response to colours [20, 21], or to stereoscopy and monoscopy [22]. The obtained results show interesting correspondences among some cerebral rhythms and the creative activity.

Currently, the research focuses on the comprehension of the cognitive mechanisms at the basis of creativity, of emotional intelligence and expression, but the technological cognitive tools at disposition today show, evidently, an enormous potentiality, in giving the possibility to verify, as Vygotskij stated, how the human ontogenesis is strongly influenced by cultural (technological, in our Era) tools at disposition in the historical and social context.

#### TECHNOLOGICAL EXPRESSION OF CREATIVITY. AN EXAMPLE.

"Art does not consist in representing new things, but in representing them in a new way" ("L'arte non consiste nel rappresentare cose nuove, bensì nel rappresentare con novità."), Ugo Foscolo

Every creative act starts, generally, in finding a new, creative answer to daily problems. This process allow to think "out of the box" to find a solution, the so called "problem solving" process, strongly related to problem solving. The term problem solving intend, in fact, the activity performed by a living organism or by an artificial intelligence device to achieve a status starting from one or more given condition. It is, so, the set of processes allowing to analyse, organize and combine information to solve given problematic situations. In its work The Act of Creation [23], Arthur Koestler explain creativity through the bi-section mechanism, that is "means to join unrelated, often conflicting, information in a new way". In fact, if in our daily life we associate elements belonging to the same reference system (book-sheet, cooking-food, etc.), in the artistic, humoristic or scientific creation we realize a connection among heterogeneous reference systems, usually considered incompatible. Problem solving and creativity are, so, correlated cognitive processes and can be

analysed through Brain Imaging methods, and it is evident that we can measure, also quantitatively, the activation level of a brain, analysing the electric signal produced.

With the aim to show how creativity is related to brain activation, we wish to introduce "Mind the Chair" (or "La sedia del Pensatore"), an interactive installation focused on a real time audio/visual representation of the activity of the brain, providing the possibility to control it. The performance allows to visualize the level of concentration of an individual, through an audio/visual answer (LED lights and sound generation/manipulation). While using a BCI headset, a person is able to understand how 'control' own brain activity and therefore to increment the level of concentration, modifying the intensity of the light or filtering the associated sounds. The interactive performance aims to give awareness about how our brain works and what 'concentration' means, therefore how much brain energy is required to perform a specific task (for example, to increase the lighting), and how it can be difficult to keep our brain working for a certain amount of time.

Figure 1: "Mind the chair", Francesco Soave. The performance installation in action, as exhibited in the artistic event " Terni Festival", 18-27 Sept 2015, Terni, Italy.

The concept comes from the idea of "the chair of the thinker' where a person sits on a chair, and its own thought starting from the mind, flows through its body, along the chair and then in the surrounding space as light.

During the performance the visitors are not given physical objects to concentrate on. They are only asked to 'concentrate' in terms of 'focusing' on something close to their attitudes. The visitor is required to not distract too much from feeling his/her brain activity, or the purpose of the installation would fail. During the performance, the word 'concentration' is, then, used as the simplest way to describe the aim of the interactive installation that is testing the power of the brain. For instance, many visitors tried to relax doing some yoga exercises, but even if they were expecting a deep fall of the lights, this didn't always happen as, in fact, whatever action you perform requires brain activity, even the act of relaxing and 'freeing' the mind.

The ability to control the brain rhythms and the concentration level, depends, of course, by the experience of an individual: solving a simple equation can be an easy task for a mathematician, but can be a really difficult task for a non mathematical mind. To make the installation to react is, then, necessary to find a specific, personal task that makes our brain work, which, of course, can be different for different people. Also, the ability to control the brain changes depending on the psychophysical status of the person: if the user is really tired, he/she will experience difficulties in controlling the performance. Interesting enough, when the brain activity is peaking, individuals show more difficulties in loosing focus and slowing down the level than actually increase it.

To realize the project a Neurosky Mindwave BCI interface and specific Java code were used to detect the brain activity through the analysis of the collected brain waves. In detail, as the installation involves concentration, Beta rhythms play a key role in identifying the level of brain activity. To make the representation consistent with the real brain activity, the interactive installation does not have a 0 level (in terms of 'lights off'); this is because our brain is working at all times and what makes the difference is the amount of work required to perform a task. Then, when the level of concentration is close to zero, a random blinking will appear, to represent the floor noise which is, in fact, a random signal existing in every kind of impulse and therefore even in the electromagnetic brain rhythms.

#### CONCLUSIONS

Neuro and Cognitive Sciences study creativity and artistic inspiration to discover their cognitive process, providing, at the same time, thanks to cognitive technology, the means for new artistic expressions, allowing researchers to investigate how the brain creates and understand Art. In cognitive science, the functioning of the brain and the achievements of Art are considered together to explain our aesthetic experience. The application of the Cognitive Science approach to Art, entertainment and educational fields also represents a promising approach. In fact, the importance of a BCI-based performance like "Mind the chair" is not limited in creativity studies. We argue that the use of BCIs and similar tools, such as, for example, eye trackers, should also be useful within a pedagogical program in order to contribute improving users' ability to focus attention on abstract cognitive tasks and in translating abstract thinking in concrete operations. Many people find great difficulties in approaching similar tasks, since their attention is easily grabbed by environmental distractions and so they fail in getting important educational achievements. Furthermore, the ability to focus and maintaining attention on mental tasks may contribute to the development of multitasking abilities, now considered particularly important in different contexts. Normally, expressive arts techniques are useful in helping people to increase attention skills, but they are limited by individuals' technical competencies (e.g. drawing) or by personality traits. BCI-based tools, such as BrainArt [16], or BCI-based performances, may instead be enjoyed by everyone, without any prerequisite both in stand-alone and in group settings. Future research will test if BCIs are concretely able to increase attention-based skills and creativity, potentially increasing the penetration of art within the realm of the neuro-cognitive approach to individual empowerment.

#### References

[1] J. P. Changeux, "Art and neuroscience". Leonardo, 189-201, 1994.

[2] P.H.Lindsay and D.A. Norman, "Human Information Processing", New York, Academic Press, 1971; trad. it .: "L'uomo elaboratore di informazioni", Firenze, Giunti Barbèra, 1983.

[3] L. S. Vygotskij, "Psicologija iskusstva", 1925; trad. it. 1972, "Psicologia dell'arte", Roma, Editori Riuniti

[4] L. S. Vygotskij, (1930), "Voobraženie i torčestvo v detskom vozraste"; trad. it., "Immaginazione e creatività nell'età infantile", Roma, Editori Riuniti, 1972

[5] J. Piaget, "Structuralism.", 1970).

[6] J. Piaget, "Science of education and the psychology of the child. Trans. D. Coltman.", 1970.


## DISABILITY & DIGITAL DISCUSSION: GLOBAL, EU & ITALIAN DEVELOPMENTS IN THE NEW MILLENNUM

J. Hemsley VASARI Research Centre, Birkbeck College University of London & EVA Conferences International England jrhemsley6(@gmail.com

Abstract - There is increasing global awareness - led by the United Nations [UN] - of the challenges and opportunities in applying Digital Technologies to the needs and talents of disabled and other disadvantaged people. This paper concentrates on major relevant UN reports and the proposed EU strategy up to 2020. The Italian situation is addressed by means of questions for discussion with participants. The paper aims at stimulating a dialogue to be continued at other EVA Conferences during 2016.

#### INTRODUCTION

The principal UN documents identified for this initial discussion paper are as follows:


In addition, two important recent EU documents were also selected:


A concluding Discussion includes intended next steps and some questions for discussion regarding the Italian situation,

#### THE BASIS: THE UNITED NATIONS UNIVERSAL DECLARATION OF HUMAN RIGHTS, UDHR (1948)

The Universal Declaration of Human Rights (UDHR, 1948,) clearly affirms 'cultural rights' in Article 22:

Everyone, as a member of society, has the right to social security and is entitled to realization, through national effort and international co-operation and in accordance with the organization and resources of each State, of the economic, social and cultural rights indispensable for his dignity and the free development of his personality.

Furthermore, Article 27 states:


This influential document served to inspire the next milestone document.

## UN CONVENTION ON THE RIGHTS OF PERSONS WITH DISABILITIES [CPRD] HIGHLIGHTING TECHNOLOGY'S ROLE

A subsequent major step by the United Nations built on the UDHR and UN Convention on the Rights of Persons with Disabilities [UN CPRD] 2006.

This crucial document heralded, signposted and legitimised a reinvigorated global campaign for Disabled People. Article 9 including substantial attention to Technology and, in particular, Accessibility. It has the following key aims (author underlining):

1. To enable persons with disabilities to live independently and participate fully in all aspects of life, States Parties shall take appropriate measures to ensure to persons with disabilities access, on an equal basis with others, to the physical environment, to transportation, to information and communications, including information and communications technologies and systems, and to other facilities and services open or provided to the public, both in urban and in rural areas. These measures, which shall include the identification and elimination of obstacles and barriers to accessibility, shall apply to, inter alia:


This point thus clearly asserts the equivalence in importance of ICTS to Physical Facilities and sets out the UN injunction to member states to 'take appropriate measures' as described a few lines later by two specific types addressing the technology focus of this paper:


This seminal Convention stimulated states and UN Agencies to pursue this subject. http://www.un.org/disabilities/convention/conventionfull.shtml [2].

#### WORLD REPORT ON DISABILITY (World Health Organization & World Bank)

This substantial report [325 pages] provides a comprehensive picture of the situation of the world's health at the end of the first decade of this century. Technology has a clear role including attention to 'Increasing the use and affordability of technology'. Recommendations are made to reduce the barriers to information and communication cover: 'Legislation and legal action; Standards; Policy and programmes; Procurement; Universal design; Action by industry and the Role of nongovernmental organizations' .

Example recommendations include: Accessible information and communication: Consider a range of bottom-up and top-down legislative and policy mechanisms including: consumer protection, non-discrimination legislation covering information and communication technologies and direct obligations on those developing ICT systems, products, and services. In the public and private sector adopt policies on procurement which take into consideration accessibility criteria. Support the development of telephone relay, sign language, and Braille services. When designing and distributing ICT equipment and services, developers should ensure that people with disabilities gain the same benefits as the wider population. Producers and providers should incorporate accessibility features in the products and services they design and sell. Support the education and training of persons with disabilities to take advantage of ICT - including training to ensure digital literacy and skills.

http://www.who.int/disabilities/world\_report/2011/report.pdf\_[3]

#### UNESCO GLOBAL REPORT: OPENING NEW AVENUES FOR EMPOWERMENT. ICTs TO ACCESS INFORMATION AND KNOWLEDGE FOR PERSONS WITH DISABILITIES, PWD 2013.

In 2013 UNESCO published this major report [Principal author, Michael Blakemore] covering the entire world but based in particular on commissioned regional studies as follows: (1) Africa; (2) Arab Region & North Africa; (3) Asia Pacific; (4) Eastern Europe & Central Asia; (5) [South America, Central America, Mexico] Latin America & the Caribbean. Each study was carried out with assistance from institutions and coordinating authors, generally from the region. This report was aimed at the following reader groups:

The report is targeted specifically towards: Policy and decision makers; Local, regional and national educational institutions; Teachers, education and information professionals; Associations and organizations working with persons with disabilities; ICT professionals and industry; and Others interested in ICTs and social inclusion of persons with disabilities.

Irina Bokova, Director-General of UNESCO, wrote in the Preface:

· Over one billion people – approximately 15 percent of the world's population – live with some form of disability. Facing a wide range of barriers, including access to information, education, health care and a lack of job opportunities, persons living with disabilities struggle every day to be integrated into society'

However, a note of caution was added in the report regarding difficult issues of definition, measurement and statistical problems:

The definition of disability is complex and evolving ---- the recognition of a disability varies across countries. Interventions require a sensitive understanding of the disabilities and a systemic approach to providing ICTs in the education context.

However, there can be no doubt that Disability is a major global issue.

#### EUROPEAN UNION POLICY INITIATIVES

Early concerted [pan] European activities in the field began in 1991 with the commencement of the EC's first TIDE Programme [Pilot Action: TIDE 0 - Technology initiative for Disabled and Elderly people] followed by its successors, aiming to build on and integrate previous and ongoing national and regional initiatives. These efforts were given extra emphasis in 2003 the European Year of The Disabled - to raise awareness and stimulate further attention including contributing to international efforts.

In 2010 the European Commission produced an ambitious strategy paper in European Disability Strategy 2010-2020: A Renewed Commitment to a Barrier-Free Europe 2010 [5] . The short { { 1 pages] forceful document begins with the following clear statement in the Introduction

'One in six people in the European Union (EU) has a disability that ranges from mild to severe making around 80 million who are often prevented from taking part fully in society and the economy because of environmental and attitudinal barriers. For people with disabilities the rate of poverty is 70% higher than the average partly due to limited access to employment.

This population number of 80 million means that the European disabled - corresponded numerically in 2010 to a country the size of Germany (although caution with regard to the statistics as indicated above) . Moreover, it continues:

> Over a third of people aged over 75 have disabilities that restrict them to some extent, and over 20% are considerably restricted 3. Furthermore, these numbers are set to rise as the EU's population ages.'

#9-!&;2+=-/5!)+DD3,,3+5!34-56303-4!-3C96!D/35!/2-/,!0+2!/863+5K!69-!032,6!+0!69-,-!</,! %88-,,3M3.36E!4-035-4!/,!0+..+<,K!

*'Accessibility' is defined as meaning that people with disabilities have access, on an equal basis with others, to the physical environment, transportation, information and communications technologies and systems (ICT), and other facilities and services. There are still major barriers in all of these areas. For example, on average in the EU-27, only 5% of public websites comply fully with web accessibility standards, though more are partially accessible. Many television broadcasters still provide few subtitled and audio-described programmes'.* 

%06-2!693,!=2+D3,35C!,6-=B!69-!=2+,=-86!</,!2/3,-4!+0!/!8+22-,=+5435C!=2+=+,/.!0+2!/! ]32-863@-\_&;2+=-/5!F/<!6+!M-!=2-=/2-4!35!STUSH!#93,!=2+=+,/. </,!2-.-/,-4!35!]-8-DM-2!STUVH! 966=K\_\_-8H-;2+=/H-;\_,+83/.\_D/35HA,=./5C(4|-5\8/6(4|ln\5-<,(4|S>TTH!! #9-!=2+=+,/.!,-6,!+;6!69-!42/06!.-C/.!4+8;D-56!0+2!/==2+@/.H!I5-!,623N35C!03C;2-!3,!69/6!ME!STST! 69-!5;DM-2!+0!43,/M.-4!=-+=.-!35!69-!&\*!3,!=2+A-86-4!6+!M-!UST!D3..3+5!=-+=.-!<369!/!,3C53038/56! D/2N-6!@/.;-!0+2!,=-83/.!C++4,!/54!,-2@38-,!M;6!02/CD-56-4!M-6<--5!69-!&\*X,!Sl!8+;5623-,!4;-! 35!8+5,34-2/M.-!=/26!6+!4300-235C!5/63+5/.!/54!2-C3+5/.!./<,!/54!=2/8638-,!#9;,B!*'another half of Germany's population'*!3,!-L=-86-4!6+!M-!/44-4!35!69-!8;22-56!4-8/4-H!&\*!,;==+26-4!\$!\!]! =2+C2/DD-,B!,;89!/,!69-!"-@-569!^2/D-<+2N!12+C2/DD-!O^1!`P!/54!8;22-56.E!bI\$(Ii! STSTB!!9/@-!/54!8+5635;-!6+!D/N-!,;M,6/563/.!2-.-@/56!8+5623M;63+5,!/,!4+!/.,+!D+2-!C-5-2/.! 35363/63@-,!/54!=+.383-,!/6!69-!&;2+=-/5B!5/63+5/.B!2-C3+5/.B!;2M/5!/54!2;2/.!.-@-.,H!!

### **%\$)%/3+(\$)+2!/(9(.-.(\$)+2!F3&+.(\$)+!j!)&I.!+.&"+!!**

123583=/.!8+58.;,3+5,!358.;4-!69-!0+..+<35CK!!

?!'.+M/.!/54!&;2+=-/5!5--4,!0+2!3D=2+@-4!/,,3,6/58-!0+2!69-!]3,/M.-4!/2-!93C9!/54!3582-/,35C! /,!69-32!=+=;./63+5!/C-,B!M;6!C.+M/.!/</2-5-,,!/54!D+D-56;D!/==-/2!6+!M-!3582-/,35CH

?!#9-!\*i!2-=+26,!=2+@34-!/!M/,3,!0+2!8+5,34-2/63+5!ME!/54!0+2!69-!%26,!\!b-236/C-!,-86+2! 2-C/2435C!355+@/63@-!8+D=;6-2!6-895+.+C3-,!0+2!69-!]3,/M.-4H!!!

?!);.6;2/.!35,636;63+5,!,;89!/,!D;,-;D,B!35!8++=-2/63+5!<369!+69-2!/86+2,B!8/5!D/N-!C2-/6-2! -00+26,!0+2!69-!]3,/M.-4!;,35C!(50+2D/63+5!#-895+.+CE!/,!69-!./66-2!2/8-,!/9-/4H!

#9-2-!/2-!-@34-56.E!,3C53038/56!C/=,!35!693,!35363/.!43,8;,,3+5!=/=-2B!0+2!-L/D=.-!(56-25/63+5/.! "6/54/24,!/54!69-!8+22-,=+5435C!M+43-,!,;89!/,!("I\_(&)!/54!69-!Z-M!%88-,,3M3.36E!(5363/63@-! OZ%(PH!#9-!\*i!2-=+26,!35438/6-4!/M+@-!9/@-!8+D=2-9-5,3@-!2-0-2-58-,!/54!,+;28-,!8+@-235CB! 356-2!/.3/B!,;89!+D3,,3+5,H!#9-2-!/2-!+69-2!3D=+26/56!\*i!2-=+26,B!0+2!-L/D=.-!02+D!\*i()&^H!!!

)/,-,!/54!-L/D=.-,!02+D!(6/.EB!<369!36,!-5+2D+;,!);.6;2/.!b-236/C-B!,9+;.4!+00-2!8+5,34-2/M.-! .-,,+5,!356-25/63+5/..EB!M;6!69-!<236-2X,!N5+<.-4C-!+0!69-!(6/.3/5!,8-5-!3,!.3D36-4B!,+!2-,=+5,-,! 6+!69-!0+..+<35C!6E=-,!+0!R;-,63+5,!<3..!M-!35@36-4!/6!&J%!^.+2-58-!STUVK!!


It is hoped to deepen and extend this initial discussion paper and pursue a dialogue on these issues later in 2015, including at EVA St Petersburg (June) and EVA London (July). Improvements are still needed - including in the 'developed countries' -- as shown for example by Lisney et al. [7]

#### References

[1] The Universal Declaration of Human Rights. United Nations General Assembly in Paris on 10 December 1948 General Assembly resolution 217 A. http://www.un.org/en/universaldeclaration-human-rights\_

[2] United Nations Convention on the Rights of Persons with Disabilities [UN CPRD] 2006. http://www.un.org/disabilities/convention/conventionfull.shtml.

[3] World Report on Disability, World Health Organization & World Bank , 2011 http://www.who.int/disabilities/world\_report/2011/report.pdf.

[4] UNESCO Global Report Opening New Avenues for Empowerment ICTs to Access Information and Knowledge for Persons with Disabilities, PWD, 2013. http://unesdoc.unesco.org/images/0021/002197/219767e.pdf

[5] European Disability Strategy 2010-2020: A Renewed Commitment to a Barrier-Free Europe 2010. European Commission. http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0636:FIN:EN:PDF

[6] Proposal for a DIRECTIVE OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on the approximation of the laws, regulations and administrative provisions of the Member States as regards the accessibility requirements for products and services (2/12/2015) http://ec.europa.eu/social/main.jsp?langId=en&catId=89&newsId=2400

[7] Lisney, Eleanor; Bowen, Jonathan P; Hearn, Kirsten and Zedda, Maria. Museums and Technology: Being Inclusive Helps Accessibility for All. CURATOR: THE MUSEUM JOURNAL. Volume 56, Issue 3, July 2013, P353-361.

## TOVIVA PROJECT: DOCUMENTING THE SPANISH DEFENSE TOWERS ALONG THE VALENCIAN COAST WITH A COMPREHENSIVE DIGITAL METHODOLOGY

#### A. Pablo Rodríguez-Navarro

Departamento de Expresión Gráfica Arquitectónica Univesitat Politècnica de Valèncian Valencia, Spain. rodriguez@upv.es B. Giorgio Verdiani Dipartimento di Architettura

Università degli Studi di Firenze Firenze, Italy.

giorgio.verdiani@unifi.it

C. Teresa Gil Piqueras

Departamento de Expresión Gráfica Arquitectónica Univesitat Politècnica de Valèncian Valencia, Spain. tgil@ega.upv.es

Abstract - The Spanish Mediterranean coasts, offers a suggestive sequence of towers, they were built, since the sixteenth century, to protect the territory from pirates. Once defeated the piracy of that time in the "Mare Nostrum", the coastal towers system lost its defensive function, but remained as a testimony of history. The Toviva project, supported by the Spanish Ministry and developed by a Spanish-Italian team, is aimed to the creation of a virtual network using advanced digital tools, where the towers become components for knowledge dissemination and a link between the cultural heritage of a territory and its history.

#### INTRODUCTION

Spain, in its Mediterranean coasts, offers a suggestive sequence of towers, this system is integrated and showed in the landscape for defence and surveillance purposes. It has represented an articulated military system since the XVI century, when there was the need to protect a territory otherwise at risk of being attacked by pirates. Once the pirates were defeated in the "Mare Nostrum", the coastal towers system lost its defensive function, but remained as a historical witness. Such a heritage can now benefit from a specific digital approach. In this, as under development in the research presented here it is important to analyze what are the final research needs, while they rise a list of products to be obtained and will result from the point of view of the graphical documentation, in a number of plans to scale, photogrammetries, 3D models, pictures and videos [1]. The first step is defining an appropriate methodology which may include traditional direct surveys, photogrammetric surveys (2D correction, 3D restitution), 3D laser scanning, UAV/drones aerial shooting, photographs, spherical panoramas, videos, etc ... The Toviva project, supported by the Spanish Ministry and developed by a Spanish-Italian team, is aimed to contribute in creating a virtual network of towers as an amplification of the real buildings, where the towers become component of dissemination of knowledge and represent a link between the cultural heritage of a territory and its history.

#### Historical notes

In 1453 Constantinople was conquered by Sultan Mehmed II. This event signed the beginning of the fight to control the Mediterranean Coast. The quantity of pirate attacks

augmented and this caused the conquest of Argel in 1516 by Arrouj Barbarrosa. During the XVI century pirate attacks increased and expanded around the coast causing sacking and robbery [2]. This century was also a period of confluence as never seen before along the Valencian Coast. A complex social-political context and the technological development produced the birth of a very articulated military architecture system in the territory. In 1557, Felipe II became King and began an ambitious project which was previously traced by the Duke of Maqueda, it was based on building a network of watchtowers, fortresses, walls and defences to protect the coast. Giovanni Battista Antonelli "the elder" was entrusted for the realization of this ambitious project. He was followed by other members of his own family. In his ideas about defence it is possible to see the comparing of the coast with a wall, where towns were the doors, the villages were the bastions and watchtowers were the battlements.

#### Specific features of the Valencian Coast

The Valencian coast is characterized by mixing rough areas with sandy lands and lightly slopes. The locations of every tower followed tactical criteria, getting the higher view of the coast and controlling landing for attack or obtaining provisions. However, the higher points of flat lands were used to get the capacity of viewing the sea and towards the horizon [3].

In 1554, "Las ordenanzas de la Guardia Marítima del reino de Valencia" was published to organize the defensive system of Valencian coast and to describe the army organization that every tower needed: Guards were responsible of looking after the coast and warning in case of danger, and "Atajadores" were responsible of communicating news between towers. This strategic network spread around all the Mediterranean area, especially and with clear architectural "system" in all the areas controlled or influenced by the Spanish Reign.

#### VALENCIAN TOWERS: TYPOLOGIES

The typologies of towers along the Valencian coast have gone developing in time and they have been adapted to new necessities and technological developments of attack solutions and strategies. From the architectural point of view, building typologies changed their morphology throughout XV and XVI centuries. It is possible to recognize 3 main stages [3]: Stage 1: From the end of the XV century to the beginning of the XVI century. Towers have prismatic shape with squared or circular base, the structure is very simple, answering to the need of visual control of the territory. Stage 2: From 1520 to 1560, the towers are prebastioned, they have singular elements like alambores and esperontes (sloping and screeding systems). Stage 3: XVII century, towers have important variations regarding previous phases. The use of conical shapes allows a higher resistance against artillery. The hexagonal shape appeared widening the eyesight and enhancing the offensive functions.

Towers were composed by three parts, foundation, base and main body. Each part had a structural and defensive function. The diameter of foundation generally was bigger than the tower base and it was adapted to the terrain. Base reaches a third of the total height and its function was to avoid the access and to reject artillery shots. The main part includes watching area while the interiors were built using vaulted system. The specific elements of these typologies were: an elevated access by a ladder, holes on the shaft, loopholes, machicolations. All these elements gave to the tower the necessary support to functions for defending and controlling the surrounding areas.

#### DIGITAL SURVEY AND 3D MODELS POST PROCESSING

The first objective of the Toviva project is to provide a methodology for the graphic documentation of coastal tower architectures, where the generation of proper metadata linked to 3D models is a task empowering the possibilities of interpretation and effective sharing.

The Toviva project proposes the use of advanced technical tools for graphical representation, employing a flexible methodology, trying to simplify processes and optimizing resources while maintaining the maximum reliability and quality of results. In addition, these same results can be adapted for cultural uses of a general public.

The first inspection is established after obtaining the following basic items: Location: town and access to the tower (conditions and issues). / Cartography of the area. / Ownership: public or private. Needed Permissions. / In case of building with a current occupation: contact managers; Tourist Office, dealership / Contact with local technicians and scholars: archaeologist, architect, engineer / Conditions for the access to the interior of the tower.

During the first inspection a set of photos and sketches is realized. The photographs are taken according to a specific procedure, enriched by notes and setting the Exif data in proper ways, so that later on it is easier to work on these archives. A specific record of the structure is prepared, with the following main information:


#### Ground and UAV Photogrammetry

After the remarkable evolution of the recent years, photogrammetry and UAV/Drone survey became a significant tool for architecture survey. For photogrammetry this "rebirth" is due to new algorithms that have facilitated the production of 3D models using S.f.M. (Structure from Motion) method [4]. When working with this kind of software, the workflow starts from the alignment of the pictures. The procedure is based on the use of each pixel of the image as a point, looking for homologous points in all the pictures and thereby the relative positions of each camera. The set of camera positions, along with the pictures themselves, are used in the following phase, with the construction of the model's geometry. In the end it is possible to create a photo-realistic texture for the final 3D model, ready to be exported to other software to be used for analysis, drawing production, multimedia.

Using UAV unit (drone) for taking aerial photographs solves many accessibility problems. Aerial photographs allow a great freedom in taking pictures from any point of view. I is possible to use the same camera for ground shooting or to prefer different cameras and then mixing the images with no significant variation in the workflow. But in general, with a double sequence of pictures, one from the ground, one from the air, it is possible to reduce the post processing needs. In example, none of the pictures taken from the ground need the masking of the sky process: the presence of aerial pictures clearly the borders of a tower, allowing a perfect processing without noisy artefacts in the skyline of the architecture.

#### 3D Laser Scanner

For complex structures, especially in presence of articulated internal/external parts, the use of 3D laser scanner technology is necessary. It solves the problem posed by photo-modeling for interiors, whether for lighting reasons or for a question of space and complexity. Scanning is much more laborious, both for data acquisition and subsequent processing (alignment of the point-clouds, mesh generation, surfaces treatments, texturing, etc ... ), but when correctly operated, it ensures excellent data acquisition under any circumstances, keeping the quality of all the data gathered with the same accuracy [4,5]. A specific topographic support is operated for the 3D Laser Scanner survey, it is limited to the targets applied this survey, reducing needs for overlapping scans and making easier and robust the process of alignment of isolated rooms and of the coverage/roof.

#### A Multimedia Catalogue

The Toviva project is based on the use of 3D photo-realistic models with the inclusion of their full data, including each model in its geographic location to facilitate their individuation and support the dissemination task. The opportunity of the project is to expand the visibility of these cultural assets along the Valencian coast, providing data to the general public through a "light" computing platforms such as Smart-Phones or PDAs (using APPs for virtual and augmented reality). The access will be through the Internet, using local WiFi network or the telephone data access. The strategies are based on the use of easily accesible solutions, like the Sketchfab (www.sketchfab.com) platform, while for more complex and site specific situation it is possible to foreseen more complex and articulated solutions, based on multimedia/programming software. The will is to create a versatile standard, capable to be easily replied in other experiences in the future, but with a solid and complete archive in its background. The catalogue, now open to the public in its online version (http://toviva.blogs.upv.es/) even if still in ongoing development, shows the system of defensive elements as a tool of knowledge and tourism guide for the Valencian Mediterranean coast, a solution usable by scholars, researchers, students, tourists, curious and any kind of interested user. The contents are organized around graphical elements like pictures and (when possible) digital survey derived representations. This system will connect to the visitor position useful information about the territory, the history of the tower and the system of the fortifications. At the same time, an annual conference, the Fortmed (www.fortmed.eu), helps in connecting scholars and researcher on this subject.

#### Conclusions

The watchtowers located in along the Mediterranean coast are historical witnesses of a defensive system from the XVI and XVII centuries. The goal to use them as a base to transmit knowledge through advanced data collection finds important solutions in the digital tools for survey and representation, thus, such an archive will be quite limited if not shared. And this can be supported by interactive schemes and 3D visualizations. Multimedia information will allow to enhance the awareness about the systems of towers, to better understand how each single tower is part of a system along the Mediterranean coast, created for defending against Barbary attacks but still capable to drive a network of knowledge between past and present. At the same time, an accurate reconstruction of buildings is a significant step in the understanding of the "evolution" of this Heritage. All the previous documents, collected and organized, are the base for a correct evaluation. Proper technologies allow to explain the defensive system along Mediterranean coast connecting new experiences to consolidated knowledge. Multimedia tools may help people to take a look in a distant past, not only to obtain an idea about tower systems with high historical and patrimonial interest but also to better understand the complex system of relationships spreading all across the Mediterranean.

Fig. 1 - Four samples of the towers documented by the Toviva Project (G. Verdiani)

Fig. 2 - Towers typologies in the set analysed by Toviva Project, (R.Atzeni, V. Naldini [6])

Fig. 3 - 3D Laser Scanner and Octacopter Drone at work (G. Verdiani, S. Giraudeau)

Fig. 4 - 3D model with texturing exported in sketchfab.com (P. Rodriguez-Navarro)

Fig. 5 - The Fortmed2016 and the Toviva Websites (P. Rodriguez-Navarro, G. Verdiani)

#### ACKNOWLEDGEMENTS

The Toviva project is a part of the R&D project entitled "Surveillance and Defense Towers of the Valencian Coast. Metadata generation and 3D models for interpretation and effective enhancement" reference HAR2013-41859-P. Principal investigator: P. Rodríguez-Navarro. The project is funded by the National Program for Fostering Excellence in Scientific and Technical Research, National Sub-Program for Knowledge Generation, Ministry of Economy and Competitiveness (Government of Spain). The project is developed by an International Team, mainly composed by members of the Universitad Politecnica de Valencia (UPV) and Dipartimento di Architettura di Firenze (DiDA). The research team: P. Rodríguez-Navarro, T. Gil Piqueras, F. Juan Vidal, A. Zaragozá Catalán, S. Varela Botella, A. Soler Estrela, S. Yudici Oliver, J. Luis Menéndez Fueyo, G. Verdiani, S. Lillo Giner, In collaboration with S. Iurilli, G. Guidi, M. Pucci, R. Atzeni, V.Naldimi, M. Mangani, P. Cabezos Bernal, S. Giraudeau, A. Leonardi, T. Pignatale, S. Columbu.

#### References

[1] P. Rodríguez-Navarro, G. Verdiani and T. Gil Piqueras, "Comprehensive methodology for Documenting the defence Towers of the Valencian Coast (Spain)", in Defensive Architecture of the Mediterranean. XV to XVIII Centuries, vol. 1, p.p. 321-328. Valencia: Universitat Politècnica de València, 2015.

[2] A. García and A. Martínez-Medina, "La frontera marítima fortificada: de la vigilancia a la señalización (XVI-XIX)", in Defensive Architecture of the Mediterranean. XV to XVIII Centuries, vol. 1, p.p. 61-68. Valencia: Universitat Politècnica de València, 2015.

[3] P. Poblet González, "Torres vigía y defensa del litoral Valenciano (VII): De la torre del Cap d'Or a la torre bombarda". Published final degree, directed by Pablo Rodríguez-Navarro. Valencia: Universitat Politècnica de València, 2013.

[4] P. RODRIGUEZ-NAVARRO, "Automated Digital photogrammetry versus the systems based on active 3D sensors", in Revista EGA, nº20, p.p. 100-111. Valencia: Universitat Politècnica de València, 2012.

[5] G. Verdiani, "Il ritorno all'immagine. Nuove procedure image based per il cultural Heritage", Lulu.com, 2011.

[6] R. Atzeni and V. Naldini, "Guardando il tempo da una torre sul mare". unpublished. Master Degree Thesis, directed by Giorgio Verdiani and Pablo Rodriguez-Navarro. Firenze: Dipartamento di Architectura. Universitá degli studi Firenze 2015.

[7] J.V. Boira, "Las torres del litoral valenciano. Valencia: Consellería de Infraestructuras y transporte", 2007.

## Rio's Digital Atelier Murat under Verrochio's Spell

#### Heitor Luiz Murat de Meirelles Quintella

Stratimidia's Murat MAGIC (Maison d'Art Galerie et Imaginarium Contemporain) Atelier Rio de Janeiro, Brasil heitor.quintella@gmail.com

#### Abstract

This paper presents a pioneering experiment in digital art carried out in Brazil, UK, USA since the years 1970. An account of the origins, development of the achievements of methods, disciples and partners in the advance of the use of technology in art and art historic legacy. An analysis of the overall action philosophy identifies the convergence with the renaissance ideals of Verrochio's Bottega. A summary of main works commissioned by corporate and collector clients is presented with considerations on future activities and overall market trends. The overall framework of the atelier Murat has been used as a pattern to implement similar organizations as a consulting service. Multiple Mecenas and Clients are exploring the possibility of placing large scale order and have them fulfilled in very short time and vey high quality. Among them Builders, Hotels, Hospitals, Universities and large corporations have already become regular partners of this promising experiment in ECOLOGICAL DIGITAL ART FOR CORPORATE INTERIOR DECORATION and can be visualized in the following blog:

http://artesvisuaisdemurat.blogspot.com.br/

### INDEX

#### STRATEGIC ISSUES


#### NEW SCIENCE AND CULTURE DEVELOPMENTS & APPLICATIONS



#### **NEW TECHNICAL DEVELOPMENTS & APPLICATIONS**


#### MUSEUMS - VIRTUAL GALLERIES AND RELATED INITIATIVES


#### ACCESS TO THE CULTURE INFORMATION

