#### Corrado Cuccurulloa , Luca D'Aniello<sup>a</sup> , Massimo Ariab , Maria Spanob **Measuring the impact of healthcare indicators on academic medical centers' scientific production**

**Measuring the impact of healthcare indicators on academic medical centers' scientific production**

<sup>a</sup> Department of Economics, University of Campania Luigi Vanvitelli, Caserta, Italy. <sup>b</sup> Department of Economics and Statistics, University of Naples Federico II, Naples, Italy. Corrado Cuccurullo, Luca D'Aniello, Massimo Aria, Maria Spano

### **1. Introduction**

The Academic Health Centers (AHCs), also known as Academic Health Science Centers (AHSCs) or Academic Medical Centers (AMCs) are hospitals where the activities of scientific research, teaching, and patients care are fully integrated. These complex institutions pursue a triple mission: research, teaching, and care, having an enormous impact on society and the nation's health.

Recently, policymakers and practitioners give more and more great importance to the AMCs' scientific activity for both welfare and Country competitiveness. However, there is no commonly agreed definition of AMCs because their structure and composition are different from the context in which an AMC is located. Indeed, some scholars comment *"when you have seen one Academic Health Centre, you've seen one Academic Health Centre"* (Sanfilippo, 2009). AMC structural and operational characteristics could affect their scientific production and impact. These factors are the scope of services, the location, the size, the market and so on.

Our study aims to investigate and determine which are the possible factors impacting the research productivity and impact of AMCs. We develop a model to assess the academic value of AMCs by considering these factors and how they are related to healthcare performance, measured in terms of scientific productivity, impact, and growth. We focus our research on Italian publicowned AMCs - that is 20 public AMCs as "Aziende Ospedaliere Universitarie", 9 public AMCs as "Ex Policlinici Universitari a gestione diretta", 23 public-owned "Istituti di Ricovero e Cura a Carattere Scientifico" (IRCCS) (Ministry of Health - *www.dati.salute.gov.it*). We retrieve structural information mainly from AMC websites and research data from bibliographic indexing databases (e.g. Web of Science, PubMed) in the period 2010-2019.

Our analysis is articulated in two steps. First, we identify different groups of AMCs by applying a Hierarchical Cluster Analysis (HCA). These groups share common structural and operational characteristics. Second also test the presence statistically significant differences in terms of research productivity and impact among the resulting groups through the Analysis of Variance (ANOVA). Any group is a peculiar AMC configuration.

This work has been partially financed by the research project "Leading Change in Academic Medical Centers", funded by the competitive call for projects V:ALERE 2019. The project aims to provide evidences, advices, and remarks to support System and AMC decision-makers to address the many challenges that AMC face.

147 Corrado Cuccurullo, University of Campania Luigi Vanvitelli, Italy, corrado.cuccurullo@unicampania.it, 0000-0002-7401-8575 Luca D'Aniello, University of Campania Luigi Vanvitelli, Italy, lucadaniello94@gmail.com, 0000-0003-1019-9212 Massimo Aria, University of Naples Federico II, Italy, massimo.aria@unina.it, 0000-0002-8517-9411 Maria Spano, University of Naples Federico II, Italy, maria.spano@unina.it, 0000-0002-3103-2342

FUP Best Practice in Scholarly Publishing (DOI 10.36253/fup\_best\_practice)

Corrado Cuccurullo, Luca D'Aniello, Massimo Aria, Maria Spano, *Measuring the impact of healthcare indicators on academic medical centers' scientific production*, pp. 161-165, © 2021 Author(s), CC BY 4.0 International, DOI 10.36253/978-88-5518-461- 8.31, in Bruno Bertaccini, Luigi Fabbris, Alessandra Petrucci (edited by), *ASA 2021 Statistics and Information Systems for Policy Evaluation. Book of short papers of the on-site conference*, © 2021 Author(s), content CC BY 4.0 International, metadata CC0 1.0 Universal, published by Firenze University Press (www.fupress.com), ISSN 2704-5846 (online), ISBN 978-88-5518-461-8 (PDF), DOI 10.36253/978-88-5518-461-8

# **2. Data and methodology**

As we said above, AMC could be very different in terms of different structural and operational characteristics. Aiming at covering these several aspects we collect data from different sources, from the official websites of AMCs, from official documents published by the Italian Ministry of Health, and on Google Maps. Table 1 provides an overview of the variables considered in our study. The table shows the variable synthetic label, how it was encoded, and the source of data. It is worth noting that some variables (e.g. *Type of AMC, Geographical localization*) do not change value during the years, but some others (e.g. *Structure Dimension*) have changed the value in the reference period 2010-2019.


**Table 1** Main structural and operational characteristics of Italian public-owned AMCs

We carried out a HCA to identify homogenous groups of AMCs by minimizing their distance within groups (clusters) and, at the same time, maximizing distance among groups. HCA is a multivariate technique that allows the visualization of the association structure among statistical observations at different levels of granularity.

We choose an agglomerative algorithm where each observation is initially considered as a singleelement cluster. At each step of the agglomerative procedure, the two clusters that are the most similar are combined into a new bigger cluster, using a specific linkage criterion. This procedure is iterated until all observations are in a single cluster. The different solutions are sequentially nested and displayed in a tree structure, known as a dendrogram. Here, we used the Ward linkage algorithm (Ward, 1963) with the Gower's distance (Gower, 1971), the most popular distance for mixed-type variables.

Regarding the research activity of AMCs, we retrieved on Web of Science (WoS) indexing database – launched by the Institute for Scientific Information (ISI) and now maintained by Clarivate Analytics – all the publications from January 2010 to December 2019. To identify the publications related to each AMC, we searched by full name affiliation (e.g. "IRCCS FND MILANO" for the Fondazione IRCCS Istituto Nazionale Tumori Milano, "IRCCS Ca Granda Ospedale Maggiore Policlinico" for the Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico). We limit our search by document type and selected only Articles, Proceedings Papers, Review Articles, and Book Chapters in the English language. The records were exported into PlainText format. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was used for the selection process of the publications (Liberati *et al.*, 2009). We used three bibliometric indicators to capture the different aspects of their research activity in terms of productivity (*n. of publications/total affiliated authors*), impact *(total citations/n. of publications*), and the annual percentage growth rate for article publication (*percentage growth rate 2010-2019*).

Analysis of Variance (ANOVA) (Jaccard *et al.*, 1984) and Tukey's Post-hoc test were used to inspect differences among the clusters resulting from the HCA.

## **3. Findings and conclusion**

HCA performed on the main characteristics of AMCs returned the dendrogram in Figure 1. We choose the solution into five clusters, highlighted in different color in the graphical representation. Interestingly, there is a natural separation among healthcare institutions with respect to the variable *Type of AMC*. For instance, IRCCS are almost all included in the Cluster 5 (orange) and in the Cluster 2 (green). They differ only with respect to the *SSR* and *PAR* variables, because the Cluster 5 includes all IRCCS subjected to both Regional Health system turnaround plans [*SSR=1*] and Hospital turnaround plans [*PAR=1*].

The Cluster 1 (blue) includes the 75% of AOU and a small portion of IRCCS (13%). All these AMCs are mainly characterized by a more articulated architectural structure [*LAYOUT=1*] and by the presence of an Emergency Departement [*ED=1*]. All of them are not subjected to both Regional Health system turnaround plans [*SSR=0*] and Hospital turnaround plans [*PAR=0*].

The remaining 25% of the AOUs fall within the Cluster 3 (red). They differ from the AOUs in the Cluster 1 because of their dimension. Indeed, these AMCs are all organized in a monoblok [*LAYOUT=0*] and therefore, they have on average a lower number of beds and wards. Finally, the Cluster 4 (lighblue) includes about the 80% of AOU\_SSN, all of them localized in metropolitan areas [*GEO\_LOC=1*], with an Emergency Departement [*ED=1*] and mainly organized in pavilions [*LAYOUT=1*].

**Figure 1** Dendogram resulting from HCA of Italian public-owned AMCs

Table 2 shows the results of a one-way ANOVA. We found a statistically-significant difference in average in our clusters by N. of publications per affiliated authors (F stat = 2.994, Pvalue = 0.0281\*) and by Total citations per N. of publications (F-stat = 4.523, P-value = 0.003\*\*) but not by Growth rate (F-stat = 0.307, P-value = 0.872).


**Table 2** Mean and standard deviation by clusters and ANOVA analysis among clusters.

We noted that AMCs in Cluster 2, Cluster 5 and Cluster 1 including all the IRCCSs and the 75% of AOUs are more productive than the others with an average value of N. of publications per affiliated authors greater than 2. This result is reflected also on the impact of their research with an average value of total citations per N. of publications greater than 22. From these preliminary results we could observe that the AMCs, where the research activity is regulated by strict guidelines (IRCCS) push these institutions to produce more and more with respect to AOUs and AOU\_SSN where more time is probably devoted to teaching and patient care.

## **References**

