Showing 1–50 of 128 results for author: D'Angelo, C A

The role of non-scientific factors vis-a-vis the quality of publications in determining their scholarly impact

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Leonardo Grilli

Abstract: In the evaluation of scientific publications' impact, the interplay between intrinsic quality and non-scientific factors remains a subject of debate. While peer review traditionally assesses quality, bibliometric techniques gauge scholarly impact. This study investigates the role of non-scientific attributes alongside quality scores from peer review in determining scholarly impact. Leveraging data… ▽ More In the evaluation of scientific publications' impact, the interplay between intrinsic quality and non-scientific factors remains a subject of debate. While peer review traditionally assesses quality, bibliometric techniques gauge scholarly impact. This study investigates the role of non-scientific attributes alongside quality scores from peer review in determining scholarly impact. Leveraging data from the first Italian Research Assessment Exercise (VTR 2001-2003) and Web of Science citations, we analyse the relationship between quality scores, non-scientific factors, and publication short- and long-term impact. Our findings shed light on the significance of non-scientific elements overlooked in peer review, offering policymakers and research management insights in choosing evaluation methodologies. Sections delve into the debate, identify non-scientific influences, detail methodologies, present results, and discuss implications. △ Less

Submitted 8 April, 2024; originally announced April 2024.

Analyzing the inter-domain vs intra-domain knowledge flows

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: Similar to how innovations often find success in fields other than their original domains, in this study we explore whether the same holds true for scientific discoveries. We investigate the flow of knowledge across scientific disciplines, focusing on connections between citing and cited publications. Specifically, we analyze the connections among cited publications from 2015 indexed in the Web of… ▽ More Similar to how innovations often find success in fields other than their original domains, in this study we explore whether the same holds true for scientific discoveries. We investigate the flow of knowledge across scientific disciplines, focusing on connections between citing and cited publications. Specifically, we analyze the connections among cited publications from 2015 indexed in the Web of Science and their citing counterparts to measure rates of knowledge dissemination within and across different fields. Our study aims to address key research questions concerning the disparities between inter- and intra-domain knowledge dissemination rates, the correlation between knowledge dissemination types and scholarly impact, as well as the evolution of knowledge dissemination patterns over time. These findings deepen our understanding of knowledge flows and offer practical insights with significant implications for evaluative bibliometrics. △ Less

Submitted 2 April, 2024; originally announced April 2024.

How reliable are unsupervised author disambiguation algorithms in the assessment of research organization performance?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: The paper examines extent of bias in the performance rankings of research organisations when the assessments are based on unsupervised author-name disambiguation algorithms. It compares the outcomes of a research performance evaluation exercise of Italian universities using the unsupervised approach by Caron and van Eck (2014) for derivation of the universities' research staff, with those of a ben… ▽ More The paper examines extent of bias in the performance rankings of research organisations when the assessments are based on unsupervised author-name disambiguation algorithms. It compares the outcomes of a research performance evaluation exercise of Italian universities using the unsupervised approach by Caron and van Eck (2014) for derivation of the universities' research staff, with those of a benchmark using the supervised algorithm of D'Angelo, Giuffrida, and Abramo (2011), which avails of input data. The methodology developed could be replicated for comparative analyses in other frameworks of national or international interest, meaning that practitioners would have a precise measure of the extent of distortions inherent in any evaluation exercises using unsupervised algorithms. This could in turn be useful in informing policy-makers' decisions on whether to invest in building national research staff databases, instead of settling for the unsupervised approaches with their measurement biases. △ Less

Submitted 7 September, 2022; originally announced October 2022.

A comparison of two approaches for measuring interdisciplinary research output: the disciplinary diversity of authors vs the disciplinary diversity of the reference list

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Lin Zhang

Abstract: This study investigates the convergence of two bibliometric approaches to the measurement of interdisciplinary research: one based on analyzing disciplinary diversity in the reference list of publications, the other based on the disciplinary diversity of authors of publications. In particular we measure the variety, balance, disparity and integrated diversity index of, respectively, single-author,… ▽ More This study investigates the convergence of two bibliometric approaches to the measurement of interdisciplinary research: one based on analyzing disciplinary diversity in the reference list of publications, the other based on the disciplinary diversity of authors of publications. In particular we measure the variety, balance, disparity and integrated diversity index of, respectively, single-author, multi-author single-field, and multi-author multi-field publications. We find that, in general, the diversity of the reference list grows with the number of fields reflected in a paper's authors' list and, to a lesser extent, with the number of authors being equal the number of fields. Further, we find that when fields belonging to different disciplines are reflected in the authors' list, the disparity in the reference list is higher than in the case of fields belonging to the same discipline. However, this general tendency varies across disciplines, and noticeable exceptions are found at individual paper level. △ Less

Submitted 27 March, 2021; originally announced March 2021.

Journal ref: Journal of Informetrics, 12(4), 2018, 1182-1193

Gender differences in research performance within and between countries: Italy vs Norway

Authors: Giovanni Abramo, Dag W. Aksnes, Ciriaco Andrea D'Angelo

Abstract: In this study, the scientific performance of Italian and Norwegian university professors is analysed using bibliometric indicators. The study is based on over 36,000 individuals and their publication output during the period 2011-2015. Applying a multidimensional indicator in which several aspects of the research performance are captured, we find large differences in the performance of men and wom… ▽ More In this study, the scientific performance of Italian and Norwegian university professors is analysed using bibliometric indicators. The study is based on over 36,000 individuals and their publication output during the period 2011-2015. Applying a multidimensional indicator in which several aspects of the research performance are captured, we find large differences in the performance of men and women. These gender differences are evident across all analysed levels, such as country, field, and academic position. However, most of the gender differences can be explained by the tails of the distributions-in particular, there is a much higher proportion of men among the top 10% performing scientists. For the remaining 90% of the population, the gender differences are practically non-existent. The results of the two countries, which differ in terms of the societal role of women, are contrasting. Further, we discuss possible biases that are intrinsic in quantitative performance indicators, which might disfavour female researchers. △ Less

Submitted 27 March, 2021; originally announced March 2021.

Journal ref: Journal of Informetrics, 15(2), 101144 (2021)

A novel methodology to assess the scientific standing of nations at field level

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: The formulation of national research policies would benefit greatly from reliable strategic analysis of the scientific infrastructure, aimed at identifying the relevant strengths and weaknesses at field level. Bibliometric methodologies thus far proposed in the literature are not completely satisfactory. This work proposes a novel "output-to-input-oriented" approach, which permits identification o… ▽ More The formulation of national research policies would benefit greatly from reliable strategic analysis of the scientific infrastructure, aimed at identifying the relevant strengths and weaknesses at field level. Bibliometric methodologies thus far proposed in the literature are not completely satisfactory. This work proposes a novel "output-to-input-oriented" approach, which permits identification of research strengths and weaknesses on the basis of the ratios of top scientists and highly cited articles to research expenditures in each field. The proposed approach is applied to the Italian academic system. 2012-2016 scientific publications are analyzed, in the 218 research fields where bibliometric assessment is appropriate. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Journal ref: AJournal of Informetrics, 14(1) 100986 (2020)

Collecting large-scale publication data at the level of individual researchers: A practical proposal for author name disambiguation

Authors: Ciriaco Andrea D'Angelo, Nees Jan van Eck

Abstract: The disambiguation of author names is an important and challenging task in bibliometrics. We propose an approach that relies on an external source of information for selecting and validating clusters of publications identified through an unsupervised author name disambiguation method. The application of the proposed approach to a random sample of Italian scholars shows encouraging results, with an… ▽ More The disambiguation of author names is an important and challenging task in bibliometrics. We propose an approach that relies on an external source of information for selecting and validating clusters of publications identified through an unsupervised author name disambiguation method. The application of the proposed approach to a random sample of Italian scholars shows encouraging results, with an overall precision, recall, and F-Measure of over 96%. The proposed approach can serve as a starting point for large-scale census of publication portfolios for bibliometric analyses at the level of individual researchers. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Journal ref: Scientometrics, 123(2), 883-907 (2020)

Does the geographic proximity effect on knowledge spillovers vary across research fields?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: Policy makers are interested in the influence of geographic distance on knowledge flows, however these can be expected to vary across research fields. The effects of geographic distance on flows are analyzed by means of citations to scientific literature. The field of observation consists of the 2010-2012 Italian publications and relevant citations up to the close of 2017. The geographic proximity… ▽ More Policy makers are interested in the influence of geographic distance on knowledge flows, however these can be expected to vary across research fields. The effects of geographic distance on flows are analyzed by means of citations to scientific literature. The field of observation consists of the 2010-2012 Italian publications and relevant citations up to the close of 2017. The geographic proximity effect is analyzed at national, continental, and intercontinental level in 244 fields, and results as evident at national level and in some cases at continental level, but not at intercontinental level. For flows between Italian municipalities, citations decrease with distance in all fields. At continental level, four fields are identified having knowledge flows that grow with distance; at intercontinental level, this occurs in 26 fields. The influence of distance is more limited in the fields of Humanities and Social sciences, much more significant in the Sciences, mainly in the Natural sciences. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Comments: arXiv admin note: text overlap with arXiv:2002.00742, arXiv:2103.13819

Journal ref: Scientometrics, 123(2), 1021-1036 (2020)

Predicting the future success of scientific publications through social network and semantic analysis

Authors: Andrea Fronzetti Colladon, Ciriaco Andrea D'Angelo, Peter A. Gloor

Abstract: Citations acknowledge the impact a scientific publication has on subsequent work. At the same time, deciding how and when to cite a paper, is also heavily influenced by social factors. In this work, we conduct an empirical analysis based on a dataset of 2010-2012 global publications in chemical engineering. We use social network analysis and text mining to measure publication attributes and unders… ▽ More Citations acknowledge the impact a scientific publication has on subsequent work. At the same time, deciding how and when to cite a paper, is also heavily influenced by social factors. In this work, we conduct an empirical analysis based on a dataset of 2010-2012 global publications in chemical engineering. We use social network analysis and text mining to measure publication attributes and understand which variables can better help predicting their future success. Controlling for intrinsic quality of a publication and for the number of authors in the byline, we are able to predict scholarly impact of a paper in terms of citations received six years after publication with almost 80 percent accuracy. Results suggest that, all other things being equal, it is better to co-publish with rotating co-authors and write the papers' abstract using more positive words, and a more complex, thus more informative, language. Publications that result from the collaboration of different social groups also attract more citations. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Journal ref: Scientometrics, 124(1), 357-377(2020)

Knowledge spillovers: does the geographic proximity effect decay over time? A discipline-level analysis, accounting for cognitive proximity, with and without self-citations

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: This work analyzes the variation over time of the effect of geographic distance on knowledge flows. The flows are measured through the citations exchanged between scientific publications, including and excluding self-citations. To calculate geographic distances between citing and cited publication, each publication is associated with a "prevailing" territory, according to the authors' affiliations… ▽ More This work analyzes the variation over time of the effect of geographic distance on knowledge flows. The flows are measured through the citations exchanged between scientific publications, including and excluding self-citations. To calculate geographic distances between citing and cited publication, each publication is associated with a "prevailing" territory, according to the authors' affiliations. We then apply a gravity model to account for the research size of the territories, in terms of cognitive proximity of citing-cited publications. The field of observation is the 2010-2017 world publications citing the 2010-2012 Italian publications, as indexed in the Web of Science. The results show that in domestic knowledge flows, geographic proximity remains an influential factor through time, although with differences among disciplines and trends of attenuating effects. Finally, we replicate the analyses of knowledge flows but with the exclusion of self-citations: in this manner the effect of geographic proximity seems reduced, particularly at the national scale, but the differences (with vs without self-citations) lessen through time. As shown in previous works, the effect of distance on continental flows is modest (imperceptible for intercontinental flows), yet here too time has some influence, including concerning exclusion of self-citations. △ Less

Submitted 25 March, 2021; originally announced March 2021.

Journal ref: Journal of Informetrics, 14(4), 2020, 101072

Informed peer review for publication assessments: Are improved impact measures worth the hassle?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Giovanni Felici

Abstract: In this work we ask whether and to what extent applying a predictor of publications' impact better than early citations, has an effect on the assessment of research performance of individual scientists. Specifically, we measure the total impact of Italian professors in the sciences and economics in a period of time, valuing their publications first by early citations and then by a weighted combina… ▽ More In this work we ask whether and to what extent applying a predictor of publications' impact better than early citations, has an effect on the assessment of research performance of individual scientists. Specifically, we measure the total impact of Italian professors in the sciences and economics in a period of time, valuing their publications first by early citations and then by a weighted combination of early citations and impact factor of the hosting journal. As expected, scores and ranks by the two indicators show a very strong correlation, but there occur also significant shifts in many fields, mainly in Economics and statistics, and Mathematics and computer science. The higher the share of uncited professors in a field and the shorter the citation time window, the more recommendable the recourse to the above combination. △ Less

Submitted 25 March, 2021; originally announced March 2021.

Journal ref: Quantitative Science Studies, 1(3), 1321-1333 (2020)

The domestic localization of knowledge flows as evidenced by publication citation: The case of Italy

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: This work applies a new approach to measure knowledge flows. Assuming that citation linkages between articles imply a flow of knowledge from the cited to the citing authors, we investigate the geographic flows of scientific knowledge produced in Italy across its regions, at both overall and field level. Furthermore, we measure the the specialization indexes for outflows and inflows of knowledge by… ▽ More This work applies a new approach to measure knowledge flows. Assuming that citation linkages between articles imply a flow of knowledge from the cited to the citing authors, we investigate the geographic flows of scientific knowledge produced in Italy across its regions, at both overall and field level. Furthermore, we measure the the specialization indexes for outflows and inflows of knowledge by a given region. Findings show that larger regions in terms of research output are more likely net exporters of new knowledge. At the same time, we register a positive correlation between the share of intraregional flows and the size of overall scientific output of a region. △ Less

Submitted 25 March, 2021; originally announced March 2021.

Comments: arXiv admin note: text overlap with arXiv:1909.08911

Journal ref: The case of Italy. Scientometrics, 125(2), 1305-1329

On the relation between the degree of internationalization of cited and citing publications: A field level analysis, including and excluding self-citations

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The growing complexity of scientific challenges demands increasingly intense research collaboration, both domestic and international. The resulting trend affects not only the modes of producing new knowledge, but also the way it is disseminated within scientific communities. This paper analyses the relationship between the "degree of internationalization" of a country's scientific production and t… ▽ More The growing complexity of scientific challenges demands increasingly intense research collaboration, both domestic and international. The resulting trend affects not only the modes of producing new knowledge, but also the way it is disseminated within scientific communities. This paper analyses the relationship between the "degree of internationalization" of a country's scientific production and that of the relevant citing publications. The empirical analysis is based on 2010-2012 Italian publications. Findings show: i) the probability of being cited increases with the degree of internationalization of the research team; ii) totally domestic research teams tend to cite to a greater extent totally domestic publications; iii) vice versa, publications resulting from international collaborations tend to be more cited by totally foreign publications rather than by publications including domestic authors. These results emerge both at overall and at discipline level. Findings might inform research policies geared towards internationalization. △ Less

Submitted 25 March, 2021; originally announced March 2021.

Journal ref: Journal of Informetrics, 2021, 15(1), 101101

Do the propensity and drivers of academics' engagement in research collaboration with industry vary over time?

Authors: Giovanni Abramo, Francesca Apponi, Ciriaco Andrea D'Angelo

Abstract: This study is about public-private research collaboration. In particular, we want to measure how the propensity of academics to collaborate with their colleagues from private firms varies over time and whether the typical profile of such academics change. Furthermore, we investigate the change of the weights of main drivers underlying the academics' propensity to collaborate with industry. In orde… ▽ More This study is about public-private research collaboration. In particular, we want to measure how the propensity of academics to collaborate with their colleagues from private firms varies over time and whether the typical profile of such academics change. Furthermore, we investigate the change of the weights of main drivers underlying the academics' propensity to collaborate with industry. In order to achieve such goals, we apply an inferential model on a dataset of professors working in Italian universities in two subsequent periods, 2010-2013 and 2014-2017. Results can be useful for supporting the definition of policies aimed at fostering public-private research collaborations, and should be taken into account when assessing their effectiveness afterwards. △ Less

Submitted 10 February, 2021; originally announced February 2021.

Gendered impact of COVID-19 pandemic on research production: a cross-country analysis

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Ida Mele

Abstract: The massive shock of the COVID-19 pandemic is already showing its negative effects on economies around the world, unprecedented in recent history. COVID-19 infections and containment measures have caused a general slowdown in research and new knowledge production. Because of the link between R&D spending and economic growth, it is to be expected then that a slowdown in research activities will slo… ▽ More The massive shock of the COVID-19 pandemic is already showing its negative effects on economies around the world, unprecedented in recent history. COVID-19 infections and containment measures have caused a general slowdown in research and new knowledge production. Because of the link between R&D spending and economic growth, it is to be expected then that a slowdown in research activities will slow in turn the global recovery from the pandemic. Many recent studies also claim an uneven impact on scientific production across gender. In this paper, we investigate the phenomenon across countries, analysing preprint depositions. Differently from other works, that compare the number of preprint depositions before and after the pandemic outbreak, we analyse the depositions trends across geographical areas, and contrast after-pandemic depositions with expected ones. Differently from common belief and initial evidence, in few countries female scientists increased their scientific output while males plunged. △ Less

Submitted 10 February, 2021; originally announced February 2021.

The effects of citation-based research evaluation schemes on self-citation behavior

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Leonardo Grilli

Abstract: We investigate the changes in the self-citation behavior of Italian professors following the introduction of a citation-based incentive scheme, for national accreditation to academic appointments. Previous contributions on self-citation behavior have either focused on small samples or relied on simple models, not controlling for all confounding factors. The present work adopts a complex statistics… ▽ More We investigate the changes in the self-citation behavior of Italian professors following the introduction of a citation-based incentive scheme, for national accreditation to academic appointments. Previous contributions on self-citation behavior have either focused on small samples or relied on simple models, not controlling for all confounding factors. The present work adopts a complex statistics model implemented on bibliometric individual data for over 15,000 Italian professors. Controlling for a number of covariates (number of citable papers published by the author; presence of international authors; number of co-authors; degree of the professor's specialization), the average increase in self-citation rates following introduction of the ASN is of 9.5%. The increase is common to all disciplines and academic ranks, albeit with diverse magnitude. Moreover, the increase is sensitive to the relative incentive, depending on the status of the scholar with respect to the scientific accreditation. A further analysis shows that there is much heterogeneity in the individual patterns of self-citing behavior, albeit with very few outliers. △ Less

Submitted 10 February, 2021; originally announced February 2021.

The different responses of universities to introduction of performance-based research funding

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: Governments and organizations design performance-based research funding systems (PBFRS) for strategic aims, such as to selectively allocate scarce resources and stimulate research efficiency. In this work we analyze the relative change in research productivity of Italian universities after the introduction of such a system, featuring financial and reputational incentives. Using a bibliometric appr… ▽ More Governments and organizations design performance-based research funding systems (PBFRS) for strategic aims, such as to selectively allocate scarce resources and stimulate research efficiency. In this work we analyze the relative change in research productivity of Italian universities after the introduction of such a system, featuring financial and reputational incentives. Using a bibliometric approach, we compare the relative research performance of universities before and after introduction of PBFRS, at the overall, discipline and field levels. The findings show convergence in the universities' performance, due above all to the remarkable improvement of the lowest performers. Geographically, the universities of the south (versus central and northern Italy) achieved the greatest improvement in relative performance. The methodology, and results, should be of use to university management and policy-makers. △ Less

Submitted 10 February, 2021; originally announced February 2021.

A bibliometric methodology to unveil territorial inequities in the scientific wealth to combat COVID-19

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: In this paper we develop a methodology to assess the scientific wealth of territories at field level. Our methodology uses a bibliometric approach based on the observation of academic research performance and overall scientific production in each territory. We apply it to assess disparities in the Italian territories in the medical specialties at the front line of the COVID-19 emergency. Italy has… ▽ More In this paper we develop a methodology to assess the scientific wealth of territories at field level. Our methodology uses a bibliometric approach based on the observation of academic research performance and overall scientific production in each territory. We apply it to assess disparities in the Italian territories in the medical specialties at the front line of the COVID-19 emergency. Italy has been the first among western countries to be severely affected by the onset of the COVID-19 pandemic. The study reveals remarkable inequities across territories, with scientific weaknesses concentrated in the south. Policies for rebalancing the north-south divide should also consider, in addition to tangible assets, the gap in production and availability of quality medical knowledge. △ Less

Submitted 16 November, 2020; originally announced November 2020.

Comments: 25 Pages, 5 Figures, 4 Tables

Unveiling the distinctive traits of a nation's research performance: the case of Italy and Norway

Authors: Giovanni Abramo, Dag W. Aksnes, Ciriaco Andrea D'Angelo

Abstract: In this study we are analysing the research performance of Italian and Norwegian professors using constituent components of the Fractional Scientific Strength (FSS) indicator. The main focus is on differences across fields in publication output and citation impact. The overall performance (FSS) of the two countries, which differ considerably in research size and profile, is remarkedly similar. How… ▽ More In this study we are analysing the research performance of Italian and Norwegian professors using constituent components of the Fractional Scientific Strength (FSS) indicator. The main focus is on differences across fields in publication output and citation impact. The overall performance (FSS) of the two countries, which differ considerably in research size and profile, is remarkedly similar. However, an in-depth analysis shows that there are large underlying performance differences. An average Italian professor publishes more papers than a Norwegian, while the citation impact of the research output is higher for the Norwegians. In addition, at field level the pattern varies along both dimensions, and we analyse in which fields each country have their relative strengths. Overall, this study contributes to further insights on how the research performance of different countries may be analysed and compared, to inform research policy. △ Less

Submitted 16 April, 2020; originally announced April 2020.

Comments: arXiv admin note: text overlap with arXiv:1907.02043

The role of geographic proximity in knowledge diffusion, measured by citations to scientific literature

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: This paper analyses the influence of geographic distance on knowledge flows, measured through citations to scientific publications. Previous works using the same approach are limited to single disciplines. In this study, we analyse the Italian scientific production in all disciplines matured in the period 2010-2012. To calculate the geographic distances between citing and cited publications, each… ▽ More This paper analyses the influence of geographic distance on knowledge flows, measured through citations to scientific publications. Previous works using the same approach are limited to single disciplines. In this study, we analyse the Italian scientific production in all disciplines matured in the period 2010-2012. To calculate the geographic distances between citing and cited publications, each one is associated with a "prevalent" territory on the basis of the authors' affiliations. The results of the application of a gravity model, estimated using ordinary least squares regression, show that despite the spread of IT, geographic distance continues to be an influential factor in the process of knowledge flows between territories. In particular, the analysis reveals that the effect of geographic distance on knowledge flows is significant at the national level, not negligible at the continental level, but completely irrelevant at the intercontinental level. △ Less

Submitted 3 February, 2020; originally announced February 2020.

The balance of knowledge flows

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Massimiliano Carloni

Abstract: In analogy to the technology balance of payments, in this paper we propose a possible way to set up a "balance of knowledge flows" (BKF), recording world flows of knowledge within the scientific community. Adopting a pure bibliometric approach, the "knowledge" traced in the BKF is that produced and exchanged by the scientific community by means of publications and relevant citations. A description… ▽ More In analogy to the technology balance of payments, in this paper we propose a possible way to set up a "balance of knowledge flows" (BKF), recording world flows of knowledge within the scientific community. Adopting a pure bibliometric approach, the "knowledge" traced in the BKF is that produced and exchanged by the scientific community by means of publications and relevant citations. A description of the theoretical foundation of such a tool is presented together with its empirical testing over the scientific production of four different countries. The BKF can be part of yearly reports of science and technology indicators, aimed at informing research policy. △ Less

Submitted 19 September, 2019; originally announced September 2019.

Journal ref: The balance of knowledge flows. Journal of Informetrics, 13(1), 1-9 (2019)

Predicting long-term publication impact through a combination of early citations and journal impact factor

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Giovanni Felici

Abstract: The ability to predict the long-term impact of a scientific article soon after its publication is of great value towards accurate assessment of research performance. In this work we test the hypothesis that good predictions of long-term citation counts can be obtained through a combination of a publication's early citations and the impact factor of the hosting journal. The test is performed on a c… ▽ More The ability to predict the long-term impact of a scientific article soon after its publication is of great value towards accurate assessment of research performance. In this work we test the hypothesis that good predictions of long-term citation counts can be obtained through a combination of a publication's early citations and the impact factor of the hosting journal. The test is performed on a corpus of 123,128 WoS publications authored by Italian scientists, using linear regression models. The average accuracy of the prediction is good for citation time windows above two years, decreases for lowly-cited publications, and varies across disciplines. As expected, the role of the impact factor in the combination becomes negligible after only two years from publication. △ Less

Submitted 19 September, 2019; originally announced September 2019.

Journal ref: Journal of Informetrics, 13(1), 32-49 (2019)

The collaboration behavior of top scientists

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The intention of this work is to analyze top scientists' collaboration behavior at the "international", "domestic extramural" and "intramural" levels, and compare it to that of their lesser performing colleagues. The field of observation consists of the entire faculty of the Italian academic system, and so the coauthorship of scientific publications by over 12,000 professors. The broader aim is to… ▽ More The intention of this work is to analyze top scientists' collaboration behavior at the "international", "domestic extramural" and "intramural" levels, and compare it to that of their lesser performing colleagues. The field of observation consists of the entire faculty of the Italian academic system, and so the coauthorship of scientific publications by over 12,000 professors. The broader aim is to improve understanding of the causal nexus between research collaboration and performance. The analysis is thus longitudinal, over two successive five-year periods. Results show a strong increase in the propensity to collaborate at domestic level (both extramural and intramural), however this is less for scientists who remain or become top, than it is for their lower-performing colleagues. In contrast, the increase in international collaboration behavior is greater for scientists who become or remain top than it is for their peers. The increase in productivity by those who acquire top scientist status is due precisely to the greater average impact of the publications achieved in collaboration with foreign colleagues. △ Less

Submitted 18 September, 2019; originally announced September 2019.

Comments: arXiv admin note: text overlap with arXiv:1906.00754

Journal ref: cientometrics, 118(1), 215-232 (2019)

Testing for universality of Mendeley readership distributions

Authors: Ciriaco Andrea D'Angelo, Samuele Di Russo

Abstract: Altmetrics promise useful support for assessing the impact of scientific works, including beyond the scholarly community and with very limited citation windows. Unfortunately, altmetrics scores are currently available only for recent articles and cannot be used as covariates in predicting long term impact of publications. However, the study of their statistical properties is a subject of evident i… ▽ More Altmetrics promise useful support for assessing the impact of scientific works, including beyond the scholarly community and with very limited citation windows. Unfortunately, altmetrics scores are currently available only for recent articles and cannot be used as covariates in predicting long term impact of publications. However, the study of their statistical properties is a subject of evident interest to scientometricians. Applying the same approaches used in the literature to assess the universality of citation distributions, the intention here is to test whether the universal distribution also holds for Mendeley readerships. Results of the analysis carried out on a sample of publications randomly extracted from the Web of Science confirm that readerships seem to share similar shapes across fields and can be rescaled to a common and universal form. Such rescaling results as not particularly effective on the right tails. In other regions, rescaling causes a good collapse of field specific distributions, even for very recent publications. △ Less

Submitted 18 September, 2019; originally announced September 2019.

Journal ref: Journal of Informetrics, 13(2), 726-737 (2019)

A nation's foreign and domestic professors: which have better research performance? (The Italian case)

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: This work investigates the research performance of foreign faculty in the Italian academic system. Incoming professors compose l'1% of total faculty across the sciences, although with variations by discipline. Their scientific performance measured over 2010-2014 is on average better than that of their Italian colleagues: the greatest difference is for associate professors. Psychology is the discip… ▽ More This work investigates the research performance of foreign faculty in the Italian academic system. Incoming professors compose l'1% of total faculty across the sciences, although with variations by discipline. Their scientific performance measured over 2010-2014 is on average better than that of their Italian colleagues: the greatest difference is for associate professors. Psychology is the discipline with the greatest concentration of top foreign scientists. However there are notable shares of unproductive foreign professors or of those with mediocre performance. The findings stimulate reflection on issues of national policy concerning attractiveness of the higher education system to skilled people from abroad, given the ongoing heavy Italian brain drain. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Comments: arXiv admin note: text overlap with arXiv:1907.02043, arXiv:1810.12207

Journal ref: Higher Education, 77(5), 917-930 (2019)

Peer review vs bibliometrics: which method better predicts the scholarly impact of publications?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Emanuela Reale

Abstract: In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an e… ▽ More In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an early citation-based indicator. We find that the latter shows stronger predictive power, i.e., it more reliably predicts late citations in all the disciplinary areas examined, and for any citation time window starting one year after publication. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Journal ref: Scientometrics, 121(1), 537-554 (2019)

Diversification versus specialization in scientific research: which strategy pays off?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The current work addresses a theme previously unexplored in the literature: that of whether the results arising from research activity in fields other than the scientist's pri-mary field have greater value than the others. Operationally, the authors proceed by identifying: the scientific production of each researcher under observation; field classifi-cation of the publications; the field containin… ▽ More The current work addresses a theme previously unexplored in the literature: that of whether the results arising from research activity in fields other than the scientist's pri-mary field have greater value than the others. Operationally, the authors proceed by identifying: the scientific production of each researcher under observation; field classifi-cation of the publications; the field containing the greatest number of the researcher's publications; attribution of value of each publication. The results show that diversifica-tion at the aggregate level does not pay off, although there are some exceptions at the level of individual disciplines. The implications at policy level are notable. Since the in-centive systems of research organizations are based on the impact of scientific output, the scientists concerned could resist engaging in multidisciplinary projects. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Journal ref: Technovation, 82-83, 51-57 (2019)

Comparison of research productivity of Italian and Norwegian professors and universities

Authors: Giovanni Abramo, Dag W. Aksnes, Ciriaco Andrea D'Angelo

Abstract: This is the first ever attempt of application in a country other than Italy of a research efficiency indicator (FSS), to assess and compare the performance of professors and universities, within and between countries. A special attention has been devoted to the presentation of the methodology developed to set up a common field classification scheme of professors, and to overcome the limited availa… ▽ More This is the first ever attempt of application in a country other than Italy of a research efficiency indicator (FSS), to assess and compare the performance of professors and universities, within and between countries. A special attention has been devoted to the presentation of the methodology developed to set up a common field classification scheme of professors, and to overcome the limited availability of comparable input data. Results of the comparison between countries, carried out in the 2011-2015 period, show similar average performances of professors, but noticeable differences in the distributions, whereby Norwegian professors are more concentrated in the tails. Norway shows notable higher performance in Mathematics and Earth and Space Sciences, while Italy in Biomedical Research and Engineering. △ Less

Submitted 3 July, 2019; originally announced July 2019.

A gender analysis of top scientists' collaboration behavior: evidence from Italy

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: This work analyzes the differences in collaboration behavior between males and females among a particular type of scholars: top scientists, and as compared to non top scientists. The field of observation consists of the Italian academic system and the co-authorships of scientific publications by 11,145 professors. The results obtained from a cross-sectional analysis covering the five-year period 2… ▽ More This work analyzes the differences in collaboration behavior between males and females among a particular type of scholars: top scientists, and as compared to non top scientists. The field of observation consists of the Italian academic system and the co-authorships of scientific publications by 11,145 professors. The results obtained from a cross-sectional analysis covering the five-year period 2006-2010 show that there are no significant differences in the overall propensity to collaborate in the top scientists of the two genders. At the level of single disciplines there are no differences in collaboration behavior, except in the case of: i) international collaborations, for Mathematics and Chemistry - where the propensity for collaboration is greater for males; and ii) extramural domestic collaborations in Physics, in which it is the females that show greater propensity for collaboration. Because international collaboration is positively correlated to research performance, findings can inform science policy aimed at increasing the representation of female top performers. △ Less

Submitted 3 June, 2019; originally announced June 2019.

Authorship analysis of specialized vs diversified research output

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The present work investigates the relations between amplitude and type of collaboration (intramural, extramural domestic or international) and output of specialized versus diversified research. By specialized or diversified research, we mean within or beyond the author's dominant research topic. The field of observation is the scientific production over five years from about 23,500 academics. The… ▽ More The present work investigates the relations between amplitude and type of collaboration (intramural, extramural domestic or international) and output of specialized versus diversified research. By specialized or diversified research, we mean within or beyond the author's dominant research topic. The field of observation is the scientific production over five years from about 23,500 academics. The analyses are conducted at the aggregate and disciplinary level. The results lead to the conclusion that in general, the output of diversified research is no more frequently the fruit of collaboration than is specialized research. At the level of the particular collaboration types, international collaborations weakly underlie the specialized kind of research output; on the contrary, extramural domestic and intramural collaborations are weakly associated with diversified research. While the weakness of association remains, exceptions are observed at the level of the individual disciplines. △ Less

Submitted 7 May, 2019; originally announced May 2019.

Journal ref: Journal of Informetrics, 13(2), 564-573 (2019)

Gender bias in academic recruitment

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Francesco Rosati

Abstract: It is well known that women are underrepresented in the academic systems of many countries. Gender discrimination is one of the factors that could contribute to this phenomenon. This study considers a recent national academic recruitment campaign in Italy, examining whether women are subject to more or less bias than men. The findings show that no gender-related differences occur among the candida… ▽ More It is well known that women are underrepresented in the academic systems of many countries. Gender discrimination is one of the factors that could contribute to this phenomenon. This study considers a recent national academic recruitment campaign in Italy, examining whether women are subject to more or less bias than men. The findings show that no gender-related differences occur among the candidates who benefit from positive bias, while among those candidates affected by negative bias, the incidence of women is lower than that of men. Among the factors that determine success in a competition for an academic position, the number of the applicant's career years in the same university as the committee members assumes greater weight for male candidates than for females. Being of the same gender as the committee president is also a factor that assumes greater weight for male applicants. On the other hand, for female applicants, the presence of a full professor in the same university with the same family name as the candidate assumes greater weight than for male candidates. △ Less

Submitted 21 December, 2018; originally announced January 2019.

Comments: arXiv admin note: text overlap with arXiv:1810.13236, arXiv:1810.12207

Journal ref: Scientometrics, 106(1), 119-141

Specialization vs diversification in research activities: the extent, intensity and relatedness of field diversification by individual scientists

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: We investigate whether and in what measure scientists tend to diversify their research activity, and if this tendency varies according to their belonging to different disciplinary areas. We analyze the nature of research diversification along three dimensions: extent of diversification, intensity of diversification, and degree of relatedness of topics in which researchers diversifies. For this pur… ▽ More We investigate whether and in what measure scientists tend to diversify their research activity, and if this tendency varies according to their belonging to different disciplinary areas. We analyze the nature of research diversification along three dimensions: extent of diversification, intensity of diversification, and degree of relatedness of topics in which researchers diversifies. For this purpose we propose three bibliometric indicators, based on the disciplinary placement of scientific output of individual scientists. The empirical investigation shows that the extent of diversification is lowest for scientists in Mathematics and highest in Chemistry; intensity of diversification is lowest in Earth sciences and highest in Industrial and information engineering; and degree of relatedness is lowest in Earth sciences and highest in Chemistry. △ Less

Submitted 21 December, 2018; originally announced December 2018.

Comments: arXiv admin note: text overlap with arXiv:1810.12633

Journal ref: Scientometrics, 112(3), 1403-1418

Assessing the accuracy of the h and g indexes for measuring researchers' productivity

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Fulvio Viel

Abstract: Bibliometric indicators are increasingly used in support of decisions for recruitment, career advancement, rewarding and selective funding for scientists. Given the importance of the applications, bibliometricians are obligated to carry out empirical testing of the robustness of the indicators, in simulations of real contexts. In this work we compare the results of national-scale research assessme… ▽ More Bibliometric indicators are increasingly used in support of decisions for recruitment, career advancement, rewarding and selective funding for scientists. Given the importance of the applications, bibliometricians are obligated to carry out empirical testing of the robustness of the indicators, in simulations of real contexts. In this work we compare the results of national-scale research assessments at the individual level, based on three different indexes: the h-index, g-index and "fractional scientific strength", or FSS, an indicator previously proposed by the authors. For each index, we construct and compare rankings lists of all Italian academic researchers working in the hard sciences over the period 2001-2005. The analysis quantifies the shifts in ranks that occur when researchers' productivity rankings by simple indicators such as h- or g-index are compared with that by more accurate FSS. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Comments: arXiv admin note: text overlap with arXiv:1810.12844

Journal ref: Abramo, G., D'Angelo, C.A., Viel, F. (2013). Assessing the accuracy of the h- and g-indexes for measuring researchers' productivity. Journal of the American Society for Information Science and Technology, 64(6), 1224-1234

Identifying interdisciplinarity through the disciplinary classification of co-authors of scientific publications

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The growing complexity of challenges involved in scientific progress demands ever more frequent application of competencies and knowledge from different scientific fields. The present work analyzes the degree of collaboration among scientists from different disciplines in order to identify the most frequent "combinations of knowledge" in research activity. The methodology adopts an innovative bibl… ▽ More The growing complexity of challenges involved in scientific progress demands ever more frequent application of competencies and knowledge from different scientific fields. The present work analyzes the degree of collaboration among scientists from different disciplines in order to identify the most frequent "combinations of knowledge" in research activity. The methodology adopts an innovative bibliometric approach based on the disciplinary affiliation of publication co-authors. The field of observation includes all publications (173,134) indexed in the Science Citation Index Expanded (SCI-E) for the five years 2004-2008, authored by all scientists in the hard sciences (43,223) at Italian universities (68). The analysis examines 205 research fields grouped in nine disciplines. Identifying the fields with the highest potential of interdisciplinary collaboration is useful to inform research polices at national and regional levels, as well as management strategies at the institutional level. △ Less

Submitted 21 December, 2018; originally announced December 2018.

Journal ref: JASIST, 63(11), 2206-2222

The alignment of public research supply and industry demand for effective technology transfer: the case of Italy

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: Italy lags quite behind vis-a'-vis other industrialized countries, in public to private technology transfer. One of the possible causes might be the mismatch between new knowledge supplied by public research and industry demand. We test this hypothesis through a survey of leading public research scientists in four high-tech sectors. The findings show that most research project results seem to be o… ▽ More Italy lags quite behind vis-a'-vis other industrialized countries, in public to private technology transfer. One of the possible causes might be the mismatch between new knowledge supplied by public research and industry demand. We test this hypothesis through a survey of leading public research scientists in four high-tech sectors. The findings show that most research project results seem to be of immediate industrial interest, which contrasts with the low patent and licensing performances of Italian public research institutions. For one third of all the results of the research, there are no Italian companies able to exploit them. The same, however, is not true for the remaining results, which shows that the misalignment between public supply and industry demand alone cannot account for poor technology transfer. What emerges from our investigation is that a closer coordination of research policy and industrial policy is required, as well as closer attention to initiatives which may support the transfer of public research results to domestic industry. △ Less

Submitted 21 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A. (2009). The alignment of public research supply and industry demand for effective technology transfer: the case of Italy. Science and Public Policy, 36(1), 2-14

Measuring science: irresistible temptations, easy shortcuts and dangerous consequences

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: In benchmarking international research, although publication and citation analyses should not be used to compare different disciplines, scientometrists frequently fail to resist the temptation to present rankings based on total publications and citations. Such measures are affected by significant distortions, due to the uneven fertility across scientific disciplines and the dishomogeneity of scien… ▽ More In benchmarking international research, although publication and citation analyses should not be used to compare different disciplines, scientometrists frequently fail to resist the temptation to present rankings based on total publications and citations. Such measures are affected by significant distortions, due to the uneven fertility across scientific disciplines and the dishomogeneity of scientific specialisation among nations and universities. In this paper, we provide an indication of the extent of the distortions when comparative bibliometric analyses fail to recognise the range of levels of scientific fertility, not only within a given major disciplinary area but also within the different scientific disciplines encompassed by the same area. △ Less

Submitted 21 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A. (2007). Measuring science: Irresistible temptations, easy shortcuts and dangerous consequences. Current Science, 93(6), 762-766

A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments

Authors: Ciriaco Andrea D'Angelo, Cristiano Giuffrida, Giovanni Abramo

Abstract: National exercises for the evaluation of research activity by universities are becoming regular practice in ever more countries. These exercises have mainly been conducted through the application of peer-review methods. Bibliometrics has not been able to offer a valid large-scale alternative because of almost overwhelming difficulties in identifying the true author of each publication. We will add… ▽ More National exercises for the evaluation of research activity by universities are becoming regular practice in ever more countries. These exercises have mainly been conducted through the application of peer-review methods. Bibliometrics has not been able to offer a valid large-scale alternative because of almost overwhelming difficulties in identifying the true author of each publication. We will address this problem by presenting a heuristic approach to author name disambiguation in bibliometric datasets for large-scale research assessments. The application proposed concerns the Italian university system, consisting of 80 universities and a research staff of over 60,000 scientists. The key advantage of the proposed approach is the ease of implementation. The algorithms are of practical application and have considerably better scalability and expandability properties than state-of-the-art unsupervised approaches. Moreover, the performance in terms of precision and recall, which can be further improved, seems thoroughly adequate for the typical needs of large-scale bibliometric research assessments. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Journal ref: D'Angelo, C.A., Giuffrida C., Abramo, G. (2011). A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments. JASIST, 62(2), 257-269

The collaboration behaviors of scientists in Italy: a field level analysis

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Gianluca Murgia

Abstract: The analysis of research collaboration by field is traditionally conducted beginning with the classification of the publications from the context of interest. In this work we propose an alternative approach based on the classification of the authors by field. The proposed method is more precise if the intended use is to provide a benchmark for the evaluation of individual propensity to collaborate… ▽ More The analysis of research collaboration by field is traditionally conducted beginning with the classification of the publications from the context of interest. In this work we propose an alternative approach based on the classification of the authors by field. The proposed method is more precise if the intended use is to provide a benchmark for the evaluation of individual propensity to collaborate. In the current study we apply the new methodology to all Italian university researchers in the hard sciences, measuring the propensity to collaborate for the various fields: in general, and specifically with intramural colleagues, extramural domestic and extramural foreign organizations. Using a simulation, we show that the results present substantial differences from those obtained through application of traditional approaches. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Murgia, G. (2013). The collaboration behaviors of scientists in Italy: a field level analysis. Journal of Informetrics, 7(2), 442-454

Individual research performance: a proposal for comparing apples to oranges

Authors: Giovanni Abramo, Tindaro Cicero, Ciriaco Andrea D'Angelo

Abstract: The evaluation of performance at the individual level is of fundamental importance in informing management decisions. The literature provides various indicators and types of measures, however a problem that is still unresolved and little addressed is how to compare the performance of researchers working in different fields (apples to oranges). In this work we propose a solution, testing various sc… ▽ More The evaluation of performance at the individual level is of fundamental importance in informing management decisions. The literature provides various indicators and types of measures, however a problem that is still unresolved and little addressed is how to compare the performance of researchers working in different fields (apples to oranges). In this work we propose a solution, testing various scaling factors for the distributions of research productivity in 174 scientific fields. The analysis is based on the observation of scientific production by all Italian university researchers active in the hard sciences over the period 2004-2008, as indexed by the Web of Science. The most effective scaling factor is the average of the productivity distribution of researchers with productivity above zero. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., Cicero, T., D'Angelo, C.A. (2013). Individual research performance: a proposal for comparing apples to oranges. Journal of Informetrics, 7(2), 528-539

A new bibliometric approach to assess the scientific specialization of regions

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The objective of the current work is to identify the territorial scientific specializations present in Italy, at the levels of regions and provinces (NUTS2 and NUTS3). To do this, we take a bibliometric approach based on the scientific production of the entire public research system in the hard sciences sphere, for the five years 2006-2010. In particular, we apply a new index of scientific special… ▽ More The objective of the current work is to identify the territorial scientific specializations present in Italy, at the levels of regions and provinces (NUTS2 and NUTS3). To do this, we take a bibliometric approach based on the scientific production of the entire public research system in the hard sciences sphere, for the five years 2006-2010. In particular, we apply a new index of scientific specialization (Scientific Specialization Index, SSI) that takes account of both the quantity and quality of scientific production achieved by the research institutions of a given territory. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Di Costa, F. (2014). A new bibliometric approach to assess the scientific specialization of regions. Research Evaluation, 23(2), 183-194

The measurement of italian universities' research productivity by a non parametric-bibliometric methodology

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Fabio Pugini

Abstract: This paper presents a methodology for measuring the technical efficiency of research activities. It is based on the application of data envelopment analysis to bibliometric data on the Italian university system. For that purpose, different input values (research personnel by level and extra funding) and output values (quantity, quality and level of contribution to actual scientific publications) a… ▽ More This paper presents a methodology for measuring the technical efficiency of research activities. It is based on the application of data envelopment analysis to bibliometric data on the Italian university system. For that purpose, different input values (research personnel by level and extra funding) and output values (quantity, quality and level of contribution to actual scientific publications) are considered. Our study aims at overcoming some of the limitations connected to the methodologies that have so far been proposed in the literature, in particular by surveying the scientific production of universities by authors' name. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Pugini, F. (2008). The measurement of Italian universities' research productivity by a non parametric-bibliometric methodology. Scientometrics, 76(2), 225-244

Gender differences in research productivity: a bibliometric analysis of the Italian academic system

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Alessandro Caprasecca

Abstract: The literature dedicated to analysis of the difference in research productivity between the sexes tends to agree in indicating better performance for men. This study enters in the vein of work on the subject. Through bibliometric examination of the entire population of research personnel working in the scientific-technological disciplines of Italian university system, it confirms the presence of s… ▽ More The literature dedicated to analysis of the difference in research productivity between the sexes tends to agree in indicating better performance for men. This study enters in the vein of work on the subject. Through bibliometric examination of the entire population of research personnel working in the scientific-technological disciplines of Italian university system, it confirms the presence of significant differences in productivity between men and women. But such differences result as being smaller than reported in a large part of the literature, confirming an ongoing tendency towards decline, and are also seen as more noticeable for quantitative performance indicators than other indicators. The gap between the sexes presents important sectorial specificities. In spite of the generally better performance of men, it can not be ignored that there are a significant number of scientific sectors in which the performance of women does not result as inferior. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Caprasecca, A. (2009). Gender differences in research productivity: a bibliometric analysis of the Italian academic system. Scientometrics, 79(3), 517-539

Research collaboration and productivity: is there correlation?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The incidence of extramural collaboration in academic research activities is increasing as a result of various factors. These factors include policy measures aimed at fostering partnership and networking among the various components of the research system, policies which are in turn justified by the idea that knowledge sharing could increase the effectiveness of the system. Over the last two decad… ▽ More The incidence of extramural collaboration in academic research activities is increasing as a result of various factors. These factors include policy measures aimed at fostering partnership and networking among the various components of the research system, policies which are in turn justified by the idea that knowledge sharing could increase the effectiveness of the system. Over the last two decades, the scientific community has also stepped up activities to assess the actual impact of collaboration intensity on the performance of research systems. This study draws on a number of empirical analyses, with the intention of measuring the effects of extramural collaboration on research performance and, indirectly, verifying the legitimacy of policies that support this type of collaboration. The analysis focuses on the Italian academic research system. The aim of the work is to assess the level of correlation, at institutional level, between scientific productivity and collaboration intensity as a whole, both internationally and with private organizations. This will be carried out using a bibliometric type of approach, which equates collaboration with the co-authorship of scientific publications. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Di Costa, F. (2009). Research collaboration and productivity: is there correlation? Higher Education, 57(2), 155-171

Assessing technical and cost efficiency of research activities: A case study of the Italian university system

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: This paper employs data envelopment analysis (DEA) to assess both technical and cost efficiency of research activities of the Italian university system. Differently from both peer review and the top-down discipline-invariant bibliographic approaches used elsewhere, a bottom-up bibliometric methodology is applied. Publications are assigned first to authors and then to one of nine scientific and tec… ▽ More This paper employs data envelopment analysis (DEA) to assess both technical and cost efficiency of research activities of the Italian university system. Differently from both peer review and the top-down discipline-invariant bibliographic approaches used elsewhere, a bottom-up bibliometric methodology is applied. Publications are assigned first to authors and then to one of nine scientific and technical university disciplinary areas. Inputs are specified in terms of the numbers of full, associate and assistant professors and outputs as the number of publications, contributions to publications and their scientific impact as variously measured across the disciplines included. DEA is undertaken cross-sectionally using the averages of these inputs and outputs over the period 2001-2003. The results typically show much variation in the rankings of the disciplinary areas within and across universities, depending on the efficiency indicator employed. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A. (2009). Assessing technical and cost efficiency of research activities: a case study of the Italian university system. Research Evaluation, 18(1), 61-70

Allocative efficiency in public research funding:can bibliometrics help?

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Alessandro Caprasecca

Abstract: The use of outcome control modes of research evaluation exercises is ever more frequent. They are conceived as tools to stimulate increased levels of research productivity, and to guide choices in allocating components of government research budgets for publicly funded institutions. There are several contributions in the literature that compare the different methodological approaches that policy m… ▽ More The use of outcome control modes of research evaluation exercises is ever more frequent. They are conceived as tools to stimulate increased levels of research productivity, and to guide choices in allocating components of government research budgets for publicly funded institutions. There are several contributions in the literature that compare the different methodological approaches that policy makers could adopt for these exercises, however the comparisons are limited to only a few disciplines. This work, examining the case of the whole of the "hard sciences" of the Italian academic system, makes a comparison between results obtained from peer review type of evaluations (as adopted by the Ministry of Universities and Research) and those possible from a bibliometric approach (as developed by the authors). The aim is to understand to what extent bibliometric methodology, which is noted as relatively inexpensive, time-saving and exhaustive, can complement and integrate peer review methodology in research evaluation. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Caprasecca, A. (2009). Allocative efficiency in public research funding: can bibliometrics help? Research Policy, 38(1), 206-215

Map** excellence in national research systems: the case of Italy

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: The study of the concept of "scientific excellence" and the methods for its measurement and evaluation is taking on increasing importance in the development of research policies in many nations. However, scientific excellence results as difficult to define in both conceptual and operational terms, because of its multi-dimensional and highly complex character. The literature on the theme is limited… ▽ More The study of the concept of "scientific excellence" and the methods for its measurement and evaluation is taking on increasing importance in the development of research policies in many nations. However, scientific excellence results as difficult to define in both conceptual and operational terms, because of its multi-dimensional and highly complex character. The literature on the theme is limited to few studies of an almost pioneering character. This work intends to contribute to the state of the art by exploring a bibliometric methodology which is effective, simple and inexpensive, and which further identifies "excellent" centers of research by beginning from excellence of the individual researchers affiliated with such centers. The study concentrates on the specific case of public research organizations in Italy, analyzing 109 scientific categories of research in the so called "hard" sciences and identifying 157 centers of excellence operating in 60 of these categories. The findings from this first application of the methodology should be considered exploratory and indicative. With a longer period of observation and the addition of further measurements, making the methodology more robust, it can be extended and adapted to a variety of national and supranational contexts, aiding with policy decisions at various levels. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Di Costa, F. (2009). Map** excellence in national research systems: the case of Italy. Evaluation Review, 33(2), 159-188

University-industry collaboration in Italy: a bibliometric examination

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa, Marco Solazzi

Abstract: This work investigates public-private research collaboration between Italian universities and domestic industry, applying a bibliometric type of approach. The study is based on an exhaustive listing of all co-authored publications in international journals that are jointly realized by Italian university scientists and researchers in the private sector: this listing permits the development of a nat… ▽ More This work investigates public-private research collaboration between Italian universities and domestic industry, applying a bibliometric type of approach. The study is based on an exhaustive listing of all co-authored publications in international journals that are jointly realized by Italian university scientists and researchers in the private sector: this listing permits the development of a national map** system for public-private collaboration, which results unique for its extensive and representative character. It is shown that, in absolute terms, most collaborations occur in medicine and chemistry, while it is industrial and information engineering which shows the highest percentage of co-authored articles out of all articles in the field. In addition, the investigation empirically examines and tests several hypotheses concerning the qualitative-quantitative impact of collaboration on the scientific production of individual university researchers. The analyses demonstrate that university researchers who collaborate with those in the private sector show research performance that is superior to that of colleagues who are not involved in such collaboration. But the impact factor of journals publishing academic articles co-authored by industry is generally lower than that concerning co-authorships with other entities. Finally, a further specific elaboration also reveals that publications with public-private co-authorship do not show a level of multidisciplinarity that is significantly different than that of other publications. △ Less

Submitted 21 November, 2018; originally announced November 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Di Costa, F., Solazzi, M. (2009). University-industry collaboration in Italy: a bibliometric examination. Technovation, 29(6-7), 498-507

The contribution of star scientists to overall sex differences in research productivity

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Alessandro Caprasecca

Abstract: The state of the art on the issue of sex differences in research efficiency agrees in recognizing higher performances for males, however there are divergences in explaining the possible causes. One of the causes advanced is that there are sex differences in the availability of aptitude at the "high end". By comparing sex differences in concentration and performance of Italian academic star scienti… ▽ More The state of the art on the issue of sex differences in research efficiency agrees in recognizing higher performances for males, however there are divergences in explaining the possible causes. One of the causes advanced is that there are sex differences in the availability of aptitude at the "high end". By comparing sex differences in concentration and performance of Italian academic star scientists to the case in the population complement, this work aims to verify if star, or "high-end", scientists play a preponderant role in determining higher performance among males. The study reveals the existence of a greater relative concentration of males among star scientists, as well as a performance gap between male and female star scientists that is greater than for the rest of the population. In the latter subpopulation the performance gap between the two sexs is seen as truly marginal. △ Less

Submitted 21 November, 2018; originally announced November 2018.

Journal ref: Abramo, G., D'Angelo, C.A., Caprasecca, A. (2009). The contribution of star scientists to overall sex differences in research productivity. Scientometrics, 81(1), 137-156

A Decision Support System for Public Research Organizations Participating in National Research Assessment Exercises

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo

Abstract: We are witnessing a rapid trend towards the adoption of exercises for evaluation of national research systems, generally based on the peer review approach. They respond to two main needs: stimulating higher efficiency in research activities by public laboratories, and realizing better allocative efficiency in government funding of such institutions. However the peer review approach is typified by… ▽ More We are witnessing a rapid trend towards the adoption of exercises for evaluation of national research systems, generally based on the peer review approach. They respond to two main needs: stimulating higher efficiency in research activities by public laboratories, and realizing better allocative efficiency in government funding of such institutions. However the peer review approach is typified by several limitations that raise doubts for the achievement of the ultimate objectives. In particular, subjectivity of judgment, which occurs during the step of selecting research outputs to be submitted for the evaluations, risks heavily distorting both the final ratings of the organizations evaluated and the ultimate funding they receive. These distortions become ever more relevant if the evaluation is limited to small samples of the scientific production of the research institutions. The objective of the current study is to propose a quantitative methodology based on bibliometric data that would provide a reliable support for the process of selecting the best products of a laboratory, and thus limit distortions. Benefits are twofold: single research institutions can maximize the probability of receiving a fair evaluation coherent with the real quality of their research. At the same time, broader adoptions of this approach could also provide strong advantages at the macroeconomic level, since it guarantees financial allocations based on the real value of the institutions under evaluation. In this study, the proposed methodology has been applied to the hard science sectors of the Italian university research system for the period 2004-2006. △ Less

Submitted 21 November, 2018; originally announced November 2018.

Journal ref: Abramo, G., D'Angelo, C.A. (2009). A decision support system for public research organizations participating in national research assessment exercises. JASIST, 60(10), 2095-2106

Testing the trade-off between productivity and quality in research activities

Authors: Giovanni Abramo, Ciriaco Andrea D'Angelo, Flavia Di Costa

Abstract: In recent years there has been an increasingly pressing need for the evaluation of results from public sector research activity, particularly to permit the efficient allocation of ever scarcer resources. Many of the studies and evaluation exercises that have been conducted at the national and international level emphasize the quality dimension of research output, while neglecting that of productiv… ▽ More In recent years there has been an increasingly pressing need for the evaluation of results from public sector research activity, particularly to permit the efficient allocation of ever scarcer resources. Many of the studies and evaluation exercises that have been conducted at the national and international level emphasize the quality dimension of research output, while neglecting that of productivity. This work is intended to test for the possible existence of correlation between quantity and quality of scientific production and determine whether the most productive researchers are also those that achieve results that are qualitatively better than those of their colleagues. The analysis proposed refers to the entire Italian university system and is based on the observation of production in the hard sciences by above 26,000 researchers in the period 2001 to 2005. The results show that the output of more productive researchers is superior in quality than that of less productive researchers. The relation between productivity and quality results as largely insensitive to the types of indicators or the test methods applied and also seems to differ little among the various disciplines examined. △ Less

Submitted 5 November, 2018; originally announced November 2018.

Journal ref: Journal of the American Society for Information Science and Technology, 61(1), 132-140 (2010)