Showing 1–50 of 288 results for author: Leydesdorff, L

A discussion of measuring the top-1 percent most-highly cited publications: Quality and impact of Chinese papers

Authors: Caroline S. Wagner, Lin Zhang, Loet Leydesdorff

Abstract: The top 1 percent most highly cited articles are watched closely as the vanguards of the sciences. Using Web of Science data, one can find that China had overtaken the USA in the relative participation in the top 1 percent in 2019, after outcompeting the EU on this indicator in 2015. However, this finding contrasts with repeated reports of Western agencies that the quality of Chinese output in sci… ▽ More The top 1 percent most highly cited articles are watched closely as the vanguards of the sciences. Using Web of Science data, one can find that China had overtaken the USA in the relative participation in the top 1 percent in 2019, after outcompeting the EU on this indicator in 2015. However, this finding contrasts with repeated reports of Western agencies that the quality of Chinese output in science is lagging other advanced nations, even as it has caught up in numbers of articles. The difference between the results presented here and the previous results depends mainly upon field normalizations, which classify source journals by discipline. Average citation rates of these subsets are commonly used as a baseline so that one can compare among disciplines. However, the expected value of the top 1 percent of a sample of N papers is N 100, ceteris paribus. Using the average citation rates as expected values, errors are introduced by using the mean of highly skewed distributions and a specious precision in the delineations of the subsets. Classifications can be used for the decomposition, but not for the normalization. When the data is thus decomposed, the USA ranks ahead of China in biomedical fields such as virology. Although the number of papers is smaller, China outperforms the US in the field of Business and Finance in the Social Sciences Citation Index when p is less than .05. Using percentile ranks, subsets other than indexing based classifications can be tested for the statistical significance of differences among them. △ Less

Submitted 1 February, 2022; originally announced February 2022.

Comments: 26 pages, 8 figures accepted for publication in Scientometrics

Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA

Authors: Loet Leydesdorff, Caroline S. Wagner, Lin Zhang

Abstract: Purpose: We address the question of whether differences are statistically significant in the rankings of universities. We propose methods measuring the statistical significance among different universities and illustrate the results by empirical data. Design/methodology/approach: Based on z-testing and overlap** confidence intervals, and using data about 205 Chinese universities included in the… ▽ More Purpose: We address the question of whether differences are statistically significant in the rankings of universities. We propose methods measuring the statistical significance among different universities and illustrate the results by empirical data. Design/methodology/approach: Based on z-testing and overlap** confidence intervals, and using data about 205 Chinese universities included in the Leiden Rankings 2020, we argue that three main groups of Chinese research universities can be distinguished. Findings: When the sample of 205 Chinese universities is merged with the 197 US universities included in Leiden Rankings 2020, the results similarly indicate three main groups: high, middle, low. Using this data (Leiden Rankings and Web-of-Science), the z-scores of the Chinese universities are significantly below those of the US universities albeit with some overlap. Research limitations: We show empirically that differences in ranking may be due to changes in the data, the models, or the modeling effects on the data. The scientometric grou**s are not always stable when we use different methods. R&D policy implications: Differences among universities can be tested for their statistical significance. The statistics relativize the values of decimals in the rankings. One can operate with a scheme of low/middle/high in policy debates and leave the more fine-grained rankings of individual universities to operational management and local settings. Originality/value: In the discussion about the rankings of universities, the question of whether differences are statistically significant, is, in our opinion, insufficiently addressed. △ Less

Submitted 17 November, 2020; originally announced November 2020.

Does the $h_α$ index reinforce the Matthew effect in science? Agent-based simulations using Stata and R

Authors: Lutz Bornmann, Christian Ganser, Alexander Tekles, Loet Leydesdorff

Abstract: Recently, Hirsch (2019a) proposed a new variant of the h index called the $h_α$ index. He formulated as follows: "we define the $h_α$ index of a scientist as the number of papers in the h-core of the scientist (i.e. the set of papers that contribute to the h-index of the scientist) where this scientist is the $α$-author" (p. 673). The $h_α$ index was criticized by Leydesdorff, Bornmann, and Opthof… ▽ More Recently, Hirsch (2019a) proposed a new variant of the h index called the $h_α$ index. He formulated as follows: "we define the $h_α$ index of a scientist as the number of papers in the h-core of the scientist (i.e. the set of papers that contribute to the h-index of the scientist) where this scientist is the $α$-author" (p. 673). The $h_α$ index was criticized by Leydesdorff, Bornmann, and Opthof (2019). One of their most important points is that the index reinforces the Matthew effect in science. We address this point in the current study using a recently developed Stata command (h_index) and R package (hindex), which can be used to simulate h index and $h_α$index applications in research evaluation. The user can investigate under which conditions $h_α$ reinforces the Matthew effect. The results of our study confirm what Leydesdorff et al. (2019) expected: the $h_α$ index reinforces the Matthew effect. This effect can be intensified if strategic behavior of the publishing scientists and cumulative advantage effects are additionally considered in the simulation. △ Less

Submitted 27 May, 2019; originally announced May 2019.

Within-Journal Self-citations and the Pinski-Narin Influence Weights

Authors: Gangan Prathap, Loet Leydesdorff

Abstract: The Journal Impact Factor (JIF) is linearly sensitive to self-citations because each self-citation adds to the numerator, whereas the denominator is not affected. Pinski & Narin (1976) derived the Influence Weight (IW) as an alternative to Garfield's JIF. Whereas the JIF is based on raw citation counts normalized by the number of publications, IWs are based on the eigenvectors in the matrix of agg… ▽ More The Journal Impact Factor (JIF) is linearly sensitive to self-citations because each self-citation adds to the numerator, whereas the denominator is not affected. Pinski & Narin (1976) derived the Influence Weight (IW) as an alternative to Garfield's JIF. Whereas the JIF is based on raw citation counts normalized by the number of publications, IWs are based on the eigenvectors in the matrix of aggregated journal-journal citations without a reference to size: the cited and citing sides are combined by a matrix approach. IWs emerge as a vector after recursive iteration of the normalized matrix. Before recursion, IW is a (vector-based) non-network indicator of impact, but after recursion (i.e. repeated improvement by iteration), IWs can be considered a network measure of prestige among the journals in the (sub)graph as a representation of a field of science. As a consequence (not intended by Pinski & Narin in 1976), the self-citations are integrated at the field level and no longer disturb the analysis as outliers. In our opinion, this is a very desirable property of a measure of quality or impact. As illustrations, we use data of journal citation matrices already studied in the literature, and also the complete set of data in the Journal Citation Reports 2017 (n = 11,579 journals). The values of IWs are sometimes counter-intuitive and difficult to interpret. Furthermore, iterations do not always converge. Routines for the computation of IWs are made available at http://www.leydesdorff.net/iw. △ Less

Submitted 15 August, 2019; v1 submitted 5 May, 2019; originally announced May 2019.

Which are the influential publications in the Web of Science subject categories over a long period of time? CRExplorer software used for big-data analyses in bibliometrics

Authors: Andreas Thor, Lutz Bornmann, Robin Haunschild, Loet Leydesdorff

Abstract: What are the landmark papers in scientific disciplines? On whose shoulders does research in these fields stand? Which papers are indispensable for scientific progress? These are typical questions which are not only of interest for researchers (who frequently know the answers - or guess to know them), but also for the interested general public. Citation counts can be used to identify very useful pa… ▽ More What are the landmark papers in scientific disciplines? On whose shoulders does research in these fields stand? Which papers are indispensable for scientific progress? These are typical questions which are not only of interest for researchers (who frequently know the answers - or guess to know them), but also for the interested general public. Citation counts can be used to identify very useful papers, since they reflect the wisdom of the crowd; in this case, the scientists using the published results for their own research. In this study, we identified with recently developed methods for the program CRExplorer landmark publications in nearly all Web of Science subject categories (WoSSCs). These are publications which belong more frequently than other publications across the citing years to the top-per mill in their subject category. The results for three subject categories "Information Science and Library Science", "Computer Science, Information Systems", and "Computer Science, Software Engineering" are exemplarily discussed in more detail. The results for the other WoSSCs can be found online at http://crexplorer.net. △ Less

Submitted 25 January, 2019; originally announced January 2019.

How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators

Authors: Lutz Bornmann, Alexander Tekles, Loet Leydesdorff

Abstract: Recently, the integrated impact indicator (I3) indicator was introduced where citations are weighted in accordance with the percentile rank class of each publication in a set of publications. I3 can also be used as a field-normalized indicator. Field-normalization is common practice in bibliometrics, especially when institutions and countries are compared. Publication and citation practices are so… ▽ More Recently, the integrated impact indicator (I3) indicator was introduced where citations are weighted in accordance with the percentile rank class of each publication in a set of publications. I3 can also be used as a field-normalized indicator. Field-normalization is common practice in bibliometrics, especially when institutions and countries are compared. Publication and citation practices are so different among fields that citation impact is normalized for cross-field comparisons. In this study, we test the ability of the indicator to discriminate between quality levels of papers as defined by Faculty members at F1000Prime. F1000Prime is a post-publication peer review system for assessing papers in the biomedical area. Thus, we test the convergent validity of I3 (in this study, we test I3/N - the size-independent variant of I3 where I3 is divided by the number of papers) using assessments by peers as baseline and compare its validity with several other (field-normalized) indicators: the mean-normalized citation score (MNCS), relative-citation ratio (RCR), citation score normalized by cited references (CSNCR), characteristic scores and scales (CSS), source-normalized citation score (SNCS), citation percentile, and proportion of papers which belong to the x% most frequently cited papers (PPtop x%). The results show that the PPtop 1% indicator discriminates best among different quality levels. I3 performs similar as (slightly better than) most of the other field-normalized indicators. Thus, the results point out that the indicator could be a valuable alternative to other indicators in bibliometrics. △ Less

Submitted 18 February, 2019; v1 submitted 4 January, 2019; originally announced January 2019.

The Integrated Impact Indicator (I3) Revisited: A Non-Parametric Alternative to the Journal Impact Factor

Authors: Loet Leydesdorff, Lutz Bornmann, Jonathan Adams

Abstract: We propose the I3* indicator as a non-parametric alternative to the Journal Impact Factor (JIF) and h-index. We apply I3* to more than 10,000 journals. The results can be compared with other journal metrics. I3* is a promising variant within the general scheme of non-parametric indicators I3 introduced previously: it provides a single metric which correlates with both impact in terms of citations… ▽ More We propose the I3* indicator as a non-parametric alternative to the Journal Impact Factor (JIF) and h-index. We apply I3* to more than 10,000 journals. The results can be compared with other journal metrics. I3* is a promising variant within the general scheme of non-parametric indicators I3 introduced previously: it provides a single metric which correlates with both impact in terms of citations (c) and output in terms of publications (p). We argue for weighting using four percentile classes: the top-1% and top-10% as excellence indicators; the top-50% and bottom-50% as output indicators. Like the h-index, which also incorporates both c and p, I3*-values are size-dependent; however, division of I3* by the number of publications (I3*/N) provides a size-independent indicator which correlates strongly with the two- and five-year Journal Impact Factors (JIF2 and JIF5). Unlike the h-index, I3* correlates significantly with both the total number of citations and publications. The values of I3* and I3*/N can be statistically tested against the expectation or against one another using chi-square tests or effect sizes. A template (in Excel) is provided online for relevant tests. △ Less

Submitted 17 March, 2019; v1 submitted 9 December, 2018; originally announced December 2018.

Does the public discuss other topics on climate change than researchers? A comparison of explorative networks based on author keywords and hashtags

Authors: Robin Haunschild, Loet Leydesdorff, Lutz Bornmann, Iina Hellsten, Werner Marx

Abstract: Twitter accounts have already been used in many scientometric studies, but the meaningfulness of the data for societal impact measurements in research evaluation has been questioned. Earlier research focused on social media counts and neglected the interactive nature of the data. We explore a new network approach based on Twitter data in which we compare author keywords to hashtags as indicators o… ▽ More Twitter accounts have already been used in many scientometric studies, but the meaningfulness of the data for societal impact measurements in research evaluation has been questioned. Earlier research focused on social media counts and neglected the interactive nature of the data. We explore a new network approach based on Twitter data in which we compare author keywords to hashtags as indicators of topics. We analyze the topics of tweeted publications and compare them with the topics of all publications (tweeted and not tweeted). Our exploratory study is based on a comprehensive publication set of climate change research. We are interested in whether Twitter data are able to reveal topics of public discussions which can be separated from research-focused topics. We find that the most tweeted topics regarding climate change research focus on the consequences of climate change for humans. Twitter users are interested in climate change publications which forecast effects of a changing climate on the environment and to adaptation, mitigation and management issues rather than in the methodology of climate-change research and causes of climate change. Our results indicate that publications using scientific jargon are less likely to be tweeted than publications using more general keywords. Twitter networks seem to be able to visualize public discussions about specific topics. △ Less

Submitted 12 March, 2019; v1 submitted 17 October, 2018; originally announced October 2018.

Comments: 31 pages, 1 table, and 7 figures

Journal ref: Jounral of Informetrics, 13, 695-707 (2019)

hα: The Scientist as Chimpanzee or Bonobo

Authors: Loet Leydesdorff, Lutz Bornmann, Tobias Opthof

Abstract: In a recent paper, Hirsch (2018) proposes to attribute the credit for a co-authored paper to the α-author--the author with the highest h-index--regardless of his or her actual contribution, effectively reducing the role of the other co-authors to zero. The indicator hα inherits most of the disadvantages of the h-index from which it is derived, but adds the normative element of reinforcing the Matt… ▽ More In a recent paper, Hirsch (2018) proposes to attribute the credit for a co-authored paper to the α-author--the author with the highest h-index--regardless of his or her actual contribution, effectively reducing the role of the other co-authors to zero. The indicator hα inherits most of the disadvantages of the h-index from which it is derived, but adds the normative element of reinforcing the Matthew effect in science. Using an example, we show that hα can be extremely unstable. The empirical attribution of credit among co-authors is not captured by abstract models such as h, h_bar , or hα. △ Less

Submitted 17 October, 2018; v1 submitted 16 October, 2018; originally announced October 2018.

Revisiting Relative Indicators and Provisional Truths

Authors: Loet Leydesdorff, Tobias Opthof

Abstract: Following discussions in 2010 and 2011, scientometric evaluators have increasingly abandoned relative indicators in favor of comparing observed with expected citation ratios. The latter method provides parameters with error values allowing for the statistical testing of differences in citation scores. A further step would be to proceed to non-parametric statistics (e.g., the top-10%) given the ext… ▽ More Following discussions in 2010 and 2011, scientometric evaluators have increasingly abandoned relative indicators in favor of comparing observed with expected citation ratios. The latter method provides parameters with error values allowing for the statistical testing of differences in citation scores. A further step would be to proceed to non-parametric statistics (e.g., the top-10%) given the extreme skewness (non-normality) of the citation distributions. In response to a plea for returning to relative indicators in the previous issue of this newsletter, we argue in favor of further progress in the development of citation impact indicators. △ Less

Submitted 29 August, 2018; originally announced August 2018.

Comments: ISSI Newsletter #55, September 2018

Interdisciplinarity as Diversity in Citation Patterns among Journals: Rao-Stirling Diversity, Relative Variety, and the Gini coefficient

Authors: Loet Leydesdorff, Caroline S. Wagner, Lutz Bornmann

Abstract: Questions of definition and measurement continue to constrain a consensus on the measurement of interdisciplinarity. Using Rao-Stirling (RS) Diversity produces sometimes anomalous results. We argue that these unexpected outcomes can be related to the use of "dual-concept diversity" which combines "variety" and "balance" in the definitions (ex ante). We propose to modify RS Diversity into a new ind… ▽ More Questions of definition and measurement continue to constrain a consensus on the measurement of interdisciplinarity. Using Rao-Stirling (RS) Diversity produces sometimes anomalous results. We argue that these unexpected outcomes can be related to the use of "dual-concept diversity" which combines "variety" and "balance" in the definitions (ex ante). We propose to modify RS Diversity into a new indicator (DIV) which operationalizes variety, balance, and disparity independently and then combines them ex post. "Balance" can be measured using the Gini coefficient. We apply DIV to the aggregated citation patterns of 11,487 journals covered by the Journal Citation Reports 2016 of the Science Citation Index and the Social Sciences Citation Index as an empirical domain and, in more detail, to the citation patterns of 85 journals assigned to the Web-of-Science category "information science & library science" in both the cited and citing directions. We compare the results of the indicators and show that DIV provides improved results in terms of distinguishing between interdisciplinary knowledge integration (citing) versus knowledge diffusion (cited). The new diversity indicator and RS diversity measure different features. A routine for the measurement of the various operationalizations of diversity (in any data matrix) is made available online. △ Less

Submitted 29 August, 2018; v1 submitted 11 July, 2018; originally announced July 2018.

Topic Modelling of Empirical Text Corpora: Validity, Reliability, and Reproducibility in Comparison to Semantic Maps

Authors: Tobias Hecking, Loet Leydesdorff

Abstract: Using the 6,638 case descriptions of societal impact submitted for evaluation in the Research Excellence Framework (REF 2014), we replicate the topic model (Latent Dirichlet Allocation or LDA) made in this context and compare the results with factor-analytic results using a traditional word-document matrix (Principal Component Analysis or PCA). Removing a small fraction of documents from the sampl… ▽ More Using the 6,638 case descriptions of societal impact submitted for evaluation in the Research Excellence Framework (REF 2014), we replicate the topic model (Latent Dirichlet Allocation or LDA) made in this context and compare the results with factor-analytic results using a traditional word-document matrix (Principal Component Analysis or PCA). Removing a small fraction of documents from the sample, for example, has on average a much larger impact on LDA than on PCA-based models to the extent that the largest distortion in the case of PCA has less effect than the smallest distortion of LDA-based models. In terms of semantic coherence, however, LDA models outperform PCA-based models. The topic models inform us about the statistical properties of the document sets under study, but the results are statistical and should not be used for a semantic interpretation - for example, in grant selections and micro-decision making, or scholarly work-without follow-up using domain-specific semantic maps. △ Less

Submitted 4 June, 2018; originally announced June 2018.

Regions, Innovation Systems, and the North-South Divide in Italy

Authors: Loet Leydesdorff, Ivan Cucco

Abstract: Using firm-level data collected by Statistics Italy for 2008, 2011, and 2015, we examine the Triple-Helix synergy among geographical and size distributions of firms, and the NACE codes attributed to these firms, at the different levels of regional and national government. At which levels is innovation-systemness indicated? The contributions of regions to the Italian innovation system have increase… ▽ More Using firm-level data collected by Statistics Italy for 2008, 2011, and 2015, we examine the Triple-Helix synergy among geographical and size distributions of firms, and the NACE codes attributed to these firms, at the different levels of regional and national government. At which levels is innovation-systemness indicated? The contributions of regions to the Italian innovation system have increased, but synergy generation between regions and supra-regionally has remained at almost 45%. As against the statistical classification of Italy into twenty regions or into Northern, Central, and Southern Italy, the greatest synergy is retrieved by considering the country in terms of Northern and Southern Italy as two sub-systems, with Tuscany included as part of Northern Italy. We suggest that separate innovation strategies should be developed for these two parts of the country. The current focus on regions for innovation policies may to some extent be an artifact of the statistics and EU policies. In terms of sectors, both medium- and high-tech manufacturing (MHTM) and knowledge-intensive services (KIS) are proportionally integrated in the various regions. △ Less

Submitted 30 May, 2018; originally announced May 2018.

Diversity and Interdisciplinarity: How Can One Distinguish and Recombine Disparity, Variety, and Balance?

Authors: Loet Leydesdorff

Abstract: The dilemma which remained unsolved using Rao-Stirling diversity, namely of how variety and balance can be combined into "dual concept diversity" (Stirling, 1998, pp. 48f.) can be clarified by using Nijssen et al.'s (1998) argument that the Gini coefficient is a perfect indicator of balance. However, the Gini coefficient is not an indicator of variety; this latter term can be operationalized indep… ▽ More The dilemma which remained unsolved using Rao-Stirling diversity, namely of how variety and balance can be combined into "dual concept diversity" (Stirling, 1998, pp. 48f.) can be clarified by using Nijssen et al.'s (1998) argument that the Gini coefficient is a perfect indicator of balance. However, the Gini coefficient is not an indicator of variety; this latter term can be operationalized independently as relative variety. The three components of diversity--variety, balance, and disparity--can thus be clearly distinguished and independently operationalized as measures varying between zero and one. The new diversity indicator ranges with more resolving power in the empirical case. △ Less

Submitted 7 June, 2018; v1 submitted 25 March, 2018; originally announced March 2018.

Comments: Scientometrics, accepted June 7, 2018

Discontinuities in Citation Relations among Journals: Self-organized Criticality as a Model of Scientific Revolutions and Change

Authors: Loet Leydesdorff, Caroline S. Wagner, Lutz Bornmann

Abstract: Using three-year moving averages of the complete Journal Citation Reports 1994-2016 of the Science Citation Index and the Social Sciences Citation Index (combined), we analyze links between citing and cited journals in terms of (1) whether discontinuities among the networks of consecutive years have occurred; (2) are these discontinuities relatively isolated or networked? (3) Can these discontinui… ▽ More Using three-year moving averages of the complete Journal Citation Reports 1994-2016 of the Science Citation Index and the Social Sciences Citation Index (combined), we analyze links between citing and cited journals in terms of (1) whether discontinuities among the networks of consecutive years have occurred; (2) are these discontinuities relatively isolated or networked? (3) Can these discontinuities be used as indicators of novelty, change, and innovation in the sciences? We examine each of the N2 links among the N journals across the years. We find power-laws for the top 10,000 instances of change, which we suggest interpreting in terms of "self-organized criticality": co-evolutions of avalanches in aggregated citation relations and meta-stable states in the knowledge base can be expected to drive the sciences towards the edges of chaos. The flux of journal-journal citations in new manuscripts may generate an avalanche in the meta-stable networks, but one can expect the effects to remain local (for example, within a specialty). The avalanches can be of any size; they reorient the relevant citation environments by inducing a rewrite of history in the affected partitions. △ Less

Submitted 1 March, 2018; originally announced March 2018.

The negative effects of citing with a national orientation in terms of recognition: national and international citations in natural-sciences papers from Germany, the Netherlands, and the UK

Authors: Lutz Bornmann, Jonathan Adams, Loet Leydesdorff

Abstract: Nations can be distinguished in terms of whether domestic or international research is cited. We analyzed the research output in natural sciences of three leading European research economies (Germany, the Netherlands, and the UK) and ask where their researchers look for the knowledge that underpins their most highly-cited papers. Is one internationally oriented or is citation limited to national r… ▽ More Nations can be distinguished in terms of whether domestic or international research is cited. We analyzed the research output in natural sciences of three leading European research economies (Germany, the Netherlands, and the UK) and ask where their researchers look for the knowledge that underpins their most highly-cited papers. Is one internationally oriented or is citation limited to national resources? Do the citation patterns reflect a growing differentiation between the domestic and international research enterprise? To evaluate change over time, we include natural-sciences papers published in the countries from three publication years: 2004, 2009, and 2014. The results show that articles co-authored by researchers from Germany or the Netherlands are less likely to be among the globally most highly-cited articles if they also cite "domestic" research (i.e. research authored by authors from the same country). To put this another way, less well-cited research is more likely to stand on domestic shoulders and research that becomes more highly-cited is more likely to stand on international shoulders. A possible reason for the results is that researchers "over-cite" the papers from their own country - lacking the focus on quality in citing. However, these differences between domestic and international shoulders are not visible for the UK. △ Less

Submitted 26 July, 2018; v1 submitted 3 February, 2018; originally announced February 2018.

Data-mining the Foundational Patents of Photovoltaic Materials: An application of Patent Citation Spectroscopy

Authors: Jordan Comins, Loet Leydesdorff

Abstract: We apply Patent Citation Spectroscopy (PCS)--originally developed as Reference Publication Year Spectroscopy for studying landmarks and milestones in scientific literature--to patent literature classified into the nine Y-subclasses of the Cooperative Patent Classification (CPC) that describe material photovoltaic technologies. For this study we extended the routine with the option to use the advan… ▽ More We apply Patent Citation Spectroscopy (PCS)--originally developed as Reference Publication Year Spectroscopy for studying landmarks and milestones in scientific literature--to patent literature classified into the nine Y-subclasses of the Cooperative Patent Classification (CPC) that describe material photovoltaic technologies. For this study we extended the routine with the option to use the advanced search queries at PatentsView. On the basis of two normalizations of the longitudinal distribution of the publication years of the patents cited by the retrieved patents, the routine (at http://www.leydesdorff.net/comins/pcs/index.html) provides a best guess of the foundational patent for the subject specified in the string. In five of the nine cases, we found corroborating evidence for the foundational character of the patent indicated by the routine. △ Less

Submitted 10 April, 2018; v1 submitted 29 January, 2018; originally announced January 2018.

The relative influences of government funding and international collaboration on citation impact

Authors: Loet Leydesdorff, Lutz Bornmann, Caroline S. Wagner

Abstract: In a recent publication in Nature, Wagner & Jonkers (2017) report that public R&D funding is only weakly correlated with the citation impact of a nation's papers as measured by the field-weighted citation index (FWCI; defined by Scopus). On the basis of the supplementary data, we upscaled the design using Web-of-Science data for the decade 2003-2013 and OECD funding data for the corresponding deca… ▽ More In a recent publication in Nature, Wagner & Jonkers (2017) report that public R&D funding is only weakly correlated with the citation impact of a nation's papers as measured by the field-weighted citation index (FWCI; defined by Scopus). On the basis of the supplementary data, we upscaled the design using Web-of-Science data for the decade 2003-2013 and OECD funding data for the corresponding decade assuming a two-year delay (2001-2011). Using negative binomial regression analysis, we find very small coefficients, but the effects of international collaboration are positive and statistically significant, whereas the effects of government funding are negative, an order of magnitude smaller, and statistically non-significant (in two of three analyses). In other words, international collaboration improves the impact of average research papers, whereas more government funding tends to have a small adverse effect when comparing OECD countries. △ Less

Submitted 13 December, 2017; originally announced December 2017.

Automated Analysis of Topic-Actor Networks on Twitter: New approach to the analysis of socio-semantic networks

Authors: Iina Hellsten, Loet Leydesdorff

Abstract: Social-media data provides increasing opportunities for automated analysis of large sets of textual documents. So far, automated tools have been developed to account for either the social networks between the participants of the debates, or to analyze the content of those debates. Less attention has been paid to map** co-occurring actors (participants) and topics (content) in online debates that… ▽ More Social-media data provides increasing opportunities for automated analysis of large sets of textual documents. So far, automated tools have been developed to account for either the social networks between the participants of the debates, or to analyze the content of those debates. Less attention has been paid to map** co-occurring actors (participants) and topics (content) in online debates that form socio-semantic networks. We propose a new, automated approach that uses a whole matrix approach of co-addressed topics and the actors. We show the advantages of the new approach with the analysis of a large set of English-language Twitter messages at the Rio+20 meeting, in June 2012 (72,077 tweets), and a smaller data set of Dutch-language Twitter messages on bird flu related to poultry farming in 2015-2017 (2,139 tweets). We discuss the theoretical, methodological and substantive implications of our approach, also for the analysis of other social-media data. △ Less

Submitted 22 November, 2017; originally announced November 2017.

Statistical Significance and Effect Sizes of Differences among Research Universities at the Level of Nations and Worldwide based on the Leiden Rankings

Authors: Loet Leydesdorff, Lutz Bornmann, John Mingers

Abstract: The Leiden Rankings can be used for grou** research universities by considering universities which are not statistically significantly different as homogeneous sets. The groups and intergroup relations can be analyzed and visualized using tools from network analysis. Using the so-called "excellence indicator" PPtop-10%--the proportion of the top-10% most-highly-cited papers assigned to a univers… ▽ More The Leiden Rankings can be used for grou** research universities by considering universities which are not statistically significantly different as homogeneous sets. The groups and intergroup relations can be analyzed and visualized using tools from network analysis. Using the so-called "excellence indicator" PPtop-10%--the proportion of the top-10% most-highly-cited papers assigned to a university--we pursue a classification using (i) overlap** stability intervals, (ii) statistical-significance tests, and (iii) effect sizes of differences among 902 universities in 54 countries; we focus on the UK, Germany, Brazil, and the USA as national examples. Although the grou**s remain largely the same using different statistical significance levels or overlap** stability intervals, these classifications are uncorrelated with those based on effect sizes. Effect sizes for the differences between universities are small (w <.2). The more detailed analysis of universities at the country level suggests that distinctions beyond three or perhaps four groups of universities (high, middle, low) may not be meaningful. Given similar institutional incentives, isomorphism within each eco-system of universities should not be underestimated. Our results suggest that networks based on overlap** stability intervals can provide a first impression of the relevant grou**s among universities. However, the clusters are not well-defined divisions between groups of universities. △ Less

Submitted 13 October, 2018; v1 submitted 30 October, 2017; originally announced October 2017.

Comments: Journal of the Association for Information Science and Technology (JASIST), forthcoming 2018

Synergy in the Knowledge Base of U.S. Innovation Systems at National, State, and Regional Levels: The Contributions of High-Tech Manufacturing and Knowledge-Intensive Services

Authors: Loet Leydesdorff, Caroline S. Wagner, Igone Porto-Gomez, Jordan A. Comins, Fred Phillips

Abstract: Using information theory, we measure innovation systemness as synergy among size-classes, zip-codes, and technological classes (NACE-codes) for 8.5 million American companies. The synergy at the national level is decomposed at the level of states, Core-Based Statistical Areas (CBSA), and Combined Statistical Areas (CSA). We zoom in to the state of California and in more detail to Silicon Valley. O… ▽ More Using information theory, we measure innovation systemness as synergy among size-classes, zip-codes, and technological classes (NACE-codes) for 8.5 million American companies. The synergy at the national level is decomposed at the level of states, Core-Based Statistical Areas (CBSA), and Combined Statistical Areas (CSA). We zoom in to the state of California and in more detail to Silicon Valley. Our results do not support the assumption of a national system of innovations in the U.S.A. Innovation systems appear to operate at the level of the states; the CBSA are too small, so that systemness spills across their borders. Decomposition of the sample in terms of high-tech manufacturing (HTM), medium-high-tech manufacturing (MHTM), knowledge-intensive services (KIS), and high-tech services (HTKIS) does not change this pattern, but refines it. The East Coast -- New Jersey, Boston, and New York -- and California are the major players, with Texas a third one in the case of HTKIS. Chicago and industrial centers in the Midwest also contribute synergy. Within California, Los Angeles contributes synergy in the sectors of manufacturing, the San Francisco area in KIS. Knowledge-intensive services in Silicon Valley and the Bay area -- a CSA composed of seven CBSA -- spill over to other regions and even globally. △ Less

Submitted 19 November, 2018; v1 submitted 30 October, 2017; originally announced October 2017.

Comments: Journal of the Association of Information Science and Technology (JASIST, forthcoming)

Patent Citation Spectroscopy (PCS): Algorithmic retrieval of landmark patents

Authors: Jordan A Comins, Stephanie A Carmack, Loet Leydesdorff

Abstract: One essential component in the construction of patent landscapes in biomedical research and development (R&D) is identifying the most seminal patents. Hitherto, the identification of seminal patents required subject matter experts within biomedical areas. In this brief communication, we report an analytical method and tool, Patent Citation Spectroscopy (PCS), for rapidly identifying landmark paten… ▽ More One essential component in the construction of patent landscapes in biomedical research and development (R&D) is identifying the most seminal patents. Hitherto, the identification of seminal patents required subject matter experts within biomedical areas. In this brief communication, we report an analytical method and tool, Patent Citation Spectroscopy (PCS), for rapidly identifying landmark patents in user-specified areas of biomedical innovation. PCS mines the cited references within large sets of patents and provides an estimate of the most historically impactful prior work. The efficacy of PCS is shown in two case studies of biomedical innovation with clinical relevance: (1) RNA interference and (2) cholesterol. PCS mined and analyzed 4,065 cited references related to patents on RNA interference and correctly identified the foundational patent of this technology, as independently reported by subject matter experts on RNAi intellectual property. Secondly, PCS was applied to a broad set of patents dealing with cholesterol - a case study chosen to reflect a more general, as opposed to expert, patent search query. PCS mined through 11,326 cited references and identified the seminal patent as that for Lipitor, the groundbreaking medication for treating high cholesterol as well as the pair of patents underlying Repatha. These cases suggest that PCS provides a useful method for identifying seminal patents in areas of biomedical innovation and therapeutics. The interactive tool is free-to-use at: www.leydesdorff.net/pcs/. △ Less

Submitted 14 October, 2017; v1 submitted 9 October, 2017; originally announced October 2017.

The geography of references in elite articles: What countries contribute to the archives of knowledge

Authors: Lutz Bornmann, Caroline Wagner, Loet Leydesdorff

Abstract: This study is intended to find an answer for the question on which national "shoulders" the worldwide top-level research stands. Traditionally, national scientific standings are evaluated in terms of the number of citations to their papers. We raise a different question: instead of analyzing the citations to the countries' articles (the forward view), we examine referenced publications from specif… ▽ More This study is intended to find an answer for the question on which national "shoulders" the worldwide top-level research stands. Traditionally, national scientific standings are evaluated in terms of the number of citations to their papers. We raise a different question: instead of analyzing the citations to the countries' articles (the forward view), we examine referenced publications from specific countries cited in the most elite publications (the backward-citing-view). "Elite publications" are operationalized as the top-1% most-highly cited articles. Using the articles published during the years 2004 to 2013, we examine the research referenced in these works. Our results confirm the well-known fact that China has emerged to become a major player in science. However, China still belongs to the low contributors when countries are ranked as contributors to the cited references in top-1% articles. Using this perspective, the results do not point to a decreasing trend for the USA; in fact, the USA exceeds expectations (compared to its publication share) in terms of contributions to cited references in the top-1% articles. Switzerland, Sweden, and the Netherlands also are shown at the top of the list. However, the results for Germany are lower than statistically expected. △ Less

Submitted 19 September, 2017; originally announced September 2017.

Reference Publication Year Spectroscopy (RPYS) of Eugene Garfield's publications

Authors: Lutz Bornmann, Robin Haunschild, Loet Leydesdorff

Abstract: Which studies, theories, and ideas have influenced Eugene Garfield's scientific work? Recently, the method reference publication year spectroscopy (RPYS) has been introduced, which can be used to answer this and related questions. Since then, several studies have been published dealing with the historical roots of research fields and scientists. The program CRExplorer (http://www.crexplorer.net) w… ▽ More Which studies, theories, and ideas have influenced Eugene Garfield's scientific work? Recently, the method reference publication year spectroscopy (RPYS) has been introduced, which can be used to answer this and related questions. Since then, several studies have been published dealing with the historical roots of research fields and scientists. The program CRExplorer (http://www.crexplorer.net) was specifically developed for RPYS. In this study, we use this program to investigate the historical roots of Eugene Garfield's oeuvre. △ Less

Submitted 15 August, 2017; originally announced August 2017.

Comments: 16 pages, 3 tables, and 3 figures

Probing Multivariate Indicators for Academic Evaluation

Authors: Helen F. Xue, Loet Leydesdorff, Fred Y. Ye

Abstract: We combine the Integrated Impact Indicator (I3) and the h-index into the I3-type framework and introduce the publication vector X = (X1, X2, X3) and the citation vector Y = (Y1, Y2, Y3) , the publication score I3X=X1+X2+X3 and the citation score I3Y=Y1+Y2+Y3, and alternative indicators based on percentile classes generated by the h-index. These multivariate indicators can be used for academic eval… ▽ More We combine the Integrated Impact Indicator (I3) and the h-index into the I3-type framework and introduce the publication vector X = (X1, X2, X3) and the citation vector Y = (Y1, Y2, Y3) , the publication score I3X=X1+X2+X3 and the citation score I3Y=Y1+Y2+Y3, and alternative indicators based on percentile classes generated by the h-index. These multivariate indicators can be used for academic evaluation. The empirical studies show that the h-core distribution is suitable to evaluate scholars, the X1 and Y1 are applied to measure core impact power of universities, and I3X and I3Y are alternatives of journal impact factor (JIF). The multivariate indicators provide a multidimensional view of academic evaluation with using the advantages of both the h-index and I3. △ Less

Submitted 6 July, 2017; originally announced July 2017.

Comments: Journal of Library Science in China (in press). This is the English version of a Chinese article. DOI:10.13530/j.cnki.jlis.170018

Betweenness and Diversity in Journal Citation Networks as Measures of Interdisciplinarity -- A Tribute to Eugene Garfield --

Authors: Loet Leydesdorff, Caroline S. Wagner, Lutz Bornmann

Abstract: Journals were central to Eugene Garfield's research interests. Among other things, journals are considered as units of analysis for bibliographic databases such as the Web of Science (WoS) and Scopus. In addition to disciplinary classifications of journals, journal citation patterns span networks across boundaries to variable extents. Using betweenness centrality (BC) and diversity, we elaborate o… ▽ More Journals were central to Eugene Garfield's research interests. Among other things, journals are considered as units of analysis for bibliographic databases such as the Web of Science (WoS) and Scopus. In addition to disciplinary classifications of journals, journal citation patterns span networks across boundaries to variable extents. Using betweenness centrality (BC) and diversity, we elaborate on the question of how to distinguish and rank journals in terms of interdisciplinarity. Interdisciplinarity, however, is difficult to operationalize in the absence of an operational definition of disciplines, the diversity of a unit of analysis is sample-dependent. BC can be considered as a measure of multi-disciplinarity. Diversity of co-citation in a citing document has been considered as an indicator of knowledge integration, but an author can also generate trans-disciplinary--that is, non-disciplined--variation by citing sources from other disciplines. Diversity in the bibliographic coupling among citing documents can analogously be considered as diffusion of knowledge across disciplines. Because the citation networks in the cited direction reflect both structure and variation, diversity in this direction is perhaps the best available measure of interdisciplinarity at the journal level. Furthermore, diversity is based on a summation and can therefore be decomposed, differences among (sub)sets can be tested for statistical significance. In an appendix, a general-purpose routine for measuring diversity in networks is provided. △ Less

Submitted 14 May, 2017; v1 submitted 9 May, 2017; originally announced May 2017.

Map** Patent Classifications: Portfolio and Statistical Analysis, and the Comparison of Strengths and Weaknesses

Authors: Loet Leydesdorff, Dieter Franz Kogler, Bowen Yan

Abstract: The Cooperative Patent Classifications (CPC) jointly developed by the European and US Patent Offices provide a new basis for map** and portfolio analysis. This update provides an occasion for rethinking the parameter choices. The new maps are significantly different from previous ones, although this may not always be obvious on visual inspection. Since these maps are statistical constructs based… ▽ More The Cooperative Patent Classifications (CPC) jointly developed by the European and US Patent Offices provide a new basis for map** and portfolio analysis. This update provides an occasion for rethinking the parameter choices. The new maps are significantly different from previous ones, although this may not always be obvious on visual inspection. Since these maps are statistical constructs based on index terms, their quality--as different from utility--can only be controlled discursively. We provide nested maps online and a routine for portfolio overlays and further statistical analysis. We add a new tool for "difference maps" which is illustrated by comparing the portfolios of patents granted to Novartis and MSD in 2016. △ Less

Submitted 14 October, 2017; v1 submitted 24 February, 2017; originally announced February 2017.

Comments: Scientometrics 112(3) (2017) 1573-1591; http://link.springer.com/article/10.1007/s11192-017-2449-0

Toward a Calculus of Redundancy: The feedback arrow of expectations in knowledge-based systems

Authors: Loet Leydesdorff, Mark W. Johnson, Inga Ivanova

Abstract: This paper considers the relationships among meaning generation, selection, and the dynamics of discourse from a variety of perspectives ranging from information theory and biology to sociology. Following Husserl's idea of a horizon of meaning in intersubjective communication, we propose a way in which, using Shannon's equations, the generation and selection of meanings from a horizon of possibili… ▽ More This paper considers the relationships among meaning generation, selection, and the dynamics of discourse from a variety of perspectives ranging from information theory and biology to sociology. Following Husserl's idea of a horizon of meaning in intersubjective communication, we propose a way in which, using Shannon's equations, the generation and selection of meanings from a horizon of possibilities can be considered probabilistically. The information-theoretical dynamics we articulate considers a process of meaning generation within cultural evolution: information is imbued with meaning, and through this process, the number of options for the selection of meaning in discourse proliferates. The redundancy of possible meanings contributes to a codification of expectations within the discourse. Unlike hard-wired DNA, the codes of non-biological systems can co-evolve with the variations. Spanning horizons of meaning, the codes structure the communications as selection environments that shape discourses. Discursive knowledge can be considered as meta-coded communication which enables us to translate among differently coded communications. The dynamics of discursive knowledge production can thus infuse the historical dynamics with a cultural evolution by adding options, that is, by increasing redundancy. A calculus of redundancy is presented as an indicator whereby these dynamics of discourse and meaning may be explored empirically. △ Less

Submitted 24 March, 2018; v1 submitted 10 January, 2017; originally announced January 2017.

Comments: accepted for publication by the Journal of the Association for Information Science and Technology, 24 March 2018

Growth of International Cooperation in Science: Revisiting Six Case Studies

Authors: Caroline S. Wagner, Travis Whetsell, Loet Leydesdorff

Abstract: International collaboration in science continues to grow at a remarkable rate, but little agreement exists about dynamics of growth and organization at the discipline level. Some suggest that disciplines differ in their collaborative tendencies, reflecting their epistemic culture. This study examines collaborative patterns in six previously studied specialties to add new data and conduct analyses… ▽ More International collaboration in science continues to grow at a remarkable rate, but little agreement exists about dynamics of growth and organization at the discipline level. Some suggest that disciplines differ in their collaborative tendencies, reflecting their epistemic culture. This study examines collaborative patterns in six previously studied specialties to add new data and conduct analyses over time. Our findings show that the global network of collaboration continues to add new nations and new participants; each specialty has added many new nations to its lists of collaborating partners since 1990. We also find that the scope of international collaboration is positively related to impact. Network characteristics for the six specialties are notable in that instead of reflecting underlying culture, they tend towards convergence. This observation suggests that the global level may represent next-order dynamics that feed back to the national and local levels (as subsystems) in a complex, networked hierarchy. △ Less

Submitted 20 December, 2016; originally announced December 2016.

Comments: data on figsharae

Patent Portfolio Analysis of Cities: Statistics and Maps of Technological Inventiveness

Authors: Dieter Franz Kogler, Gaston Heimeriks, Loet Leydesdorff

Abstract: Cities are engines of the knowledge-based economy, because they are the primary sites of knowledge production activities that subsequently shape the rate and direction of technological change and economic growth. Patents provide a wealth of information to analyse the knowledge specialization at specific places, such as technological details and information on inventors and entities involved, inclu… ▽ More Cities are engines of the knowledge-based economy, because they are the primary sites of knowledge production activities that subsequently shape the rate and direction of technological change and economic growth. Patents provide a wealth of information to analyse the knowledge specialization at specific places, such as technological details and information on inventors and entities involved, including address information. The technology codes on each patent document indicate the specialization and scope of the underlying technological knowledge of a given invention. In this paper we introduce tools for portfolio analysis in terms of patents that provide insights into the technological specialization of cities. The map** and analysis of patent portfolios of cities using data of the Unites States Patent and Trademark Office (USPTO) website (at http://www.uspto.gov) and dedicated tools (at http://www.leydesdorff.net/portfolio) can be used to analyse the specialisation patterns of inventive activities among cities. The results allow policy makers and other stakeholders to identify promising areas of further knowledge development and 'smart specialisation' strategies. △ Less

Submitted 17 December, 2016; originally announced December 2016.

Full and Fractional Counting in Bibliometric Networks

Authors: Loet Leydesdorff, Han Woo Park

Abstract: In their study entitled "Constructing bibliometric networks: A comparison between full and fractional counting," Perianes-Rodriguez, Waltman, & van Eck (2016; henceforth abbreviated as PWvE) provide arguments for the use of fractional counting at the network level as different from the level of publications. Whereas fractional counting in the latter case divides the credit among co-authors (countr… ▽ More In their study entitled "Constructing bibliometric networks: A comparison between full and fractional counting," Perianes-Rodriguez, Waltman, & van Eck (2016; henceforth abbreviated as PWvE) provide arguments for the use of fractional counting at the network level as different from the level of publications. Whereas fractional counting in the latter case divides the credit among co-authors (countries, institutions, etc.), fractional counting at the network level can normalize the relative weights of links and thereby clarify the structures in the network. PWvE, however, propose a counting scheme for fractional counting that is one among other possible ones. Alternative schemes proposed by Batagelj and Cerinšek (2013) and Park, Yoon, & Leydesdorff (2016; henceforth abbreviated as PYL) are discussed in an appendix. However, our approach is not correctly identified as identical to their Equation A3. Here below, we distinguish three approaches analytically; routines for applying these approaches to bibliometric data are also provided. △ Less

Submitted 27 November, 2016; v1 submitted 21 November, 2016; originally announced November 2016.

Comments: accepted for publication in the Journal of Informetrics, 27 November 2016

Skewness of citation impact data and covariates of citation distributions: A large-scale empirical analysis based on Web of Science data

Authors: Lutz Bornmann, Loet Leydesdorff

Abstract: Using percentile shares, one can visualize and analyze the skewness in bibliometric data across disciplines and over time. The resulting figures can be intuitively interpreted and are more suitable for detailed analysis of the effects of independent and control variables on distributions than regression analysis. We show this by using percentile shares to analyze so-called "factors influencing cit… ▽ More Using percentile shares, one can visualize and analyze the skewness in bibliometric data across disciplines and over time. The resulting figures can be intuitively interpreted and are more suitable for detailed analysis of the effects of independent and control variables on distributions than regression analysis. We show this by using percentile shares to analyze so-called "factors influencing citation impact" (FICs; e.g., the impact factor of the publishing journal) across year and disciplines. All articles (n= 2,961,789) covered by WoS in 1990 (n= 637,301), 2000 (n= 919,485), and 2010 (n= 1,405,003) are used. In 2010, nearly half of the citation impact is accounted for by the 10% most-frequently cited papers; the skewness is largest in the humanities (68.5% in the top-10% layer) and lowest in agricultural sciences (40.6%). The comparison of the effects of the different FICs (the number of cited references, number of authors, number of pages, and JIF) on citation impact shows that JIF has indeed the strongest correlations with the citation scores. However, the correlation between FICs and citation impact is lower, if citations are normalized instead of using raw citation counts. △ Less

Submitted 1 December, 2016; v1 submitted 7 November, 2016; originally announced November 2016.

Citation algorithms for identifying research milestones driving biomedical innovation

Authors: Jordan A. Comins, Loet Leydesdorff

Abstract: Scientific activity plays a major role in innovation for biomedicine and healthcare. For instance, fundamental research on disease pathologies and mechanisms can generate potential targets for drug therapy. This co-evolution is punctuated by papers which provide new perspectives and open new domains. Despite the relationship between scientific discovery and biomedical advancement, identifying thes… ▽ More Scientific activity plays a major role in innovation for biomedicine and healthcare. For instance, fundamental research on disease pathologies and mechanisms can generate potential targets for drug therapy. This co-evolution is punctuated by papers which provide new perspectives and open new domains. Despite the relationship between scientific discovery and biomedical advancement, identifying these research milestones that truly impact biomedical innovation can be difficult and is largely based solely on the opinions of subject matter experts. Here, we consider whether a new class of citation algorithms that identify seminal scientific works in a field, Reference Publication Year Spectroscopy (RPYS) and multi-RPYS, can identify the connections between innovation (e.g. therapeutic treatments) and the foundational research underlying them. Specifically, we assess whether the results of these analytic techniques converge with expert opinions on research milestones driving biomedical innovation in the treatment of Basal Cell Carcinoma. Our results show that these algorithms successfully identify the majority of milestone papers detailed by experts (Wong and Dlugosz 2014) thereby validating the power of these algorithms to converge on independent opinions of seminal scientific works derived by subject matter experts. These advances offer an opportunity to identify scientific activities enabling innovation in biomedicine. △ Less

Submitted 5 November, 2016; originally announced November 2016.

Generating Clustered Journal Maps: An Automated System for Hierarchical Classification

Authors: Loet Leydesdorff, Lutz Bornmann, Caroline S. Wagner

Abstract: Journal maps and classifications for 11,359 journals listed in the combined Journal Citation Reports 2015 of the Science and Social Sciences Citation Indexes are provided at http://www.leydesdorff.net/jcr15. A routine using VOSviewer for integrating the journal map** and their hierarchical clustering is also made available. In this short communication, we provide background on the journal mappin… ▽ More Journal maps and classifications for 11,359 journals listed in the combined Journal Citation Reports 2015 of the Science and Social Sciences Citation Indexes are provided at http://www.leydesdorff.net/jcr15. A routine using VOSviewer for integrating the journal map** and their hierarchical clustering is also made available. In this short communication, we provide background on the journal map**/clustering and an explanation and instructions about the routine. We compare 2015 journal maps with those for 2014 and show the delineations among fields and subfields to be sensitive to fluctuations. Labels for fields and sub-fields are not provided by the routine, but can be added by an analyst for pragmatic or intellectual reasons. The routine provides a means for testing one's assumptions against a baseline without claiming authority, clusters of related journals can be visualized to understand communities. The routine is generic and can be used for any 1-mode network. △ Less

Submitted 10 January, 2017; v1 submitted 12 October, 2016; originally announced October 2016.

Comments: Accepted for publication in Scientometrics. Full set of journal clusters at http://www.leydesdorff.net/jcr15/

Measuring the match between evaluators and evaluees: Cognitive distances between panel members and research groups at the journal level

Authors: A. I. M. Jakaria Rahman, Raf Guns, Loet Leydesdorff, Tim C. E. Engels

Abstract: When research groups are evaluated by an expert panel, it is an open question how one can determine the match between panel and research groups. In this paper, we outline two quantitative approaches that determine the cognitive distance between evaluators and evaluees, based on the journals they have published in. We use example data from four research evaluations carried out between 2009 and 2014… ▽ More When research groups are evaluated by an expert panel, it is an open question how one can determine the match between panel and research groups. In this paper, we outline two quantitative approaches that determine the cognitive distance between evaluators and evaluees, based on the journals they have published in. We use example data from four research evaluations carried out between 2009 and 2014 at the University of Antwerp. While the barycenter approach is based on a journal map, the similarity-adapted publication vector (SAPV) approach is based on the full journal similarity matrix. Both approaches determine an entity's profile based on the journals in which it has published. Subsequently, we determine the Euclidean distance between the barycenter or SAPV profiles of two entities as an indicator of the cognitive distance between them. Using a bootstrap** approach, we determine confidence intervals for these distances. As such, the present article constitutes a refinement of a previous proposal that operates on the level of Web of Science subject categories. △ Less

Submitted 22 September, 2016; originally announced September 2016.

Professional and Citizen Bibliometrics: Complementarities and ambivalences in the development and use of indicators

Authors: Loet Leydesdorff, Paul Wouters, Lutz Bornmann

Abstract: Bibliometric indicators such as journal impact factors, h-indices, and total citation counts are algorithmic artifacts that can be used in research evaluation and management. These artifacts have no meaning by themselves, but receive their meaning from attributions in institutional practices. We distinguish four main stakeholders in these practices: (1) producers of bibliometric data and indicator… ▽ More Bibliometric indicators such as journal impact factors, h-indices, and total citation counts are algorithmic artifacts that can be used in research evaluation and management. These artifacts have no meaning by themselves, but receive their meaning from attributions in institutional practices. We distinguish four main stakeholders in these practices: (1) producers of bibliometric data and indicators; (2) bibliometricians who develop and test indicators; (3) research managers who apply the indicators; and (4) the scientists being evaluated with potentially competing career interests. These different positions may lead to different and sometimes conflicting perspectives on the meaning and value of the indicators. The indicators can thus be considered as boundary objects which are socially constructed in translations among these perspectives. This paper proposes an analytical clarification by listing an informed set of (sometimes unsolved) problems in bibliometrics which can also shed light on the tension between simple but invalid indicators that are widely used (e.g., the h-index) and more sophisticated indicators that are not used or cannot be used in evaluation practices because they are not transparent for users, cannot be calculated, or are difficult to interpret. △ Less

Submitted 23 September, 2016; v1 submitted 15 September, 2016; originally announced September 2016.

Comments: Scientometrics (in press)

"Open Innovation" and "Triple Helix" Models of Innovation: Can Synergy in Innovation Systems Be Measured?

Authors: Loet Leydesdorff, Inga Ivanova

Abstract: The model of "Open Innovations" (OI) can be compared with the "Triple Helix of University-Industry-Government Relations" (TH) as attempts to find surplus value in bringing industrial innovation closer to public R&D. Whereas the firm is central in the model of OI, the TH adds multi-centeredness: in addition to firms, universities and (e.g., regional) governments can take leading roles in innovation… ▽ More The model of "Open Innovations" (OI) can be compared with the "Triple Helix of University-Industry-Government Relations" (TH) as attempts to find surplus value in bringing industrial innovation closer to public R&D. Whereas the firm is central in the model of OI, the TH adds multi-centeredness: in addition to firms, universities and (e.g., regional) governments can take leading roles in innovation eco-systems. In addition to the (transversal) technology transfer at each moment of time, one can focus on the dynamics in the feedback loops. Under specifiable conditions, feedback loops can be turned into feedforward ones that drive innovation eco-systems towards self-organization and the auto-catalytic generation of new options. The generation of options can be more important than historical realizations ("best practices") for the longer-term viability of knowledge-based innovation systems. A system without sufficient options, for example, is locked-in. The generation of redundancy -- the Triple Helix indicator -- can be used as a measure of unrealized but technologically feasible options given a historical configuration. Different coordination mechanisms (markets, policies, knowledge) provide different perspectives on the same information and thus generate redundancy. Increased redundancy not only stimulates innovation in an eco-system by reducing the prevailing uncertainty; it also enhances the synergy in and innovativeness of an innovation system. △ Less

Submitted 28 July, 2017; v1 submitted 24 May, 2016; originally announced July 2016.

Comments: Journal of Open Innovations: Technology, Market and Complexity, 2(1) (2016) 1-12; doi:10.1186/s40852-016-0039-7

Cited References and Medical Subject Headings (MeSH) as Two Different Knowledge Representations: Clustering and Map**s at the Paper Level

Authors: Loet Leydesdorff, Jordan A. Comins, Aaron A. Sorensen, Lutz Bornmann, Iina Hellsten

Abstract: For the biomedical sciences, the Medical Subject Headings (MeSH) make available a rich feature which cannot currently be merged properly with widely used citing/cited data. Here, we provide methods and routines that make MeSH terms amenable to broader usage in the study of science indicators: using Web-of-Science (WoS) data, one can generate the matrix of citing versus cited documents; using PubMe… ▽ More For the biomedical sciences, the Medical Subject Headings (MeSH) make available a rich feature which cannot currently be merged properly with widely used citing/cited data. Here, we provide methods and routines that make MeSH terms amenable to broader usage in the study of science indicators: using Web-of-Science (WoS) data, one can generate the matrix of citing versus cited documents; using PubMed/MEDLINE data, a matrix of the citing documents versus MeSH terms can be generated analogously. The two matrices can also be reorganized into a 2-mode matrix of MeSH terms versus cited references. Using the abbreviated journal names in the references, one can, for example, address the question whether MeSH terms can be used as an alternative to WoS Subject Categories for the purpose of normalizing citation data. We explore the applicability of the routines in the case of a research program about the amyloid cascade hypothesis in Alzheimer's disease (AD). One conclusion is that referenced journals provide archival structures, whereas MeSH terms indicate mainly variation (including novelty) at the research front. Furthermore, we explore the option of using the citing/cited matrix for main-path analysis as a by-product of the software. △ Less

Submitted 11 September, 2016; v1 submitted 21 July, 2016; originally announced July 2016.

Comments: accepted for publication in Scientometrics, August 2016

New features of CitedReferencesExplorer (CRExplorer)

Authors: Andreas Thor, Werner Marx, Loet Leydesdorff, Lutz Bornmann

Abstract: CRExplorer version 1.6.7 was released on July 5, 2016. This version includes the following new features and improvements: Scopus: Using "File" - "Import" - "Scopus", CRExplorer reads files from Scopus. The file format "CSV" (including citations, abstracts and references) should be chosen in Scopus for downloading records. Export facilities: Using "File" - "Export" - "Scopus", CRExplorer exports fi… ▽ More CRExplorer version 1.6.7 was released on July 5, 2016. This version includes the following new features and improvements: Scopus: Using "File" - "Import" - "Scopus", CRExplorer reads files from Scopus. The file format "CSV" (including citations, abstracts and references) should be chosen in Scopus for downloading records. Export facilities: Using "File" - "Export" - "Scopus", CRExplorer exports files in the Scopus format. Using "File" - "Export" - "Web of Science", CRExplorer exports files in the Web of Science format. These files can be imported in other bibliometric programs (e.g. VOSviewer). Space bar: Select a specific cited reference in the cited references table, press the space bar, and all bibliographic details of the CR are shown. Internal file format: Using "File" - "Save", working files are saved in the internal file format "*.cre". The files include all data including matching results and manual matching corrections. The files can be opened by using "File" - "Open". △ Less

Submitted 20 July, 2016; v1 submitted 5 July, 2016; originally announced July 2016.

Comments: Accepted for publication in Scientometrics

What is the effect of synergy in international collaboration on regional economies?

Authors: Inga Ivanova, Oivind Strand, Loet Leydesdorff

Abstract: We analyze the effects of relative increments of mutual information among the geographical, technological, and organizational distributions of firms on the relative augmentation of regional summary turnover in terms of synergies. How do increases in synergy in international cooperation affect regional turnover? The methodological contribution of this study is that we translate the synergy (abstrac… ▽ More We analyze the effects of relative increments of mutual information among the geographical, technological, and organizational distributions of firms on the relative augmentation of regional summary turnover in terms of synergies. How do increases in synergy in international cooperation affect regional turnover? The methodological contribution of this study is that we translate the synergy (abstractly measured in bits of information) into more familiar economic terms, such as turnover for the special case of domestic-foreign collaborations. The analysis is based on Norwegian data, as Norway is a small country with an open and export-oriented economy. Data for Norway is publicly available in great detail. △ Less

Submitted 17 October, 2016; v1 submitted 19 May, 2016; originally announced May 2016.

Journal ref: Journal of the Knowledge Economy 10(1) (2019) 18-34

The Normalization of Co-authorship Networks in the Bibliometric Evaluation: The Government Stimulation Programs of China and Korea

Authors: Han Woo Park, Jungwon Yoon, Loet Leydesdorff

Abstract: Using co-authored publications between China and Korea in Web of Science (WoS) during the one-year period of 2014, we evaluate the government stimulation program for collaboration between China and Korea. In particular, we apply dual approaches, full integer vs. fractional counting, to collaborative publications in order to better examine both the patterns and contents of Sino-Korean collaboration… ▽ More Using co-authored publications between China and Korea in Web of Science (WoS) during the one-year period of 2014, we evaluate the government stimulation program for collaboration between China and Korea. In particular, we apply dual approaches, full integer vs. fractional counting, to collaborative publications in order to better examine both the patterns and contents of Sino-Korean collaboration networks in terms of individual countries and institutions. We first conduct a semi-automatic network analysis of Sino-Korean publications based on the full-integer counting method, and then compare our categorization with contextual rankings using the fractional technique; routines for fractional counting of WoS data are made available at http://www.leydesdorff.net/software/fraction . Increasing international collaboration leads paradoxically to lower numbers of publications and citations using fractional counting for performance measurement. However, integer counting is not an appropriate measure for the evaluation of the stimulation of collaborations. Both integer and fractional analytics can be used to identify important countries and institutions, but with other research questions. △ Less

Submitted 11 May, 2016; originally announced May 2016.

Comments: Accepted for publication in Scientometrics

Referenced Publication Year Spectroscopy (RPYS) and Algorithmic Historiography: The Bibliometric Reconstruction of András Schubert's Œuvre

Authors: Loet Leydesdorff, Lutz Bornmann, Jordan Comins, Werner Marx, Andreas Thor

Abstract: Referenced Publication Year Spectroscopy (RPYS) was recently introduced as a method to analyze the historical roots of research fields and groups or institutions. RPYS maps the distribution of the publication years of the cited references in a document set. In this study, we apply this methodology to the œuvre of an individual researcher on the occasion of a Festschrift for András Schubert's 70th… ▽ More Referenced Publication Year Spectroscopy (RPYS) was recently introduced as a method to analyze the historical roots of research fields and groups or institutions. RPYS maps the distribution of the publication years of the cited references in a document set. In this study, we apply this methodology to the œuvre of an individual researcher on the occasion of a Festschrift for András Schubert's 70th birthday. We discuss the different options of RPYS in relation to one another (e.g. Multi-RPYS), and in relation to the longer-term research program of algorithmic historiography (e.g., HistCite) based on Schubert's publications (n=172) and cited references therein as a bibliographic domain in scientometrics. Main path analysis and Multi-RPYS of the citation network are used to show the changes and continuities in Schubert's intellectual career. Diachronic and static decomposition of a document set can lead to different results, while the analytically distinguishable lines of research may overlap and interact over time, and intermittent. △ Less

Submitted 16 April, 2016; originally announced April 2016.

Comments: Leydesdorff, L., Bornmann, L., Comins, J. A., Marx, W., & Thor, A. (2016). Referenced Publication Year Spectrography (RPYS) and Algorithmic Historiography: A Bibliometric Reconstruction of András Schubert's Œuvre. In W. Glänzel & B. Schlemmer (Eds.), András Schubert--A World of Models and Metrics (pp. 79-96). Louvain: ISSI

Construction of a Pragmatic Base Line for Journal Classifications and Maps Based on Aggregated Journal-Journal Citation Relations

Authors: Loet Leydesdorff, Lutz Bornmann, ** Zhou

Abstract: A number of journal classification systems have been developed in bibliometrics since the launch of the Citation Indices by the Institute of Scientific Information (ISI) in the 1960s. These systems are used to normalize citation counts with respect to field-specific citation patterns. The best known system is the so-called "Web-of-Science Subject Categories" (WCs). In other systems papers are clas… ▽ More A number of journal classification systems have been developed in bibliometrics since the launch of the Citation Indices by the Institute of Scientific Information (ISI) in the 1960s. These systems are used to normalize citation counts with respect to field-specific citation patterns. The best known system is the so-called "Web-of-Science Subject Categories" (WCs). In other systems papers are classified by algorithmic solutions. Using the Journal Citation Reports 2014 of the Science Citation Index and the Social Science Citation Index (n of journals = 11,149), we examine options for develo** a new system based on journal classifications into subject categories using aggregated journal-journal citation data. Combining routines in VOSviewer and Pajek, a tree-like classification is developed. At each level one can generate a map of science for all the journals subsumed under a category. Nine major fields are distinguished at the top level. Further decomposition of the social sciences is pursued for the sake of example with a focus on journals in information science (LIS) and science studies (STS). The new classification system improves on alternative options by avoiding the problem of randomness in each run that has made algorithmic solutions hitherto irreproducible. Limitations of the new system are discussed (e.g. the classification of multi-disciplinary journals). The system's usefulness for field-normalization in bibliometrics should be explored in future studies. △ Less

Submitted 21 July, 2016; v1 submitted 10 April, 2016; originally announced April 2016.

Comments: accepted for publication in the Journal of Informetrics, 20 July 2016

Citations: Indicators of Quality? The Impact Fallacy

Authors: Loet Leydesdorff, Lutz Bornmann, Jordan Comins, Staša Milojević

Abstract: We argue that citation is a composed indicator: short-term citations can be considered as currency at the research front, whereas long-term citations can contribute to the codification of knowledge claims into concept symbols. Knowledge claims at the research front are more likely to be transitory and are therefore problematic as indicators of quality. Citation impact studies focus on short-term c… ▽ More We argue that citation is a composed indicator: short-term citations can be considered as currency at the research front, whereas long-term citations can contribute to the codification of knowledge claims into concept symbols. Knowledge claims at the research front are more likely to be transitory and are therefore problematic as indicators of quality. Citation impact studies focus on short-term citation, and therefore tend to measure not epistemic quality, but involvement in current discourses in which contributions are positioned by referencing. We explore this argument using three case studies: (1) citations of the journal Soziale Welt as an example of a venue that tends not to publish papers at a research front, unlike, for example, JACS; (2) Robert Merton as a concept symbol across theories of citation; and (3) the Multi-RPYS ("Multi-Referenced Publication Year Spectroscopy") of the journals Scientometrics, Gene, and Soziale Welt. We show empirically that the measurement of "quality" in terms of citations can further be qualified: short-term citation currency at the research front can be distinguished from longer-term processes of incorporation and codification of knowledge claims into bodies of knowledge. The recently introduced Multi-RPYS can be used to distinguish between short-term and long-term impacts. △ Less

Submitted 21 July, 2016; v1 submitted 28 March, 2016; originally announced March 2016.

Comments: accepted for publication in Frontiers in Research Metrics and Analysis; doi: 10.3389/frma.2016.00001

Economic and Technological Complexity: A Model Study of Indicators of Knowledge-based Innovation Systems

Authors: Inga Ivanova, Oivind Strand, Duncan Kushnir, Loet Leydesdorff

Abstract: The Economic Complexity Index (ECI; Hidalgo & Hausmann, 2009) measures the complexity of national economies in terms of product groups. Analogously to ECI, a Patent Complexity Index (PatCI) can be developed on the basis of a matrix of nations versus patent classes. Using linear algebra, the three dimensions: countries, product groups, and patent classes can be combined into a measure of "Triple He… ▽ More The Economic Complexity Index (ECI; Hidalgo & Hausmann, 2009) measures the complexity of national economies in terms of product groups. Analogously to ECI, a Patent Complexity Index (PatCI) can be developed on the basis of a matrix of nations versus patent classes. Using linear algebra, the three dimensions: countries, product groups, and patent classes can be combined into a measure of "Triple Helix" complexity (THCI) including the trilateral interaction terms between knowledge production, wealth generation, and (national) control. THCI can be expected to capture the extent of systems integration between the global dynamics of markets (ECI) and technologies (PatCI) in each national system of innovation. We measure ECI, PatCI, and THCI during the period 2000-2014 for the 34 OECD member states, the BRICS countries, and a group of emerging and affiliated economies (Argentina, Hong Kong, Indonesia, Malaysia, Romania, and Singapore). The three complexity indicators are correlated between themselves; but the correlations with GDP per capita are virtually absent. Of the world's major economies, Japan scores highest on all three indicators, while China has been increasingly successful in combining economic and technological complexity. We could not reproduce the correlation between ECI and average income that has been central to the argument about the fruitfulness of the economic complexity approach. △ Less

Submitted 7 December, 2016; v1 submitted 7 February, 2016; originally announced February 2016.

Journal ref: Technological Forecasting and Social Change 120 (July 2017) 77-89

RPYS i/o: A web-based tool for the historiography and visualization of citation classics, slee** beauties, and research fronts

Authors: Jordan A. Comins, Loet Leydesdorff

Abstract: Reference Publication Year Spectroscopy (RPYS) and Multi-RPYS provide algorithmic approaches to reconstructing the intellectual histories of scientific fields. With this brief communication, we describe a technical advancement for develo** research historiographies by introducing RPYS i/o, an online tool for performing standard RPYS and Multi-RPYS analyses interactively (at http://comins.leydesd… ▽ More Reference Publication Year Spectroscopy (RPYS) and Multi-RPYS provide algorithmic approaches to reconstructing the intellectual histories of scientific fields. With this brief communication, we describe a technical advancement for develo** research historiographies by introducing RPYS i/o, an online tool for performing standard RPYS and Multi-RPYS analyses interactively (at http://comins.leydesdorff.net/). The tool enables users to explore seminal works underlying a research field and to plot the influence of these seminal works over time. This suite of visualizations offers the potential to analyze and visualize the myriad of temporal dynamics of scientific influence, such as citation classics, slee** beauties, and the dynamics of research fronts. We demonstrate the features of the tool by analyzing--as an example--the references in documents published in the journal Philosophy of Science. △ Less

Submitted 9 February, 2016; v1 submitted 5 February, 2016; originally announced February 2016.

Introducing CitedReferencesExplorer (CRExplorer): A program for Reference Publication Year Spectroscopy with Cited References Standardization

Authors: Andreas Thor, Werner Marx, Loet Leydesdorff, Lutz Bornmann

Abstract: We introduce a new tool - the CitedReferencesExplorer (CRExplorer, www.crexplorer.net) - which can be used to disambiguate and analyze the cited references (CRs) of a publication set downloaded from the Web of Science (WoS). The tool is especially suitable to identify those publications which have been frequently cited by the researchers in a field and thereby to study for example the historical r… ▽ More We introduce a new tool - the CitedReferencesExplorer (CRExplorer, www.crexplorer.net) - which can be used to disambiguate and analyze the cited references (CRs) of a publication set downloaded from the Web of Science (WoS). The tool is especially suitable to identify those publications which have been frequently cited by the researchers in a field and thereby to study for example the historical roots of a research field or topic. CRExplorer simplifies the identification of key publications by enabling the user to work with both a graph for identifying most frequently cited reference publication years (RPYs) and the list of references for the RPYs which have been most frequently cited. A further focus of the program is on the standardization of CRs. It is a serious problem in bibliometrics that there are several variants of the same CR in the WoS. In this study, CRExplorer is used to study the CRs of all papers published in the Journal of Informetrics. The analyses focus on the most important papers published between 1980 and 1990. △ Less

Submitted 16 February, 2016; v1 submitted 6 January, 2016; originally announced January 2016.

Comments: Accepted for publication in the Journal of Informetrics

Identification of long-term concept-symbols among citations: Can documents be clustered in terms of common intellectual histories?

Authors: Jordan A. Comins, Loet Leydesdorff

Abstract: "Citation classics" are not only highly cited, but also cited during several decades. We test whether the peaks in the spectrograms generated by Reference Publication Years Spectroscopy (RPYS) indicate such long-term impact by comparing across RPYS for subsequent time intervals. Multi-RPYS enables us to distinguish between short-term citation peaks at the research front that decay within ten years… ▽ More "Citation classics" are not only highly cited, but also cited during several decades. We test whether the peaks in the spectrograms generated by Reference Publication Years Spectroscopy (RPYS) indicate such long-term impact by comparing across RPYS for subsequent time intervals. Multi-RPYS enables us to distinguish between short-term citation peaks at the research front that decay within ten years versus historically constitutive (long-term) citations that function as concept symbols (Small, 1978). Using these constitutive citations, one is able to cluster document sets (e.g., journals) in terms of intellectually shared histories. We test this premise by clustering 40 journals in the Web of Science Category of Information and Library Science using multi-RPYS. It follows that RPYS can not only be used for retrieving roots of sets under study (cited), but also for algorithmic historiography of the citing sets. Significant references are historically rooted symbols among other citations that function as currency. △ Less

Submitted 3 January, 2016; originally announced January 2016.

A Triple Helix Model of Medical Innovation: Supply, Demand, and Technological Capabilities in terms of Medical Subject Headings

Authors: Alexander M. Petersen, Daniele Rotolo, Loet Leydesdorff

Abstract: We develop a model of innovation that enables us to trace the interplay among three key dimensions of the innovation process: (i) demand of and (ii) supply for innovation, and (iii) technological capabilities available to generate innovation in the forms of products, processes, and services. Building on triple helix research, we use entropy statistics to elaborate an indicator of mutual informatio… ▽ More We develop a model of innovation that enables us to trace the interplay among three key dimensions of the innovation process: (i) demand of and (ii) supply for innovation, and (iii) technological capabilities available to generate innovation in the forms of products, processes, and services. Building on triple helix research, we use entropy statistics to elaborate an indicator of mutual information among these dimensions that can provide indication of reduction of uncertainty. To do so, we focus on the medical context, where uncertainty poses significant challenges to the governance of innovation. We use the Medical Subject Headings (MeSH) of MEDLINE/PubMed to identify publications classified within the categories "Diseases" (C), "Drugs and Chemicals" (D), "Analytic, Diagnostic, and Therapeutic Techniques and Equipment" (E) and use these as knowledge representations of demand, supply, and technological capabilities, respectively. Three case-studies of medical research areas are used as representative 'entry perspectives' of the medical innovation process. These are: (i) human papilloma virus, (ii) RNA interference, and (iii) magnetic resonance imaging. We find statistically significant periods of synergy among demand, supply, and technological capabilities (C-D-E) that point to three-dimensional interactions as a fundamental perspective for the understanding and governance of the uncertainty associated with medical innovation. Among the pairwise configurations in these contexts, the demand-technological capabilities (C-E) provided the strongest link, followed by the supply-demand (D-C) and the supply-technological capabilities (D-E) channels. △ Less

Submitted 4 January, 2016; v1 submitted 22 December, 2015; originally announced December 2015.

Comments: Accepted for publication in Research Policy (in press)

Journal ref: Research Policy 45(3), 666-681 (2016)

The Globalization of Academic Entrepreneurship? The Recent Growth (2009-2014) in University Patenting Decomposed

Authors: Loet Leydesdorff, Henry Etzkowitz, Duncan Kushnir

Abstract: The contribution of academia to US patents has become increasingly global. Following a pause, with a relatively flat rate, from 1998 to 2008, the long-term trend of university patenting rising as a share of all patenting has resumed, driven by the internationalization of academic entrepreneurship and the persistence of US university technology transfer. We disaggregate this recent growth in univer… ▽ More The contribution of academia to US patents has become increasingly global. Following a pause, with a relatively flat rate, from 1998 to 2008, the long-term trend of university patenting rising as a share of all patenting has resumed, driven by the internationalization of academic entrepreneurship and the persistence of US university technology transfer. We disaggregate this recent growth in university patenting at the US Patent and Trademark Organization (USPTO) in terms of nations and patent classes. Foreign patenting in the US has almost doubled during the period 2009-2014, mainly due to patenting by universities in Taiwan, Korea, China, and Japan. These nations compete with the US in terms of patent portfolios, whereas most European countries--with the exception of the UK--have more specific portfolios, mainly in the bio-medical fields. In the case of China, Tsinghua University holds 63% of the university patents in USPTO, followed by King Fahd University with 55.2% of the national portfolio. △ Less

Submitted 14 December, 2015; originally announced December 2015.