Cobertura de fontes secundárias e indicadores de visibilidade: um estudo de caso sobre a produção científica da Universidade Federal do Rio de Janeiro

Abstract

With an increase in scientific production, quantitative indicators such as the number of articles published in specialized journals have become increasingly used in evaluation processes for scientific institutions, which has an impact on funding for projects and scholarships. These indicators are directly influenced by the characteristics of the secondary sources used to calculate them. The aim of this thesis is to investigate the impact of the secondary source on the calculation of visibility indicators for one institution: the Federal University of Rio de Janeiro. The work is a case study and has a quantitative approach. Four multidisciplinary sources were used to select and retrieve UFRJ records: Scopus, Web of Science, Dimensions and Lens. The set of UFRJ documents retrieved from these sources totals 376,281, which were analyzed using the R software (version 4.1.2). We compared, among other characteristics, the coverage of UFRJ's scientific output, as well as visibility indicators (h, e, g, hc and i10 indices) calculated for the four databases. Our results include the development of an R package called biblioverlap, which makes it possible to automate coverage analyses between bibliographic datasets. This program was used in the coverage analysis of the UFRJ data and showed a relatively high degree of overlap in the production retrieved by the sources: 28% of the total documents analyzed are covered by all four sources, a percentage that increases to 36% when only articles are analyzed and to 49% when only highly cited articles are analyzed. This suggests that coverage is less important when choosing the source used in most evaluation scenarios, especially those that focus on articles. The visibility indicators showed considerable variation between sources and each other. It was observed that a greater number of indexed records does not necessarily imply higher indicator values. The set of results suggests that both source and indicator choice can have an impact on the results of an institutional evaluation process. It is therefore imperative that managers and professionals involved in these processes are familiar with secondary sources, their characteristics and their limitations. This is essential for choosing suitable sources and/or indicators for each scenario.