Monitoring the broader impact of the journal publication output on institutional level

A case study for the University of Vienna

Abstract

This study serves as a monitoring practice example concerning the broader impact of journal publication output on institutional level. All publications affiliated to the University of Vienna within the timeframe 2014-2016 and indexed in Web of Science Core Collection were analysed in PlumX. Furthermore, the most current and active knowledge areas were identified and analysed by means of different metrics and their evolution according to their publication year. Finally, we explored potential correlations: first, between same metrics collected in different data sources, and second, between different metrics collected in the same data sources.  The obtained results are very similar to those already gained on country level in a previous study and illustrate the prominent role of the University of Vienna among all organisations contributing to the Austrian scientific output. They also reinforce the importance of usage metrics particularly in the Arts & Humanities. The percentage of publications with social media scores (Total Social Media), especially tweets, has significantly increased within the three reported publication years. The highest values are observed for the Health and Life Sciences, followed by Engineering & Technology and the Social Sciences. The relative insignificance in the Arts & Humanities is noteworthy.  Finally, our study shows very low correlation values between the different measures traced in PlumX and supports the hypothesis that these should rather be considered as complementary sources. Observed correlations between different metrics should be taken with a pinch of salt, due to their different obsolescence patterns.  From a technical point of view, PlumX has proven to be a very useful tool in order to monitor the broader impact of the publication output at institution level. Unfortunately, PlumX does not offer the possibility to select different measuring windows. Therefore, temporal monitoring currently only works by archiving obtained results and a later comparison of different time intervals. Apart from surveys monitoring exercises have proven to be the most practicable and state-of-the-art to study the further development and acceptance of these tools by the scholar community, which is particularly interesting for own organisations and countries.