New opportunities for Institutional Repositories: the evaluation challenge. A case study

Susanna Mornati
CILEA

Andrea Bollini
CILEA

     Full text: PDF
     Last modified: October 11, 2007

Abstract
The population of Institutional Repositories (IRs) is one of the most controversial issues within the OAI community. Ideological approaches, based exclusively on the authors. awareness that scholarly communication should become controlled by the scholarly community, have so far proved to be largely unsatisfactory. Relying on open-access deposit mandates from institutions, research funders, governments, looks not entirely appropriate if imposition is the only reason to deposit. To overcome the limits of the present communication system, present stakeholders that get advantages from the their dominant position (e.g. the big publishers) are replaced by stakeholders that get advantages from a reversed situation (e.g. the authors, from a wider circulation of knowledge). But this lies partly beyond the capabilities of any single institution. An easier strategy for change is from inside the system, accepting its rules but acting on its weak points. Change management analysis unfolds two key elements that we may affect: levers and constraints. Both elements are well known in the Open Access (OA) environment: levers have been extensively explored, such as author awareness, spiralling journal costs, etc., while constraints are harder to remove. A main constraint lies in the relation between research evaluation and commercial publishing. In many countries, including Italy, evaluation is based on the ISI Impact Factor (IF), biasing the communication system towards a restricted number of channels that are the most cited journals, mainly belonging to the commercial circuit. Even though in some disciplinary fields the IF cannot be calculated, in others it constitutes the only evaluation parameter. In order to substitute it with more reliable and article-based metrics, these have to be more deeply investigated, particularly as far as their relevance, appropriateness, and usability are concerned. IRs and the software packages on which they are based may become building blocks of a new infrastructure for metrics, a network for the production and analysis of data about usage, cross citations, and any other parameters and indicators that can contribute to assess the quality of a research work. The IR has to be the centre for institutional internal evaluation activities, and consequently the tool to collect evaluation data at the national level, guaranteeing both the governance and control on the institutional research activities and their output, and the capability to undergo external evaluation by exposing institutional records.