Direct and joint assessment of Automatic Indexing Systems by Assignment

Abstract

This work proposes a methodological pathway for the direct and joint evaluation of automatic indexing systems by
assignment, as well as applying the proposed methodological pathway in the evaluation of the quality of indexing of the
SISA (Sistema de Indización Semi-Automático) and MAUI (Multi-purpose Automatic Topic Indexing) for automatic indexing
by assignment of scientific articles written in Portuguese in the area of Information Science. The research is exploratory
and empirical, involving a case study and data collection through an indexing experiment. The experiment consists of
the automatic indexing of scientific articles from a reference corpus composed of 60 articles from scientific journals
indexed intellectually using the Brazilian Thesaurus for Information Science as an indexing language. It concludes that the
methodological pathway proved to be adequate and that both systems presented good results, however MAUI stood out for
presenting better performance indexes.

Keywords: Automatic indexing systems. Automatic indexing by assignment. Information Science. Evaluation of automatic
indexing. SISA.