In recent years, named entity linking (NEL) tools were primarily developed as general approaches, whereas today numerous tools are focusing on specific domains such as e.g. the mapping of persons and organizations only, or the annotation of locations or events in microposts. However, the available benchmark datasets used for the evaluation of NEL tools do not reflect this focalizing trend. We have analyzed the evaluation process applied in the NEL benchmarking framework GERBIL [16] and its benchmark datasets. Based on these insights we extend the GERBIL framework to enable a more fi ne grained evaluation and in deep analysis of the used benchmark datasets according to diff erent emphases. In this paper, we present the implementation of an adaptive fi lter for arbitrary entities as well as a system to automatically measure benchmark dataset properties, such as the extent of content-related ambiguity and diversity. The implementation as well as a result visualization are integrated in the publicly available GERBIL framework.
Henrik Jürges is master student in computer science at University of Potsdam. He is also student co-worker at the research group for Semantic Technologies and Multimedia Retrieval at Hasso Plattner Institute for IT-Systems Engineering. He earned his bachelor degree in computer science in July 2016 at the University of Potsdam. The presented talk at Semantics 2016 is an excerpt of his bachelor thesis.
