Linking corpus characteristics to performance of semantic annotation systems for biosystematic descriptions

Digitizing and repurposing taxonomic descriptions of living organisms is an urgent task facing biodiversity informatics researchers. Semantic annotation is the essential technology that makes taxonomic descriptions' reuse and repurpose possible. However, annotation systems performance often vary by collections. Given large content and structural variations inherent in different collections of taxonomic descriptions, this paper looks into corpus characteristic measures in an attempt to establish a performance prediction model which, when given a small set of samples, predicts a system's performance for a collection. The predication model helps deepen our understanding of strengths and weaknesses of an annotation system, but more importantly provides a valuable decision-making tool for end users. We started this research by using MARTT (Markuper for Taxonomic Treatments) system as a base. Although an universal performance predication model for all systems and all corpora may not be possible at this time, we hope more and more individual systems would offer such tools as a regular component in their delivery package.