Benchmarking your research community: Setting up an international (reasoning) competition 9/11/16

9th November 2016 at 2pm in Kilburn L.T. 1.5

Benchmarking your research community: Setting up an international (reasoning) competition. 
Speaker: Nicolas Matentzoglu.

The OWL Reasoner Evaluation (ORE) Competition is an annual competition (with an associated workshop) which pits OWL 2 compliant reasoning systems against each other on various standard reasoning tasks over naturally occurring OWL ontologies. In this talk, we outline the design of the competition and present the infrastructure used for its execution: the corpora of ontologies, the competition framework, and the submitted systems. We will share our experiences from inception of the competition through its design to its uptake by the research community and hope to encourage other research communities to pursue the set up of similar events.

The paper describing the ORE resources can be found here: http://link.springer.com/chapter/10.1007/978-3-319-46547-0_17