- Program Chair:
- Sameer Pradhan
This volume contains a description of the CoNLL-2011 Shared Task and the participating systems. This year, the shared task was based on the English portion of OntoNotes 4.0 corpus. The goal was to identify anaphoric mentions - both entities and events - and perform coreference resolution to create clusters of mentions representing the same entity or event in the text.
The OntoNotes data spans five genres and multiple layers of annotation in addition to coreference, including parses, semantic roles, word sense, and named entities, making it a rich and diverse corpus. One of the challenges for the shared task participants (though they were limited by the time constraints of the task) and also for continuing research going forward is to find effective ways to bring these multiple layers of information to bear on the coreference task to improve upon the current state of the art.
As is traditional with CoNLL, we had two tracks - an open and a closed track. Since world knowledge is an important factor in coreference resolution, even in the closed task participants were allowed to use some limited, outside sources, including WordNet and a pre-computed table predicting number and gender information for noun phrases. For the open task, as usual, participants were allowed to use any other source of information, such as Wikipedia, gazetteers, etc., that did not violate the evaluation criteria designed to protect the test set.
A total of 23 participants submitted system outputs and 21 of them submitted system description papers. We hope that this data set will provide a useful benchmark and spur further research in this important sub-field of language processing.
Proceeding Downloads
CoNLL-2011 shared task: modeling unrestricted coreference in OntoNotes
The CoNLL-2011 shared task involved predicting coreference using OntoNotes data. Resources in this field have tended to be limited to noun phrase coreference, often on a restricted set of entities, such as ace entities. OntoNotes provides a large-scale ...
Stanford's multi-pass sieve coreference resolution system at the CoNLL-2011 shared task
This paper details the coreference resolution system submitted by Stanford at the CoNLL-2011 shared task. Our system is a collection of deterministic coreference resolution models that incorporate lexical, syntactic, semantic, and discourse information. ...
RelaxCor participation in CoNLL shared task on coreference resolution
This paper describes the participation of RelaxCor in the CoNLL-2011 shared task: "Modeling Unrestricted Coreference in Ontonotes". RELAXCOR is a constraint-based graph partitioning approach to coreference resolution solved by relaxation labeling. The ...
Inference protocols for coreference resolution
This paper presents Illinois-Coref, a system for coreference resolution that participated in the CoNLL-2011 shared task. We investigate two inference methods, Best-Link and All-Link, along with their corresponding, pairwise and structured, learning ...
Exploring lexicalized features for coreference resolution
In this paper, we describe a coreference solver based on the extensive use of lexical features and features extracted from dependency graphs of the sentences. The solver uses Soon et al. (2001)'s classical resolution algorithm based on a pairwise ...
Rule and tree ensembles for unrestricted coreference resolution
In this paper, we describe a machine learning system based on rule and tree ensembles for unrestricted coreference resolution. We use Entropy Guided Transformation Learning (ETL) and Decision Trees as the base learners, and, respectively, ETL Committee ...
Unrestricted coreference resolution via global hypergraph partitioning
We present our end-to-end coreference resolution system, COPA, which implements a global decision via hypergraph partitioning. In constrast to almost all previous approaches, we do not rely on separate classification and clustering steps, but perform ...
Multi-metric optimization for coreference: the UniTN/IITP/Essex submission to the 2011 CoNLL Shared Task
Because there is no generally accepted metric for measuring the performance of anaphora resolution systems, a combination of metrics was proposed to evaluate submissions to the 2011 CONLL Shared Task (Pradhan et al., 2011). We investigate therefore ...
Combining syntactic and semantic features by SVM for unrestricted coreference resolution
The paper presents a system for the CoNLL-2011 share task of coreference resolution. The system composes of two components: one for mentions detection and another one for their coreference resolution. For mentions detection, we adopted a number of ...
Supervised coreference resolution with SUCRE
In this paper we present SUCRE (Kobdani and Schütze, 2010) that is a modular coreference resolution system participating in the CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNote (Pradhan et al., 2011). The SUCRE's modular ...
ETS: an error tolerable system for coreference resolution
This paper presents our error tolerable system for coreference resolution in CoNLL-2011 (Pradhan et al., 2011) shared task (closed track). Different from most previous reported work, we detect mention candidates based on packed forest instead of single ...
An incremental model for coreference resolution with restrictive antecedent accessibility
We introduce an incremental model for coreference resolution that competed in the CoNLL 2011 shared task (open regular). We decided to participate with our baseline model, since it worked well with two other datasets. The benefits of an incremental over ...
Narrative schema as world knowledge for coreference resolution
In this paper we describe the system with which we participated in the CoNLL-2011 Shared Task on modelling coreference. Our system is based on a cluster-ranking model proposed by Rahman and Ng (2009), with novel semantic features based on recent ...
Hybrid approach for coreference resolution
This paper describes our participation in the CoNLL-2011 shared task for closed task. The approach used combines refined salience measure based pronominal resolution and CRFs for non-pronominal resolution. In this work we also use machine learning based ...
Poly-co: a multilayer perceptron approach for coreference detection
This paper presents the coreference resolution system Poly-co submitted to the closed track of the CoNLL-2011 Shared Task. Our system integrates a multilayer perceptron classifier in a pipeline approach. We describe the heuristic used to select the ...
Mention detection: heuristics for the OntoNotes annotations
Our submission was a reduced version of the system described in Haghighi and Klein (2010), with extensions to improve mention detection to suit the OntoNotes annotation scheme. Including exact matching mention detection in this shared task added a new ...
Coreference resolution with loose transitivity constraints
Our system treats coreference resolution as an integer linear programming (ILP) problem. Extending Denis and Baldridge (2007) and Finkel and Manning (2008)'s work, we exploit loose transitivity constraints on coreference pairs. Instead of enforcing ...
UBIU: a robust system for resolving unrestricted coreference
In this paper, we discuss the application of UBIU to the CoNLL-2011 shared task on "Modeling Unrestricted Coreference" in OntoNotes. The shared task concentrates on the detection of coreference not only in noun phrases but also involving verbs. The ...
A machine learning-based coreference detection system for OntoNotes
In this paper, we describe the algorithms and experimental results of Brandeis University in the participation of the CoNLL Task 2011 closed track. We report the features used in our system, and describe a novel cluster-based chaining algorithm to ...
Reconciling OntoNotes: unrestricted coreference resolution in OntoNotes with Reconcile
This paper describes our entry to the 2011 CoNLL closed task (Pradhan et al., 2011) on modeling unrestricted coreference in OntoNotes. Our system is based on the Reconcile coreference resolution research platform. Reconcile is a general software ...
Coreference resolution system using maximum entropy classifier
In this paper, we present our supervised learning approach to coreference resolution in ConLL corpus. The system relies on a maximum entropy-based classifier for pairs of mentions, and adopts a rich linguisitically motivated feature set, which mostly ...
Link type based pre-cluster pair model for coreference resolution
This paper presents our participation in the CoNLL-2011 shared task, Modeling Unrestricted Coreference in OntoNotes. Coreference resolution, as a difficult and challenging problem in NLP, has attracted a lot of attention in the research community for a ...