Learning-Based Named Entity Recognition for Morphologically-Rich, Resource-Scarce Languages

Kazi Saidul Hasan, Md. Altaf ur Rahman and Vincent Ng.
Proceedings of the Twelfth Conference of the European Chapter of the Association for Computational Linguistics, pp. 354-362, 2009.

Click here for the PostScript or PDF version. The talk slides are available here.

Abstract

Named entity recognition for morphologically rich, case-insensitive languages, including the majority of semitic languages, Iranian languages, and Indian languages, is inherently more difficult than its English counterpart. Worse still, progress on machine learning approaches to named entity recognition for many of these languages is currently hampered by the scarcity of annotated data and the lack of an accurate part-of-speech tagger. While it is possible to rely on manually-constructed gazetteers to combat data scarcity, this gazetteer-centric approach has the potential weakness of creating irreproducible results, since these name lists are not publicly available in general. Motivated in part by this concern, we present a learning-based named entity recognizer that does not rely on manually-constructed gazetteers, using Bengali as our representative resource-scarce, morphologically-rich language. Our recognizer achieves a relative improvement of 7.5% in F-measure over a baseline recognizer. Improvements arise from (1) using induced affixes, (2) extracting information from online lexical databases, and (3) jointly modeling part-of-speech tagging and named entity recognition.

BibTeX entry

@InProceedings{Hasan+Rahman+Ng:09a,
  author = {Hasan, Kazi Saidul and Altaf Rahman and Vincent Ng},
  title = {Learning-Based Named Entity Recognition for Morphologically-Rich, Resource-Scarce Languages},
  booktitle = {Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics},
  pages = {354--362},
  year = 2009
}