The word-based regular expressions in computational linguistics

Abstract

In this paper we propose a unified notation for representing rules in the natural language processing (NLP) and information retrieval (IR) systems. Our notation is both flexible and compact and can be used in a wide variety of tasks. This technique was successfully used in different NLP and IR modules such as input text preformatter, sentence boundary disambiguation, set phrase extraction, constraint grammar, rule based part-of-speech tagging, case sensitive tagging, shallow parsing, transformational grammar, noun phrase normalization, uninformative noun phrase filter, word sense disambiguation, anaphora resolution and others.