Neural nets for a language processing task: tag disambiguation

Abstract

This paper first shows how part-of-speech tags cen be ambiguous and why it is necessary to disambiguate them. Prototypes which can do this are developed in a limited natural language domain. The representation of syntactic data is discussed. An algorithm to disambiguate tags, using supervised training with a neural net, is presented. The single layer HODYNE net, which takes higher order input, is described and its performance on the processing task examined. Using the simplest text up to 95% of tags can be successfully disambiguated, up to 88% in slightly more complex text. It is shown how altering the language representation and training parameters can affect performance. The results from Hodyne are compared to those obtained from a back propagation net with one hidden layer and found to be comparable, demonstrating that the higher order input data is linearly separable. The work described here shows that there are syntactic patterns in natural language that neural nets can detect, and use for a langiage processing task.