Using single layer networks for discrete, sequential data: an example from natural language processing. [extended version]

Abstract

Supervised, feed-forward networks will, in general, need more than one layer to process data. However, if they can be used, single layer networks offer advantages of functional transparency and operational speed. Now, in some cases data can be pre-processed and then presented in a linearly separable form for processing by a single layer net. In effect, processing at different stages can be de-coupled. The critical issue is finding the pre-processing function to convert data into an appropriate form. For characteristic linguistic data this can be done, and a natural language parser which has been successfully implemented is used to investigate the approach. Single layer nets can then be trained by finding weight adjustments based on (a) factors proportional to the input, as in the Perceptron, (b) factors proportional to the existing weights,and (c) an error minimization mathod. In our experiments generalization ability varies little; method (b) has been used for the prototype parser.