Entropy and Generalization in Feedforward Nets

Citation

Smith L (1992) Entropy and Generalization in Feedforward Nets. In: Aleksander I & Taylor J (eds.) Artificial Neural Networks, 1992: Proceedings of the 1992 International Conference on Artificial Neural Networks (ICANN–92), Brighton, United Kingdom, 4–7 September, 1992. 1992 International Conference on Artificial Neural Networks (ICANN–92), Brighton, 04.09.1992-07.09.1992. Amsterdam, The Netherlands: Elsevier, pp. 999-1002. http://www.sciencedirect.com/science/article/pii/B9780444894885500385

Abstract
Hidden layers in a feedforward net are considered in terms of the entropy of their storage of different training patterns, and in terms of their total information storage capacity. Two forms of generalisation are identified, and the implication for low-resolution output implementations briefly discussed.