Multimodal Learning and the SHOSLIF Approach

Comprehensive sensory learning is the treatment of theories and techniques for computer systems to automatically learn to understand comprehensive visual, auditory and other sensory information with minimal human-imposed rules about the world. The concept of comprehensive learning here implies two coverages: comprehensive coverage of the sensory world and comprehensive coverage of the recognition algorithm. The SHOSLIF is a framework aiming to provide a unified theory and methodology for comprehensive sensor-actuator learning. Its objective is not just to attack a particular sensory or actuator problem, but a variety of such problems. It addresses critical problems such as how to automatically select the most useful features; how to automatically organize sensory and control information using a coarse-to-fine space partition tree which results in a very low, logarithmic time complexity for content-based retrieving from a large visual knowledge base, how to handle invariance based on learning, how to enable on-line incremental learning, how to conduct autonomous learning etc.

SHOSLIF in AAAI97 robot exhibition.


J. Weng, ``Cresceptron and SHOSLIF: Toward Comprehensive Visual Learning,'' in S. K. Nayar and T. Poggio (eds.), Early Visual Learning, Oxford University Press, New York, pp. 183 - 214, 1996. Click here to down load the paper (PostScript).
J. Weng, ``SHOSLIF: A framework for sensor-based learning for high-dimensional complex systems,'' invited paper in Proc. IEEE Workshop on Architectures for Semiotic Modeling and situation analysis in Large Complex Systems, Monterey, CA, Aug. 27-29, 1995. Click here to down load the paper (PostScript).
J. Weng, ``On comprehensive visual learning,'' invited paper in Proc. NSF/ARPA Workshop on Performance vs. Methodology in Computer Vision, Seattle, WA, pp. 152-166, June 24-25, 1994. Click here to down load the paper (PostScript).
J. Weng, ``SHOSLIF: A framework for object recognition from images,'' invited paper in Proc. IEEE International Conference on Neural Networks, Orlando, FL, pp. 4204-4209, June 28 - July 2, 1994.
J. Weng, ``SHOSLIF: A Learning System for Vision and Control,'' invited paper in Proc. IEEE Annual Workshop on Architectures for Intelligent Control Systems, Columbus, Ohio, August 16, 1994.
 Back To Weng's Home Page: