SAIL and Dav Developmental Robot Projects:
the Developmental Approach to Machine Intelligence
for integrated vision, audition, touch, speech, language, reasoning, robotics and the mind

SAIL on Oct. 1, 1998	SAIL on Jan. 22, 1999	SAIL on Aug. 7, 2000	Dav on Jan. 11, 2002	Dav on May 30, 2002	Dav on Aug. 24, 2003

SAIL originally stands for Self-organizing Autonomous Incremental Learner.  Dav is a variant of "Development" and it is the second developmental robot after SAIL.  For the availability of Dav, contact weng@cse.msu.edu.

Demonstrations

Video segments for demonstration of SAIL and Dav developmental robots:

• Movie 1: Vision-guided indoor navigation and learning-based motion tracking (supervised learning) (MPEG 36.7MB),
• Movie 2: Simple language acquisition and vision-guided indoor navigation (supervised, reinforcement and communicative learning) (MPEG 48.9 MB),
• Movie 3: Vision-guided outdoor navigation (supervised learning) (MPEG 16.8MB)
• Movie 4: Audition-based action chaining (task transfer) (supervised and communicative learning) (MPEG 29.6MB)
• Movie 5: Integration of vision and audition in a dynamic world via online dialogue (supervised and communicative learning) (MPEG 26.4MB)
• Movie 6: Dav starts its moves (programmed-in behaviors, not learned yet) (MPEG 26MB)
• Movie 7: Drawbridge experiment ("Innate" knowledge about object permanence or perceptual novelty instead?) (MPEG 24MB)
• Movie 8: Learning early speech production (reinforcement learning in high dimensional action and state spaces) (MPEG 35mB)
• Movie 9: Dav performs collision avoidance (online-learning and attention-based approach to obstacle avoidance using range finder) (MPEG 14mB)

Mental development

For humans, the developmental program is in the genes.  It starts to run at the time of conception of each human individual. This program is responsible for whatever can happen through the entire life span of that individual.  For machines, the developmental program starts to run at the ``birth'' time of the machine, which enables the machine to develop its mental skills (including perception, cognition, behaviors and motivation) through direct interactions with its environment using its sensors and effectors.  For machines to truly understand the world, the environment must be the physical world which includes the human teachers and the robot itself.

The concept of a developmental program does not mean just to make machines grow from small to big and from simple to complex. It must enable the machine to learn new tasks that a human programmer does not know about at the time of programming.   This implies that the representation of any task that the robot learns must be generated by the robot itself, a well known holy grail in AI and fundamental for machine understanding.

Development does not mean learning from tabular rasa.  Innate behaviors such as those present at birth can greatly facilitate early mental development.

The basic nature of developmental learning plays a central role in enabling a human being to incrementally scale his or her level of intelligence from the ground up. In order to scale up the machine's capability to understand what happens around it, the learning mechanism embedded in a developmental program must perform systematic self-organization, according to what it sensed, what it did, the actions imposed by the human when necessary, the reward it received from the humans, and the context.  As a fundamental requirement of scaling up, the robot must develop a value system (also called a motivational system).

In contrast with traditional thoughts that artificial intelligence should be studied within a narrow scope and that otherwise the complexity is out of control, the developmental approach aims to provide a broad and unified developmental framework, which is applicable to a wide variety of perceptual capabilities (e.g., vision, audition and touch), cognitive capabilities (e.g., situation awareness, language understanding, reasoning, planning, communication, decision making, task execution), behavioral capabilities (e.g., speaking, dancing, walking, playing music), motivational capabilities (e.g., pain avoidance, pleasure seeking, what is right and what is wrong) and the fusion of these capabilities. By the very nature of autonomous development, a developmental program does not require humans to manually model task-specific representation.   Some recent evidence in neuroscience has suggested that the developmental mechanisms in our brain are probably very similar across different sensing modalities.    This is good news since it means that the task of designing a developmental program is probably more tractable than traditional task-specific programming.

Eight requirements for practical AMD

A developmental robot that is capable of practical  autonomous mental development (AMD) must deal with the following eight requirements:

1. Environmental openness:  Due to the task-nonspecificity, AMD must deal with unknown and uncontrolled environments, including various human environments.
2. High-dimensional sensors: The dimension of a sensor is the number of scalar values per unit time.  AMD must directly deal with continuous raw signals from high-dimensional sensors (e.g., vision, audition and taction).  
3. Completeness in using sensory information.  Due to the environmental openness and task nonspecificity, it is not desirable for a developmental program to discard, at the program design stage, sensory information that may be useful for some future, unknown tasks. Of course, its task-specific representation autonomously derived after birth does discard information that is not useful for a particular task.
4. Online processing:  At each time instant, what the machine will sense next depends on what the machine does now.
5. Real-time speed:  The sensory/memory refreshing rate must be high enough so that each physical event (e.g., motion and speech) can be temporally sampled and processed in real time (e.g., about 15Hz for vision).   This speed must be maintained even when a full (very large but finite) physical "machine brain size''  is used.  It must handle one-instance learning: learning from one instance of experience.
6. Incremental processing:  Acquired skills must be used to assist in the acquisition of new skills, as a form of ``scaffolding.''  This requires incremental processing.  Thus, batch processing is not practical for AMD.  Each new observation must be used to update the current complex representation and the raw sensory data must be discarded after it is used for updating.
7. Perform while learning:  Conventional machines perform after they are built.  An AMD machine must perform while it ``builds'' itself "mentally.''
8. Scale up to large memory: For large perceptual and cognitive tasks, an AMD machine must handle multimodal contexts, large long-term memory and generalization, and capabilities for increasing maturity, all in real time speed. 

Why AMD?

Why do we pursue this developmental approach?  Traditional approaches to machine intelligence require human designers to explicitly program task-specific representation, perception and behaviors, according to the tasks that the machine is supposed to execute.  However, AI tasks require capabilities such as vision, speech, language, and motivation which been proved to be too muddy to program effectively by hand.  Although a developmental program is by no means simple, the new developmental approach does not require human programmers to understand the domain of tasks nor to predict them. Therefore, this approach does not only drastically reduce the programming burden, but also enables machines to develop capabilities or skills that the programmer does not have or are too muddy to be adequately understood by the programmer.

Some Research Issues

We are moving ahead to overcome some major technical challenges that this new field faces.   They include the design of a developmental architecture,  the developmental mechanisms for an artificial cortex, representation in high-dimensional sensory/state/effector space required by distributed representation (known as distributed neuronal population coding in neuroscience), real-time incremental self-organization of very large memory (our memory engine is called IHDR), different levels of sensory and effector integration, and the computational nature of a developing motivational system.   We do not intend to fully duplicate a biological developmental program, since it is impractical.  However, our work is motivated by some known results in neuroscience and psychology.  We are also working on the body of developmental robots that is suited for interactive development in human environments.

Miscellaneous

How long can a developmental robot  live? When the hardware of a developmental robot is worn or broken, the developmental program with its learned "brain" can be downloaded from the robot and uploaded to a new robot body. Therefore, unlike a biological brain, a developmental robot can live "mentally" as long as we humans like.  It can have a very old "mental age" but a very young "body age."

This line of work is supported in part by NSF, DARPA, Microsoft Research, Siemens Corporate Research, GM, and Zyvex.

Here is a tutorial: Autonomous Mental Development for Robots, presented at ICRA 2001 and ICDL 2002.
For an introduction to computational study of mental development by robots and animals, read a paper that appeared in Science.
For importance and predicted impact of this new field, read a white paper from the Workshop on Development and Learning.

Software download

Some related publications:

General philosophy:

General theory:

J. Weng and W. S. Hwang, "From Neural Networks to the Brain: Autonomous Mental Development''  IEEE Computational Intelligence Magazine, vol. 1, no. 3, pp. 15-31, August 2006. PDF file.
J. Weng, "Developmental Robotics: Theory and Experiments''  International Journal of Humanoid Robotics, vol. 1, no. 2, 2004. PDF file.
J. Weng, "A Theory for Mentally Developing Robots,''  in Proc. 2nd International Conference on Development and Learning, June 12 - 15, MIT, Cambridge, MA, IEEE Computer Society Press, 2002. PS file or PDF file.

Task muddiness and performance matrics:

J. Weng, "Muddy Tasks and the Necessity of Autonomous Mental Development," in Proc. 2005 AAAI Spring Symposium Series, Developmental Robotics Symposium, Stanford University, March 21-23, 2005. PDF file.

Overview of related projects at the EI Lab:

J. Weng and Y. Zhang, ``Developmental Robots: A New Paradigm,''  an invited paper in Proc. Second International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, Edinburgh, Scotland, August 10 - 11, 2002.  PDF file.

Architecture:

LCA: Biologically inspired single-layer network:

J. Weng and N. Zhang, ``In-Place Learning and the Lobe Component Analysis,''  in Proc. IEEE World Congress on Computational Intelligence, International Joint Conference on Neural Networks, Vancouver, BC, Canada, July 16-21, 2006. PDF file.
Matlab program.  Program package with test set. 
J. Weng, Y. Zhang and W. Hwang, ``Candid Covariance-free Incremental Principal Component Analysis,'' IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, 2003. PDF file.

MILN: Biologically inspired sensorimotor pathway:

J. Weng, H. Lu, T. Luwang and X. Xue, ``Multilayer In-place Learning Networks for Modeling Functional Layers in the Laminar Cortex''  Neural Networks, vol. 21, no.2-3, pp. 150-159, 2008. PDF file.
J. Weng, H. Lu, T. Luwang and X. Xue, ``A Multilayer In-Place Learning Network for Development of General Invariances,''  International Journal of Humanoid Robotics, vol. 4, no. 2, 2007. PDF file.
J. Weng, H. Lu, T. Luwang and X. Xue, ``In-Place Learning for Positional and Scale Invariance,''  in Proc. IEEE World Congress on Computational Intelligence , International Joint Conference on Neural Networks, Vancouver, BC, Canada, July 16-21, 2006. PDF file.
J. Weng and Matthew D. Luciw, ``Optimal In-Place Self-Organization for Cortical Development: Limited Cells, Sparse Coding and Cortical Topology,''  in Proc. 5th International Conference on Development and Learning (ICDL'06) , Bloomington, IN, USA, May 31 - June 3, 2006. PDF file.

Motivational system, reinforcement learning:

X. Huang and J. Weng, ``Inherent Value Systems for Autonomous Mental Development,'' International Journal of Humanoid Robotics, vol. 4, no. 2, pp. 407-433, 2007.   PDF file.
X. Huang and J. Weng, "Novelty and Reinforcement Learning in the Value System of Developmental Robots,"  in Proc. Second International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, Edinburgh, Scotland, August 10 - 11, 2002. PDF file.

Skill transfer:

Y. Zhang and J. Weng, ``Task Transfer by a Developmental Robot,'' IEEE Transactions on Evolutionary Computation, vol. 11, no. 2, pp. 226-248, 2007. PDF file.
Y. Zhang and J. Weng, ``Action Chaining by a Developmental Robot with a Value System,''  in Proc. 2nd International Conference on Development and Learning, June 12 - 15, MIT, Cambridge, MA, IEEE Computer Society Press, 2002. PDF file.

Sensory mapping engine (early processing for local feature extraction with receptive fields):

N. Zhang, J. Weng and Z. Zhang, ``A Developing Sensory Mapping for Robots,''  in Proc. 2nd International Conference on Development and Learning, June 12 - 15, MIT, Cambridge, MA, IEEE Computer Society Press, 2002. PDF file.
J. Weng and M. D. Luciw, ``Optimal In-Place Self-Organization for Cortical Development: Limited Cells, Sparse Coding and Cortical Topography,''  in Proc. 5th International Conference on Development and Learning, May 30 - June 3, Bloomgton, IN, 2006. PDF file.

Cognitive mapping engine (later processing for high-dimensional regression):

J. Weng and W. Hwang, "Incremental Hierarchical Discriminant Regression,"  IEEE Transactions on Neural Networks,  vol. 18, no. 2, pp. 397-415, 2007.   PDF file.

W. Hwang and J. Weng, "Hierarchical Discriminant Regression'', IEEE Trans. Pattern Analysis and Machine Intelligence, vol.  22, no. 11, pp. 1277-1293, November 2000.  (Batch HDR.) PDF file.

J. Weng and W. Hwang, "An incremental learning algorithm with automatically derived discriminating features'',  in Proc. Asian Conference on Computer Vision,  Taipei, Taiwan, pp. 426 - 431, Jan. 8 - 9, 2000. (Incremental HDR.) PDF file.

J. Weng and W. Hwang, "Online Image Classification Using IHDR,"  International Journal on Document Analysis and Recognition,  vol. 5, no. 2-3, pp. 118-125, 2003.  PDF file.

In situ co-acquisition of audition and speech recognition behavior：

Y. Zhang, J. Weng and W. Hwang, "Auditory Learning: A Developmental Method," IEEE Transactions on Neural Networks, vol. 16, no. 3, pp. 601-616, 2005. PDF file.

In situ co-acquisition of vision, early spoken language and behavior：

Y. Zhang and J. Weng, "Conjunctive Visual and Auditory Development via Real-Time Dialogue,''  in Proc. 3rd International Workshop on Epigenetic Robotics, Boston, MA, pp. 974 - 980, August 4-5, 2003.  PDF file.

Learning to speak (high-dimensional motor space using reinforcement learning):

A. Joshi and J. Weng, "Autonomous mental development in high dimensional context and action spaces,'' Neural Networks, vol. 16, no. 5-6, pp. 701-710, 2003. PDF file.

Application of Developmental Vision to Part Inspection in Manufacturing:

G. Abramovich, J. Weng and D. Dutta, "Adaptive Part Inspection through Developmental Vision,"  Journal of Manufacturing Science and Engineering, vol. 127, no. 4, pp. 846-856, Nov. 2005. PDF file.

The "innate physical world knowledge" issue and multimodal interactions to learn physical world knowledge:

J. Weng, Y. Zhang and Y. Chen, "Developing Early Senses about the World: `Object Permanence' and Visuoauditory Real-time
Learning,'' in Proc. International Joint Conf. on Neural Networks, Portland, OR, pp. 2710 - 2715, July 20-24, 2003.   PDF file.

Dav:

S. Zeng and J. Weng, ``Online-learning and attention-based approach to obstacle avoidance using a range finder," Journal of Intelligent and Robotic Systems, vol. 43, no. 2, June 2005. PDF file.
S. Zeng and J. Weng, ``Obstacle Avoidance through Incremental Learning with Attention Selection,''  in Proc. IEEE Int'l Conf. on Robotics and Automation, New Orleans, Louisiana, pp. 115-121, April 26 - May 1, 2004. PDF file.
J. D. Han, S. W. Zeng, K. Y. Tham, M. Badgero and J. Weng, ``Dav: A Humanoid Robot Platform for Autonomous Mental Development,''  in Proc. 2nd International Conference on Development and Learning, June 12 - 15, MIT, Cambridge, MA, IEEE Computer Society Press, 2002. PDF file.

SAIL-2:

Y. Zhang and J. Weng, ``Grounded Auditory Development by a Developmental Robot,''  in Proc. INNS/IEEE International Joint Conference of Neural Networks 2001 (IJCNN 2001), Washington DC, pp. 1059-1064,  July 14-19, 2001. PDF file.

J. Weng, W. S. Hwang, Y. Zhang, C. Yang and R. Smith, ``Developmental Humanoids: Humanoids that Develop Skills Automatically,''  in the Proc. the first IEEE-RAS International Conference on Humanoid Robots, Cambridge, MIT, Sept. 7-8, 2000. PDF file.

J. Weng, C. H. Evans and W. S. Hwang, ``An Incremental Learning Method for Face Recognition under Continuous Video Stream,'' in Proc. Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France. March 28 - 30, 2000. PDF file.

J. Weng, W. S. Hwang, Y. Zhang and C. Evans, ``Developmental robots: Theory, Method and Experimental Results,'' in Proc. 2nd Int'l Symposium on Humanoid Robots, Tokyo, Japan, pp. 57- 64, Oct. 8- 9, 1999. PDF file.

SAIL-1:

J. Weng, C. Evans, W. S. Hwang, and Y. B. Lee, ``The Developmental Approach to Artificial Intelligence: Concepts, Developmental Algorithms and Experimental Results'' in Proc. NSF Design & Manufacturing Grantees Conference, , Queen Mary, Long Beach, CA, Jan 5-8, 1999. Also, Technical Report MSU-CPS-98-25, July, 1998. PDF.

J. Weng, Y. B. Lee and C. H. Evans, ``The developmental approach to multimedia speech learning,'' in Proc. IEEE Int'l Conf. on Acoustics, Speech, and Signal Processing, Phoenix, Arizona, vol. 6, pp. 3093 - 3096, March 15 - 19, 1999. PDF file.

To Weng's Home Page: http://www.cse.msu.edu/~weng/