Introduction To Continual Learning

With the world changing at a rapid pace, data is now available at unprecedented rates and is continuously changing over time. Therefore, the need for agents to continuously adapt to the ever-changing environment is growing more and more. To illustrate the necessity for an agent to be a lifelong learner, consider the autonomous driving application. The perception models deployed in the car must continuously adapt to not only different weather, lighting and road conditions but also learn new sets of object instances. For instance, a model trained only on Netherlands’ traffic signs, will have to adapt and accurately predict traffic signs that differ in appearance, while also learning new and unseen traffic signs. Training the models from scratch each time a new object must be learned is a resource and time expensive. A more efficient and sustainable approach is to develop models that can adapt and learn the new objects without forgetting the previously learned objects. Further, as the models interact with the environment and make decisions, they should also be able to utilize the feedback from the car and driver to remodel their behavior and continually evolve to become efficient lifelong learners.

Challenges with traditional Deep Neural Network approach

Lifelong learning in the brain

On the other hand, lifelong learning in the brain is enabled by a rich set of neurophysiological principles which enable incremental learning by acquiring, fine-tuning, and transferring knowledge and skills throughout the lifespan [2]. Continual learning in the brain is mediated by twin objectives: learning and memorization [1]. The former task is characterized by the extraction of the statistical structure of the perceived events with the aim to generalize to novel situations.

The latter, conversely, requires the collection of separated episodic-like events. While it is true that we tend to gradually forget previously learned information, only rarely does the learning of novel information catastrophically interfere with consolidated knowledge. This ability to continually learn from a dynamic environment without catastrophic forgetting is a hallmark of human intelligence, currently missing in artificial systems.

Combining human and machine intelligence

This was soon followed by the inculcation of task specialization in the agents, inspired by modular and specialized units in the brain. Later works have further incorporated the replay action in the brain — the reactivation of and relearning on information of past experiences — into the agents. Finally, recent methods have focused on combining these approaches, and extending them to multiple memory systems which focus on learning general-purpose or global information (for e.g. shapes of roads and cars, which are similar everywhere) and task-specific or local information (for e.g. signboard content, which can differ between countries), separately. These methods have achieved impressive performance in continual learning, making them candidates for deployment in critical applications such as in self-driving cars.

Conclusion

References:

[1] Parisi, German I., et al. “Continual lifelong learning with neural networks: A review.” Neural Networks 113 (2019): 54–71.

[2] Lewkowicz, David J. “Early experience and multisensory perceptual narrowing.” Developmental psychobiology 56.2 (2014): 292–315.

[3] Kudithipudi, D., Aguilar-Simon, M., Babb, J. et al. Biological underpinnings for lifelong learning machines. Nat Mach Intell 4, 196–210 (2022). https://doi.org/10.1038/s42256-022-00452-0

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
NavInfo Europe

Helping companies power their future with intelligent solutions in AI, Simulation, Map Data Services, and Cybersecurity.