Lifelong-Learning

Survey

• Continual Learning with Neural Networks: A Review. (ACM 2019) [paper]
• Continual Lifelong Learning in Natural Language Processing: A Survey (COLING 2020)[paper]
• Class-incremental learning: survey and performance evaluation (arXiv 2020) [paper]
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (arXiv 2020) [paper]
• Continual learning: A comparative study on how to defy forgetting in classification tasks (arXiv 2019) [paper]
• Continual Lifelong Learning with Neural Networks: A Review (Neural Networks) [paper]

Papers

2022

2021

ICCV2021

• Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process. [paper][poster]
• Continual Prototype Evolution: Learning Online From Non-Stationary Data Streams. [paper]
• RECALL: Replay-Based Continual Learning in Semantic Segmentation. [paper]
• Online Continual Learning With Natural Distribution Shifts: An Empirical Study With Visual Data. [paper]
• Wanderlust: Online Continual Object Detection in the Real World. [paper]
• Co2L: Contrastive Continual Learning. [paper]
• Few-Shot and Continual Learning With Attentive Independent Mechanisms. [paper]
• Structure-From-Sherds: Incremental 3D Reassembly of Axially Symmetric Pots From Unordered and Mixed Fragment Collections. [paper]
• Synthesized Feature Based Few-Shot Class-Incremental Learning on a Mixture of Subspaces. [paper]
• Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning. [paper]
• Striking a Balance Between Stability and Plasticity for Class-Incremental Learning. [paper]

CVPR2021

• PLOP: Learning without Forgetting for Continual Semantic Segmentation. [paper][code]
• Rainbow Memory: Continual Learning with a Memory of Diverse Samples. [paper][code]
• Towards Open World Object Detection(ORE). [paper][code]
• Image De-raining via Continual Learning. [paper][code]
• DER: Dynamically Expandable Representation for Class Incremental Learning. [paper][code]

2020

ICRA2020

• OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning. [paper]

AAAI2020

• Learning on the Job: Online Lifelong and Continual Learning. [paper]
• Lifelong Learning with a Changing Action Set [paper]
• Lifelong Spectral Clustering [paper]
• Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control [paper]
• Bi-objective Continual Learning: Learning New' while Consolidating Known' [paper]
• Learning from the Past: Continual Meta-Learning with Bayesian Graph Neural Networks [paper]
• Residual Continual Learning [paper]

NeurIPS2020

• Meta-Consolidation for Continual Learning (NeurIPS2020) [paper]
• Understanding the Role of Training Regimes in Continual Learning (NeurIPS2020) [paper]
• Continual Learning with Node-Importance based Adaptive Group Sparse Regularization (NeurIPS2020) [paper]
• Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning (NeurIPS2020) [paper]
• Coresets via Bilevel Optimization for Continual Learning and Streaming (NeurIPS2020) [paper]
• RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning (NeurIPS2020) [paper]
• Continual Deep Learning by Functional Regularisation of Memorable Past (NeurIPS2020) [paper]
• Dark Experience for General Continual Learning: a Strong, Simple Baseline (NeurIPS2020) [paper] [code]
• GAN Memory with No Forgetting (NeurIPS2020) [paper]
• Calibrating CNNs for Lifelong Learning (NeurIPS2020) [paper]

BMVC2020

• Initial Classifier Weights Replay for Memoryless Class Incremental Learning (BMVC2020) [paper]

ECCV2020

• Adversarial Continual Learning (ECCV2020) [paper] [code]
• REMIND Your Neural Network to Prevent Catastrophic Forgetting (ECCV2020) [paper] [code]
• Incremental Meta-Learning via Indirect Discriminant Alignment (ECCV2020) [paper]
• Memory-Efficient Incremental Learning Through Feature Adaptation (ECCV2020) [paper]
• PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning (ECCV2020) [paper] [code]
• Reparameterizing Convolutions for Incremental Multi-Task Learning Without Task Interference (ECCV2020) [paper]
• Learning latent representions across multiple data domains using Lifelong VAEGAN (ECCV2020) [paper]
• Online Continual Learning under Extreme Memory Constraints (ECCV2020) [paper]
• Class-Incremental Domain Adaptation (ECCV2020)[paper]
• More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning (ECCV2020) [[paper](http://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123710698.pdf]
• Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation (ECCV2020) [paper]
• GDumb: A Simple Approach that Questions Our Progress in Continual Learning (ECCV2020) [paper]
• Imbalanced Continual Learning with Partitioning Reservoir Sampling (ECCV2020) [paper]
• Topology-Preserving Class-Incremental Learning (ECCV2020) [paper]

CIKM2020

• GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems (CIKM2020)[paper]

IJCNN2020

• OvA-INN: Continual Learning with Invertible Neural Networks (IJCNN2020) [paper]

ICLM2020

• XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning (ICLM2020) [paper]
• Optimal Continual Learning has Perfect Memory and is NP-HARD (ICML2020) [paper]
• Neural Topic Modeling with Continual Lifelong Learning (ICML2020) [paper]

CVPR2020

• Semantic Drift Compensation for Class-Incremental Learning (CVPR2020) [paper] [code]
• Few-Shot Class-Incremental Learning (CVPR2020) [paper]
• Modeling the Background for Incremental Learning in Semantic Segmentation (CVPR2020) [paper]
• Incremental Few-Shot Object Detection (CVPR2020) [paper]
• Incremental Learning In Online Scenario (CVPR2020) [paper]
• Maintaining Discrimination and Fairness in Class Incremental Learning (CVPR2020) [paper]
• Conditional Channel Gated Networks for Task-Aware Continual Learning (CVPR2020) [paper]
• Continual Learning with Extended Kronecker-factored Approximate Curvature (CVPR2020) [paper]
• iTAML : An Incremental Task-Agnostic Meta-learning Approach (CVPR2020) [paper] [code]
• Mnemonics Training: Multi-Class Incremental Learning without Forgetting (CVPR2020) [paper] [code]

WACV2020

• ScaIL: Classifier Weights Scaling for Class Incremental Learning (WACV2020) [paper]

ICLR2020

• Scalable and Order-robust Continual Learning with Additive Parameter Decomposition (ICLR2020) [paper]
• Continual Learning with Adaptive Weights (CLAW) (ICLR2020) [paper]
• Continual Learning with Bayesian Neural Networks for Non-Stationary Data (ICLR2020) [paper]
• Uncertainty-guided Continual Learning with Bayesian Neural Networks (ICLR2020) [paper]
• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning (ICLR2020) [paper]
• Functional Regularisation for Continual Learning with Gaussian Processes (ICLR2020) [paper]
• Continual learning with hypernetworks (ICLR2020) [paper]
• Compositional Continual Language Learning (ICLR2020) [paper]

Natrue Communications 2020

• Brain-inspired replay for continual learning with artificial neural networks (Natrue Communications 2020) [paper] [code]

2019

NeurIPS2019

• Compacting, Picking and Growing for Unforgetting Continual Learning (NeurIPS2019) [paper][code]
• Random Path Selection for Incremental Learning (NeurIPS2019) [paper]
• Online Continual Learning with Maximal Interfered Retrieval (NeurIPS2019) [paper]

ICMR2019

• Increasingly Packing Multiple Facial-Informatics Modules in A Unified Deep-Learning Model via Lifelong Learning (ICMR2019) [paper][code]

Neural Computation 2019

• Towards Training Recurrent Neural Networks for Lifelong Learning (Neural Computation 2019) [paper]

ICCV2019

• IL2M: Class Incremental Learning With Dual Memory (ICCV2019) [paper]
• Incremental Learning Using Conditional Adversarial Networks (ICCV2019) [paper]
• Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild (ICCV2019) [paper]
• Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation (ICCV2019) [paper]
• Lifelong GAN: Continual Learning for Conditional Image Generation (ICCV2019) [paper]

KDD2019

• Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability (KDD2019) [paper]

CVPR2019

• Large Scale Incremental Learning (CVPR2019) [paper] [code]
• Learning a Unified Classifier Incrementally via Rebalancing (CVPR2019) [paper] [code]
• Learning Without Memorizing (CVPR2019) [paper]
• Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning (CVPR2019) [paper]
• Task-Free Continual Learning (CVPR2019) [paper]

Nature Machine Intelligence 2019

• Continual learning of context-dependent processing in neural networks (Nature Machine Intelligence 2019) [paper] [code]

ICML2019

• Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting (ICML2019) [paper]

ICLR2019

• Efficient Lifelong Learning with A-GEM (ICLR2019) [paper] [code]
• Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference (ICLR2019) [paper] [code]
• Overcoming Catastrophic Forgetting via Model Adaptation (ICLR2019) [paper]
• A comprehensive, application-oriented study of catastrophic forgetting in DNNs (ICLR2019) [paper]

2018

NIPS2018

• Memory Replay GANs: learning to generate images from new categories without forgetting (NIPS2018) [paper] [code]
• Reinforced Continual Learning (NIPS2018) [paper] [code]
• Online Structured Laplace Approximations for Overcoming Catastrophic Forgetting (NIPS2018) [paper]

ICPR2018

• Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting (R-EWC) (ICPR2018) [paper] [code]

BMVC2018

• Exemplar-Supported Generative Reproduction for Class Incremental Learning (BMVC2018) [paper] [code]

ECCV2018

• End-to-End Incremental Learning (ECCV2018)[paper][code]
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence (ECCV2018)[paper]
• Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights (ECCV2018) [paper] [code]
• Memory Aware Synapses: Learning what (not) to forget (ECCV2018) [paper] [code]
• Lifelong Learning via Progressive Distillation and Retrospection (ECCV2018) [paper]

CVPR2018

• PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning (CVPR2018) [paper] [code]

ICML2018

• Overcoming Catastrophic Forgetting with Hard Attention to the Task (ICML2018) [paper][code]

ICLR2018

• Lifelong Learning with Dynamically Expandable Networks (ICLR2018) [paper]
• FearNet: Brain-Inspired Model for Incremental Learning (ICLR2018) [paper]

2017

ICCV2017

• Incremental Learning of Object Detectors Without Catastrophic Forgetting (ICCV2017) [paper]
• Encoder Based Lifelong Learning (ICCV2017) [paper]

PNAS2017

• Overcoming catastrophic forgetting in neural networks (EWC) (PNAS2017) [paper] [code][code]

ICML2017

• Continual Learning Through Synaptic Intelligence (ICML2017) [paper] [code]

NIPS2017

• Gradient Episodic Memory for Continual Learning (NIPS2017) [paper] [code]
• Continual Learning with Deep Generative Replay (NIPS2017) [paper] [code]
• Overcoming Catastrophic Forgetting by Incremental Moment Matching (NIPS2017) [paper] [code]

CVPR2017

• iCaRL: Incremental Classifier and Representation Learning (CVPR2017) [paper] [code]
• Expert Gate: Lifelong Learning with a Network of Experts (CVPR2017) [paper]

Workshops

1. CVPR 2022 Workshop on Continual Learning
2. Workshop on Continual Learning at ICML 2020
3. Continual Learning in Computer Vision Workshop CVPR 2020
4. Continual learning workshop NeurIPS 2018

Labs

1. Eric Eaton et al. Department of Computer and Information Science GRASP (General Robotics, Automation, Sensing & Perception) Lab University of Pennsylvania Lifelong Machine Learning [projects]
2. Liu Bing et al. Department of Computer Science University of Illinois at Chicago (UIC). Lifelong machine learning
3. Vincenzo Lomonaco ContinualAI Lab

Challenge or Competitions

1. Lifelong Robotic Vision Challenge IROS 2019
2. Continual Learning in Computer Vision Challenge CVPR 2020

Benchmark Comparisons & Codes

1. Continual-Learning-Benchmark (pytorch)[code]
2. Framework for Analysis of Class-Incremental Learning with 12 state-of-the-art methods and 3 baselines. [code]
3. A continual learning survey: Defying forgetting in classification tasks. (TPAMI) [code]

---

Thanks

The above lists are based on Xialei Liu's Awesome Incremental Learning / Lifelong learning.