Incremental learning is related to continual learning, lifelong learning, transfer learning, multitask learning, and never-ending learning. These algorithms' main goal is not to forget the previously learned task and learn increasingly over time. Incremental learning is, therefore, an important area in the study of machine learning, as there is an increasing number of demands for systems that constantly update the information acquired, yet preserve the prior knowledge. This demand has sparked research on various learning paradigms, such as task-incremental learning, class-incremental learning, and domain-incremental learning [1].

In the domain of image processing, class incremental learning has been researched, and numerous methods have been proposed. Researchers have proposed several strategies to mitigate catastrophic forgetting, which can be broadly categorized into the following techniques: 1) parameter regularization, 2) knowledge distillation, 3) exemplar-based methods, and 4) synthetic data generation. In parameter regularization methods, the task knowledge is preserved by penalizing updates to parameters deemed important for earlier tasks. Focus on preserving the knowledge of previous tasks by penalizing significant updates to parameters deemed important for earlier tasks. Elastic Weight Consolidation (EWC) [1], [2] exploits the Fisher information matrix to calculate the importance of the parameters, and significant changes are prevented through regularization. Synaptic Intelligence (SI) [3] calculates a measure of the contribution of the parameters to performance on each task as training progresses, allowing for selective updates of less critical weights. Learning without Forgetting (LwF) [4] penalizes deviations in earlier tasks’ predictions when training on new ones. Knowledge distillation is the most broadly adopted technique in class-incremental learning frameworks. The “teacher-student” model architecture is employed. Here, the teacher is learned from earlier tasks, and such distilled knowledge from the teacher is used as guidance for the student, which is learned explicitly for newer tasks without forgetting previously gained knowledge. Logit-based distillation [5] allows aligning the output probability of the teacher and the student models to retain knowledge. Feature-level distillation [6] ensures that the student learns intermediate feature representations like the teacher. Exemplar-based methods concentrate on a small memory buffer that maintains exemplars from previous tasks, and these exemplars are replayed while learning new classes. The Incremental Classifier and Representation Learning (iCaRL) [7] method addresses the forgetting challenge by employing a nearest-mean-of-exemplars classification strategy that dynamically selects exemplars from incoming data, allowing the model to maintain performance across observed classes while learning new information. Herding strategies are based on the selection of exemplars based on their proximity to the class mean in feature space; hence, it is representative. Another alternative for storing real exemplars is synthetic data generation by exploiting generative models such as GANs [6] or VAEs [8] for simulating data distributions from prior tasks.

This paper [9] presents a domain-incremental learning that could minimize catastrophic forgetting through the utilization of Random Vector Functional Link (RVFL) networks. RVFL is a lightweight neural network with fewer parameters but more computationally intense, hence its success in incremental learning settings. An approach to the core of Gram Matrix accumulation could be a way of retaining the previous knowledge learned without requiring access to past data. Important information regarding the previous domains is thus being encoded by the Gram Matrix, which enables the model to learn new domains while conserving performance on earlier ones. This paper compares the proposed method’s performance with the joint training approach.

Another work [10] combines Bi-directional Guided Modulation (BGM) and meta-learning principles. The model is trained first on a large corpus of base classes to provide a robust starting point for recognition while getting the model ready for incremental sessions. To have the network be both plastic and stable, BGM presents a modulation network that computes gating masks in order to modulate activations throughout the classification network. These masks are guided through attention maps drawn from the weights of the network, to maintain significant parameters for previously acquired tasks when accommodating new classes. A meta-learning framework based on Model-Agnostic Meta-Learning (MAML) is employed for simulating incremental learning during training. The MAML is through bi-level optimization. The meta-learning framework has produced promising results on benchmark datasets such as CIFAR100, Mini-ImageNet, and CUB200. However, it can be prone to overfitting on particular task orders and depends on a known number of classes and examples.

The paper [11] has merged a variety of approaches: Knowledge distillation, exemplar management, and attention mechanisms to retain all knowledge learned before acquiring new tasks. Knowledge distillation techniques align feature representations of the current model with those of the previous model, weighted by importance masks that weigh critical temporal and channel dimensions in feature maps. Furthermore, exemplar selection strategies, such as the herding approach, are utilized to maintain representative samples in ways to approximate class distributions without excessive memory usage. In an effort to further enhance feature distinctiveness and counter representation drift, orthogonality regularization is applied to ensure that the temporal features are independent. TSM-based models have been used to process spatiotemporal data. Class-incremental learning on video-based action recognition by distillation of various knowledge [12] uses two approaches to maintain network information in combination, which are network sharing and network knowledge distillation. The architecture used is a teacher-student framework. The teacher model is first trained on an initial set of classes, and then a student model is trained incrementally. Knowledge distillation is applied at the classification, spatial, and temporal levels to transfer knowledge from the teacher to the student. The loss function combines classification and distillation losses, ensuring that previously learned classes are not forgotten. This approach allows efficient learning of new classes without retraining from scratch. Three-stream architecture with pretrained AlexNet for spatial processing and a separate motion-CNN (TV-L1) for optical flow (Temporal processing), combined with an LSTM for temporal modeling. The proposed paper [13] focuses on a prompt-based incremental learning technique adapted for image-based tasks by using CLIP, a pretrained vision-language model. The methodology is in two stages. Task-specific prompts are optimized independently for each task to integrate new knowledge and retain prior information while keeping the pre-trained model frozen during inference. Task-agnostic prompt matching is used to dynamically select relevant prompts for the test example, enabling accurate classification based on visual-text feature similarity. Task-specific prompts consist of multi-granular prompts (Spatial, temporal, and comprehensive) stored in prompt pools. These are selected through feature matching using K-Means centroids and combined via operations like addition for task-specific integration. Task-agnostic prompts, designed to model temporal relationships across video frames, are selected using a frame-level attention mechanism to ensure contextual consistency across frames. These prompts are then prepended to the video embeddings, allowing the pre-trained model to understand temporal variations better.

An important addition to the Class-Incremental Learning (CIL) methods is Class-Incremental Masking (CIM), introduced [14]. The proposed paper method focuses on adaptive exemplar compression, where only non-discriminative pixels in the image are downsampled while preserving discriminative features, allowing for more exemplars to be stored within the same memory budget. CIM utilizes Class Activation Maps (CAM) to locate discriminative pixels and employs bilevel optimization to tune masks through incremental steps while maintaining the model’s ability to retain existing class information while acquiring new classes— comprehensive experiments on benchmarks such as ImageNet-100 and Food-101.

The 101 action classes of the UCF101 dataset are dealt with incrementally. In every step, a subset of the classes is added with a balance across all the classes. A video is broken down into fixed frames, resized to 224 times 224 pixels, and normalized to [−1, 1] to maintain consistency. Data augmentation is employed for model generalization enhancement, comprising temporal augmentation (Random frame selection with frame repetition for shorter videos) and spatial augmentation (Random cropping, flipping, brightness change, and contrast). The dataset is then divided into training, validation, and testing subsets for every incremental step.

The base model is a 3D ResNet-18 architecture, optimized for action recognition. It employs 3D convolutional layers to capture both spatial and temporal information from videos. These layers decompose convolutions using Conv2Plus1D, which sequentially applies spatial and temporal convolutions for enhanced feature extraction. The network integrates residual connections to improve gradient flow and stability during training. Layer normalization and ReLU activation functions are used to strengthen feature learning. A final dense layer maps the extracted features to action classes. The model is initially trained on a subset of classes from the UCF101 dataset, establishing a baseline before incremental learning begins. Implementation is carried out using TensorFlow/Keras, with custom layers designed for specific operations.

To enable incremental learning, the model is expanded by adding new output neurons for newly incorporated classes. Previous layers are frozen to preserve previous knowledge and update only the new, added layers. To further address catastrophic forgetting, a knowledge distillation scheme is used. A previously trained model (Teacher network) passes its knowledge to an expanded model (Student network) using response-based and feature-based distillation.

I. Response-based distillation: Response-based distillation ensures that the student model aligns its predictions with the teacher model. The loss function is defined as follows:

L = − ∑ P (y ) log (P (y )), (1)

response−distil teacher i student i

where pteacher(yi) represents the probability distribution predicted by the teacher model for class yi, and pstudent(yi) is the corresponding prediction from the student model.

II. Feature-based distillation: Feature-based distillation ensures that the student model retains internal feature representations learned by the teacher model. The feature-based distillation is achieved by minimizing the difference between their intermediate feature maps. The feature-based distillation loss is defined as:

2 (2) L feature−distill = ∑‖Fteacher(Xi) − fstudent(xi)‖ ,

where fteacher(xi) and fstudent(xi) represent the extracted feature embeddings for input xi, and ‖. ‖ denotes the squared Euclidean norm.

The UCF101 dataset, a widely used benchmark for action recognition, consists of 101 action classes across 13,320 videos. To facilitate incremental learning, the dataset was partitioned into staged configurations, where a subset of classes was introduced incrementally, such as 10 × 5 or 5 × 10. Each video was resized to 224 × 224 pixels and normalized within the range of [−1, 1]. The dataset was divided into training, validation, and testing subsets. To enhance generalization, data augmentation techniques, including random cropping, flipping, brightness adjustment, and temporal frame sampling, were applied during training.

The suggested system, developed with TensorFlow, is based on a 3D ResNet model with (2+1)D convolutions to effectively learn spatiotemporal features for action recognition with minimized computational complexity. This architecture separates 3D kernels into 2D spatial and 1D temporal convolutions to promote better temporal dynamics. Residual connections enable effective gradient flow and training of deep networks. The model was first trained on 10 classes for 100 epochs with Adam optimizer (Learning rate 0.0001, batch size 2) with categorical cross-entropy loss.

To simulate real-world scenarios, extra classes were added gradually in groups of 10. A knowledge distillation framework was applied to mitigate catastrophic forgetting, where the previous model (Teacher) provides softened probability distributions to guide the current model (Student). Two distillation techniques were used: response-based, which aligns output logits, and feature-based, which aligns intermediate features using L2 loss.

We adopt the Top-1 accuracy on the validation set after each incremental update as our primary evaluation metric. To assess forgetting, we record the model’s performance on previously learned classes before and after the addition of new classes. The forgetting rate is calculated as the difference in accuracy on old classes before and after each incremental step.

We evaluate the performance of both response-based and feature-based distillation techniques under varying incremental configurations. The effectiveness of each method is assessed in terms of overall accuracy and forgetting rate across configurations.