Video Class-Incremental Learning for Action Recognition
Downloads
Abstract
In the domain of video-based action recognition, overcoming catastrophic forgetting while continuously learning new classes remains a major challenge. We propose a Video Class-Incremental Learning (VCIL) framework that addresses this issue by employing a teacher-student knowledge distillation strategy. Our approach leverages both response-based distillation, which aligns the student model’s predictions with the teacher’s softened outputs, and feature-based distillation, which ensures the student retains internal feature representations learned by the teacher. With the UCF101 action recognition dataset and a 3D ResNet backbone, our approach extracts spatiotemporal features to recognize actions in multiple incremental steps. Our model is tested with various settings (10×5, 5×10, 2×25) and has high accuracy for retaining knowledge of past classes and learning new courses. The results show that our approach is efficient in preventing forgetting and maintaining high performance on new tasks.
Keywords
Incremental learning Action recognition Knowledge distillation Deep learningReferences (14)
- [1] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., … ., & Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521–3526. https://doi.org/10.1073/pnas.1611835114
- [2] Feng, T., Wang, M., & Yuan, H. (2022). Overcoming catastrophic forgetting in incremental object detection via elastic response distillation. 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 9427–9436). IEEE. https://doi.org/10.1109/CVPR52688.2022.00921
- [3] Zenke, F., Poole, B., & Ganguli, S. (2017). Continual learning through synaptic intelligence. Proceedings of the 34th international conference on machine learning (pp. 3987–3995). PMLR. https://proceedings.mlr.press/v70/zenke17a.html
- [4] Li, Z., & Hoiem, D. (2018). Learning without Forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12), 2935–2947. https://doi.org/10.1109/TPAMI.2017.2773081
- [5] Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. https://doi.org/10.48550/arXiv.1503.02531
- [6] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … ., & Bengio, Y. (2020). Generative adversarial networks. Communications of the acm, 63(11), 139–144. https://doi.org/10.1145/3422622
- [7] Rebuffi, S. A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017). iCaRL: Incremental classifier and representation learning. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2001–2010). IEEE. https://doi.org/10.1109/CVPR.2017.587
- [8] Kingma, D. P. (2017). Variational inference & deep learning. [Thesis]. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://pure.uva.nl/ws/files/17891313/Thesis.pdf
- [9] Liu, C., Wang, Y., Li, D., & Wang, X. (2024). Domain-incremental learning without forgetting based on random vector functional link networks. Pattern recognition, 151, 110430. https://doi.org/10.1016/j.patcog.2024.110430
- [10] Chi, Z., Gu, L., Liu, H., Wang, Y., Yu, Y., & Tang, J. (2022). Metafscil: A meta-learning approach for few-shot class incremental learning. 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 14166–14175). IEEE. https://doi.org/10.1109/CVPR52688.2022.01377
- [11] Park, J., Kang, M., & Han, B. (2021). Class-incremental learning for action recognition in videos. 2021 IEEE/CVF international conference on computer vision (ICCV) (pp. 13698–13707). IEEE. https://doi.org/10.1109/ICCV48922.2021.01344
- [12] Maraghi, V. O., & Faez, K. (2022). Class-incremental learning on video-based action recognition by distillation of various knowledge. Computational intelligence and neuroscience, 2022(1), 4879942. https://doi.org/10.1155/2022/4879942
- [13] Pei, Y., Qing, Z., Zhang, S., Wang, X., Zhang, Y., Zhao, D., & Qian, X. (2023). Space-time prompting for video class-incremental learning. 2023 IEEE/CVF international conference on computer vision (ICCV) (pp. 11932–11942).IEEE. https://doi.org/10.1109/ICCV51070.2023.01096
- [14] Luo, Z., Liu, Y., Schiele, B., & Sun, Q. (2023). Class-incremental exemplar compression for class-incremental learning. 2023 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 11371–11380). IEEE. https://doi.org/10.1109/CVPR52729.2023.01094
Article Metrics
- Published 2025-06-17
- Issue Vol. 2 No. 2 (2025): Soft Computing Fusion with Applications
- Section Articles
- Document Type Research Article
- Views 696
- Downloads 259
