Replay-Based Continual Learning on Sequential Digit Tasks: Buffer Size, Task Similarity, and the Effect of Interleaving

Abstract

We study replay-based mitigation of forgetting on five binary digit tasks using the 1797-sample subset of UCI Optical Recognition of Handwritten Digits distributed by scikit-learn. A shared 64-16-16 ReLU network with task-specific sigmoid heads is trained under sequential learning, experience replay (ER), past-task-balanced interleaved replay (IR), and joint training with all tasks available from the outset. Reported means over ten seeds give backward transfer (BWT) -0.170 and final accuracy 0.861 for naive training, versus joint-reference accuracy 0.996. In the natural-layout ER condition, nominal per-class capacities 25 to 200 improve BWT from -0.035 to -0.010 and accuracy from 0.969 to 0.988; capacity 200 exceeds the available training examples per class and stores the full observed class data. IR does not significantly outperform ER in BWT at any capacity by the reported unadjusted tests (p ≥ 0.058). ER and IR also differ in current-task inclusion and expected past-task rehearsal dose, so this contrast does not isolate temporal interleaving alone. Per-task feature permutations reduce measured mean-image correlation and are associated with less forgetting in several exploratory contrasts. The twenty-six reported tests are unadjusted; applying Holm to their rounded p-values leaves none below 0.05. Results characterize one small task-incremental benchmark, and mechanism, execution, and reproducibility claims require the original code and per-run outputs.