Continual Learning and Unknown Object Discovery in 3D Scenes via Self-distillationShow others and affiliations
2025 (English)In: COMPUTER VISION - ECCV 2024, PT LXXIII, SPRINGER INTERNATIONAL PUBLISHING AG , 2025, Vol. 15131, p. 416-431Conference paper, Published paper (Refereed)
Abstract [en]
Open-world 3D instance segmentation is a recently introduced problem with diverse applications, notably in continually learning embodied agents. This task involves segmenting unknown instances and learning new instances when their labels are introduced. However, prior research in the open-world domain has traditionally addressed the two sub-problems, namely continual learning and unknown object identification, separately. This approach has resulted in limited performance on unknown instances and cannot effectively mitigate catastrophic forgetting. Additionally, these methods bypass the utilization of the information stored in the previous version of the continual learning model, instead relying on a dedicated memory to store historical data samples, which inevitably leads to an expansion of the memory budget. In this paper, we argue that continual learning and unknown object identification are desired to be tackled in conjunction. To this end, we propose a new exemplar-free approach for 3D continual learning and unknown object discovery through continual self-distillation. Our approach, named OpenDistill3D, leverages the pseudo-labels generated by the model from the preceding task to improve the unknown predictions during training while simultaneously mitigating catastrophic forgetting. By integrating these pseudo-labels into the continual learning process, we achieve enhanced performance in handling unknown objects. We validate the efficacy of the proposed approach via comprehensive experiments on various splits of the ScanNet200 dataset, showcasing superior performance in contin vual learning and unknown object retrieval compared to the state-of-the-art. Code and model are available at github.com/aminebdj/OpenDistill3D.
Place, publisher, year, edition, pages
SPRINGER INTERNATIONAL PUBLISHING AG , 2025. Vol. 15131, p. 416-431
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349
National Category
Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:liu:diva-212399DOI: 10.1007/978-3-031-73464-9_25ISI: 001416935000025Scopus ID: 2-s2.0-85212292313ISBN: 9783031734632 (print)ISBN: 9783031734649 (electronic)OAI: oai:DiVA.org:liu-212399DiVA, id: diva2:1945831
Conference
18th European Conference on Computer Vision (ECCV), Milan, ITALY, sep 29-oct 04, 2024
Note
Funding Agencies|Alvis - Swedish Research Council [2022-06725]; Knut and Alice Wallenberg Foundation at the NSC
2025-03-192025-03-192025-03-19