Trace3D: Consistent Segmentation Lifting via
Gaussian Instance Tracing

ICCV 2025

Hongyu Shen ^1,2,* Junfeng Ni^2,3,*, Yixin Chen^✉2, Weishuo Li², Mingtao Pei¹, Siyuan Huang^✉2
✉ indicates corresponding author    ^* these authors contributed equally to this work    ¹Beijing Institute of Technology
²State Key Laboratory of General Artificial Intelligence, BIGAI    ³Tsinghua University

Trace3D leverages the proposed Gaussian Instance Tracing to enhance multi-view consistency and reduce ambiguous Gaussians, resulting in high-quality 3D instance segmentation.

Installation

• Tested System: Ubuntu 22.04, CUDA 11.8
• Tested GPUs: RTX4090

1. Basic environment

conda create -n trace3d python=3.10
conda activate trace3d
pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

2. SAM for segmentation

git clone https://github.com/facebookresearch/segment-anything.git
cd segment-anything
pip install -e .
mkdir sam_ckpt; cd sam_ckpt
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

Data

    data
    ├── nerf_llff_data  # Link: https://drive.google.com/drive/folders/14boI-o5hGO9srnWaaogTU5_ji7wkX2S7
    │   └── [fern|flower|fortress|horns|leaves|orchids|room|trex]
    │       ├── [sparse/0] (colmap results)
    │       └── [images|images_2|images_4|images_8]
    │
    └── replica		# Link: https://www.dropbox.com/sh/9yu1elddll00sdl/AAC-rSJdLX0C6HhKXGKMOIija?dl=0
        └── [office_0|room_0|...]
            ├── traj_w_c.txt
            ├── [sparse/0] (colmap results)
            └── [rgb|depth|sam|]

Training

Get SAM masks

python get_sam_masks.py --sam_checkpoint {SAM_CKPT_PATH} --file_path {IMAGE_FOLDER}

Before running: please specify the information in the scripts (e.g. replica.sh). More options can be found in conf/ and arguments/ and you can them adjusted in config file.

#--- Edit the config file replica.sh
dataset=replica_900
path=./data/${dataset}
scene='room_0' 

Scene reconstruction

bash replica.sh train_rgb

Merge patch masks

bash replica.sh merge_patches

Delete Ambiguous Gaussians

bash replica.sh remove_ab_gaus

Contrastive lifting

bash replica.sh train_contra

Evaluation

3D Object Extraction

bash replica.sh eval_3d

Novel View 2D Instance Segmentation

bash replica.sh eval         

Acknowledgements

Some codes are borrowed from Egolifter, SA3D, Omniseg3D, FlashSplat and Gaussian-Editor. We thank all the authors for their great work.

Citation

@inproceedings{shen2025trace3d,
  title={Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing},
  author={Shen, Hongyu and Ni, Junfeng, and Chen, Yixin and Li, Weishuo and Pei, Mingtao and Huang, Siyuan},
  booktitle=ICCV,
  year={2025}
}