The SCANnotate++ Dataset

arXiv

This repository provides CAD model and pose annotations for objects in the ScanNet++v1 dataset. The annotations were automatically generated using SCANnotate and HOC-Search. The quality of these annotations was verified in several verification passes, with manual re-annotations performed for outliers to ensure that final annotations are of high quality.

Details about Annotations

For the public ScanNet++v1 dataset, we provide:

• 5290 CAD model annotations for objects in the ScanNet++v1 dataset
• Accurate 9D pose for each CAD model
• 3D semantic object instance segmentation corresponding to the annotated objects
• Extracted view parameters (selected RGB-D images and camera poses) for each object, which can be used for image-based optimization

CAD Model and Pose Annotations

Our annotations for ScanNet++v1 are provided as .pkl files, which contain additional information about the annotated objects, e.g. view parameters for render-and-compare and the corresponding 3D instance segmentation of the pointcloud data.

Note that in order to use any of the provided annotations correctly, it is necessary to preprocess the ShapeNet CAD models (center and scale-normalize all CAD models) as explained below, to generate clean CAD models which are then compatible with our annotations.

Preliminaries: Download ShapeNet and ScanNet++v1 examples

• Download the ScanNet++v1 example scene here. Extract the data and copy them to ./data/ScanNetpp/data. Note that by downloading this example data you agree to the ScanNet++ Terms of Use. To download the full ScanNet++ dataset follow the instructions on the ScanNet++ webpage.

• Download the ShapenetV2 dataset by signing up on the website. Extract ShapeNetCore.v2.zip to ./data/ShapeNet.

• Download our annotations for the ScanNet++v1 dataset here. Extract the data and copy them to ./data/ScanNetpp/annotations.

Preprocessing ShapeNet CAD Models

To center and scale-normalize the downloaded ShapeNet CAD models, run:

bash run_shapenet_prepro.sh gpu=0

The gpu argument specifies which GPU should be used for processing. By default, code is executed on CPU.

After the above-mentioned steps the ./data folder should contain the following directories:

- data
    - ScanNetpp
        - annotations
            - 30966f4c6e
            - ...
        - data
            - 30966f4c6e
    - ShapeNet
        - ShapeNet_preprocessed            
        - ShapeNetCore.v2

Installation Requirements and Setup

• Clone this repository. Install PyTorch3D by following the instructions from the official installation guide.

After installing Pytorch3D, run the following command:

pip install scikit-image matplotlib imageio plotly opencv-python open3d trimesh==3.10.2

Visualization of Annotations

Use the following command to visualize the annotations:

bash visualize_annotations.sh

Citation

If you find our dataset useful, please consider citing our paper:

@article{rao2025leveraging,
  author    = {Rao, Yuchen and Ainetter, Stefan and Stekovic, Sinisa and Lepetit, Vincent and Fraundorfer, Friedrich},
  title     = {Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understanding},
  journal   = {arXiv preprint arXiv:2504.13580},
  year      = {2025}
}