Stardust SDK

Stardust SDK is an end-to-end toolkit for converting, validating, and visualising multi-sensor perception datasets.

The SDK powers the data infrastructure behind Stardust.ai and distils those production pipelines into a reusable Python library and CLI. It lets computer-vision teams standardise Rosetta exports, compare regressions across formats, and ship annotated datasets with confidence.

---

🔗 Quick links

• Highlights
• Supported formats
• Quickstart
• Architecture
• Testing
• Contributing

✨ Highlights

• Unified perception schema – Parse Rosetta/MorningStar payloads into a canonical frame representation shared across converters, metrics, and quality checks.
• Multi-format export parity – Generate Stardust JSONL, COCO, KITTI, PandaSet, and raw Rosetta assets using the same API surface (Python or CLI).
• Review-ready visualisation – Build AnnotationTool payloads with fallbacks for legacy keys so historical projects stay compatible with modern viewers.
• Automation first – Every workflow is scriptable. Pair the CLI with CI systems or trigger conversions from notebooks without bespoke glue code.
• Extensible internals – Modular geometry utilities, calibration helpers, and format adapters make it simple to extend the SDK to bespoke pipelines.

🧬 Supported export targets

Target	Entrypoint	Description
Stardust	stardust.utils.convert.read_rosetta	Native Stardust JSONL payloads used throughout the platform.
Rosetta	stardust.utils.convert.read_rosetta	Consume raw Rosetta exports for custom post-processing.
COCO	stardust.convertion.to_coco.to_coco.coco_export	Generate COCO instance annotations for detection models.
KITTI	stardust.convertion.to_kitti.to_kitti.kitti_export	Produce KITTI-style label files for autonomous stacks.
PandaSet	stardust.convertion.to_pandaset.to_pandaset.to_pandaset	Emit PandaSet-compatible multimodal datasets.

Check the demo/ directory for runnable scripts that showcase end-to-end conversions and visualisation payloads.

🚀 Quickstart

Prerequisites

• Python 3.9+
• Optional: conda for reproducible environments
• Access to Rosetta/MorningStar exports if running against private datasets

Installation

# Create and activate an isolated environment
conda create -n stardust_sdk python=3.9 -y
conda activate stardust_sdk

# Install dependencies and the SDK in editable mode
pip install -r requirements.txt
pip install -e .

Prefer a traditional install? Swap the final command with python setup.py install.

Configure credentials

Runtime credentials, endpoints, and OSS buckets live in stardust/config.ini. The tracked template contains placeholder values—replace them with your Rosetta keys, service URLs, and storage paths before exporting. Load the configuration via stardust.config.

First export (Python)

from pathlib import Path

from stardust.utils.convert import read_rosetta
from stardust.convertion.to_kitti.to_kitti import kitti_export

frames = read_rosetta(
    project_id=1507,
    input_path="/tmp/stardust",  # Directory for downloaded Rosetta JSON files
    pool_lst=[48904, 48905],
)

output_dir = Path("/tmp/stardust/kitti")
kitti_export(frames, str(output_dir))
print(f"Saved KITTI labels to {output_dir}")

First export (CLI)

stardust export \
  --type coco \
  --project_id 1507 \
  --pool 48904,48905 \
  --output /tmp/stardust_exports \
  --overwrite

The CLI protects existing directories by default. Pass --overwrite if you intentionally want to replace previous runs.

Visualisation smoke test

Demo fixtures in demo/visualization_demo map 1:1 to build_visualization_payload. Regenerate an AnnotationTool payload without uploading to OSS by running:

from pathlib import Path
from stardust.visualization.visual import ToAnnotationTool

demo_path = Path("demo/visualization_demo/tracking.json")
ToAnnotationTool(str(demo_path), "tracking", results_dir="./demo_results", upload=False)

🧱 Architecture overview

stardust/
├── command/            # CLI entry points (export workflows, utilities)
├── components/         # Data classes describing media, tasks, annotations
├── convertion/         # Format converters (COCO, KITTI, PandaSet, ...)
├── geometry/           # 3D geometry helpers for calibration and projection
├── metric/             # Evaluation metrics for detection and tracking
├── ms/ and rosetta/    # Integrations with MorningStar and Rosetta services
├── utils/              # IO helpers, readers, and shared utilities
└── visualization/      # Rendering templates and AnnotationTool builders

Demo assets live in demo/, while publishable docs live under docs/.

🧪 Testing & quality gates

• Unit and smoke tests live in tests/. Run them before committing:

  python -m unittest discover tests

• Fixture-driven visualisation checks ensure backwards compatibility for legacy tracking keys.
• Export smoke tests assert that COCO, KITTI, and Stardust pipelines produce non-empty payloads and that the CLI honours the --overwrite safety guard.
• Release builds should also run python -m compileall stardust to verify bytecode generation across modules.

🤝 Contributing

1. Fork the repository and create a feature branch.
2. Install dependencies with pip install -r requirements.txt followed by pip install -e ..
3. Update or create demo fixtures when adding new visual tasks so that the regression suite stays green.
4. Run python -m unittest discover tests (and any relevant integration demos) before submitting a PR.
5. Open a pull request describing the change, test strategy, and any follow-up work.

📮 Community & support

• File bugs or feature requests through GitHub Issues.
• For security disclosures, email security@stardust.ai.
• Need help? Start a discussion or reach us via the issue tracker.

📄 License

Stardust SDK is released under the MIT License.