Skip to content

🔨 SegmentationRoutine supports OOD detection #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 18, 2025
Merged
85 changes: 46 additions & 39 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,40 +1,47 @@
repos:
- repo: local
hooks:
- id: end-of-file-fixer
name: end-of-file-fixer
entry: end-of-file-fixer
language: system
types: [text]
- id: check-toml
name: check-toml
entry: check-toml
language: system
types: [toml]
- id: check-yaml
name: check-yaml
language: system
entry: check-yaml
types: [yaml]
- id: check-merge-conflict
name: check-merge-conflict
entry: check-merge-conflict
language: system
- id: check-added-large-files
name: check-added-large-files
entry: check-added-large-files
language: system
- id: ruff-check
name: ruff-check
entry: ruff check
language: python
types_or: [python, pyi]
require_serial: true
args: [--force-exclude, --fix, --exit-non-zero-on-fix, --output-format, concise]
exclude: ^auto_tutorial_source/
- id: ruff-format
name: ruff-format
entry: ruff format
language: python
types_or: [python, pyi]
exclude: ^auto_tutorials_source/
- repo: local
hooks:
- id: end-of-file-fixer
name: end-of-file-fixer
entry: end-of-file-fixer
language: system
types: [text]
- id: check-toml
name: check-toml
entry: check-toml
language: system
types: [toml]
- id: check-yaml
name: check-yaml
language: system
entry: check-yaml
types: [yaml]
- id: check-merge-conflict
name: check-merge-conflict
entry: check-merge-conflict
language: system
- id: check-added-large-files
name: check-added-large-files
entry: check-added-large-files
language: system
- id: ruff-check
name: ruff-check
entry: ruff check
language: python
types_or: [python, pyi]
require_serial: true
args:
[
--force-exclude,
--fix,
--exit-non-zero-on-fix,
--output-format,
concise,
]
exclude: ^auto_tutorial_source/
- id: ruff-format
name: ruff-format
entry: ruff format
language: python
types_or: [python, pyi]
exclude: ^auto_tutorials_source/
20 changes: 17 additions & 3 deletions docker/DOCKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,37 @@
This Docker image is designed for users and contributors who want to run experiments with `torch-uncertainty` on remote virtual machines with GPU support. It is particularly useful for those who do not have access to a local GPU and need a pre-configured environment for development and experimentation.

---

## How to Use The Docker Image

### Step 1: Fork the Repository

Before proceeding, ensure you have forked the `torch-uncertainty` repository to your own GitHub account. You can do this by visiting the [torch-uncertainty GitHub repository](https://github.com/ENSTA-U2IS-AI/torch-uncertainty) and clicking the **Fork** button in the top-right corner.

Once forked, clone your forked repository to your local machine:

```bash
git clone git@github.com:<your-username>/torch-uncertainty.git
cd torch-uncertainty
```

> ### ⚠️ IMPORTANT NOTE: Keep Your Fork Synced
>
>
> **To ensure that you are working with the latest stable version and bug fixes, you must manually sync your fork with the upstream repository before building the Docker image. Failure to sync your fork may result in outdated dependencies or missing bug fixes in the Docker image.**

### Step 2: Build the Docker image locally

Build the modified image locally and push it to a Docker registry:
```

```bash
docker build -t my-torch-uncertainty-docker:version .
docker push my-dockerhub-user/my-torch-uncertainty-image:version
```

### Step 3: Set environment variables on your VM

Connect to you VM and set the following environment variables:

```bash
export VM_SSH_PUBLIC_KEY="$(cat ~/.ssh/id_rsa.pub)"
export GITHUB_SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)"
Expand All @@ -36,6 +44,7 @@ export USE_COMPACT_SHELL_PROMPT=true
```

Here is a brief explanation of the environment variables used in the Docker setup:

- **`VM_SSH_PUBLIC_KEY`**: The public SSH key used to authenticate with the container via SSH.
- **`GITHUB_SSH_PRIVATE_KEY`**: The private SSH key used to authenticate with GitHub for cloning and pushing repositories.
- **`GITHUB_USER`**: The GitHub username used to clone the repository during the first-time setup.
Expand All @@ -44,8 +53,10 @@ Here is a brief explanation of the environment variables used in the Docker setu
- **`USE_COMPACT_SHELL_PROMPT`** (optional): Enables a compact and colorized shell prompt inside the container if set to `"true"`.

### Step 4: Run the Docker container

First, authenticate with your Docker registry if you use a private registry.
Then run the following command to run the Docker image from your docker registriy

```bash
docker run --rm -it --gpus all -p 8888:8888 -p 22:22 \
-e VM_SSH_PUBLIC_KEY \
Expand All @@ -58,10 +69,13 @@ docker run --rm -it --gpus all -p 8888:8888 -p 22:22 \
```

### Step 5: Connect to your container

Once the container is up and running, you can connect to it via SSH:

```bash
ssh -i /path/to/private_key root@<VM_HOST> -p <VM_PORT>
```

Replace `<VM_HOST>` and `<VM_PORT>` with the host and port of your VM,
and `/path/to/private_key` with the private key that corresponds to `VM_SSH_PUBLIC_KEY`.

Expand All @@ -74,7 +88,7 @@ If using a cloud provider, ensure your network volume is correctly attached to a
## Remote Development

This Docker setup also allows for remote development on the VM, since GitHub SSH access is set up and the whole repo is cloned to the VM from your GitHub fork.
For example, you can seamlessly connect your VS Code editor to your remote VM and run experiments, as if on your local machine but with the GPU acceleration of your VM.
For example, you can seamlessly connect your VS Code editor to your remote VM and run experiments, as if on your local machine but with the GPU acceleration of your VM.
See [VS Code Remote Development](https://code.visualstudio.com/docs/remote/remote-overview) for further details.

## Streamline setup with your Cloud provider of choice
Expand Down
20 changes: 10 additions & 10 deletions experiments/depth/kitti/configs/bts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ trainer:
name: standard
default_hp_metric: false
callbacks:
- class_path: lightning.pytorch.callbacks.ModelCheckpoint
init_args:
monitor: val/reg/SILog
mode: min
save_last: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.ModelCheckpoint
init_args:
monitor: val/reg/SILog
mode: min
save_last: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
model:
loss:
loss:
class_path: torch_uncertainty.metrics.SILog
init_args:
sqrt: true
Expand All @@ -34,7 +34,7 @@ model:
data:
root: ./data
batch_size: 4
crop_size:
crop_size:
- 352
- 704
eval_size:
Expand Down
20 changes: 10 additions & 10 deletions experiments/depth/nyu/configs/bts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ trainer:
name: standard
default_hp_metric: false
callbacks:
- class_path: lightning.pytorch.callbacks.ModelCheckpoint
init_args:
monitor: val/reg/SILog
mode: min
save_last: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.ModelCheckpoint
init_args:
monitor: val/reg/SILog
mode: min
save_last: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
model:
loss:
loss:
class_path: torch_uncertainty.metrics.SILog
init_args:
sqrt: true
Expand All @@ -34,7 +34,7 @@ model:
data:
root: ./data
batch_size: 8
crop_size:
crop_size:
- 416
- 544
eval_size:
Expand Down
26 changes: 13 additions & 13 deletions experiments/regression/uci_datasets/configs/boston/mlp/laplace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,22 @@ trainer:
name: standard
default_hp_metric: false
callbacks:
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
init_args:
probabilistic: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/NLL
patience: 1000
check_finite: true
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
init_args:
probabilistic: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/NLL
patience: 1000
check_finite: true
model:
output_dim: 1
in_features: 13
hidden_dims:
- 50
hidden_dims:
- 50
loss: torch_uncertainty.losses.DistributionNLLLoss
version: std
distribution: laplace
Expand Down
26 changes: 13 additions & 13 deletions experiments/regression/uci_datasets/configs/boston/mlp/normal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,22 @@ trainer:
name: standard
default_hp_metric: false
callbacks:
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
init_args:
probabilistic: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/NLL
patience: 1000
check_finite: true
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
init_args:
probabilistic: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/NLL
patience: 1000
check_finite: true
model:
output_dim: 1
in_features: 13
hidden_dims:
- 50
hidden_dims:
- 50
loss: torch_uncertainty.losses.DistributionNLLLoss
version: std
distribution: normal
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,20 @@ trainer:
name: standard
default_hp_metric: false
callbacks:
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/MSE
patience: 1000
check_finite: true
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/MSE
patience: 1000
check_finite: true
model:
output_dim: 1
in_features: 13
hidden_dims:
- 50
hidden_dims:
- 50
loss: MSELoss
version: std
data:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,22 @@ trainer:
name: standard
default_hp_metric: false
callbacks:
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
init_args:
probabilistic: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/NLL
patience: 1000
check_finite: true
- class_path: torch_uncertainty.callbacks.TURegCheckpoint
init_args:
probabilistic: true
- class_path: lightning.pytorch.callbacks.LearningRateMonitor
init_args:
logging_interval: step
- class_path: lightning.pytorch.callbacks.EarlyStopping
init_args:
monitor: val/reg/NLL
patience: 1000
check_finite: true
model:
output_dim: 1
in_features: 8
hidden_dims:
- 50
hidden_dims:
- 50
loss: torch_uncertainty.losses.DistributionNLLLoss
version: std
distribution: laplace
Expand Down
Loading