SAM2 just released !

[jeanchristopheruel]

Segment Anything 2.0 model and weights just released from meta under CC BY 4.0.

Everything's here

[jeanchristopheruel]

I just release this PR #8243 to integrate SAM 2.0 🚀🚀🚀
For GPU machines only.

1. Launch CVAT with docker compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
2. Launch the serverless SAM 2.0 with cd serverless && ./deploy_gpu.sh pytorch/facebookresearch/sam2/nuclio/

[bilalcodehub]

@jeanchristopheruel: I’ve successfully configured SAM with CVAT and it’s working like a magic. If I am not wrong, the latest CVAT has the earlier version of SAM.

I wanted to check if I can migrate to SAM 2.0. At the moment, I can't locate a sam2 directory under serverless/pytorch/facebookresearch/ as referenced in your second deployment command.

Could you please clarify whether SAM 2.0 is still supported and guide me on how to set it up properly?

Thanks in advance!

[FraidOfWater]

@bilalcodehub

Get the sam2 files here:

git clone -branch feature/sam2 https://github.com/hashJoe/cvat/tree/feature/sam2

Close and delete any old images and instances of cvat, and the serverless functions.

Overwrite your cvat files with these.
You can now run

# in WSL (cvat directory)
CLIENT_PLUGINS=plugins/sam2 CVAT_HOST=localhost CVAT_VERSION=v2.21.2 docker compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml -p cvat up -d --build

# on cpu
./serverless/deploy_cpu.sh serverless/pytorch/facebookresearch/sam2

# on gpu (if you have 
./serverless/deploy_gpu.sh serverless/pytorch/facebookresearch/sam2

Make sure you have nuctl
(wget might have to be run with sudo)

wsl --install
wsl -d ubuntu
wget https://github.com/nuclio/nuclio/releases/download/1.13.0/nuctl-1.13.0-linux-amd64
sudo chmod +x nuctl-1.13.0-linux-amd64
sudo ln -sf ${pwd}/nuctl-1.13.0-linux-amd64 /usr/local/bin/nuctl
sudo ln -sf $(pwd)/nuctl-1.13.0-linux-amd64 /usr/local/bin/nuctl

(One of the pwds should work)

and nvidia support: (must restart docker afterwards)

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

Note: For some reason I have to run

CLIENT_PLUGINS=plugins/sam2 CVAT_HOST=localhost CVAT_VERSION=v2.21.2 docker compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml -p cvat up -d --build
./serverless/deploy_gpu.sh serverless/pytorch/facebookresearch/sam2

Every single time I close the docker container.

[joaointech]

Does this work with video annotation in the frontend?
Thanks