Should the macOS installation and setup work?

[bkuhlmann]

Hello. 👋

I've been trying to get Tabby to work locally on my macOS M3 machine and nothing seems to work. I tried Docker, Docker Compose, and Homebrew and none of them work.

I ended up using the following Docker Compose configuration after cloning the project and trying to build Tabby that way:

services:
  tabby:
    build:
      context: .
      dockerfile: docker/Dockerfile.rocm
    environment:
      - RUST_BACKTRACE=1
    platform: linux/amd64
    command: serve --model StarCoder-3B --device cpu
    volumes:
      - "./tabby:/data"
    ports:
      - 8080:8080

Basically, I'd mess with the build by either using the GitHub registry for the image and/or pointing to the local Dockerfile.

In all situations, there always seems to be some kind of build failure with the Docker image.

[wsxiaoys]

Hi - could you share the error message you encounter with homebrew installation?

[bkuhlmann]

Hey Meng. 👋 Sorry, I can't seem to recreate this now. I updated all of my Homebrew dependencies and things work now using Homebrew.

...but I can't get Docker Compose to work. Here's the details:

Docker Compose Configuration

NOTE : I had to remove cuda and switch to cpu in order to get this far. Otherwise, I'll get this error:

Error response from daemon: could not select device driver "nvidia" with capabilities: [[gpu]]

Here's the Docker Compose configuration I'm using.

services:
  tabby:
    image: registry.tabbyml.com/tabbyml/tabby
    environment:
      - RUST_BACKTRACE=1
    platform: linux/amd64
    command: serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cpu
    volumes:
      - "$HOME/.tabby:/data"
    ports:
      - 8080:8080

Console Output

[+] Running 1/1
 ✔ Container tabby-tabby-1  Recreated                                                                                                                                                                           0.0s
Attaching to tabby-1
tabby-1  | Recent llama-cpp errors:
tabby-1  |
tabby-1  | /opt/tabby/bin/llama-server: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory
⠼     0.336 s	Starting...2025-06-20T17:52:16.261157Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
tabby-1  | 2025-06-20T17:52:16.262449Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:164: Attempting to restart the llama-server...
⠦     1.354 s	Starting...2025-06-20T17:52:17.285904Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
tabby-1  | 2025-06-20T17:52:17.285956Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:164: Attempting to restart the llama-server...
⠇     2.357 s	Starting...2025-06-20T17:52:18.309526Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
tabby-1  | 2025-06-20T17:52:18.309687Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:164: Attempting to restart the llama-server...
⠋     3.351 s	Starting...2025-06-20T17:52:19.330164Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
tabby-1  | 2025-06-20T17:52:19.330214Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:164: Attempting to restart the llama-server...
⠸     4.441 s	Starting...2025-06-20T17:52:20.351210Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
tabby-1  | 2025-06-20T17:52:20.351254Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:164: Attempting to restart the llama-server...
⠴     5.452 s	Starting...2025-06-20T17:52:21.374247Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
tabby-1  | llama-server <embedding> encountered a fatal error. Exiting service. Please check the above logs and suggested solutions for details.
tabby-1 exited with code 1
⠼     0.344 s	Starting...2025-06-20T17:52:22.922109Z  WARN llama_cpp_server::supervisor: crates/llama-cpp-server/src/supervisor.rs:124: llama-server <embedding> exited with status code 127, args: `Command { std: "/opt/tabby/bin/llama-server" "-m" "/data/models/TabbyML/Nomic-Embed-Text/ggml/model-00001-of-00001.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`

[wsxiaoys]

Hi - it's expected that docker won't work for mac os. As the distribution is x86 w/ cuda bundled.

[bkuhlmann]

Thanks. Any chance the Docker image build be matrixed with x86 support? I ask, because this makes for a nice use case for macOS solo engineering. I could submit a feature request (if not already requested by others).