Activate CLBlast support Intel IGPU

[ernesst]

Hello,

First of all, thank you for this work !

I need your lights to activate intel IGPU into the Localai Docker.

So far what I was able / try to achieve :

• Build the docker with clblas option

in .env
BUILD_TYPE=clblas build
CLBLAST_DIR=/mnt/clblast
#LLAMA_CLBLAST=1

• Authorize the docker to access the card
  in the docker-compose file

    devices:
      - "/dev/dri:/dev/dri"                         # Intel QSV

• Build the clblast lib

CLBlast_built/:
include  lib

• add it to the docker
  in the docker-compose file

    volumes:
      - ./models:/models:cached
      - ./images/:/tmp/generated/images/
      - ./CLBlast_built:/mnt/clblast

• add the intel driver into the docker image
  in the entrypoint.sh

mkdir /tmp/neo && \
cd /tmp/neo && \
wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.13700.14/intel-igc-core_1.0.13700.14_amd64.deb && \
wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.13700.14/intel-igc-opencl_1.0.13700.14_amd64.deb && \
wget https://github.com/intel/compute-runtime/releases/download/23.13.26032.30/intel-level-zero-gpu-dbgsym_1.3.26032.30_amd64.ddeb && \
wget https://github.com/intel/compute-runtime/releases/download/23.13.26032.30/intel-level-zero-gpu_1.3.26032.30_amd64.deb && \
wget https://github.com/intel/compute-runtime/releases/download/23.13.26032.30/intel-opencl-icd-dbgsym_23.13.26032.30_amd64.ddeb && \
wget https://github.com/intel/compute-runtime/releases/download/23.13.26032.30/intel-opencl-icd_23.13.26032.30_amd64.deb && \
wget https://github.com/intel/compute-runtime/releases/download/23.13.26032.30/libigdgmm12_22.3.0_amd64.deb && \
dpkg -i ./*.deb && \
apt-get install clinfo

cd /build

# If we have set EXTRA_BACKENDS, then we need to prepare the backends

As result :
The container detect correctly the card

 Platform Name                                   Intel(R) OpenCL HD Graphics
Number of devices                                 1
  Device Name                                     Intel(R) UHD Graphics 730
  Device Vendor                                   Intel(R) Corporation
  Device Vendor ID                                0x8086
  Device Version                                  OpenCL 3.0 NEO 

The debug gives :

api_1  | I local-ai build info:
api_1  | I BUILD_TYPE: clblas build
api_1  | I GO_TAGS: 
api_1  | I LD_FLAGS: -X "github.com/go-skynet/LocalAI/internal.Version=v2.2.0-7-g939187a" -X "github.com/go-skynet/LocalAI/internal.Commit=939187a12969153dd6e4d6a73add54cdb1713b25"
api_1  | CGO_LDFLAGS="" go build -ldflags "-X "github.com/go-skynet/LocalAI/internal.Version=v2.2.0-7-g939187a" -X "github.com/go-skynet/LocalAI/internal.Commit=939187a12969153dd6e4d6a73add54cdb1713b25"" -tags "" -o local-ai ./
api_1  | 11:05PM INF Starting LocalAI using 15 threads, with models path: /models
api_1  | 11:05PM INF LocalAI version: v2.2.0-7-g939187a (939187a12969153dd6e4d6a73add54cdb1713b25)
api_1  | 
api_1  |  ┌───────────────────────────────────────────────────┐ 
api_1  |  │                   Fiber v2.50.0                   │ 
api_1  |  │               http://127.0.0.1:8080               │ 
api_1  |  │       (bound on host 0.0.0.0 and port 8080)       │ 
api_1  |  │                                                   │ 
api_1  |  │ Handlers ............ 74  Processes ........... 1 │ 
api_1  |  │ Prefork ....... Disabled  PID ............... 254 │ 
api_1  |  └───────────────────────────────────────────────────┘ 
api_1  | 
api_1  | 11:05PM DBG Extracting backend assets files to /tmp/localai/backend_data

When loading the model, I don't see any reference to GPU.

I've some doubt that my localai is compile correctly, for instance I don't see any reference to the initialisation of clblas.
Also, I should get the Go flag as :
CGO_LDFLAGS="-lOpenCL -lclblast"

Any inputs are welcome :)

My sources so far :
#1182
#404

[ksullivan86]

ever get this to work?