failed to load model with internal loader: grpc service not ready

[Chard0212]

LocalAI version: v2.29.0

Environment, CPU architecture, OS, and Version: Ubuntu 22.04.5 LTS (GNU/Linux 5.15.0-142-generic x86_64)

Describe the bug Occurs when I prompt model to generate image, also text generation not working

To Reproduce

Expected behavior

Logs

Additional context

[johndev168]

I have the same issue. A reboot helped but it keeps popping up sometimes.

If that occurs then this is looping until I stop the process:

11:48AM DBG Loading from the following backends (in order): [llama-cpp llama-cpp-fallback silero-vad whisper huggingface]
11:48AM INF Trying to load the model 'gemma-3-4b-it-qat' with the backend '[llama-cpp llama-cpp-fallback silero-vad whisper huggingface]'
11:48AM INF [llama-cpp] Attempting to load
11:48AM INF BackendLoader starting backend=llama-cpp modelID=gemma-3-4b-it-qat o.model=google_gemma-3-4b-it-qat-Q4_0.gguf
11:48AM DBG Loading model in memory from file: /users/xyz/localai/models/google_gemma-3-4b-it-qat-Q4_0.gguf
11:48AM DBG Loading Model gemma-3-4b-it-qat with gRPC (file: /users/xyz/localai/models/google_gemma-3-4b-it-qat-Q4_0.gguf) (backend: llama-cpp): {backendString:llama-cpp model:google_gemma-3-4b-it-qat-Q4_0.gguf modelID:gemma-3-4b-it-qat assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0x1400a108008 externalBackends:map[] grpcAttempts:20 grpcAttemptsDelay:2 parallelRequests:false}
11:48AM DBG [llama-cpp-fallback] llama-cpp variant available
11:48AM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama-cpp-fallback
11:48AM DBG GRPC Service for gemma-3-4b-it-qat will be running at: '127.0.0.1:50660'
11:48AM DBG GRPC Service state dir: /tmp/go-processmanager1939193735
11:48AM DBG GRPC Service Started
11:48AM DBG Wait for the service to start up
11:48AM DBG Options: ContextSize:8192 Seed:1311559892 NBatch:512 F16Memory:true MMap:true NGPULayers:256 Threads:10 MMProj:"mmproj-google_gemma-3-4b-it-qat-f16.gguf"
11:49AM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:50660: connect: connection refused\""
11:49AM DBG GRPC Service NOT ready
11:49AM ERR [llama-cpp] Failed loading model, trying with fallback 'llama-cpp-fallback', error: failed to load model with internal loader: grpc service not ready