Challenges and Help

[shimjunho]

What Stanford Spezi module is your challenge related to?

Spezi

Description

Hi i tried to install a new model on speziLLM and that is https://huggingface.co/mlx-community/EXAONE-Deep-2.4B-4bit
I heard that we can use MLX model on this code and i chose this one...however, when I wrote custom id, i got this result like a pic

I am korean so i really wanna use a model which can support korean well
Can you help me how to add a new model?

Reproduction

import Foundation
import SpeziLLMLocal

enum ModelChoice: String, CaseIterable, Identifiable {
// 현재 정의된 케이스 이름들
case llama1B = "Llama 3.2 - 1B"
case llama3B = "Llama 3.2 - 3B"
case llama8B = "Llama 3 - 8B"
case phi3 = "Phi-3 Mini"
case gemma2B = "Gemma 2B"
case smolLM_135M = "SmolLM 135M"
case mistral7B = "Mistral 7B"
case deepseek_qwen_1_5B = "DeepSeek-R1-Qwen-1.5B"
case deepseek_llama_8B = "DeepSeek-R1-Llama-8B"
case exaoneDeep = "EXAONE-Deep-2.4B" // 추가된 모델

var id: String { self.rawValue }

/// SpeziLLM 로컬 모델과 매핑
var localModel: LLMLocalModel {
    switch self {
    case .llama1B:
        return .llama3_2_1B_4bit
    case .llama3B:
        return .llama3_2_3B_4bit
    case .llama8B:
        return .llama3_8B_4bit
    case .phi3:
        return .phi3_4bit
    case .deepseek_qwen_1_5B:
        return .deepseek_r1_distill_qwen_1_5b_8bit
    case .deepseek_llama_8B:
        return .deepseek_r1_distill_llama_8b_4bit
    case .gemma2B:
        return .gemma_2_2b_it_4bit
    case .mistral7B:
        return .mistral7B4bit
    case .smolLM_135M:
        return .smolLM_135M_4bit
    case .exaoneDeep:
        return .custom(id: "mlx-community/EXAONE-Deep-2.4B-4bit")
    }
}

// 모델 파일 이름 (허깅페이스 ID 경로 포함)
var modelFileName: String {
    switch self {
    case .llama1B:
        return "mlx-community/Llama-3.2-1B-Instruct-4bit"
    case .llama3B:
        return "mlx-community/Llama-3.2-3B-Instruct-4bit"
    case .llama8B:
        return "mlx-community/Meta-Llama-3-8B-Instruct-4bit"
    case .phi3:
        return "mlx-community/Phi-3-mini-4k-instruct-4bit-no-q-embed"
    case .gemma2B:
        return "mlx-community/gemma-2-2b-it-4bit"
    case .smolLM_135M:
        return "mlx-community/SmolLM-135M-Instruct-4bit"
    case .mistral7B:
        return "mlx-community/Mistral-7B-Instruct-v0.3-4bit"
    case .deepseek_qwen_1_5B:
        return "mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-8bit"
    case .deepseek_llama_8B:
        return "mlx-community/DeepSeek-R1-Distill-Llama-8B-4bit-mlx"
        // 문제가 되는 코드
        case .exaoneDeep(let "mlx-community/EXAONE-Deep-2.4B-4bit"):
            return"mlx-community/EXAONE-Deep-2.4B-4bit"
    }
}

// 기존 속성명 호환성 유지
var fileName: String {
    return modelFileName
}

//we tried to change a LLMLocalModel file like this 

How can i fix it?

Expected behavior

I really wanna know how to add a new model here

Additional context

No response

Code of Conduct

• I agree to follow this projects's Code of Conduct and Contributing Guidelines

[PSchmiedmayer]

@philippzagar Would be able to provide some more context on this.

[philippzagar]

Hi @shimjunho,

Thank you for reaching out and for integrating a custom model into SpeziLLM! It’s great to see you’re working with Korean language support using a Hugging Face MLX model 🚀

You’re correct that SpeziLLMLocal allows using custom models from Hugging Face via the .custom(id: String) case in LLMLocalModel. There’s no need to add a new enum case to your model selection (via a fork) — you can directly use .custom("mlx-community/EXAONE-Deep-2.4B-4bit") when downloading/configuring the model.
However, it seems that the model may not have been downloaded correctly before usage in your example shown above.

Here’s how to properly integrate and run your the EXAONE model in SpeziLLM:

Step 1: Download the Model Using LLMLocalDownloadView

Before you can use the model, it must be downloaded from Hugging Face and stored locally on the device:

struct LLMLocalOnboardingDownloadView: View {
    var body: some View {
        LLMLocalDownloadView(
            model: .custom("mlx-community/EXAONE-Deep-2.4B-4bit"),
            downloadDescription: "Downloading the EXAONE-Deep model for Korean language support."
        ) {
            // Action after download is complete (e.g. navigation to main app screen)
        }
    }
}

You can include this view in your onboarding flow, or call it wherever you want users to initiate model setup.

Step 2: Configure the App with LLMLocalPlatform

Ensure that the app is using the local LLM execution backend by registering the LLMLocalPlatform inside your SpeziAppDelegate:

class TestAppDelegate: SpeziAppDelegate {
    override var configuration: Configuration {
        Configuration {
            LLMRunner {
                LLMLocalPlatform()
            }
        }
    }
}

This sets up SpeziLLM to run locally downloaded MLX models.

Step 3: Use the Model in Chat View

Once the model is downloaded and the local runner is set up, you can launch the chat interface using:

struct LLMLocalChatView: View {
    var body: some View {
        LLMChatViewSchema(
            with: LLMLocalSchema(
                model: .custom("mlx-community/EXAONE-Deep-2.4B-4bit")
            )
        )
    }
}

Note

To learn more about the usage of SpeziLLMLocal, please refer to the comprehensive DocC documentation.

Let us know if you’re still facing issues or if you need help with any other part of the integration!