How to use TGI for a finetuned LORA adapter of llama-3.3-70B using unsloth?

[InderjeetVishnoi]

Hi ,

I've fine-tuned a LoRA adapter on the LLaMA-3.3-70B model for my task and am exploring using TGI for optimized inference. From what I understand, TGI doesn’t natively support loading adapters directly for inference — is there a workaround for this?

Do I need to merge the adapter with the base model before serving it via TGI? I’ve been trying to avoid that due to compute constraints.

Any guidance or suggestions would be greatly appreciated.