Hugging Face Public Inference Endpoint

[zthafir]

Feature Description

I'm trying to use the meta-llama/Llama-3.3-70B-Instruct model from Hugging Face via the public inference API (which is already integrated). However, whenever I attempt to run the model, it returns a 404 error on the Hugging Face side.

After investigating, I found that Hugging Face now provides a new method to use the public inference endpoint, as described in this blog
. When I tried this new approach, it worked without issues.

Is there any plan to upgrade the Hugging Face inference integration to support the latest version and update the code so we can use new models via this new method?

Feature Category

Integration

Problem Statement

When using the public inference api it returns a 404 error on the Hugging Face side.

Proposed Solution

Upgrade the Hugging Face inference package to the latest one and reflect the changes to the code.