Some HuggingFace models return 'Bad Request, status code 400'

[Captain-Ironbump]

Hello Everyone,
when I'm trying to make a REST call to the LLM using the HuggingFace dependency of quarkus-langchain4j, some of those models only use the 'inputs' variable for the body. Now, please correct me here if I'm wrong, but the @RegisterAiService appends additional parameters ('parameters', 'options') to the body of the REST call, which will fail some calls because they only expect the 'inputs' parameter.

Is there a option for disabling those parameters of do I need to build a RestClient myself?

for Example the Inference Endpoint of the model 'google/flan-t5-large' example payload looks like this:

curl https://router.huggingface.co/hf-inference/models/google/flan-t5-large \ -X POST \ -H 'Authorization: Bearer hf_xxxxxxxxxxxxxxxxxxxxxxxx' \ -H 'Content-Type: application/json' \ -d '{ "inputs": "\"The answer to the universe is\"" }'

and this is the call created from the extension:

2025-03-26 09:15:41,396 INFO [io.qua.lan.hug.QuarkusHuggingFaceClientFactory$HuggingFaceClientLogger] (vert.x-eventloop-thread-1) Request:

• method: POST
• url: https://api-inference.huggingface.co/models/google/flan-t5-large
• headers: [Accept: application/json], [Authorization: Bearer hf_xxxxxxxxxxx], [Content-Type: application/json], [User-Agent: Quarkus REST Client], [content-length: 1556]
• body: {
  "inputs" : "This is a placeholder text",
  "parameters" : {
  "temperature" : 1.0,
  "return_full_text" : false
  },
  "options" : {
  "wait_for_model" : true
  }
  }

Thanks in advance :)

[geoand]

Any chance you can add a sample I can run and see the issue in action?

[Captain-Ironbump]

Here is my example

@Path("/hello")
public class GreetingResource {
    private final String query3 = """
                Hello World!
                """;

    
    @Inject
    LlmService llmService;

    @POST
    @Path("/hello")
    public String hello() {
        return llmService.sendData(query3);
    } 
}

@RegisterAiService
public interface LlmService {

    @UserMessage(
        """
        [{query}]
        """
    )
    String sendData(String query);
}

and here what happens when using the curl call

2025-03-26 09:57:28,616 INFO  [io.qua.lan.hug.QuarkusHuggingFaceClientFactory$HuggingFaceClientLogger] (vert.x-eventloop-thread-0) Request:
- method: POST
- url: https://api-inference.huggingface.co/models/google/flan-t5-large
- headers: [Accept: application/json], [Authorization: Bearer hf_xxxx], [Content-Type: application/json], [User-Agent: Quarkus REST Client], [content-length: 166]
- body: {
  "inputs" : "[Hello World!\n]\n",
  "parameters" : {
    "temperature" : 1.0,
    "return_full_text" : false
  },
  "options" : {
    "wait_for_model" : true
  }
}

2025-03-26 09:57:28,758 INFO  [io.qua.lan.hug.QuarkusHuggingFaceClientFactory$HuggingFaceClientLogger] (vert.x-eventloop-thread-0) Response:
- status code: 400
- headers: [Content-Type: application/json], [Transfer-Encoding: chunked], [Connection: keep-alive], [Date: Wed, 26 Mar 2025 08:57:28 GMT], [X-Powered-By: huggingface-moon], [cross-origin-opener-policy: same-origin], [Referrer-Policy: strict-origin-when-cross-origin], [x-request-id: Root=1-67e3c178-52b17eb26219f10d7705befc], [vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers], [Access-Control-Allow-Origin: *], [access-control-expose-headers: x-compute-type, x-compute-time], [x-inference-id: -mMhZqaeT-G55NPfG_35_], [x-sha: 0613663d0d48ea86ba8cb3d7a44f0f65dc596a2a], [access-control-allow-credentials: true], [x-compute-time: 0.003], [x-compute-type: cpu], [server: uvicorn], [X-Cache: Error from cloudfront], [Via: 1.1 13ad77f5993668d67b8168f460ba6368.cloudfront.net (CloudFront)], [X-Amz-Cf-Pop: FRA60-P6], [X-Amz-Cf-Id: Lk7ojPTFfd_DPqgheZ4Yx23XlNWOr1FCTetZtgb9H2B1FobDG0BhlQ==]
- body: {"error":"The following `model_kwargs` are not used by the model: ['return_full_text'] (note: typos in the generate arguments will also show up in this list)","warnings":["There was an inference error: The following `model_kwargs` are not used by the model: ['return_full_text'] (note: typos in the generate arguments will also show up in this list)"]}

[geoand]

Thanks. There isn't an option to disable those parameters, but we could certainly add one. Would you be interested in contributing that?

[Captain-Ironbump]

That depends on how much time I would need to find the code and implement that feature.

[geoand]

The code is here