New Feature: Multiple Llamafile agent support

[fastzombies]

Problem

This is an important feature request. There are times where one must work at a client/customer site in an environment where no internet access or sudo are possible but VSCode is available. In this case the best, and perhaps only option is to bring in some vsix and llamafiles.

Proposed

Due to the limited resource nature of this scenario that forces use of of locally served models, it may be helpful to support multiple llamafiles for different agents:

• Plan
• Act
• FIM (Fill In the Middle / code completion)

This way context windows can be preserved as much as possible, particularly of the planning agent as it may be asking a lot for even a 16K window in these circumstances.

Notes

Attempted

I set it up with OpenAI-Compatible and http://127.0.0.1:8080/v1, NO_KEY for API key and I cannot change the model. It wants to work, but it gets passed the wrong type.

API Request Failed
$0.0000
500 500 Internal Server Error
[json.exception.type_error.302] type must be string, but is array

I can use this and get a response:

❯ curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "developer",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'

It seems like it would not take much to make this work.

Duplicates

This is a duplicate of #1041 but I wanted to create and emphasize my own requirements and justification.

[varungl1969]

A Request to include connectivity for LLAMAFILE llamafile in Cline. Pls add the same, to run LLMs locally. Thanks in advance.