Add support for LLM routing using developer preferences

[adilhafeez]

Related GitHub Issue

Closes: #5362

Roo Code Task Context (Optional)

Description

This change adds support for archgw router. This gateway allows developers to choose between models based on developer preferences.

This PR is influenced from Lite-LLM PR #3242

Test Procedure

1. Ensure that you have installed pre-requisites
2. Start archgw with preference_based_routing demo

$ cd demos/use_cases/preference_based_routing
$ archgw up --service archgw --foreground

2025-07-22 12:29:00,763 - cli.main - INFO - Starting archgw cli version: 0.3.5
2025-07-22 12:29:00,763 - cli.main - INFO - Validating /Users/adilhafeez/src/intelligent-prompt-gateway/demos/use_cases/preference_based_routing/arch_config.yaml
2025-07-22 12:29:01,845 - cli.core - INFO - Starting arch gateway, image name: archgw, tag: katanemo/archgw:0.3.5
...
[2025-07-22 19:29:05.909][66][debug][wasm] [source/extensions/common/wasm/wasm.cc:100] Thread-Local Wasm created 24 now active

3. In providers select "Arch LLM Gateway"
4. Select "use preference based routing" option and use following configuration for routing. Note: model name must match with model names listed in arch_config.yaml.

  - model: openai/gpt-4o
    routing_preferences:
      - name: code understanding
        description: understand and explain code

  - model: openai/gpt-4.1
    routing_preferences:
      - name: code generation
        description: generating new code

At this point you are ready. Fire away your queries and see arch router use dynamic models based on query type.

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes (if applicable).
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

Video screenshot - reddit post

Documentation Updates

Does this PR necessitate updates to user-facing documentation?

• No documentation updates are required.
• Yes, documentation updates are required. (Please describe what needs to be updated or link to a PR in the docs repository).

RooCodeInc/Roo-Code-Docs#262

Additional Notes

• API Key requirement: Right now API Key is required in this code change which is not required for arch gateway. But I am not able to make it optional. When I make it optional I cannot get past welcome screen. I do need some help here on how to make the api key optional so that I can get past welcome screen.

Get in Touch

@mrubens

---

Important

Adds support for Arch Gateway routing using developer preferences, including configuration, validation, and tests.

• Behavior:
  • Adds support for Arch Gateway routing using developer preferences in archgw.ts.
  • Introduces archgwPreferenceConfig and archgwUsePreferences in provider-settings.ts.
  • Validates archgwPreferenceConfig in validate.ts.
• Tests:
  • Adds tests for ArchGwHandler in archgw.spec.ts.
  • Adds tests for validateArchGwPreferenceConfig in validate.test.ts.
• Misc:
  • Updates i18n files for new Arch Gateway settings.

^{This description was created by for d72040c. You can customize this summary. It will automatically update as commits are pushed.}