SambaNova Integration #6077
Description
What specific problem does this solve?
Currently, Roo Code supports integration with several open source LLM providers; however, some of them, like groq, do not support certain large models, such as DeepSeek R1 (non-distilled version). This creates a gap for users who require access to models not available. And even though those models are available in other providers, there are quite limited in inference speed
SambaNova cloud offers competitive inference speeds (often outperforming other provider depending on the model, as seen here) and hosts a broader selection of full-sized models, including all listed here and a lot more in their sambastack and sambamanaged version
This integration would allow Roo Code users to:
Access higher quality outputs from full-size models
Benchmark across different inference providers
Reduce latency in select models thanks to SambaNova's inference speed
Additional context (optional)
artificial analysis benchmarks sambanova speeds
comparison for llama4 maverick:
Roo Code Task Links (Optional)
No response
Request checklist
- I've searched existing Issues and Discussions for duplicates
- This describes a specific problem with clear impact and context
Interested in implementing this?
- Yes, I'd like to help implement this feature
Implementation requirements
- I understand this needs approval before implementation begins
How should this be solved? (REQUIRED if contributing, optional otherwise)
Add SambaNova as a selectable inference provider in the model selection interface.
Implement backend support for forwarding messages to SambaNova’s API
Map model availability for SambaNova.
How will we know it works? (Acceptance Criteria - REQUIRED if contributing, optional otherwise)
Given a user is selecting a model in the AI setup screen
When SambaNova is selected as a provider
Then sambanova models appear in the list
And the inference requests are sent to SambaNova's API
And the response returns valid outputs without delay or errors
Technical considerations (REQUIRED if contributing, optional otherwise)
Add Sambanova provider inference handler in src/api/providers inheriting from BaseOpenAiCompatibleProvider to avoid unnecessary maintenance, also adding its corresponding types, locales, and component for provider in web view
Trade-offs and risks (REQUIRED if contributing, optional otherwise)
Alternative, use OpenAI compatible provider from selector, but this adds extra steps in the set up for users. Also model list will not be present