Skip to content

SambaNova Integration #6077

@snova-jorgep

Description

@snova-jorgep

What specific problem does this solve?

Currently, Roo Code supports integration with several open source LLM providers; however, some of them, like groq, do not support certain large models, such as DeepSeek R1 (non-distilled version). This creates a gap for users who require access to models not available. And even though those models are available in other providers, there are quite limited in inference speed

SambaNova cloud offers competitive inference speeds (often outperforming other provider depending on the model, as seen here) and hosts a broader selection of full-sized models, including all listed here and a lot more in their sambastack and sambamanaged version

This integration would allow Roo Code users to:

Access higher quality outputs from full-size models
Benchmark across different inference providers
Reduce latency in select models thanks to SambaNova's inference speed

Additional context (optional)

artificial analysis benchmarks sambanova speeds

Image

comparison for llama4 maverick:

Image

Roo Code Task Links (Optional)

No response

Request checklist

  • I've searched existing Issues and Discussions for duplicates
  • This describes a specific problem with clear impact and context

Interested in implementing this?

  • Yes, I'd like to help implement this feature

Implementation requirements

  • I understand this needs approval before implementation begins

How should this be solved? (REQUIRED if contributing, optional otherwise)

Add SambaNova as a selectable inference provider in the model selection interface.

Implement backend support for forwarding messages to SambaNova’s API

Map model availability for SambaNova.

How will we know it works? (Acceptance Criteria - REQUIRED if contributing, optional otherwise)

Given a user is selecting a model in the AI setup screen
When SambaNova is selected as a provider
Then sambanova models appear in the list
And the inference requests are sent to SambaNova's API
And the response returns valid outputs without delay or errors

Technical considerations (REQUIRED if contributing, optional otherwise)

Add Sambanova provider inference handler in src/api/providers inheriting from BaseOpenAiCompatibleProvider to avoid unnecessary maintenance, also adding its corresponding types, locales, and component for provider in web view

Trade-offs and risks (REQUIRED if contributing, optional otherwise)

Alternative, use OpenAI compatible provider from selector, but this adds extra steps in the set up for users. Also model list will not be present

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue/PR - TriageNew issue. Needs quick review to confirm validity and assign labels.enhancementNew feature or requestproposal

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions