Inference Gateway UI

The Inference Gateway UI is a Next.js application that provides a user-friendly interface to interact with AI models through the Inference Gateway service. It enables easy access to various language models through a consistent interface, streamlining the process of sending requests and receiving responses.

Table of Contents

• Key Features
• Development
• Configuration
  • General Settings
  • Storage Settings
  • Authentication Settings
  • Keycloak Auth Provider
• Docker
• Kubernetes Deployment
  • Quick Installation
  • Deployment Options
    • 1. Combined Deployment with Inference Gateway Backend
    • 2. UI-Only Deployment (Connecting to External Gateway)
    • 3. Deployment with Ingress for External Access
  • Key Configuration Parameters
  • Complete Example
• Deployment
• Related Projects
• License
• Contributing

Key Features

• 🎨 Modern Interface: Clean, responsive design built with Next.js 15 and Tailwind CSS
• 🔌 Seamless API Integration: Connects directly to the Inference Gateway backend API
• 🛠️ Model Selection: Support for multiple language models through an intuitive model selector
• 💬 Chat Interface: Intuitive chat UI for interacting with AI models
• 🧰 Tool Support: Enables AI models to use tools, including web search capability
• 🔍 Web Search: Integrated web search functionality for models to retrieve current information
• 🔒 Authentication: Optional authentication using NextAuth.js
• 📱 Responsive Design: Works on desktop and mobile devices
• 🐳 Docker Support: Easy containerization and deployment
• ☸️ Kubernetes Ready: Includes configurations for Kubernetes deployment
• 🧩 Component Library: Built with shadcn/ui components and Radix UI primitives
• 🔄 State Management: Efficient state management with React hooks

Development

# Install dependencies
npm install

# Run development server
npm run dev

The development server will be available at http://localhost:3000.

Configuration

The UI can be configured using the following environment variables:

General Settings

Environment Variable	Default Value	Description
NODE_ENV	development	Node environment
PORT	3000	Port to run the application on
HOSTNAME	0.0.0.0	Hostname to bind to
INFERENCE_GATEWAY_URL	http://localhost:8080/v1	URL of the Inference Gateway API
LOG_LEVEL	debug	Server-side logging level
NEXT_PUBLIC_LOG_LEVEL	debug	Client-side logging level

Storage Settings

Environment Variable	Default Value	Description
NEXT_PUBLIC_INFERENCE_GATEWAY_UI_STORAGE_TYPE	local	Storage type for chat history

Authentication Settings

Environment Variable	Default Value	Description
ENABLE_AUTH	false	Enable authentication
SECURE_COOKIES	false	Use secure cookies (set to true for HTTPS)
NEXTAUTH_URL	http://localhost:3000	URL of this application (for NextAuth)
NEXTAUTH_SECRET	-	Secret used to encrypt session cookies
NEXTAUTH_TRUST_HOST	true	Trust the host header from the proxy
NEXTAUTH_REFRESH_TOKEN_ENABLED	true	Enable refresh token rotation

Keycloak Auth Provider

Environment Variable	Default Value	Description
KEYCLOAK_ID	app-client	Keycloak client ID
KEYCLOAK_SECRET	-	Keycloak client secret
KEYCLOAK_ISSUER	http://localhost:8080/realms/app-realm	Keycloak issuer URL

Docker

Pre-built container images are available on the GitHub Container Registry. You can use these images directly:

# Pull the pre-built image
docker pull ghcr.io/inference-gateway/ui:latest

# Run the container with the pre-built image
docker run -p 3000:3000 \
  -e INFERENCE_GATEWAY_URL=http://localhost:8080/v1 \
  ghcr.io/inference-gateway/ui:latest

Alternatively, you can build the image locally:

# Build the Docker image locally
docker build -t inference-gateway-ui --target dev .

# Run the container with the locally built image
docker run -p 3000:3000 \
  -v $(pwd):/app \
  -e INFERENCE_GATEWAY_URL=http://localhost:8080/v1 \
  inference-gateway-ui

Kubernetes Deployment

The Inference Gateway UI provides a Helm chart for easy deployment to Kubernetes environments. The chart is available as an OCI artifact, which can be deployed directly without adding a Helm repository.

Quick Installation

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --version 0.7.1 \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway

Deployment Options

The Helm chart supports multiple deployment scenarios:

1. Combined Deployment with Inference Gateway Backend

Deploy the UI with the Inference Gateway backend as a dependency (recommended):

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway

2. UI-Only Deployment (Connecting to External Gateway)

Deploy the UI separately and connect it to an existing Inference Gateway instance:

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=false \
  --set-string "env[0].name=INFERENCE_GATEWAY_URL" \
  --set-string "env[0].value=http://your-gateway-service:8080/v1"

3. Deployment with Ingress for External Access

Enable ingress for accessing the UI from outside the cluster:

helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --create-namespace \
  --namespace inference-gateway \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway \
  --set ingress.enabled=true \
  --set ingress.className=nginx \
  --set "ingress.hosts[0].host=ui.inference-gateway.local" \
  --set "ingress.hosts[0].paths[0].path=/" \
  --set "ingress.hosts[0].paths[0].pathType=Prefix" \
  --set "ingress.tls[0].secretName=inference-gateway-ui-tls" \
  --set "ingress.tls[0].hosts[0]=ui.inference-gateway.local"

Key Configuration Parameters

Parameter	Description	Default
replicaCount	Number of UI replicas to deploy	1
image.repository	UI container image repository	ghcr.io/inference-gateway/ui
image.tag	UI container image tag (defaults to chart appVersion if not specified)	""
gateway.enabled	Deploy the Inference Gateway backend with the UI	true
ingress.enabled	Enable ingress for external access	false
ingress.className	Ingress controller class to use	nginx
resources	CPU/Memory resource limits and requests	See values.yaml
config.*	Environment variables for the UI configuration	See values.yaml

Complete Example

For a complete example including a local Kubernetes setup with k3d, ingress configuration, and more, refer to the Kubernetes example directory.

# Example with comprehensive configuration
helm upgrade --install inference-gateway-ui \
  oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
  --version 0.7.1 \
  --create-namespace \
  --namespace inference-gateway \
  --set replicaCount=1 \
  --set gateway.enabled=true \
  --set gateway.envFrom.secretRef=inference-gateway \
  --set resources.limits.cpu=500m \
  --set resources.limits.memory=512Mi \
  --set resources.requests.cpu=100m \
  --set resources.requests.memory=128Mi \
  --set-string "env[0].name=NODE_ENV" \
  --set-string "env[0].value=production" \
  --set-string "env[1].name=NEXT_TELEMETRY_DISABLED" \
  --set-string "env[1].value=1"

Deployment

The application is automatically packaged as a Docker image and published to GitHub Container Registry (ghcr.io) when a new release is created.

To pull the latest release:

docker pull ghcr.io/inference-gateway/ui:latest

Related Projects

• Inference Gateway - The main gateway service
• Documentation - Project documentation

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please see our Contributing Guide for details on how to get started, coding standards, development workflow, and more.