Skip to content

@deepinfra Deep Infra

Change the repository type filter

All

    Repositories list

    31 repositories

    • TensorRT-LLM

      Public
      TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
      C++
      •
      Apache License 2.0
      •1.7k•0•0•1•Updated Aug 16, 2025Aug 16, 2025

    • kilocode

      Public
      Open Source AI coding assistant for planning, building, and fixing code. We're a superset of Roo, Cline, and our own features. Follow us: kilocode.ai/social
      TypeScript
      •
      Apache License 2.0
      •672•0•0•0•Updated Aug 15, 2025Aug 15, 2025

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •9.4k•0•0•0•Updated Aug 11, 2025Aug 11, 2025

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •2.6k•0•0•0•Updated Aug 11, 2025Aug 11, 2025

    • ocr-tools

      Public
      Python
      •1•4•1•0•Updated Aug 2, 2025Aug 2, 2025

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •30k•0•0•0•Updated Jul 29, 2025Jul 29, 2025

    • olmocr

      Public
      Toolkit for linearizing PDFs for LLM datasets/training
      Python
      •
      Apache License 2.0
      •1k•0•0•0•Updated Jul 9, 2025Jul 9, 2025

    • Kokoro-FastAPI

      Public
      Dockerized FastAPI wrapper for Kokoro-82M text-to-speech model w/CPU ONNX and NVIDIA GPU PyTorch support, handling, and auto-stitching
      Python
      •
      Apache License 2.0
      •548•0•0•0•Updated May 28, 2025May 28, 2025

    • tensorrtllm_backend

      Public
      The Triton TensorRT-LLM Backend
      Python
      •
      Apache License 2.0
      •129•0•0•0•Updated May 8, 2025May 8, 2025

    • deepinfra-chat

      Public
      Sample Next.js ai chat app using Deep Infra inference and Vercel ai sdk
      TypeScript
      •2•1•0•0•Updated Mar 17, 2025Mar 17, 2025

    • cutlass

      Public
      CUDA Templates for Linear Algebra Subroutines
      C++
      •
      Other
      •1.4k•0•0•0•Updated Mar 15, 2025Mar 15, 2025

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1.9k•0•0•0•Updated Feb 20, 2025Feb 20, 2025

    • Zonos

      Public
      Python
      •
      Apache License 2.0
      •784•0•0•0•Updated Feb 12, 2025Feb 12, 2025

    • Pyramid-Flow

      Public
      Code of Pyramidal Flow Matching for Efficient Video Generative Modeling
      Python
      •
      MIT License
      •298•0•0•0•Updated Oct 21, 2024Oct 21, 2024

    • llama-stack

      Public
      Model components of the Llama Stack APIs
      Python
      •
      MIT License
      •1.1k•0•0•0•Updated Oct 10, 2024Oct 10, 2024

    • ngx-http-auth-jwt-module

      Public
      Secure your NGINX locations with JWT
      Shell
      •
      MIT License
      •131•0•0•0•Updated Jun 17, 2024Jun 17, 2024

    • deepctl

      Public
      Command line tool for Deep Infra cloud ML inference service
      Rust
      •
      Apache License 2.0
      •3•32•2•0•Updated Jun 10, 2024Jun 10, 2024

    • langchainjs

      Public
      🦜🔗 Build context-aware reasoning applications 🦜🔗
      TypeScript
      •
      MIT License
      •2.7k•0•0•0•Updated May 31, 2024May 31, 2024

    • deepinfra-node

      Public
      Official TypeScript wrapper for DeepInfra Inference API
      javascriptapiwrappertypescriptdeep-learningapi-clientllmllm-inference
      TypeScript
      •
      MIT License
      •3•14•5•2•Updated May 13, 2024May 13, 2024

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •2.6k•0•0•0•Updated Apr 29, 2024Apr 29, 2024

    • langchain

      Public
      ⚡ Building applications with LLMs through composability ⚡
      Python
      •
      MIT License
      •19k•1•0•0•Updated Jan 22, 2024Jan 22, 2024

    • litellm

      Public
      Call all LLM APIs using the OpenAI format. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs)
      Python
      •
      MIT License
      •3.8k•0•0•0•Updated Jan 8, 2024Jan 8, 2024

    • text-generation-inference

      Public
      Large Language Model Text Generation Inference
      Python
      •
      Apache License 2.0
      •1.2k•9•0•6•Updated Dec 15, 2023Dec 15, 2023

    • fetch-stream-parser

      Public
      fetch-stream
      JavaScript
      •
      Apache License 2.0
      •1•0•0•0•Updated Nov 6, 2023Nov 6, 2023

    • fetch-event-source

      Public
      A better API for making Event Source requests, with all the features of fetch()
      TypeScript
      •
      MIT License
      •170•0•0•0•Updated Aug 18, 2023Aug 18, 2023

    • cog

      Public
      Containers for machine learning
      Go
      •
      Apache License 2.0
      •627•0•0•0•Updated Aug 1, 2023Aug 1, 2023

    • cog-llama-2

      Public
      A cog for running llama-2 using llama.cpp server
      Python
      •0•0•0•0•Updated Aug 1, 2023Aug 1, 2023

    • superfans-gpu-controller

      Public
      NVIDIA GPU-based FAN controller for SUPERMICRO server
      Python
      •
      MIT License
      •3•0•0•0•Updated Apr 25, 2023Apr 25, 2023

    • whisper-timestamped

      Public
      Multilingual Automatic Speech Recognition with word-level timestamps and confidence
      Python
      •
      GNU Affero General Public License v3.0
      •194•0•0•0•Updated Mar 7, 2023Mar 7, 2023

    • sentence-transformers

      Public
      Multilingual Sentence & Image Embeddings with BERT
      Python
      •
      Apache License 2.0
      •2.7k•0•0•0•Updated Feb 28, 2023Feb 28, 2023