🔍 CodeIndexer

An open-source code indexing and semantic search tool implemented by Milvus vector database and popular embedding models. You can build your AI Coding IDE or code search plugin with it.

🌟 Why CodeIndexer?

In the AI-first development era, traditional keyword-based search is no longer sufficient for modern software development:

🚀 The AI Coding Revolution

AI-Powered IDEs like Cursor and GitHub Copilot are transforming development workflows
Growing demand for intelligent code assistance and semantic understanding
Modern codebases contain millions of lines across hundreds of files, making manual navigation inefficient

❌ Current Limitations

Regex and keyword-based search miss contextual relationships
Developers waste time navigating large codebases manually
Knowledge transfer between team members is inefficient
Traditional search tools can't bridge the gap between human intent and code implementation

✅ Our Solution

CodeIndexer bridges the gap between human understanding and code discovery through:

Semantic search with natural language queries like "find authentication functions"
AI-powered understanding of code meaning and relationships
Universal integration across multiple platforms and development environments

💡 Find code by describing functionality, not just keywords - Discover existing solutions before writing duplicate code.

✨ Features

🔍 Semantic Code Search: Ask questions like "find functions that handle user authentication" instead of guessing keywords
📁 Intelligent Indexing: Automatically index entire codebases and build semantic vector databases with contextual understanding
🎯 Context-Aware Discovery: Find related code snippets based on meaning, not just text matching
⚡ Incremental File Synchronization: Efficient change detection using Merkle trees to only re-index modified files
🧩 Smart Chunking: AST-based code splitting that preserves context and structure
🚀 Developer Productivity: Significantly reduce time spent searching for relevant code and discovering existing solutions
🔧 Embedding Providers: Support for OpenAI, VoyageAI, Ollama as embedding providers
💾 Vector Storage: Integrated with Milvus/Zilliz Cloud for efficient storage and retrieval
🛠️ VSCode Integration: Built-in VSCode extension for seamless development workflow
🤖 MCP Support: Model Context Protocol integration for AI agent interactions
📊 Progress Tracking: Real-time progress feedback during indexing operations
🎨 Customizable: Configurable file extensions, ignore patterns, and embedding models

🏗️ Architecture

CodeIndexer is a monorepo containing three main packages:

Core Components

@code-indexer/core: Core indexing engine with embedding and vector database integration
VSCode Extension: Semantic Code Search extension for Visual Studio Code
@code-indexer/mcp: Model Context Protocol server for AI agent integration

Supported Technologies

Embedding Providers: OpenAI, VoyageAI, Ollama
Vector Databases: Milvus or Zilliz Cloud(fully managed vector database as a service)
Code Splitters: AST-based splitter (with automatic fallback), LangChain character-based splitter
Languages: TypeScript, JavaScript, Python, Java, C++, C#, Go, Rust, PHP, Ruby, Swift, Kotlin, Scala, Markdown
Development Tools: VSCode, Model Context Protocol

🚀 Quick Start (for the core package)

Prerequisites

Node.js >= 20.0.0
pnpm >= 10.0.0
Milvus database
OpenAI or VoyageAI API key

Installation

# Using npm
npm install @code-indexer/core

# Using pnpm
pnpm add @code-indexer/core

# Using yarn
yarn add @code-indexer/core

Prepare Environment Variables

OpenAI API key

See OpenAI Documentation for more details to get your API key.

OPENAI_API_KEY=your-openai-api-key

Milvus configuration

Optional 1: Self-hosted Milvus See Milvus Documentation for more details to install Milvus.

MILVUS_ADDRESS is the address of your Milvus instance
(Optional)MILVUS_TOKEN is the token of your Milvus instance, which can be left empty if you don't use token-based authentication.

MILVUS_ADDRESS=localhost:19530
MILVUS_TOKEN=your-milvus-token

Optional 2: Zilliz Cloud(fully managed vector database as a service, you can use it for free)

MILVUS_ADDRESS is the Public Endpoint of your Zilliz Cloud instance
MILVUS_TOKEN is the token of your Zilliz Cloud instance.

MILVUS_ADDRESS=https://xxx-xxxxxxxxxxxx.serverless.gcp-us-west1.cloud.zilliz.com
MILVUS_TOKEN=xxxxxxx

Basic Usage

@code-indexer/core Core indexing engine that provides the fundamental functionality for code indexing and semantic search. Handles embedding generation, vector storage, and search operations.

import { CodeIndexer, MilvusVectorDatabase, OpenAIEmbedding } from '@code-indexer/core';

// Initialize embedding provider
const embedding = new OpenAIEmbedding({
    apiKey: process.env.OPENAI_API_KEY || 'your-openai-api-key',
    model: 'text-embedding-3-small'
});

// Initialize vector database
const vectorDatabase = new MilvusVectorDatabase({
    address: process.env.MILVUS_ADDRESS || 'localhost:19530',
    token: process.env.MILVUS_TOKEN || ''
});

// Create indexer instance
const indexer = new CodeIndexer({
    embedding,
    vectorDatabase
});

// Index your codebase with progress tracking
const stats = await indexer.indexCodebase('./your-project', (progress) => {
    console.log(`${progress.phase} - ${progress.percentage}%`);
});
console.log(`Indexed ${stats.indexedFiles} files, ${stats.totalChunks} chunks`);

// Perform semantic search
const results = await indexer.semanticSearch('./your-project', 'vector database operations', 5);
results.forEach(result => {
    console.log(`File: ${result.relativePath}:${result.startLine}-${result.endLine}`);
    console.log(`Score: ${(result.score * 100).toFixed(2)}%`);
    console.log(`Content: ${result.content.substring(0, 100)}...`);
});

📦 Built on Core

All the following packages are built on top of the @code-indexer/core engine, extending its capabilities to different platforms and use cases. They leverage the core's semantic search and indexing functionality to provide specialized interfaces and integrations.

📖 Each package has its own detailed documentation and usage examples. Click the links below to learn more.

@code-indexer/mcp

Model Context Protocol (MCP) server that enables AI assistants and agents to interact with CodeIndexer through a standardized protocol. Exposes indexing and search capabilities via MCP tools.

Cursor

Go to: Settings -> Cursor Settings -> MCP -> Add new global MCP server

Pasting the following configuration into your Cursor ~/.cursor/mcp.json file is the recommended approach. You may also install in a specific project by creating .cursor/mcp.json in your project folder. See Cursor MCP docs for more info.

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["-y", "@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

Claude Desktop

Add to your Claude Desktop configuration:

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

Claude Code

Use the command line interface to add the CodeIndexer MCP server:

# Add the CodeIndexer MCP server
claude mcp add code-indexer -e OPENAI_API_KEY=your-openai-api-key -e MILVUS_ADDRESS=localhost:19530 -- npx @code-indexer/mcp@latest

See the Claude Code MCP documentation for more details about MCP server management.

Windsurf

Windsurf supports MCP configuration through a JSON file. Add the following configuration to your Windsurf MCP settings:

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["-y", "@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

VS Code

The CodeIndexer MCP server can be used with VS Code through MCP-compatible extensions. Add the following configuration to your VS Code MCP settings:

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["-y", "@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

Cherry Studio

Cherry Studio allows for visual MCP server configuration through its settings interface. While it doesn't directly support manual JSON configuration, you can add a new server via the GUI:

Navigate to Settings → MCP Servers → Add Server.
Fill in the server details:
- Name: code-indexer
- Type: STDIO
- Command: npx
- Arguments: ["@code-indexer/mcp@latest"]
- Environment Variables:
  - OPENAI_API_KEY: your-openai-api-key
  - MILVUS_ADDRESS: localhost:19530
Save the configuration to activate the server.

Cline

Cline uses a JSON configuration file to manage MCP servers. To integrate the provided MCP server configuration:

Open Cline and click on the MCP Servers icon in the top navigation bar.
Select the Installed tab, then click Advanced MCP Settings.
In the cline_mcp_settings.json file, add the following configuration:

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

Save the file.

Augment

To configure Code Indexer MCP in Augment Code, you can use either the graphical interface or manual configuration.

A. Using the Augment Code UI

Click the hamburger menu.
Select Settings.
Navigate to the Tools section.
Click the + Add MCP button.
Enter the following command:
```
npx @code-indexer/mcp@latest
```
Name the MCP: Code Indexer.
Click the Add button.

B. Manual Configuration

Press Cmd/Ctrl Shift P or go to the hamburger menu in the Augment panel
Select Edit Settings
Under Advanced, click Edit in settings.json
Add the server configuration to the mcpServers array in the augment.advanced object

"augment.advanced": { 
  "mcpServers": [ 
    { 
      "name": "code-indexer", 
      "command": "npx", 
      "args": ["-y", "@code-indexer/mcp@latest"] 
    } 
  ] 
}

Gemini CLI

Gemini CLI requires manual configuration through a JSON file:

Create or edit the ~/.gemini/settings.json file.
Add the following configuration:

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

Save the file and restart Gemini CLI to apply the changes.

Roo Code

Roo Code utilizes a JSON configuration file for MCP servers:

Open Roo Code and navigate to Settings → MCP Servers → Edit Global Config.
In the mcp_settings.json file, add the following configuration:

{
  "mcpServers": {
    "code-indexer": {
      "command": "npx",
      "args": ["@code-indexer/mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key",
        "MILVUS_ADDRESS": "localhost:19530"
      }
    }
  }
}

Save the file to activate the server.

Other MCP Clients

The server uses stdio transport and follows the standard MCP protocol. It can be integrated with any MCP-compatible client by running:

npx @code-indexer/mcp@latest

VSCode Extension

Visual Studio Code extension that integrates CodeIndexer directly into your IDE. Provides an intuitive interface for semantic code search and navigation.

Direct Link: Install from VS Code Marketplace
Manual Search:
- Open Extensions view in VSCode (Ctrl+Shift+X or Cmd+Shift+X on Mac)
- Search for "Semantic Code Search"
- Click Install

🛠️ Development

Setup Development Environment

# Clone repository
git clone https://github.com/zilliztech/CodeIndexer.git
cd CodeIndexer

# Install dependencies
pnpm install

# Build all packages
pnpm build

# Start development mode
pnpm dev

Building

# Build all packages
pnpm build

# Build specific package
pnpm build:core
pnpm build:vscode
pnpm build:mcp

Running Examples

# Development with file watching
cd examples/basic-usage
pnpm dev

Supported File Extensions

By default, CodeIndexer supports:

Programming languages: .ts, .tsx, .js, .jsx, .py, .java, .cpp, .c, .h, .hpp, .cs, .go, .rs, .php, .rb, .swift, .kt, .scala, .m, .mm
Documentation: .md, .markdown

Ignore Patterns

Common directories and files are automatically ignored:

node_modules/**, dist/**, build/**
.git/**, .vscode/**, .idea/**
*.log, *.min.js, *.map

📖 Examples

Check the /examples directory for complete usage examples:

Basic Usage: Simple indexing and search example

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details on how to get started.

Package-specific contributing guides:

🗺️ Roadmap

AST-based code analysis for improved understanding
Support for additional embedding providers
Agent-based interactive search mode
Enhanced code chunking strategies
Search result ranking optimization
Robust Chrome Extension

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Name		Name	Last commit message	Last commit date
Latest commit History 41 Commits
.github/workflows		.github/workflows
.vscode		.vscode
assets		assets
examples		examples
packages		packages
python		python
.env.example		.env.example
.eslintrc.js		.eslintrc.js
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json

License

zilliztech/CodeIndexer

Folders and files

Latest commit

History

Repository files navigation