Skip to content

Commit d5975f7

Browse files
FindHaofacebook-github-bot
authored andcommitted
Add Comprehensive CI Scripts and Documentation (#26)
Summary: ## 🚀 Add Comprehensive CI Scripts and Documentation ### 📋 Overview This PR introduces a complete CI infrastructure for tritonparse, including modular scripts for environment setup, dependency installation, and test execution, along with comprehensive documentation. ### 🆕 New Features #### CI Scripts (`.ci/` directory): - **`setup.sh`** - Sets up conda environment, installs system dependencies, and configures CUDA/cuDNN - **`install-triton.sh`** - Installs Triton from source with proper libstdc++ handling - **`install-cudnn.sh`** - Downloads and installs cuDNN with architecture detection - **`install-project.sh`** - Installs tritonparse in editable mode with test dependencies - **`run-tests.sh`** - Runs test suite with configurable options (CPU/CUDA/all, coverage) - **`README.md`** - Comprehensive documentation for all CI scripts #### Key Script Features: - **Environment Variables**: Configurable defaults (Python 3.11, CUDA 12.8, cuDNN 9.10.2.21) - **Error Handling**: All scripts use `set -e` for fail-fast behavior - **Modular Design**: Each script has single responsibility and can be used independently - **Verbose Logging**: Detailed output for debugging and monitoring - **Architecture Support**: Automatic detection for x86_64 and ARM64/aarch64 #### Test Configuration: - **CPU-only tests**: `TEST_TYPE=cpu` - **CUDA-only tests**: `TEST_TYPE=cuda` - **All tests**: `TEST_TYPE=all` (default) - **Coverage reporting**: `COVERAGE=true` ### 📚 Documentation - Complete workflow examples for CI and local development - Environment variable reference for all scripts - Usage examples with different configurations - Script dependencies and assumptions clearly documented ### 🎯 Benefits - **Reproducible Builds**: Same environment setup for CI and local development - **Easy Maintenance**: Modular scripts that can be updated independently - **Flexible Testing**: Support for different test types and coverage reporting - **Developer Experience**: Clear documentation and examples for all use cases Pull Request resolved: #26 Test Plan: #### Github Actions: https://github.com/pytorch-labs/tritonparse/actions/workflows/test.yml #### Manual Test: ```bash # Complete setup workflow CONDA_ENV=tritonparse bash .ci/setup.sh CONDA_ENV=tritonparse bash .ci/install-triton.sh CONDA_ENV=tritonparse bash .ci/install-project.sh CONDA_ENV=tritonparse bash .ci/run-tests.sh # Run specific test types TEST_TYPE=cuda COVERAGE=true bash .ci/run-tests.sh ``` Reviewed By: xuzhao9 Differential Revision: D77978115 Pulled By: FindHao fbshipit-source-id: bc91c2c62f8e24585708692a6e8498e73c1ae6b9
1 parent 021641c commit d5975f7

File tree

6 files changed

+782
-0
lines changed

6 files changed

+782
-0
lines changed

.ci/README.md

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# CI Scripts
2+
3+
This directory contains scripts for setting up and running the tritonparse CI environment.
4+
5+
## Scripts Overview
6+
7+
### `setup.sh`
8+
Sets up the conda environment, installs dependencies, configures CUDA, and installs cuDNN.
9+
10+
**Environment Variables:**
11+
- `CONDA_ENV`: Conda environment name (default: "tritonparse")
12+
- `PYTHON_VERSION`: Python version (default: "3.11")
13+
- `CUDA_VERSION`: CUDA version (default: "12.8")
14+
- `CUDNN_VERSION`: cuDNN version (default: "9.10.2.21")
15+
16+
**Usage:**
17+
```bash
18+
CONDA_ENV=tritonparse PYTHON_VERSION=3.11 bash .ci/setup.sh
19+
```
20+
21+
> **Note:**
22+
> `setup.sh` will automatically download and execute the official PyTorch cuDNN installation script from:
23+
> https://github.com/pytorch/pytorch/blob/main/.ci/docker/common/install_cudnn.sh
24+
> There is no need to maintain a local cuDNN install script.
25+
26+
### `install-triton.sh`
27+
Installs Triton from source by cloning the repository and building it.
28+
29+
**Environment Variables:**
30+
- `CONDA_ENV`: Conda environment name (required)
31+
32+
**Usage:**
33+
```bash
34+
CONDA_ENV=tritonparse bash .ci/install-triton.sh
35+
```
36+
37+
### `install-project.sh`
38+
Installs the tritonparse project in editable mode with test dependencies.
39+
40+
**Environment Variables:**
41+
- `CONDA_ENV`: Conda environment name (required)
42+
43+
**Usage:**
44+
```bash
45+
CONDA_ENV=tritonparse bash .ci/install-project.sh
46+
```
47+
48+
### `run-tests.sh`
49+
Runs the test suite with proper environment setup.
50+
51+
**Environment Variables:**
52+
- `CONDA_ENV`: Conda environment name (required)
53+
- `TEST_TYPE`: Type of tests to run (default: "all")
54+
- `cpu`: CPU tests only
55+
- `cuda`: CUDA tests only
56+
- `all`: All tests
57+
- `VERBOSE`: Enable verbose output (default: "true")
58+
- `COVERAGE`: Enable coverage reporting (default: "false")
59+
60+
**Usage:**
61+
```bash
62+
# Run all tests
63+
CONDA_ENV=tritonparse bash .ci/run-tests.sh
64+
65+
# Run CPU tests only with coverage
66+
CONDA_ENV=tritonparse TEST_TYPE=cpu COVERAGE=true bash .ci/run-tests.sh
67+
68+
# Run CUDA tests only
69+
CONDA_ENV=tritonparse TEST_TYPE=cuda bash .ci/run-tests.sh
70+
```
71+
72+
## Complete Workflow
73+
74+
For a complete setup and test run:
75+
76+
```bash
77+
# 1. Setup environment (includes cuDNN installation)
78+
CONDA_ENV=tritonparse PYTHON_VERSION=3.11 bash .ci/setup.sh
79+
80+
# 2. Install Triton
81+
CONDA_ENV=tritonparse bash .ci/install-triton.sh
82+
83+
# 3. Install project
84+
CONDA_ENV=tritonparse bash .ci/install-project.sh
85+
86+
# 4. Run tests
87+
CONDA_ENV=tritonparse TEST_TYPE=all COVERAGE=true bash .ci/run-tests.sh
88+
```
89+
90+
## Local Development
91+
92+
For local development, you can use these scripts to set up the same environment as CI:
93+
94+
```bash
95+
# Setup local environment (includes cuDNN installation)
96+
CONDA_ENV=tritonparse-local bash .ci/setup.sh
97+
98+
# Install Triton
99+
CONDA_ENV=tritonparse-local bash .ci/install-triton.sh
100+
101+
# Install project
102+
CONDA_ENV=tritonparse-local bash .ci/install-project.sh
103+
104+
# Run tests
105+
CONDA_ENV=tritonparse-local bash .ci/run-tests.sh
106+
```
107+
108+
## Script Features
109+
110+
- **Error handling**: All scripts use `set -e` to stop on errors
111+
- **Environment validation**: Scripts check for required environment variables
112+
- **Verbose output**: Detailed logging for debugging
113+
- **Modular design**: Each script has a single responsibility
114+
- **Reusable**: Scripts can be used in different contexts (CI, local development)
115+
116+
## Dependencies
117+
118+
The scripts assume:
119+
- Linux environment
120+
- Git available
121+
- Internet access for downloading packages
122+
- Sufficient disk space for conda and packages

.ci/install-project.sh

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/bash
2+
3+
# Install tritonparse project dependencies
4+
# This script installs the project in editable mode with test dependencies
5+
6+
set -e
7+
8+
echo "Installing tritonparse project dependencies..."
9+
10+
# Ensure we're in the conda environment
11+
if [ -z "$CONDA_ENV" ]; then
12+
echo "ERROR: CONDA_ENV is not set"
13+
exit 1
14+
fi
15+
16+
# Activate conda environment
17+
source /opt/miniconda3/etc/profile.d/conda.sh
18+
conda activate "$CONDA_ENV"
19+
20+
# Upgrade pip
21+
echo "Upgrading pip..."
22+
python -m pip install --upgrade pip
23+
24+
# Install project in editable mode with test dependencies
25+
echo "Installing tritonparse in editable mode..."
26+
pip install -e ".[test]"
27+
28+
# Verify installation
29+
echo "Verifying installation..."
30+
python -c "import tritonparse; print(f'tritonparse installed successfully')"
31+
python -c "import coverage; print(f'coverage version: {coverage.__version__}')"
32+
33+
echo "Project installation completed successfully!"

.ci/install-triton.sh

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
#!/bin/bash
2+
3+
# Install Triton from source
4+
# This script clones and installs Triton from the main repository
5+
6+
set -e
7+
8+
echo "🚀 Installing Triton from source..."
9+
START_TIME=$(date +%s)
10+
11+
# Function to show elapsed time
12+
show_elapsed() {
13+
CURRENT_TIME=$(date +%s)
14+
ELAPSED=$((CURRENT_TIME - START_TIME))
15+
echo "⏱️ Elapsed time: ${ELAPSED}s"
16+
}
17+
18+
# Pre-flight checks
19+
echo "🔍 Running pre-flight checks..."
20+
21+
# Set Triton version/commit for cache consistency
22+
TRITON_COMMIT=${TRITON_COMMIT:-"main"}
23+
echo "🎯 Target Triton commit/branch: $TRITON_COMMIT"
24+
TRITON_CACHE_DIR="/tmp/triton-cache"
25+
TRITON_SOURCE_DIR="/tmp/triton"
26+
27+
# Check disk space (need at least 10GB for Triton compilation)
28+
AVAILABLE_SPACE=$(df /tmp | tail -1 | awk '{print $4}')
29+
REQUIRED_SPACE=10485760 # 10GB in KB
30+
if [ "$AVAILABLE_SPACE" -lt "$REQUIRED_SPACE" ]; then
31+
echo "⚠️ WARNING: Low disk space. Available: $(($AVAILABLE_SPACE / 1024 / 1024))GB, Recommended: 10GB"
32+
else
33+
echo "✅ Sufficient disk space available: $(($AVAILABLE_SPACE / 1024 / 1024))GB"
34+
fi
35+
36+
# Ensure we're in the conda environment
37+
if [ -z "$CONDA_ENV" ]; then
38+
echo "ERROR: CONDA_ENV is not set"
39+
exit 1
40+
fi
41+
42+
# Activate conda environment
43+
source /opt/miniconda3/etc/profile.d/conda.sh
44+
conda activate "$CONDA_ENV"
45+
46+
# Create cache directory
47+
mkdir -p "$TRITON_CACHE_DIR"
48+
49+
# Check if we have cached source with correct commit
50+
if [ -f "$TRITON_CACHE_DIR/commit" ] && [ -d "$TRITON_SOURCE_DIR" ]; then
51+
CACHED_COMMIT=$(cat "$TRITON_CACHE_DIR/commit")
52+
if [ "$CACHED_COMMIT" = "$TRITON_COMMIT" ] && [ "$TRITON_COMMIT" != "main" ]; then
53+
echo "Found cached Triton source with correct commit ($CACHED_COMMIT)"
54+
echo "Will use cached source and re-install to new conda environment"
55+
USE_CACHED_SOURCE=true
56+
elif [ "$TRITON_COMMIT" = "main" ]; then
57+
echo "Target is 'main' branch (API fallback), will rebuild from scratch"
58+
echo "Cached commit: $CACHED_COMMIT"
59+
USE_CACHED_SOURCE=false
60+
else
61+
echo "Cached source commit mismatch: cached=$CACHED_COMMIT, target=$TRITON_COMMIT"
62+
echo "Will rebuild from scratch"
63+
USE_CACHED_SOURCE=false
64+
fi
65+
else
66+
echo "No cached source found or no commit info, will build from scratch"
67+
USE_CACHED_SOURCE=false
68+
fi
69+
70+
# Update libstdc++ to match system version
71+
# Otherwise, we get errors like:
72+
# ImportError: /opt/miniconda3/envs/tritonparse/bin/../lib/libstdc++.so.6:
73+
# version `GLIBCXX_3.4.30' not found (required by /tmp/triton/python/triton/_C/libtriton.so)
74+
echo "Updating libstdc++ to match system version..."
75+
conda install -y -c conda-forge libstdcxx-ng=12.3.0
76+
# Check if the update was successful
77+
strings /opt/miniconda3/envs/tritonparse/lib/libstdc++.so.6 | grep GLIBCXX | tail -5
78+
79+
# Uninstall existing pytorch-triton
80+
echo "Uninstalling existing pytorch-triton..."
81+
pip uninstall -y pytorch-triton || true
82+
pip uninstall -y triton || true
83+
84+
# Setup Triton repository based on cache status
85+
if [ "$USE_CACHED_SOURCE" = "true" ]; then
86+
echo "Using cached Triton source..."
87+
cd "$TRITON_SOURCE_DIR"
88+
ACTUAL_COMMIT=$(git rev-parse HEAD)
89+
echo "Using cached Triton commit: $ACTUAL_COMMIT"
90+
else
91+
echo "Setting up Triton repository from scratch..."
92+
if [ -d "$TRITON_SOURCE_DIR" ]; then
93+
echo "Removing existing source directory..."
94+
rm -rf "$TRITON_SOURCE_DIR"
95+
fi
96+
97+
echo "Cloning Triton repository..."
98+
if ! git clone https://github.com/triton-lang/triton.git "$TRITON_SOURCE_DIR"; then
99+
echo "❌ ERROR: Failed to clone Triton repository"
100+
echo "This might be due to network issues or GitHub rate limiting"
101+
exit 1
102+
fi
103+
104+
cd "$TRITON_SOURCE_DIR"
105+
106+
# Checkout specific commit for reproducibility
107+
echo "Checking out commit: $TRITON_COMMIT"
108+
if ! git checkout "$TRITON_COMMIT"; then
109+
echo "❌ ERROR: Failed to checkout commit $TRITON_COMMIT"
110+
echo "This might be due to an invalid commit hash or network issues"
111+
exit 1
112+
fi
113+
114+
ACTUAL_COMMIT=$(git rev-parse HEAD)
115+
echo "✅ Using Triton commit: $ACTUAL_COMMIT"
116+
fi
117+
118+
# Install build dependencies
119+
echo "Installing build dependencies..."
120+
pip install ninja cmake wheel pybind11
121+
122+
# Install Triton requirements
123+
echo "Installing Triton requirements..."
124+
pip install -r python/requirements.txt
125+
126+
# Set environment to use clang compiler for faster compilation
127+
echo "Setting up clang compiler for faster compilation..."
128+
export CC=clang
129+
export CXX=clang++
130+
echo "Using CC: $CC"
131+
echo "Using CXX: $CXX"
132+
133+
# Install Triton in editable mode with clang
134+
if [ "$USE_CACHED_SOURCE" = "true" ]; then
135+
echo "Installing cached Triton to new conda environment..."
136+
echo "This should be fast since build artifacts are cached"
137+
else
138+
echo "Compiling and installing Triton from scratch..."
139+
echo "This will take 30-50 minutes for compilation"
140+
fi
141+
pip install -e .
142+
show_elapsed
143+
144+
# Verify Triton installation
145+
echo "Verifying Triton installation..."
146+
if python -c "import triton; print(f'Triton version: {triton.__version__}')" 2>/dev/null; then
147+
python -c "import triton; print(f'Triton path: {triton.__file__}')"
148+
echo "✅ Triton installation verified successfully"
149+
150+
# Only save commit info after successful verification
151+
echo "$ACTUAL_COMMIT" >"$TRITON_CACHE_DIR/commit"
152+
echo "✅ Cache information saved"
153+
154+
show_elapsed
155+
echo "🎉 Triton installation completed successfully!"
156+
else
157+
echo "❌ ERROR: Failed to import triton"
158+
echo "This might be due to libstdc++ version issues"
159+
echo "Checking system libstdc++ version:"
160+
strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX | tail -5 || echo "Could not check system libstdc++"
161+
echo "Checking conda libstdc++ version:"
162+
strings /opt/miniconda3/envs/tritonparse/lib/libstdc++.so.6 | grep GLIBCXX | tail -5 || echo "Could not check conda libstdc++"
163+
164+
# Clean up cache on failure to prevent corruption
165+
echo "🧹 Cleaning up cache due to installation failure..."
166+
rm -f "$TRITON_CACHE_DIR/commit"
167+
168+
exit 1
169+
fi

0 commit comments

Comments
 (0)