You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First off, thank you for considering contributing to RubyLLM! It's people like you that make RubyLLM such a great tool.
3
+
Thank you for considering contributing to RubyLLM! We're aiming to build a high-quality, robust library, and thoughtful contributions are welcome.
4
+
5
+
## Development Setup and Workflow
6
+
7
+
Getting started and contributing follows a typical GitHub-based workflow:
8
+
9
+
1.**Fork & Clone**: Fork the repository to your own GitHub account and then clone it locally.
10
+
```bash
11
+
gh repo fork crmne/ruby_llm --clone
12
+
cd ruby_llm
13
+
```
14
+
2. **Install Dependencies**:
15
+
```bash
16
+
bundle install
17
+
```
18
+
3. **Set Up Git Hooks**: Required.
19
+
```bash
20
+
overcommit --install
21
+
```
22
+
4. **Branch**: Create a new branch for your feature or bugfix. If it relates to an existing issue, you can use the `gh` CLI to help:
23
+
```bash
24
+
gh issue develop 123 --checkout # Substitute 123 with the relevant issue number
25
+
```
26
+
5. **Code & Test**: Make your changes and ensure they are well-tested. (See "Running Tests" section for more details).
27
+
6. **Commit**: Write clear and concise commit messages.
28
+
7. **Pull Request**: Create a Pull Request (PR) against the `main` branch of the `crmne/ruby_llm` repository.
29
+
***Thoroughly review your own PR before submitting.** Check for any "vibe coding" – unnecessary files, experimental code that doesn't belong, or incomplete work.
30
+
* Write a **clear and detailed PR description** explaining the "what" and "why" of your changes. Link to any relevant issues.
31
+
* Badly/vibe-coded PRs with minimal descriptions will likely be closed or receive extensive review comments, slowing things down for everyone. Follow the existing conventions of RubyLLM. Aim for quality.
32
+
```bash
33
+
gh pr create --web
34
+
```
35
+
36
+
## Model Registry (`models.json`) & Aliases (`aliases.json`)
37
+
38
+
These files are critical for how RubyLLM identifies and uses AI models. **Both are auto-generated by rake tasks. Do not edit them manually or include manual changes to them in PRs.**
39
+
40
+
### `models.json`: The Model Catalog
41
+
42
+
* **How it's made**: The `rake models:update` task builds this file. It fetches model data directly from configured provider APIs (processing these details via each provider's `capabilities.rb` file) and also from the [Parsera LLM Specs API](https://api.parsera.org/v1/llm-specs). These lists are then merged, with Parsera's standardized data generally taking precedence forcommon models, augmented by provider-specific metadata. Models unique to a provider's API (and not yetin Parsera) are also included.
43
+
***Updating Model Information**:
44
+
***Incorrect public specs (pricing, context size, etc.)?** Parsera scrapes public provider documentation. If data for a publicly documented model is wrong or missing on Parsera, please [file an issue with Parsera](https://github.com/parsera-labs/api-llm-specs/issues). Once they update, `rake models:update` will fetch the corrections.
45
+
***Models not in public docs / Provider-specifics**: If a model isn't well-documented publicly by the provider (e.g., older or preview models) or needs provider-specific handling within RubyLLM, update the relevant `lib/ruby_llm/providers/<provider>/capabilities.rb` and potentially `models.rb`. Then run `bundle exec rake models:update`.
46
+
* **New Provider Support**: This involves more in-depth work to create the provider-specific modules and ensure integration with the `models:update` task.
47
+
48
+
### `aliases.json`: User-Friendly Shortcuts
49
+
50
+
* **Purpose**: Maps common names (e.g., `claude-3-5-sonnet`) to precise, versioned model IDs.
51
+
* **How it's made**: Generated by `rake aliases:generate` using the current `models.json`. Run this task *after*`models.json` is updated.
4
52
5
-
## Development Setup
53
+
## Running Tests
6
54
7
-
Here's how to get started:
55
+
Tests are crucial. We use RSpec and VCR.
8
56
9
57
```bash
10
-
# Clone the repository
11
-
gh repo clone crmne/ruby_llm
12
-
cd ruby_llm
13
-
14
-
# Install dependencies
15
-
bundle install
16
-
17
-
# Set up git hooks
18
-
overcommit --install
19
-
20
-
# Run the tests (uses VCR cassettes)
58
+
# Run all tests (uses existing VCR cassettes)
21
59
bundle exec rspec
22
-
```
23
-
24
-
## Development Workflow
25
-
26
-
We recommend using GitHub CLI to simplify the workflow:
27
-
28
-
```bash
29
-
# Create a new branch for your feature
30
-
gh repo fork crmne/ruby_llm --clone
31
-
cd ruby_llm
32
-
33
-
# Find or make an issue for the feature on GitHub and then:
34
-
gh issue develop 123 --checkout # Substitute 123 with the issue number
35
-
36
-
# Make your changes and test them
37
-
# ...
38
-
39
-
# Commit your changes
40
-
git commit
41
-
42
-
# Create a PR
43
-
gh pr create --web
44
-
```
45
60
46
-
## Model Naming Convention & Provider Strategy
47
-
48
-
When adding new providers to RubyLLM, please follow these guidelines:
49
-
50
-
### Normalized Model IDs
51
-
52
-
We use a consistent approach separating **what** (model) from **where** (provider):
1.**Use normalized model IDs** - Don't include provider prefixes in the model ID itself
67
-
2.**Add provider mapping** - Map the normalized IDs to your provider's specific format internally
68
-
3.**Preserve capabilities** - Ensure models accessed through your provider report the same capabilities as their native counterparts
69
-
4.**Update models.json** - Include your provider's models in models.json
70
-
5.**Update aliases.json** - Add entries to aliases.json for models accessible through your provider
71
-
6.**Implement refresh mechanism** - Ensure your provider supports the `list_models` method for refreshing
72
-
73
-
### Model Registry (`models.json`)
74
-
75
-
The `models.json` file is autogenerated. PRs shouldn't change the file manually, instead either refresh it with `rake models:update` or change the relevant `capabilities.rb` files and then refresh it.
76
-
77
-
### Model Aliases
61
+
# Run a specific test file
62
+
bundle exec rspec spec/ruby_llm/chat_spec.rb
78
63
79
-
For providers that use complex model identifiers (like Bedrock's `anthropic.claude-3-5-sonnet-20241022-v2:0:200k`), add mappings to the global aliases.json file:
64
+
# To re-record a specific test's cassette, first remove its .yml file:
65
+
rm spec/fixtures/vcr_cassettes/chat_vision_models_*_can_understand_local_images.yml # Adjust file name as needed
66
+
# Then run the specific test or test file that uses this cassette.
# Run a specific test by its description string (or part of it)
69
+
bundle exec rspec -e "can understand local images"
94
70
```
95
71
96
-
If a model can't be found with the provided ID and provider, a `ModelNotFoundError` will be raised with an informative message. Your implementation should make this error helpful by suggesting available alternatives.
97
-
98
-
When the same model has multiple versions and context windows e.g.
72
+
### Testing Philosophy & VCR
99
73
100
-
```
101
-
anthropic.claude-3-5-sonnet-20240620-v1:0
102
-
anthropic.claude-3-5-sonnet-20240620-v1:0:18k
103
-
anthropic.claude-3-5-sonnet-20240620-v1:0:200k
104
-
anthropic.claude-3-5-sonnet-20240620-v1:0:51k
105
-
anthropic.claude-3-5-sonnet-20241022-v2:0
106
-
anthropic.claude-3-5-sonnet-20241022-v2:0:18k
107
-
anthropic.claude-3-5-sonnet-20241022-v2:0:200k
108
-
anthropic.claude-3-5-sonnet-20241022-v2:0:51k
109
-
```
74
+
* New tests should generally be **end-to-end** to verify integration with actual provider APIs (via VCR).
75
+
* Keep tests **minimal and focused**. We don't need to test every single model variant for every feature if the underlying API mechanism is the same. One or two representative models per provider for a given feature is usually sufficient.
76
+
* **API Call Costs**: VCR cassettes are used to avoid hitting live APIs on every test run. However, recording these cassettes costs real money for API calls. Please be mindful of this when adding tests that would require new recordings. If you're adding extensive tests that significantly increase API usage for VCR recording, consider [sponsoring the project on GitHub](https://github.com/sponsors/crmne) to help offset these costs.
110
77
111
-
We default all aliases to the biggest context window, and the main alias (without date) to the latest version:
If your changes affect API interactions, you'll need to re-record the VCR cassettes.
131
81
132
-
Tests automatically use VCR to record and replay HTTP interactions, so you don't need real API keys for testing:
82
+
To re-record cassettes for specific providers (e.g., OpenAI and Anthropic):
133
83
134
84
```bash
135
-
# Run all tests (using existing VCR cassettes)
136
-
bundle exec rspec
85
+
# Set necessary API keys as environment variables
86
+
export OPENAI_API_KEY="your_openai_key"
87
+
export ANTHROPIC_API_KEY="your_anthropic_key"
137
88
138
-
# Run a specific test file
139
-
bundle execrspec spec/ruby_llm/chat_spec.rb
89
+
# Run the rake task, specifying providers
90
+
bundle exec rake vcr:record[openai,anthropic]
140
91
```
141
92
142
-
### Recording VCR Cassettes
143
-
144
-
When you make changes that affect API interactions, you can record new VCR cassettes.
145
-
146
-
If you have keys for all providers:
93
+
To re-record all cassettes (requires all relevant API keys to be set):
147
94
148
95
```bash
149
-
# Re-record all cassettes
150
96
bundle exec rake vcr:record[all]
151
97
```
152
98
153
-
If you only have keys for specific providers (e.g., just OpenAI):
99
+
The rake task will delete the relevant existing cassettes and re-run the tests to record fresh interactions.
154
100
155
-
```bash
156
-
# Set the API keys you have
157
-
export OPENAI_API_KEY=your_openai_key
158
-
159
-
# Find and remove only cassettes for OpenAI, then run tests to re-record them
160
-
bundle exec rake vcr:record[openai]
161
-
162
-
# You can also specify multiple providers
163
-
bundle exec rake vcr:record[openai,anthropic]
164
-
```
165
-
166
-
Important: After recording new cassettes, please **manually check** them for any sensitive information that might have been missed by the automatic filters.
167
-
168
-
## Adding New Tests
169
-
170
-
Tests automatically create VCR cassettes based on their descriptions, so make sure your test descriptions are unique and descriptive.
101
+
**CRITICAL**: After recording new or updated VCR cassettes, **manually inspect the YAML files in `spec/fixtures/vcr_cassettes/`**. Ensure that no sensitive information (API keys, personal data, etc.) has accidentally been recorded. The VCR configuration has filters for common keys, but diligence is required.
171
102
172
103
## Coding Style
173
104
174
-
We follow the [Standard Ruby](https://github.com/testdouble/standard) style. Please ensure your contributions adhere to this style.
105
+
We follow the [Standard Ruby](https://github.com/testdouble/standard) style guide.
175
106
176
107
```bash
177
108
# Check your code style
@@ -181,27 +112,27 @@ bundle exec rubocop
181
112
bundle exec rubocop -A
182
113
```
183
114
184
-
## Documentation
185
-
186
-
When adding new features, please include documentation updates:
115
+
The Overcommit pre-commit hook should help enforce this.
187
116
188
-
- Update relevant guides in the `docs/guides/` directory
189
-
- Add inline documentation using YARD comments
190
-
- Keep the README clean and focused on helping new users get started quickly
117
+
## Documentation
191
118
192
-
## Discussions and Issues
119
+
If you add new features or change existing behavior, please update the documentation:
193
120
194
-
- For questions and discussions, please use [GitHub Discussions](https://github.com/crmne/ruby_llm/discussions)
195
-
- For bugs and feature requests, please use [GitHub Issues](https://github.com/crmne/ruby_llm/issues)
121
+
* Update relevant guides in the `docs/guides/` directory.
122
+
* Ensure the `README.md` remains a concise and helpful entry point for new users.
0 commit comments