-
-
Notifications
You must be signed in to change notification settings - Fork 286
Add Content Moderation Feature #383
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A pretty great PR! Love the attention to detail and the fact that you (and/or your AI assistant) have replicated the existing patterns in RubyLLM.
That said, I left you some comments, and I believe we should also implement multi-modal moderation.
- Rename RubyLLM::Moderate class to RubyLLM::Moderation - Rename .ask() method to .moderate() for better semantic clarity - Update all references in lib/, spec/, and docs/ - Rename corresponding test files and VCR cassettes - Maintain backward compatibility through global RubyLLM.moderate method - Add demo script showing new API usage BREAKING CHANGE: RubyLLM::Moderate.ask() is now RubyLLM::Moderation.moderate()
@crmne Kindly check the fixes and let me know! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #383 +/- ##
==========================================
+ Coverage 84.29% 84.49% +0.20%
==========================================
Files 36 37 +1
Lines 1897 1922 +25
Branches 493 497 +4
==========================================
+ Hits 1599 1624 +25
Misses 298 298 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
It's great! Thank you, merged. |
What this does
This PR adds content moderation functionality to RubyLLM, allowing developers to identify potentially harmful content before sending it to LLM providers. This helps prevent API key bans and ensures safer user interactions.
New Features
flagged?
,flagged_categories
, andcategory_scores
Usage Examples
Changes Made
Core Implementation
RubyLLM::Moderate
- Main moderation interface following existing patternsmoderate()
to base Provider classOpenAI::Moderation
module with API implementationRubyLLM.moderate()
method for global accessConfiguration
default_moderation_model
configuration option (defaults toomni-moderation-latest
)Documentation
moderation.md
with examplesTesting
moderation_spec.rb
with 4 test casesType of change
Scope check
Quality check
overcommit --install
and all hooks passmodels.json
,aliases.json
)API changes
Related issues
N/A