Skip to content

API efficiency #35

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jan 29, 2025
Merged

API efficiency #35

merged 5 commits into from
Jan 29, 2025

Conversation

veganbeef
Copy link
Contributor

@veganbeef veganbeef commented Jan 28, 2025 •
edited
Loading

This PR makes the following updates in order to make the API more efficient:

  • adds rate limiting and an in-memory LRU cache for requests (should be changed to a redis cache eventually)
  • adds DB indexes to speed up common queries
  • adds smarter looping strategy, request concurrency limiting, and caching for LLM requests

@thelostone-mc thelostone-mc merged commit 354afc7 into main Jan 29, 2025
2 checks passed
@thelostone-mc thelostone-mc deleted the veganbeef/api-efficiency branch January 29, 2025 04:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thelostone-mc thelostone-mc thelostone-mc approved these changes

@0xKurt 0xKurt Awaiting requested review from 0xKurt

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@veganbeef @thelostone-mc