-
Notifications
You must be signed in to change notification settings - Fork 2
Feature/Proactive Rate Limitting in Github Client #75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/Proactive Rate Limitting in Github Client #75
Conversation
isSerge
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a significant improvement, few minor comments below.
Also, check cargo clippy - there are some warnings, and run cargo fmt. I will update CI to run these for all branches, not just main
Thanks!
src/poller/mod.rs
Outdated
| GithubError::HeaderError(_) => { | ||
| tracing::error!("Header error while polling issues for repository"); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| GithubError::HeaderError(_) => { | |
| tracing::error!("Header error while polling issues for repository"); | |
| } | |
| GithubError::HeaderError(msg) => { | |
| tracing::warn!( | |
| "Could not parse rate limit headers for repo {} (chat {}): {}. \ | |
| Skipping this repo for this cycle.", | |
| repo.name_with_owner, | |
| chat_id, | |
| msg | |
| ); | |
| } |
|
@isSerge requesting review. resolved the cargo fmt issue |
isSerge
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your hard work @riturajFi
|
Thanks for your guidance! More to come!! |
This PR implements proactive rate-limit handling for the GitHub API in DefaultGithubClient.
Previously, the application only reacted after hitting a 403 RateLimitExceeded error. With these changes, the client now:
✅ Reads X-RateLimit-Remaining and X-RateLimit-Reset headers from each GitHub API response.
✅ Tracks the remaining quota and reset time internally.
✅ Checks the quota before each request and pauses (with sleep) when the quota is near exhausted, avoiding unnecessary failures.
This improves stability and avoids API bans, especially under heavy load or with many tracked repositories.
Changes
Add RateLimitState struct inside DefaultGithubClient (with Arc<Mutex<...>>).
Update state after every response using the provided rate-limit headers.
Add rate_limit_guard() method that blocks requests if remaining quota is below a safety threshold.
Call rate_limit_guard() at the start of every execute_graphql() call.
Linked Issue
Closes #67