Login 429 Mitigation #366
Login 429 Mitigation #366
Conversation
… matching flights and running fare_checker.
other This is an attempt at fixing jdholtz#325 and improving the speed of check-ins. During check-in, what usually happens is that a bad request (400) will go through on the first attempt. This will then take ~3 seconds to finish before the next request is attempted. This change will allow multiple threads to check in at the same time, each starting one second after the other. This will reduce the time between check-in requests and hopefully reduce the total time it takes to check in.
…n-threads' into webdriver-enhacement
…ate-limiting Implemented driver.close() to close all browser windows before calling driver.quit() to avoid issues with shutil temporary folder removal
Changed retry attemp to 2 from 1 and decreased retry wait time from 20 to 10 seconds
Minor changes inclde:
- Fixing/improving tests
- Removing sleep before check-in
- This will be experimented separately on
… before fallback click (all to try avoid 429 error)
|
@dmytrokoren Is this ready for me to review? The Docker upgrade fixes the 429 issue I'm getting. The browser used to not load/work for me when upgrading the base image (why it is outdated), but that issue seems to be fixed. |
…ug in Chromium versions >119 that broke the second WebDriver instance. Resolved by downgrading Python to a specific Alpine version for compatibility.
Im not sure how it was working for you but I was getting issue with 2 accounts monitoring - the second instance would freeze on header_set. I have added my comment in issue: #277 (comment) |
Ah, I was only testing with 1 account. I see the same issue with 2 accounts. In the container, for whatever reason, the same websites are not seen in the CDP listener on the second instance--but works fine outside of the container or in an earlier version. I've tried different options of setting the user_data_dir, toggling headless, ensuring any zombie processes are cleaned up (running the container with I will try creating an Ubuntu image with Chrome to see if the issue still exists (this environment is the most supported by SeleniumBase). |
Yes, it’s definitely an issue with SeleniumBase. I’m currently using: |
|
the issue is fixed using Up to this seleniumbase version we get rid of 429 and 403 if we go further >=4.34.2 we start to see instability. If I cut a branch of v8.3 and apply the fix can we merge to 8.3 or release 8.4 but we need to skip develop branch?! |
Can confirm that works for me as well! Although, I do see stability up to 4.34.12 (4.34.13+ fail to get a login response for the second account). Due to the header URL change (I am using #372 for testing) only specifying reservations still works perfectly fine for me in the latest SB version. Can you confirm if you see the same behavior or not (both for SB v4.34.12 and reservations working with the latest SB)? On an Ubuntu 22.04 headless VM, I'm able to reproduce the 403 errors people get in #277 with v9.0. Using #372 works fine for me with both reversations and logins on the latest SB version and v4.34.12.
No, it's fine to just include these changes in v9.1. |
|
Yes I'll confirm this later tonight with SB v4.34.12 and we can then cut a minor version (v9.1) |
@jdholtz seleniumbase v4.34.12 is working fine in Docker, I have pushed the changes. |
2 participants
Working on mitigating a 429 Too Many Requests error encountered during the login process.