Allow users to run crawls with 1 or 2 browser windows #2627
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tw4l
commented
May 27, 2025
tw4l
commented
May 27, 2025
tw4l
commented
May 27, 2025
|
@tw4l Updated to set the maximum width to the screencast image width: 
|
This will keep scale and browserWindows from getting out of sync when the number of workers per pod on an instance changes.
Comment out unsetting of scale for now for easier testing on dev
…ces, then in multiples through max
SuaYoo
force-pushed
the
issue-2425-browser-windows
branch
from
8cb422e to
aab9f5a
Compare
Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
SuaYoo
reviewed
Jun 3, 2025
ikreymer
approved these changes
Jun 3, 2025
tw4l
pushed a commit
that referenced
this pull request
Jun 12, 2025
) - follow up to #2627 - use qa_num_browser_windows to set exact number of QA browsers, fallback to qa_scale - set num_browser_windows and num_browsers_per_pod using crawler / qa values depending if QA crawl - scale_from_browser_windows() accepts optional browsers_per_pod if dealing with possible QA override - store 'desiredScale' in CrawlStatus to avoid recomputing for later scale resolving - ensure status.scale is always the actual scale observed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2425
Changed
browserWindowsfromscalefor existing workflows and crawls)scalein addition tobrowserWindowsin input models for creating and updating workflows and re-adjusting live crawl scale for backwards compatibilitymax_browser_windowsvalue to Helm chart, but calculates the value frommax_crawl_scaleas fallback for users with that value already set in local chartscrawler_browser_instancesor any value belowcrawler_browser_instancesfor browser windows. For instance, withcrawler_browser_instances=4andmax_browser_windows=8, the user would be presented with the following options: 1, 2, 3, 4, 8message