-
-
Notifications
You must be signed in to change notification settings - Fork 52
Add support for autoclick #2313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Fixes #2259 This PR brings backend and frontend support for the new autoclick behavior in Browsertrix, introduces in Browsertrix 1.5.0+ On the backend, we introduce `min_autoclick_crawler_image` to `values.yaml`, with a default value of `"docker.io/webrecorder/browsertrix-crawler:1.5.0"`. If this is set and the crawler version for a new crawl is less than this value, the autoclick behavior is removed from the behaviors list in the configmap created for the crawl. The one caveat for this is that a crawler image tag like "latest" will always be parsed as greater than `min_autoclick_crawler_image`, so there is the potential for the crawler to run into issues if using a non-numeric image tag with an older version of the crawler. For production we use hardcoded specific versions of the crawler except for the dev channel, which from here on out will including autoclick support, so I think this should be okay (and is also true of the existing implementation for checking `min_qa_crawler_image`). On the frontend, I've added a checkbox (unchecked by default) in the "Limits" section just below the current checkbox for autoscroll. We might want to move these to a different section eventually - I'm not sure Limits is the right place for them - but I wanted to be consistent with things as they are. --------- Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
One thing when summing up in Crawl Setting:
I'm trying to find examples where the behaviour works, and will try to find some good cases:-) No luck, so far with, these usecases: |
@Klindten at this point, the autoclick selector will only click on anchor tags with hrefs, but if you want to try those pages with the crawler, you can use the We're adding support for this in Browsertrix too, it's just a bit behind the backend work. I'm also going to create a new issue for the Crawl Settings miss, thanks for that! |
Thanks for the clarification:-) |
Fixes #2259
This PR brings backend and frontend support for the new autoclick behavior in Browsertrix.
On the backend, we introduce
min_autoclick_crawler_image
tovalues.yaml
, with a default value of"docker.io/webrecorder/browsertrix-crawler:1.5.0"
. If this is set and the crawler version for a new crawl is less than this value, the autoclick behavior is removed from the behaviors list in the configmap created for the crawl.The one caveat for this is that a crawler image tag like "latest" will always be parsed as greater than
min_autoclick_crawler_image
, so there is the potential for the crawler to run into issues if using a non-numeric image tag with an older version of the crawler. For production we use hardcoded specific versions of the crawler except for the dev channel, which from here on out will including autoclick support, so I think this should be okay (and is also true of the existing implementation for checkingmin_qa_crawler_image
).On the frontend, I've added a checkbox (unchecked by default) in the "Limits" section just below the current checkbox for autoscroll. We might want to move these to a different section eventually - I'm not sure Limits is the right place for them - but I wanted to be consistent with things as they are.