proxy: support setting proxy via --proxyServer, PROXY_SERVER env var or PROXY_HOST + PROXY_PORT env vars #589
Conversation
… set (to be compatible with 0.12.x) accidentally always ignored proxy settings before fixes #587
as --proxyServer cli flag
ikreymer
changed the title
|
Might involve a bit of effort to spin up a local proxy server, but would be great to add some tests with this PR to ensure the feature is working correctly moving forward. |
The socks proxy feature for browsertrix will also depend on this |
|
The AsyncFetcher in recorder.ts doesn't handle proxy settings as well. As nodejs doesn't seem to support setting a global proxy, it might make sense to just add support for socks proxy like we had with pywb? |
tests: add proxy tests for socks5 and http!
- support for SOCKS5 (as supported in Brave though not Chromium) but not HTTP (not supported in any browser w/o interactive prompt) - tests: update tests to check socks5/http and html/pdf in a loop
… browser, simplify proxy init
Now fixed, supporting http and socks proxy settings for both browser and direct fetch via async fetcher. |
Tests added in |
| for (const scheme of ["socks5", "http"]) { | ||
| const port = scheme === "socks5" ? SOCKS_PORT : HTTP_PORT; | ||
|
|
||
| for (const type of ["HTML page", "PDF"]) { |
There was a problem hiding this comment.
nice homegrown parametrization here
fixes #587
The proxy env vars PROXY_HOST and PROXY_PORT were being ignored, as they were hardcoded to obsolete values in the Dockerfile.
Proxy settings can now be set, in order of precedence via:
The --proxyServer / PROXY_SERVER settings are passed to the browser via the --proxy-server flag.
AsyncFetcher / direct fetch also supports HTTP and SOCKS5 proxying.
Supported proxies are: HTTP no auth, SOCKS5 no auth, SOCKS5 with auth (supported in Brave, but not Chrome!)