WIP for Support Pillow 11 and Python up to 3.13

[alexdlaird]

This is a WIP PR, but I'm opening to highlight that this library does still work with pillow 11+ (which would allow many dependencies, like my own, to add support for Python up to 3.13 if it was re-released this way). The main issue is, the build itself is old and broken, so validating this new version in the previous, automated way is not working.

I've made changes to fix the build and test runtime. They fail due to integration issues though, it appears Amazon has changed how you trigger a Captcha, and thus running the battery of accuracy tests does not find a Captcha to validate. I think I'm close to a solution to this, but also am exploring this area of their site for the first time, so thought while I worked toward the solution, I'd open this PR in case anything jumps out as obvious to you from when you were initially implementing these accuracy tests.

Hope this gives us some good direction to go and eventually be able to release a new version to support non-EOL versions of Python. Thanks!

[alexdlaird]

No longer needed, see the top-level Makefile now and make docs

[alexdlaird]

Quality of life improvements for you and anyone else trying to develop on this. Simply run make test and your dev env will be provisioned and tests run. make local will install the built version of the library in to your local pip cache. make docs, well, docs :).

[alexdlaird]

No longer needed, see the top-level Makefile now and make docs.

[3ldar]

it appears Amazon has changed how you trigger a Captcha

The funny part is that they've changed it in a way that it's no longer a captcha. You just need to make a request by passing the values provided in the rendered form.

[alexdlaird]

Not needed anymore with latest version of Sphinx

[alexdlaird]

Not needed with latest version of Sphinx

[alexdlaird]

it appears Amazon has changed how you trigger a Captcha

The funny part is that they've changed it in a way that it's no longer a captcha. You just need to make a request by passing the values provided in the rendered form.

Via the URL used in validation in this library, yes. But they do still trigger this version of Captcha in some cases, I'm just sure how to fetch it anymore for validation purposes, like this library used to do—effectively, the URL they used to use now just shows a blank image—I'm guessing we need to identify a header to pass in to trigger a "real" Captcha to load now.

It's also possible they're going to fully move away from OCR-style Captchas fully in favor of their WAF Captcha as some point, at which point this library won't be useful anymore. Until they keep throwing both at people though, I'm hoping the dev will work with the community and continue to support it.

[3ldar]

it appears Amazon has changed how you trigger a Captcha

The funny part is that they've changed it in a way that it's no longer a captcha. You just need to make a request by passing the values provided in the rendered form.

Via the URL used in validation in this library, yes. But they do still trigger this version of Captcha in some cases, I'm just sure how to fetch it anymore for validation purposes, like this library used to do—effectively, the URL they used to use now just shows a blank image—I'm guessing we need to identify a header to pass in to trigger a "real" Captcha to load now.

It's also possible they're going to fully move away from OCR-style Captchas fully in favor of their WAF Captcha as some point, at which point this library won't be useful anymore. Until they keep throwing both at people though, I'm hoping the dev will work with the community and continue to support it.

They are already using a Cloudflare approach, but not in an aggressive way. They return 403 responses without showing any captcha. In my experience, every time the old-school captcha page was triggered, it is missing the captcha image. I have had about 100k responses like this until I fix the issue.

[alexdlaird]

They are already using a Cloudflare approach, but not in an aggressive way. They return 403 responses without showing any captcha. In my experience, every time the old-school captcha page was triggered, it is missing the captcha image. I have had about 100k responses like this until I fix the issue.

Indeed, if you step through the tests on this branch, you'll see that the initial response from https://www.amazon.com/errors/validateCaptcha is a 503 (even though it's returning what appears to be a non-error page), and if you continue to link given there, that's when you get the blank Captcha (which I believe is what you're also saying). I used to also be validating this same flow as part of amazon-orders, but removed it a few months back, since the frequency with which they were showing OCR-based Captchas was clearly reducing in favor of WAF. Perhaps since then they've taken it completely out of the flow. But in the event they still occasionally present it, I wanted to leave support for it in the library, hence the attempt to submit the PR here.

What do you mean by "until I fix the issue"—are you saying you've found a workaround solution that you could share, or are you just saying your projects are in the same place as this one, stuck until a workaround is found (or perhaps we just need to abandon this path altogether, if Amazon truly is fully eliminating this type of Captcha from their auth flow).

[3ldar]

What do you mean by "until I fix the issue"—are you saying you've found a workaround solution that you could share, or are you just saying your projects are in the same place as this one, stuck until a workaround is found (or perhaps we just need to abandon this path altogether, if Amazon truly is fully eliminating this type of Captcha from their auth flow).

I have already mentioned the solution (maybe too implicitly). There is a form field in the new page input[name=field-keywords]
which represents the captcha keyword, use this keyword, send it in the URL. This part is the same, I think. Instead of solving the captcha, you are using the field as the solution. It seems someone introduced a bug and didn't realize it yet, or Amazon completely gave up, as the captcha itself became very trivial. If you want, I can add my Go source code that handles the captcha response.

[alexdlaird]

What do you mean by "until I fix the issue"—are you saying you've found a workaround solution that you could share, or are you just saying your projects are in the same place as this one, stuck until a workaround is found (or perhaps we just need to abandon this path altogether, if Amazon truly is fully eliminating this type of Captcha from their auth flow).

I have already mentioned the solution (maybe too implicitly). There is a form field in the new page input[name=field-keywords] which represents the captcha keyword, use this keyword, send it in the URL. This part is the same, I think. Instead of solving the captcha, you are using the field as the solution. It seems someone introduced a bug and didn't realize it yet, or Amazon completely gave up, as the captcha itself became very trivial. If you want, I can add my Go source code that handles the captcha response.

Oh I understand now, yah. Very simple ... and silly. This definitely feels like a step in their progression toward them just removing this type of simple Captcha altogether, so we'll see what they do in the future. Thanks!