[S3] More comprehensive error handling

[npow]

1. Improved error handling by ensuring fatal exceptions like ExpiredToken are no longer retried.
2. Performed some code refactoring for better clarity and maintainability.

[Copilot]

Pull Request Overview

This PR enhances S3 error handling by introducing a dedicated 400 “invalid request” code and refactoring the client‐error normalization to use categorized error sets.

• Added ERROR_INVALID_REQUEST constant and mapped it in op_info, handle_client_error, get_info, and list_prefix.
• Refactored normalize_client_error to classify errors into permission, not found, range, transient, and fatal groups.
• Updated higher‐level S3 operations to raise or propagate the new 400 error.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
metaflow/plugins/datatools/s3/s3op.py	Added ERROR_INVALID_REQUEST, categorized error sets, and 400 mappings
metaflow/plugins/datatools/s3/s3.py	Raised new 400 error in _head and _one_boto_op

Comments suppressed due to low confidence (2)

metaflow/plugins/datatools/s3/s3op.py:261

• [nitpick] There are no unit tests verifying the new 400 (ERROR_INVALID_REQUEST) branch in op_info. Consider adding tests to cover this error mapping to avoid regressions.

elif error_code == 400:

metaflow/plugins/datatools/s3/s3.py:856

• [nitpick] Raising the generic MetaflowS3Exception for invalid requests may obscure this specific scenario. Consider defining a distinct exception class (e.g., MetaflowS3InvalidRequest) for clearer error handling.

elif info["error"] == s3op.ERROR_INVALID_REQUEST:

[romain-intel]

Minor nits but lgtm.

[romain-intel]

If you got these errors from a specific location (doc), could you add a link to it so we can periodically check if they haven't changed. This is much cleaner though so hopefully it will help going forward.

[npow]

added links and some more error codes

[npow]

ok i think i got them all now

[madhur-ob]

@npow adding the label to run S3 tests against minIO