Skip to content

eliminate use of __file__ in boto3.session #4445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

zsol
Copy link

@zsol zsol commented Feb 18, 2025

Python doesn't guarantee that the __file__ global is a string pointing to the name of the Python file hosting the source code of the current module, and certain Python environments (like PyOxidizer) indeed don't set it.

This PR makes boto3.session.Session still work in these environments.

@zsol zsol marked this pull request as ready for review February 18, 2025 21:19
@nateprewitt
Copy link
Contributor

Hi @zsol,

Apologies this has been lingering for a bit. I was going back through open PRs to find uses/issues about __file__ and stumbled on this. We recently dropped Python 3.8 support which lets us use some more recent interfaces from importlib.resources.

#4528 and boto/botocore#3464 take a slightly different approach to address this. Do you have any concerns with those or other issues we might be missing for your use case?

@zsol
Copy link
Author

zsol commented May 8, 2025

Hmm I'm not entirely sure that str(files(...)) will always give you a path that's on disk

@nateprewitt
Copy link
Contributor

Do you happen to have any specific use cases you were seeing variance in? From some quick testing, we weren't able to find any difference between the old and new paths being produced. I'm sure there are some edge cases we're missing though.

Or perhaps if you could point out why str(path(__package__, "data").__enter__()) as a context manager is more resilient to this issue?

@zsol
Copy link
Author

zsol commented May 8, 2025

Sorry I meant to provide an example but it's proving to be a bit tricky to come up with something short :)

The problem is that files() eventually calls reader.files(), which for example for ZipReader, simply returns a zipfile.Path, without actually unzipping the file.
OTOH path() calls into as_file(), which returns a context manager that materializes the file/directory when entering, and cleans it up after exiting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants