Skip to content

Release thread start barrier in case an error occurs #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 25, 2025

Conversation

andfoy
Copy link
Contributor

@andfoy andfoy commented Mar 20, 2025

Fixes #27

Copy link
Contributor

@ngoldbaum ngoldbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test where a worker raises an unhandled exception (e.g. KeyboardInterrupt) and verify that it deadlocks before this PR and the test passes with this PR.

worker.start()
try:
worker.start()
except Exception:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a rare case where a bare except or BaseException is the correct thing, since we probably do want to handle KeyboardInterrupt.

Rather than worrying about exactly which exceptions you want to handle, better to use a try/finally:

num_completed = 0
for worker in workers:
    try:
        worker.start()
        num_completed += 1
    finally:
        if num_completed < len(workers):
            barrier.abort()

@andfoy
Copy link
Contributor Author

andfoy commented Mar 25, 2025

Can you add a test where a worker raises an unhandled exception

I think we cannot, since the instrumented code gets executed after the barrier has been put in place, we would need to somehow create the conditions under which threads cannot be spawned, something like

monkeypatch.setattr("os.process_cpu_count", lambda: None, raising=False)

@ngoldbaum
Copy link
Contributor

Ah you're right! Good point. Never mind.

@andfoy andfoy force-pushed the release_barrier_error branch from adebc76 to 4afaf15 Compare March 25, 2025 15:22
@andfoy andfoy merged commit 80bb6d6 into main Mar 25, 2025
9 checks passed
@andfoy andfoy deleted the release_barrier_error branch March 25, 2025 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possible deadlocks if Python can't spawn threads
2 participants