Uvicorn workers don't shutdown on Ctrl+C when running as a gunicorn worker inside Docker and no TTY #2497
Unanswered
kajic
asked this question in
Potential Issue
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have been investigating an issue where Uvicorn workers don't shutdown when you issue a Ctrl+C, if you run them as Gunicorn workers inside Docker without a TTY. This StackOverflow issue which I created today describes the situation in some detail:
https://stackoverflow.com/questions/79139077/how-do-you-make-gunicorn-forward-sigint-to-uvicorn-when-running-inside-docker
I've been reading through Gunicorn's
arbiter.py
,workers/base.py
and Uvicorn'sworkers.py
, and I'm getting closer to understanding what's going on, if not why the implementation is what it is.First, when running without a TTY:
When gunicorn receives a SIGINT, it sends a SIGQUIT to its workers.
(
gunicorn/arbiter.py:handle_int
callsstop(False)
, andstop()
sends SIGQUIT whengraceful=False
.)In
uvicorn/workers.py:_install_sigquit_handler
, Uvicorn register's as callbackgunicorn/workers/base.py:handle_exit
for SIGQUIT.Interestingly, these two issues are referenced in _install_sigquit_handler.
- #1116
- benoitc/gunicorn#2604
Here I found it strange that Gunicorn doesn't forward the SIGINT, but forwards a SIGQUIT instead.
Anyway, when uvicorn receives SIGQUIT, the registered
handle_exit()
callback just setsself.alive=False
.I'm not sure how
self.alive
affects the lifecycle of a worker, but from here on nothing much happens until Gunicorn's--graceful-timeout
has elapsed, and then it sends a SIGKILL to the Uvicorn workers, and prints an error like "Worker (pid:8) was sent SIGKILL! Perhaps out of memory?".When instead you run Gunicorn+Uvicorn with a TTY, using
docker run -it ...
, and press Ctrl+C, mostly everything happens as before, but now (before Gunicorn receives SIGINT)uvicorn/server.py:handle_exit
receives SIGINT first, and it is able to shutdown successfully.I still haven't been able to figure out why Uvicorn is able to receive SIGINT when there is a TTY, but not otherwise, or why it is even able to receive the SIGINT before Gunicorn does. I'd be curious to know, but assuming this discrepancy is expected, I also don't understand why
_install_sigquit_handler
registershandle_exit
as the callback, which doesn't seem to do much more than setself.alive=False
, instead of, perhaps, usinghandle_quit
as the callback, which (I tried) does properly shutdown the workers.Sorry for the long post, but I thought I'd share my investigation in case someone with a better understanding of signal handling in Uvicorn can elaborate on why Uvicorn is able to receive SIGINT with a TTY, but not without, and why
_install_sigquit_handler
useshandle_exit
instead ofhandle_quit
to handle SIGQUIT.Thanks for your time!
Beta Was this translation helpful? Give feedback.
All reactions