Daemonized queue and pcntl

Closes #571

Author: AugustMiller

docs/5.x/reference/config/app.md

@@ -429,7 +429,7 @@ Any changes you make to the Mailer component from `config/app.php` will not be r
 
 ### Queue
 
-Craft’s job queue is powered by the [Yii2 Queue Extension](https://github.com/yiisoft/yii2-queue). By default, Craft will use a [custom queue driver](craft5:craft\queue\Queue) based on the extension’s [DB driver](https://github.com/yiisoft/yii2-queue/blob/master/docs/guide/driver-db.md).
+Craft’s [queue](../../system/queue.md) is powered by the [Yii2 Queue Extension](https://github.com/yiisoft/yii2-queue). By default, Craft will use a [custom queue driver](craft5:craft\queue\Queue) based on the extension’s [DB driver](https://github.com/yiisoft/yii2-queue/blob/master/docs/guide/driver-db.md).
 
 Switching to a different driver by overriding Craft’s `queue` component from `config/app.php` will result in a loss of visibility into the queue’s state from the control panel. Instead of replacing the entire component, set your custom queue driver config on <craft5:craft\queue\Queue::$proxyQueue>:
 
@@ -459,7 +459,27 @@ Only drivers that implement <craft5:craft\queue\QueueInterface> will be visible
 :::
 
 ::: tip
-If your queue driver supplies its own worker, set the <config5:runQueueAutomatically> config setting to `false` in `config/general.php`.
+If your queue driver supplies its own [worker](../../system/queue.md#workers), set the <config5:runQueueAutomatically> config setting to `false` in `config/general.php`.
+:::
+
+For projects that spawn many long-running jobs (as is often the case when synchronizing content from other sources), you may wish to extend the default job `ttr` or _time to reserve_:
+
+```php
+
+return [
+    'components' => [
+        'queue' => [
+            // Allow individual jobs to run for 15 minutes:
+            'ttr' => 900,
+        ],
+    ],
+];
+```
+
+This does not prevent plugins or modules from pushing jobs with a shorter `ttr`, but Craft will respect it for all built-in queue usage.
+
+::: warning
+[Daemonized queue runners](../../system/queue.md#daemon) should be configured to allow the full `ttr` duration before issuing a `KILL` signal.
 :::
 
 ### Mutex

docs/5.x/requirements.md

@@ -66,9 +66,10 @@ We recommend ImageMagick over GD for expanded [image handling options](developme
 
 Some shared hosting environments disable common PHP methods and configurations that affect Craft features.
 
-- [allow_url_fopen](http://php.net/manual/en/filesystem.configuration.php#ini.allow-url-fopen) must be enabled for updating and installing plugins from the Plugin Store.
-- [proc_*](http://php.net/manual/en/ref.exec.php) methods must be enabled in order to utilize the Plugin Store, install updates, and send emails.
-- [ignore_user_abort](https://www.php.net/manual/en/function.ignore-user-abort.php) must be enabled for the [default, web-based queue runner](config5:runQueueAutomatically) to operate.
+- [allow_url_fopen](http://php.net/manual/en/filesystem.configuration.php#ini.allow-url-fopen) — This `ini` setting must be enabled to support updating and installing plugins from the Plugin Store.
+- [proc_*](http://php.net/manual/en/ref.exec.php) — These methods are required in order to utilize the Plugin Store, install updates, and send emails.
+- [ignore_user_abort](https://www.php.net/manual/en/function.ignore-user-abort.php) — Required when using the default [web-based queue runner](system/queue.md#http) to operate.
+- [pcntl_*](https://www.php.net/manual/en/book.pcntl.php) — The _Process Control_ extension allows [daemonized queue runners](system/queue.md#daemon) to gracefully exit between jobs.
 
 ## File Permissions
 

docs/5.x/system/queue.md

@@ -124,6 +124,8 @@ ExecStart=/usr/bin/php /var/www/craft queue/listen --verbose=1 --color=0
 Restart=on-failure
 # Extend time between restart attempts after a failure:
 RestartSec=120
+# Allow more time before issuing a kill signal:
+TimeoutStopSec=300
 
 [Install]
 WantedBy=multi-user.target
@@ -139,13 +141,17 @@ command=/usr/bin/php /var/www/craft queue/listen --verbose=1 --color=0
 ; User + Group should agree with HTTP processes:
 user=www-data
 group=www-data
+# Allow more time before issuing a kill signal:
+stopwaitsecs=300
 ```
 :::
 
+The `TimeoutStopSec` and `stopwaitsecs` settings in the examples above should agree with (or exceed) [your queue’s `ttr`](../reference/config/app.md#queue), so that currently-running jobs may complete and the runner may exit gracefully when the process manager sends a `TERM` signal. At the end of that window (if the queue has not voluntarily exited), the process manager sends a `KILL` signal and may interrupt the queue in the middle of a task. This safeguard relies on the [`pcntl` PHP extension](https://www.yiiframework.com/extension/yiisoft/yii2-queue/doc/guide/2.3/en/retryable), which is _not_ a core Craft [requirement](../requirements.md), and may need to be enabled by your host.
+
 ::: warning
 Long-running processes must be restarted to pick up code and schema changes after a deployment or migration!
 
-To register your newly created service, the process manager itself may need to be restarted, or specifically told to look for new configuration.
+To register your newly created (or updated) service, the process manager itself may need to be restarted, or specifically told to look for new configuration.
 :::
 
 Your process manager also has commands for interacting with the service:
@@ -193,7 +199,7 @@ sudo systemctl status "craft-queue-worker@*"
 sudo systemctl restart "craft-queue-worker@*"
 ```
 
-This is possible due to the `@` symbol at the end of our service unit file’s name. Note that the bash “range” syntax (`{1..4}`) is _not_ surrounded by quotes, but the “glob” patterns (`*`) _are_. This is critical for the system to be able to expand and map the commands to the appropriate service instances(s).
+This is possible due to the `@` symbol at the end of our service unit file’s name. Note that the bash-specific “range” syntax (`{1..4}`) is _not_ surrounded by quotes, but the “glob” patterns (`*`) _are_. This is critical for the system to be able to expand and map the commands to the appropriate service instances(s).
 :::
 
 These setup instructions ensure your service is started whenever the host machine is rebooted, but it’s important to check your work! If your host allows, consider manually rebooting your machine and verifying that the queue is active.
@@ -226,7 +232,7 @@ Do not run the queue as root!
 
 ## Local Development
 
-If you’ve turned off <config5:runQueueAutomatically> for your live infrastructure, the queue will also be disabled in your local environment. You can [override](config/README.md#environment-overrides) the general config setting by declaring `CRAFT_RUN_QUEUE_AUTOMATICALLY=true` in your `.env`, or use the [CLI](#cli) as you would in a [daemon](#daemon):
+If you’ve turned off <config5:runQueueAutomatically> for your live infrastructure, the queue will also be disabled in your local environment. You can [override](../configure.md#environment-overrides) the general config setting by declaring `CRAFT_RUN_QUEUE_AUTOMATICALLY=true` in your `.env`, or use the [CLI](#cli) as you would in a [daemon](#daemon):
 
 ::: code
 ```bash DDEV