Skip to content

Commit a2a7751

Browse files
authored
Merge pull request #5845 from influxdata/pbarnett/processing-engine-updates
update: fix some wording and hierarchy
2 parents 8da7e7e + 002074b commit a2a7751

File tree

1 file changed

+51
-36
lines changed

1 file changed

+51
-36
lines changed

content/shared/v3-core-plugins/_index.md

Lines changed: 51 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,12 @@ for different database events.
55
{{% product-name %}} provides the InfluxDB 3 Processing engine, an embedded Python VM that can dynamically load and trigger Python plugins
66
in response to events in your database.
77

8-
## Plugins
8+
## Key Concepts
9+
10+
### Plugins
11+
12+
A Processing engine _plugin_ is Python code you provide to run tasks, such as
13+
downsampling data, monitoring, creating alerts, or calling external services.
914

1015
> [!Note]
1116
> #### Contribute and use community plugins
@@ -14,28 +19,27 @@ in response to events in your database.
1419
> and contribute example plugins.
1520
> You can reference plugins from the repository directly within a trigger configuration.
1621
17-
A Processing engine _plugin_ is Python code you provide to run tasks, such as
18-
downsampling data, monitoring, creating alerts, or calling external services.
19-
20-
## Triggers
22+
### Triggers
2123

2224
A _trigger_ is an InfluxDB 3 resource you create to associate a database
2325
event (for example, a WAL flush) with the plugin that should run.
24-
When an event occurs, the trigger passes configuration, optional arguments, and event data to the plugin.
26+
When an event occurs, the trigger passes configuration details, optional arguments, and event data to the plugin.
2527

26-
The Processing engine provides four types of plugins and triggers--each type corresponds to an event type with event-specific configuration to let you handle events with targeted logic.
28+
The Processing engine provides four types of triggerseach type corresponds to an event type with event-specific configuration to let you handle events with targeted logic.
2729

28-
- **WAL flush**: Triggered when the write-ahead log (WAL) is flushed to the object store (default is every second)
29-
- **Parquet persistence (coming soon)**: Triggered when InfluxDB 3 persists data to object store Parquet files
30-
- **Scheduled tasks**: Triggered on a schedule you specify using cron syntax
31-
- **On Request**: Bound to the HTTP API `/api/v3/engine/<CUSTOM_PATH>` endpoint and triggered by a GET or POST request to the endpoint.
30+
- **WAL Flush**: Triggered when the write-ahead log (WAL) is flushed to the object store (default is every second).
31+
- **Scheduled Tasks**: Triggered on a schedule you specify using cron syntax.
32+
- **On-request**: Triggered on a GET or POST request to the bound HTTP API endpoint at `/api/v3/engine/<CUSTOM_PATH>`.
33+
<!--
34+
- **Parquet Persistence (coming soon)**: Triggered when InfluxDB 3 persists data to object storage Parquet files.
35+
-->
3236

33-
## Activate the Processing engine
37+
### Activate the Processing engine
3438

35-
To enable the Processing engine, start the {{% product-name %}} server with the `--plugin-dir` option and a path to your plugins directory (it doesn't need to exist yet)--for example:
39+
To enable the Processing engine, start the {{% product-name %}} server with the `--plugin-dir` option and a path to your plugins directory. If the directory doesn’t exist, the server creates it.
3640

3741
```bash
38-
influxdb3 serve --node-id node0 --plugin-dir /path/to/plugins
42+
influxdb3 serve --node-id node0 --object-store [OBJECT STORE TYPE] --plugin-dir /path/to/plugins
3943
```
4044

4145
## Shared API
@@ -47,7 +51,7 @@ The shared API provides access to the following:
4751
- `query` to query data from any database
4852
- `info`, `warn`, and `error` to log messages to the database log, which is output in the server logs and captured in system tables queryable by SQL
4953

50-
### Line builder
54+
### LineBuilder
5155

5256
The `LineBuilder` is a simple API for building lines of Line Protocol to write into the database. Writes are buffered while the plugin runs and are flushed when the plugin completes. The `LineBuilder` API is available in all plugin types.
5357

@@ -198,12 +202,12 @@ influxdb3_local.query("SELECT * from foo where bar = $bar and time > now() - 'in
198202
### Logging
199203

200204
The shared API `info`, `warn`, and `error` functions log messages to the database log, which is output in the server logs and captured in system tables queryable by SQL.
201-
The `info`, `warn`, and `error` functions are available in all plugin types. The functions take an arbitrary number of arguments, convert them to strings, and then join them into a single message separated by a space.
205+
The `info`, `warn`, and `error` functions are available in all plugin types. Each function accepts multiple arguments, converts them to strings, and logs them as a single, space-separated message.
202206

203-
The following examples show to use the `info`, `warn`, and `error` logging functions:
207+
The following examples show how to use the `info`, `warn`, and `error` logging functions:
204208

205209
```python
206-
ifluxdb3_local.info("This is an info message")
210+
influxdb3_local.info("This is an info message")
207211
influxdb3_local.warn("This is a warning message")
208212
influxdb3_local.error("This is an error message")
209213

@@ -215,7 +219,7 @@ influxdb3_local.info("This is an info message with an object", obj_to_log)
215219
### Trigger arguments
216220

217221
Every plugin type can receive arguments from the configuration of the trigger that runs it.
218-
You can use this to provide runtime configuration and drive behavior of a plugin--for example:
222+
You can use this to provide runtime configuration and drive behavior of a pluginfor example:
219223

220224
- threshold values for monitoring
221225
- connection properties for connecting to third-party services
@@ -233,7 +237,7 @@ def process_writes(influxdb3_local, table_batches, args=None):
233237
influxdb3_local.warn("No threshold provided")
234238
```
235239

236-
The `args` parameter is optional and can be omitted from the trigger definition if the plugin doesn't need to use arguments.
240+
The `args` parameter is optional. If a plugin doesn’t require arguments, you can omit it from the trigger definition.
237241

238242
## Import plugin dependencies
239243

@@ -267,14 +271,17 @@ influxdb3 install package <PACKAGE_NAME>
267271
```
268272

269273
The result is an active Python virtual environment with the package installed in `<PLUGINS_DIR>/.venv`.
270-
You can pass additional options to use a `requirements.txt` file or a custom virtual environment path.
274+
You can specify additional options to install dependencies from a `requirements.txt` file or a custom virtual environment path.
271275
For more information, see the `influxdb3` CLI help:
272276

273277
```bash
274278
influxdb3 install package --help
275279
```
276280

277-
## WAL flush plugin
281+
## Configure plugin triggers
282+
Triggers define when and how plugins execute in response to database events. Each trigger type corresponds to a specific event, allowing precise control over automation within {{% product-name %}}.
283+
284+
### WAL flush trigger
278285

279286
When a WAL flush plugin is triggered, the plugin receives a list of `table_batches` filtered by the trigger configuration (either _all tables_ in the database or a specific table).
280287

@@ -302,7 +309,7 @@ def process_writes(influxdb3_local, table_batches, args=None):
302309
influxdb3_local.info("wal_plugin.py done")
303310
```
304311

305-
### WAL flush trigger Configuration
312+
#### WAL flush trigger configuration
306313

307314
When you create a trigger, you associate it with a database and provide configuration specific to the trigger type.
308315

@@ -330,9 +337,9 @@ For more information about trigger arguments, see the CLI help:
330337
influxdb3 create trigger help
331338
```
332339

333-
## Schedule Plugin
340+
### Schedule trigger
334341

335-
Schedule plugins run on a schedule specified in cron syntax. The plugin will receive the local API, the time of the trigger, and any arguments passed in the trigger definition. Here's an example of a simple schedule plugin:
342+
Schedule plugins run on a schedule specified in cron syntax. The plugin receives the local API, the time of the trigger, and any arguments passed in the trigger definition. Here's an example of a simple schedule plugin:
336343

337344
```python
338345
# see if a table has been written to in the last 5 minutes
@@ -347,7 +354,7 @@ def process_scheduled_call(influxdb3_local, time, args=None):
347354
influxdb3_local.error("No table_name provided for schedule plugin")
348355
```
349356

350-
### Schedule Trigger Configuration
357+
#### Schedule trigger configuration
351358

352359
Schedule plugins are set with a `trigger-spec` of `schedule:<cron_expression>` or `every:<duration>`. The `args` parameter can be used to pass configuration to the plugin. For example, if we wanted to use the system-metrics example from the Github repo and have it collect every 10 seconds we could use the following trigger definition:
353360

@@ -358,9 +365,12 @@ influxdb3 create trigger \
358365
--database mydb system-metrics
359366
```
360367

361-
## On Request Plugin
368+
### On Request trigger
369+
370+
On Request plugins are triggered by a request to an HTTP API endpoint.
371+
The plugin receives the shared API, query parameters `Dict[str, str]`, request headers `Dict[str, str]`, the request body (as bytes), and any arguments passed in the trigger definition.
362372

363-
On Request plugins are triggered by a request to a specific endpoint under `/api/v3/engine`. The plugin will receive the local API, query parameters `Dict[str, str]`, request headers `Dict[str, str]`, request body (as bytes), and any arguments passed in the trigger definition. Here's an example of a simple On Request plugin:
373+
#### Example: On Request plugin
364374

365375
```python
366376
import json
@@ -385,15 +395,20 @@ def process_request(influxdb3_local, query_parameters, request_headers, request_
385395
return 200, {"Content-Type": "application/json"}, json.dumps({"status": "ok", "line": line_str})
386396
```
387397

388-
### On Request Trigger Configuration
398+
#### On Request trigger configuration
389399

390-
On Request plugins are set with a `trigger-spec` of `request:<endpoint>`. The `args` parameter can be used to pass configuration to the plugin. For example, if we wanted the above plugin to run on the endpoint `/api/v3/engine/my_plugin`, we would use `request:my_plugin` as the `trigger-spec`.
400+
To create a trigger for an On Request plugin, specify the `request:<ENDPOINT>` trigger-spec.
391401

392-
Trigger specs must be unique across all configured plugins, regardless of which database they are tied to, given the path is the same. Here's an example to create a request trigger tied to the "hello-world' path using a plugin in the plugin-dir:
402+
For example, the following command creates an HTTP API `/api/v3/engine/my-plugin` endpoint for the plugin file:
393403

394-
```shell
404+
```bash
395405
influxdb3 create trigger \
396-
--trigger-spec "request:hello-world" \
397-
--plugin-filename "hellp/hello_world.py" \
398-
--database mydb hello-world
399-
```
406+
--trigger-spec "request:my-plugin" \
407+
--plugin-filename "examples/my-on-request.py" \
408+
--database mydb my-plugin
409+
410+
To run the plugin, you send an HTTP request to `<HOST>/api/v3/engine/my-plugin`.
411+
412+
Because all On Request plugins for a server share the same `<host>/api/v3/engine/` base URL,
413+
the trigger-spec you define must be unique across all plugins configured for a server,
414+
regardless of which database they are associated with.

0 commit comments

Comments
 (0)