|
| 1 | +# Redis Pipeline (Message broker) |
| 2 | +<!-- comment |
| 3 | + SPDX-FileCopyrightText: 2025 Sebastian Wagner, Intevation GmbH <sebix@sebix.at> |
| 4 | + SPDX-License-Identifier: AGPL-3.0-or-later |
| 5 | +--> |
| 6 | + |
| 7 | +The default IntelMQ Pipeline (the message exchange between bots) is [Redis](https://redis.io/) or its OpenSource successor [Valkey](https://valkey.io/). |
| 8 | + |
| 9 | +For AMQP (RabbitMQ) see [Using AMQP Message Broker](../beta-features.md#using-amqp-message-broker) in the section about Beta features. |
| 10 | + |
| 11 | +## Usage of databases |
| 12 | + |
| 13 | +You can use any redis database for any purpose. There are no hardcoded defaults or other requirements in IntelMQ. |
| 14 | + |
| 15 | +It's also possible to use the same database for two different bots. Just make sure, that their data doesn't collide. |
| 16 | + |
| 17 | +These are some of the usages of redis databases using the IntelMQ default values: |
| 18 | + |
| 19 | +- 2: Pipeline (Queues) |
| 20 | +- 4: IntelMQ Tests |
| 21 | +- 6: Deduplicator Expert |
| 22 | +- 7: Reverse DNS Expert |
| 23 | +- 8: RDAP Expert, Aggregate Expert |
| 24 | +- 10: RIPE Expert |
| 25 | +- 12: Shadowserver Reports API Collector |
| 26 | +- 15: SMTP Batch Output |
| 27 | + |
| 28 | +By default, Redis/valkey have a maximum of 16 databases (0-15). In it the Redis/valkey server configuration file, this value can be increased. |
| 29 | +While the number of maximum databases [is unlimited](https://redis.io/docs/latest/embeds/how-many-databases-software/), the practical limit is INT_32. |
| 30 | + |
| 31 | +## Setup tips |
| 32 | + |
| 33 | +SWAP space should be bigger or equal to your memory. See also [Hardware requirements](../hardware-requirements.md). |
| 34 | + |
| 35 | +### Redis memory overcommitting |
| 36 | + |
| 37 | +It is [recommended](https://redis.io/docs/latest/operate/oss_and_stack/management/admin/#linux) to enable memory overcommitting for Redis. |
| 38 | + |
| 39 | +Run `sysctl vm.overcommit_memory=1` to set it for the current session. |
| 40 | +To enable it permanently, create a file with `vm.overcommit_memory = 1` in `/etc/sysctl.d/intelmq.conf`. |
| 41 | + |
| 42 | +### Maxmemory |
| 43 | + |
| 44 | +To limit the maximum memory used by Redis and also to mitigate possible downsides of memory overcommitting, you can set a maximum memory usage in the Redis/Valkey server configuration with setting `maxmemory <bytes>`. |
0 commit comments