diff --git a/miscellaneous/glossary.md b/miscellaneous/glossary.md index fe4913b..5a17b4e 100644 --- a/miscellaneous/glossary.md +++ b/miscellaneous/glossary.md @@ -1,5 +1,5 @@ --- -last_modified_date: 2024-05-31 20:16:38 +last_modified_date: 2024-07-30 16:55:16 layout: default title: Glossary description: Glossary of GatewayD terms @@ -28,6 +28,11 @@ Each configuration file contains multiple configuration objects that correspond To enable multi-tenancy, GatewayD supports configuring multiple instances of each (configuration) object using configuration groups. All the default configuration objects have a single configuration group called `default`, except the `API`. +## Configuration block +Within each configuration group, GatewayD allows for further segmentation of settings through the use of configuration blocks. A configuration block represents a specific set of parameters within a configuration group, tailored to a particular instance. + +Configuration blocks are supported for the following configuration objects: `clients`, `pools` and `proxies`. + ## Configuration parameter A configuration object has one or many configuration parameters to set up the corresponding object. For example, the `output` parameter on the `logger` object is used to set the outputs by the `default` (configuration group) logger. diff --git a/using-gatewayd/configuration.md b/using-gatewayd/configuration.md index e05f4ef..ee86781 100644 --- a/using-gatewayd/configuration.md +++ b/using-gatewayd/configuration.md @@ -1,5 +1,5 @@ --- -last_modified_date: 2024-05-31 20:16:38 +last_modified_date: 2024-07-30 16:36:53 layout: default title: Configuration description: GatewayD is fully configurable via various sources, including default values, YAML config files, environment variables, CLI flags and plugins. @@ -74,28 +74,31 @@ metrics: clients: default: - network: tcp - address: localhost:5432 - tcpKeepAlive: False - tcpKeepAlivePeriod: 30s # duration - receiveChunkSize: 8192 - receiveDeadline: 0s # duration, 0ms/0s means no deadline - receiveTimeout: 0s # duration, 0ms/0s means no timeout - sendDeadline: 0s # duration, 0ms/0s means no deadline - dialTimeout: 60s # duration, 0ms/0s means no timeout - # Retry configuration - retries: 3 # 0 means no retry - backoff: 1s # duration - backoffMultiplier: 2.0 # 0 means no backoff - disableBackoffCaps: false + writes: # ⬅️ Configuration block + network: tcp + address: localhost:5432 + tcpKeepAlive: False + tcpKeepAlivePeriod: 30s # duration + receiveChunkSize: 8192 + receiveDeadline: 0s # duration, 0ms/0s means no deadline + receiveTimeout: 0s # duration, 0ms/0s means no timeout + sendDeadline: 0s # duration, 0ms/0s means no deadline + dialTimeout: 60s # duration, 0ms/0s means no timeout + # Retry configuration + retries: 3 # 0 means no retry + backoff: 1s # duration + backoffMultiplier: 2.0 # 0 means no backoff + disableBackoffCaps: false pools: default: - size: 10 + writes: + size: 10 proxies: default: - healthCheckPeriod: 60s # duration + writes: + healthCheckPeriod: 60s # duration servers: default: @@ -107,6 +110,11 @@ servers: certFile: "" keyFile: "" handshakeTimeout: 5s # duration + loadBalancer: + # Load balancer strategies can be found in config/constants.go + strategy: ROUND_ROBIN + consistentHash: + useSourceIp: true api: enabled: True @@ -158,24 +166,27 @@ plugins: ## Environment variables -All configuration parameters have a corresponding environment variables, except in certain cases. All environment variables are prefixed with `GATEWAYD` and are in snake case. For example, the `GATEWAYD_LOGGERS_DEFAULT_OUTPUT` environment variable can be set to the outputs required by the default logger and consists of four parts: +All configuration parameters have a corresponding environment variables, except in certain cases. All environment variables are prefixed with `GATEWAYD` and are in snake case. For example, the `GATEWAYD_CLIENTS_DEFAULT_WRITES_NETWORK` environment variable can be set to the network type for the writes in the default client configuration and consists of five parts: 1. Prefix: all environment variables are prefixed with `GATEWAYD`. -2. Object: the configuration object, in this case `LOGGERS`. +2. Object: the configuration object, in this case `CLIENTS`. 3. Group: the configuration group, in this case `DEFAULT`. -4. Parameter: the configuration parameter, in this case `OUTPUT`. +4. Block: the configuration block, in this case `WRITES` (if applicable). +5. Parameter: the configuration parameter, in this case `NETWORK`. ```mermaid flowchart TD - A(GATEWAYD_LOGGERS_DEFAULT_OUTPUT) + A(GATEWAYD_CLIENTS_DEFAULT_WRITES_NETWORK) A --> GATEWAYD - A --> LOGGERS + A --> CLIENTS A --> DEFAULT - A --> OUTPUT + A -.-> WRITES + A --> NETWORK GATEWAYD --> Prefix - LOGGERS --> Object + CLIENTS --> Object DEFAULT --> Group - OUTPUT --> Parameter + WRITES -.-> Block + NETWORK --> Parameter ``` ## Runtime configuration diff --git a/using-gatewayd/global-configuration/clients.md b/using-gatewayd/global-configuration/clients.md index 8348f83..415c284 100644 --- a/using-gatewayd/global-configuration/clients.md +++ b/using-gatewayd/global-configuration/clients.md @@ -1,5 +1,5 @@ --- -last_modified_date: 2024-05-31 20:16:38 +last_modified_date: 2024-07-29 17:35:18 layout: default title: Clients description: GatewayD client configuration @@ -10,7 +10,7 @@ grand_parent: Using GatewayD # Clients -GatewayD supports multiple client configurations. Each client in each configuration group will connect to the same database server specified in the configuration parameters and will be added to its corresponding [pool](pools) based on their configuration group, i.e. `default`. +GatewayD supports multiple client configurations. Each client within a configuration group will connect to a database server specified in the configuration parameters and will be added to its corresponding [pool](pools) based on their configuration group (e.g., `default`) and configuration block (e.g., `writes`). ## Configuration parameters @@ -33,18 +33,19 @@ GatewayD supports multiple client configurations. Each client in each configurat ```yaml clients: default: - network: tcp - address: localhost:5432 - tcpKeepAlive: False - tcpKeepAlivePeriod: 30s # duration - receiveChunkSize: 8192 - receiveDeadline: 0s # duration, 0ms/0s means no deadline - receiveTimeout: 0s # duration, 0ms/0s means no timeout - sendDeadline: 0s # duration, 0ms/0s means no deadline - dialTimeout: 60s # duration, 0ms/0s means no timeout - # Retry configuration - retries: 3 # 0 means no retry - backoff: 1s # duration - backoffMultiplier: 2.0 # 0 means no backoff - disableBackoffCaps: false + writes: + network: tcp + address: localhost:5432 + tcpKeepAlive: False + tcpKeepAlivePeriod: 30s # duration + receiveChunkSize: 8192 + receiveDeadline: 0s # duration, 0ms/0s means no deadline + receiveTimeout: 0s # duration, 0ms/0s means no timeout + sendDeadline: 0s # duration, 0ms/0s means no deadline + dialTimeout: 60s # duration, 0ms/0s means no timeout + # Retry configuration + retries: 3 # 0 means no retry + backoff: 1s # duration + backoffMultiplier: 2.0 # 0 means no backoff + disableBackoffCaps: false ``` diff --git a/using-gatewayd/global-configuration/pools.md b/using-gatewayd/global-configuration/pools.md index 401cf5e..534b222 100644 --- a/using-gatewayd/global-configuration/pools.md +++ b/using-gatewayd/global-configuration/pools.md @@ -1,5 +1,5 @@ --- -last_modified_date: 2024-05-31 20:16:38 +last_modified_date: 2024-07-29 17:37:00 layout: default title: Pools description: GatewayD pool configuration @@ -23,5 +23,6 @@ The size of the pool defines the maximum capacity of the pool. Upon start, Gatew ```yaml pools: default: - size: 10 + writes: + size: 10 ``` diff --git a/using-gatewayd/global-configuration/proxies.md b/using-gatewayd/global-configuration/proxies.md index 22db04f..6112779 100644 --- a/using-gatewayd/global-configuration/proxies.md +++ b/using-gatewayd/global-configuration/proxies.md @@ -1,5 +1,5 @@ --- -last_modified_date: 2024-05-31 20:16:38 +last_modified_date: 2024-07-30 16:34:34 layout: default title: Proxies description: GatewayD proxy configuration @@ -12,7 +12,7 @@ grand_parent: Using GatewayD The proxy object is used to proxy connections between database clients and servers. -GatewayD supports a fixed proxy that creates a pool with a fixed number of connection to the database server. It honors the pool capacity, and if the number of connections from the clients is more than the capacity, new connections will be rejected. +GatewayD supports multiple proxies, each creating its own pool of connections to the database server. Each proxy pool honors its capacity, and if the number of connections from the clients exceeds the capacity, new connections will be rejected. The PostgreSQL database expects new connections to authenticate before keeping them connected forever, thus the TCP connections from GatewayD will be timed out and dropped. A health check scheduler is started when creating connections to the database. If there are connections available in the available connections pool after the `healthCheckPeriod` is reached, it will remove and recreate new TCP connections to the database and put them in the pool. @@ -32,5 +32,6 @@ Each proxy has two pools: ```yaml proxies: default: - healthCheckPeriod: 60s # duration -``` \ No newline at end of file + writes: + healthCheckPeriod: 60s # duration +``` diff --git a/using-gatewayd/global-configuration/servers.md b/using-gatewayd/global-configuration/servers.md index 7c1d1ab..9f3ae88 100644 --- a/using-gatewayd/global-configuration/servers.md +++ b/using-gatewayd/global-configuration/servers.md @@ -1,5 +1,5 @@ --- -last_modified_date: 2024-05-31 20:16:38 +last_modified_date: 2024-07-30 16:45:29 layout: default title: Servers description: GatewayD server configuration @@ -14,24 +14,46 @@ The server object runs to listen for incoming connections from database clients. ## Configuration parameters -| Name | Type | Default value | Possible values | Description | -| ---------------- | ------- | ------------- | --------------- | ----------------------------------- | -| network | string | tcp | tcp, unix | The network protocol to use | -| address | string | 0.0.0.0:15432 | Valid host:port | The address to listen on | -| enableTicker | boolean | False | True, False | Whether to enable the ticker or not | -| tickInterval | string | 5s | Valid duration | The interval of the ticker | -| enableTLS | boolean | False | True, False | Whether to enable TLS or not | -| certFile | string | | Valid path | The path to the TLS certificate | -| keyFile | string | | Valid path | The path to the TLS key | -| handshakeTimeout | string | 5s | Valid duration | The timeout for TLS handshake | +| Name | Type | Default value | Possible values | Description | +| -------------------------------------- | ------- | ------------- | -------------------------------------------- | --------------------------------------------------------------------------- | +| network | string | tcp | tcp, unix | The network protocol to use | +| address | string | 0.0.0.0:15432 | Valid host:port | The address to listen on | +| enableTicker | boolean | False | True, False | Whether to enable the ticker or not | +| tickInterval | string | 5s | Valid duration | The interval of the ticker | +| enableTLS | boolean | False | True, False | Whether to enable TLS or not | +| certFile | string | | Valid path | The path to the TLS certificate | +| keyFile | string | | Valid path | The path to the TLS key | +| handshakeTimeout | string | 5s | Valid duration | The timeout for TLS handshake | +| loadBalancer | object | | | Configuration for the load balancer | +| loadBalancer.strategy | string | ROUND_ROBIN | ROUND_ROBIN, RANDOM, WEIGHTED_ROUND_ROBIN | The strategy used to distribute connections | +| loadBalancer.consistentHash | object | | | Configuration for consistent hash-based load balancing | +| loadBalancer.consistentHash.useSourceIp | boolean | False | True, False | Whether to use the source IP for consistent hashing | +| loadBalancer.loadBalancingRules | array | | List of rules | Optional configuration for strategies that support rules (e.g., WEIGHTED_ROUND_ROBIN) | +| loadBalancingRules.condition | string | DEFAULT | DEFAULT | Condition for the load balancing rule (currently, only "DEFAULT" is supported) | +| loadBalancingRules.distribution | array | | List of proxyName and weight pairs | Defines the weight distribution for proxies in the load balancing strategy | +| distribution.proxyName | string | | Valid proxy name | Name of the proxy server (e.g., "writes", "reads") | +| distribution.weight | integer | | Positive integer | Weight assigned to a proxy in the load balancing distribution | -## Example configuration + +### Example Configuration ```yaml servers: default: network: tcp address: 0.0.0.0:15432 + loadBalancer: + strategy: ROUND_ROBIN # ROUND_ROBIN, RANDOM, WEIGHTED_ROUND_ROBIN + consistentHash: + useSourceIp: true + # Optional configuration for strategies that support rules (e.g., WEIGHTED_ROUND_ROBIN) + # loadBalancingRules: + # - condition: "DEFAULT" # Currently, only the "DEFAULT" condition is supported + # distribution: + # - proxyName: "writes" + # weight: 70 + # - proxyName: "reads" + # weight: 30 enableTicker: False tickInterval: 5s # duration enableTLS: False