[Q&A] Fluentd S3 plugin custom endpoint

[lisazhao30]

What is a problem?

Can fluentd s3 plugin support endpoints other than the s3..amazonaws.com? If I have an object storage (SwiftStack) instance with a unique URL, can I still use the s3 plugin? I want to chunk logs to be sent to files in the object storage.

Describe the configuration of Fluentd

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
data:
  fluent.conf: |-
    <source>
      @type forward
      bind 0.0.0.0
      port 80
    </source>
    <match pattern>
      @type s3
      aws_key_id key_id
      aws_sec_key sec_key
      s3_bucket test-bucket
      path logs/
     # unique url here if possible
    </match>

Describe the logs of Fluentd

2025-07-10 23:57:58 +0000 [error]: #0 unexpected error error_class=RuntimeError error="can't call S3 API. Please check your credentials or s3_region configuration. error = #<Seahorse::Client::NetworkingError: Failed to open TCP connection to UNIQUE_URL (getaddrinfo: Try again)>"

Environment

- Fluentd version: 1.18.0
- TD Agent version:
- Fluent Package version:
- Docker image (tag):
- Operating system:
- Kernel version:

[Watson1978]

I don't know about SwiftStack,
but, it is possible by configuring s3_endpoint to connect to the storage service of localstack

Please refer to https://github.com/fluent/fluent-plugin-s3/blob/master/docs/output.md#s3_endpoint