Skip to content

Can i suppress type name for OpenSearchservice v2.3 #84

@hanedatatsufumi

Description

@hanedatatsufumi

Problem

The _type field has been removed from Amazon OpenSearchService since v2.0.

https://opensearch.org/docs/latest/breaking-changes/

For example, in the fluent-plugin-elasticsearch plugin, the suppress_type_name setting can be used to prevent the _type field from being sent to OpenSearchService, but is there a similar setting planned to be released for the fluent-plugin-aws-elasticsearch-service plugin?

https://github.com/uken/fluent-plugin-elasticsearch/#suppress_type_name

Alternatively, could you please provide a workaround?

Note that even if type_name "" is set, the _type field is still sent to OpenSearchService and the sent logs are rejected.

...

Steps to replicate

  <match os.**>
    @type aws-elasticsearch-service
    include_tag_key true
    type_name ""
    target_index_key "@log_name"
    tag_key "@log_name"
    logstash_format true
    logstash_prefix "projecta"
    logstash_dateformat "%Y.%m"
    <buffer tag>
      @type "memory"
      flush_thread_count 4
    </buffer>
    <endpoint>
      url "https://search-xxxxxxxxxx.ap-northeast-1.es.amazonaws.com"
      region "ap-northeast-1"
      access_key_id "xxxxxx"
      secret_access_key xxxxxx
    </endpoint>
  </match>

Expected Behavior or What you need to ask

...

Using Fluentd and ES plugin versions

  • OS version
    4.14.301-224.520.amzn2.x86_64
  • Fluentd
    td-agent 1.11.1
  • AWS ES Service plugin version
    fluent-plugin-aws-elasticsearch-service (2.4.0)
  • IAM policies for AWS ES Service (optional)
  • ES version (optional)
    OpenSearchservice v2.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions