-
Notifications
You must be signed in to change notification settings - Fork 48
Open
Description
Problem
The _type field has been removed from Amazon OpenSearchService since v2.0.
https://opensearch.org/docs/latest/breaking-changes/
For example, in the fluent-plugin-elasticsearch plugin, the suppress_type_name setting can be used to prevent the _type field from being sent to OpenSearchService, but is there a similar setting planned to be released for the fluent-plugin-aws-elasticsearch-service plugin?
https://github.com/uken/fluent-plugin-elasticsearch/#suppress_type_name
Alternatively, could you please provide a workaround?
Note that even if type_name "" is set, the _type field is still sent to OpenSearchService and the sent logs are rejected.
...
Steps to replicate
<match os.**>
@type aws-elasticsearch-service
include_tag_key true
type_name ""
target_index_key "@log_name"
tag_key "@log_name"
logstash_format true
logstash_prefix "projecta"
logstash_dateformat "%Y.%m"
<buffer tag>
@type "memory"
flush_thread_count 4
</buffer>
<endpoint>
url "https://search-xxxxxxxxxx.ap-northeast-1.es.amazonaws.com"
region "ap-northeast-1"
access_key_id "xxxxxx"
secret_access_key xxxxxx
</endpoint>
</match>
Expected Behavior or What you need to ask
...
Using Fluentd and ES plugin versions
- OS version
4.14.301-224.520.amzn2.x86_64 - Fluentd
td-agent 1.11.1 - AWS ES Service plugin version
fluent-plugin-aws-elasticsearch-service (2.4.0) - IAM policies for AWS ES Service (optional)
- ES version (optional)
OpenSearchservice v2.3
Metadata
Metadata
Assignees
Labels
No labels