sink elasticsearch, contains an unknown parameter [_type] status 400

[StefanSa]

Hi there,
after upgrading to last vector version, i get this error message.

2022-05-27T12:47:21.007441Z ERROR sink{component_kind="sink" component_id=out_telegraf component_type=elasticsearch component_name=out_telegraf}:request{request_id=8}: vector::sinks::util::retries: Not retriable; dropping the request. reason="client-side error, 400 Bad Request: {\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Action/metadata line [1] contains an unknown parameter [_type]\"}],\"type\":\"illegal_argument_exception\",\"reason\":\"Action/metadata line [1] contains an unknown parameter [_type]\"},\"status\":400}"

bakend is opensearch v2, but with opensearch v1.3, we did not see this error.

{
  "name" : "os-client02",
  "cluster_name" : "development-cluster",
  "cluster_uuid" : "i_yvShiNS1ScuLlB9uWVgg",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.0.0",
    "build_type" : "tar",
    "build_hash" : "bae3b4e4178c20ac24fece8e82099abe3b2630d0",
    "build_date" : "2022-05-19T00:26:04.115016552Z",
    "build_snapshot" : false,
    "lucene_version" : "9.1.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

[spencergilbert]

I expect you'd want to toggle suppress_type_name for that version of OpenSearch. I'm not overly familiar with the project but it's likely they've also deprecated that option now.

[sach3000]

Hello Spencer,
version Vector: 0.21.2, Elasticsearch 8.2.2

Check my sink section:

[sinks.d0ejnxca_api_adapter_to_all_es]
  type = "elasticsearch"
  inputs = [ "d0ejnxca_api_adapter_all_full" ]
  bulk.action = "index"
  endpoint = "http://elk-node-0:9200"
  id_key = "id"
  bulk.index = "vector-all"
  mode = "normal"
  compression = "none"
  encoding.timestamp_format = "rfc3339"

and i too received from ES that
400 Bad Request

ERROR sink{component_kind="sink" component_id=d0ejnxca_api_adapter_to_all_es component_type=elasticsearch component_name=d0ejnxca_api_adapter_to_all_es}:request{request_id=0}: vector::sinks::util::retries: Not retriable; dropping the request. reason="client-side error, 400 Bad Request: {\"error\":{\"root_cause\":[ {\"type\":\"illegal_argument_exception\",\"reason\":\"Action/metadata line [1] contains an unknown parameter [_type]\"}],\"type\":\"illegal_argument_exception\"

i'm trying apply parameter suppress_type_name and Vector skip error, but nothing comes to elasticsearch (((

[spencergilbert]

If Vector is no longer logging any errors it's likely on the Elasticsearch side. You should check Elasticsearch's logs and open a new issue/discussion with details.

[StefanSa]

@spencergilbert
thanks for your quick help, that was the right tip.