Stop reading from source (kafka) when sink (opensearch) is down in vector aggregator.

[shreyasarani23]

I have a source as a kafka and sink as a opensearch my requirement is that vector aggregator should stop reading from kafka whenever sink (opensearch) is down and should resume reading from source (kafka) once the opensearch is up. Below is my vector aggregator configuration.

source:
  kafka_logs:
     type: kafka
     bootstrap_servers: <kafka-endpoint>:9092
     topics: 
      - eks-logs
     group_id: vector-aggregator-group
     decoding:
         codec: "json"
        
 sink:
    opensearch:
          type: elasticsearch
          inputs:
            - kafka_logs
          batch:
            timeout_secs: 5
          buffer:
            type: memory
            when_full: block
          bulk:
            action: index
            index: |-
              {{ print "{{ _meta.target.index }}" }}-%Y.%m.%d
          mode: bulk
          pipeline: primary
          compression: gzip
          id_key: "_id"
          endpoints:
            - ${VECTOR_OPENSEARCH_ENDPOINT}
          auth:
            strategy: basic
            user: $${VECTOR_OPENSEARCH_USERNAME}
            password: $${VECTOR_OPENSEARCH_PASSWORD}
          healthcheck:
            enabled: true

How can I configure my vector aggregator to stop reading from the source if the sink is down ? As per the above configuration i guess the events will be stored in buffer.memory and it is used along withwhen_full:block option so does that mean vector aggregator will stop reading from kafka when buffer.max_events is full?

To stop vector aggregator reading from kafka should I add acknowledgements.enabled: true in the opensearch sink?
Does it mean that if for some reason if vector aggregator was not able to index the data to opensearch will it stop reading from source as acknowledgements.enabled: true?

[shreyasarani23]

@pront can you please suggest if enabling acknowledgements.enabled: trueat the sink will enable back pressure to source and vector aggregator will stop reading from source?