Skip to content

Commit 8f351f8

Browse files
[Pull-based Ingestion] Update ingestion source settings (#10122)
* Update ingestion source settings Signed-off-by: Varun Bharadwaj <varunbharadwaj1995@gmail.com> * Update _api-reference/document-apis/pull-based-ingestion.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Varun Bharadwaj <varunbharadwaj1995@gmail.com> --------- Signed-off-by: Varun Bharadwaj <varunbharadwaj1995@gmail.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
1 parent 18cb84f commit 8f351f8

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

_api-reference/document-apis/pull-based-ingestion.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,10 @@ The `ingestion_source` parameters control how OpenSearch pulls data from the str
7171
| `pointer.init.reset` | Determines the stream location from which to start reading. Optional. Valid values are `earliest`, `latest`, `reset_by_offset`, `reset_by_timestamp`, or `none`. See [Stream position](#stream-position). |
7272
| `pointer.init.reset.value` | Required only for `reset_by_offset` or `reset_by_timestamp`. Specifies the offset value or timestamp in milliseconds. See [Stream position](#stream-position). |
7373
| `error_strategy` | How to handle failed messages. Optional. Valid values are `DROP` (failed messages are skipped and ingestion continues) and `BLOCK` (when a message fails, ingestion stops). Default is `DROP`. We recommend using `DROP` for the current experimental release. |
74-
| `max_batch_size` | The maximum number of records to retrieve in each poll operation. Optional. |
74+
| `poll.max_batch_size` | The maximum number of records to retrieve in each poll operation. Optional. |
7575
| `poll.timeout` | The maximum time to wait for data in each poll operation. Optional. |
7676
| `num_processor_threads` | The number of threads for processing ingested data. Optional. Default is 1. |
77+
| `internal_queue_size` | The size of the internal blocking queue for advanced tuning. Valid values are from 1 to 100,000, inclusive. Optional. Default is 100. |
7778
| `param` | Source-specific configuration parameters. Required. <br>&ensp;&#x2022; The `ingest-kafka` plugin requires:<br>&ensp;&ensp;- `topic`: The Kafka topic to consume from<br>&ensp;&ensp;- `bootstrap_servers`: The Kafka server addresses<br>&ensp;&ensp;Optionally, you can provide additional standard Kafka consumer parameters (such as `fetch.min.bytes`). These parameters are passed directly to the Kafka consumer. <br>&ensp;&#x2022; The `ingest-kinesis` plugin requires:<br>&ensp;&ensp;- `stream`: The Kinesis stream name<br>&ensp;&ensp;- `region`: The AWS Region<br>&ensp;&ensp;- `access_key`: The AWS access key<br>&ensp;&ensp;- `secret_key`: The AWS secret key<br>&ensp;&ensp;Optionally, you can provide an `endpoint_override`. |
7879

7980
### Stream position

0 commit comments

Comments
 (0)