How to use vector as a lib to push to another vector instance

[rolandjitsu]

I was wondering if it is possible to use vector as a lib within my own rust program to push data to another vector endpoint.

Typically, I would just use vector as an agent and have a vector 2 vector setup, but due to some restrictions, I cannot run additional services/programs.

Right now, I've enabled the http_server source and I simple POST the data, but it would be great to use the vector source and port instead of opening up an additional port:

[my program -> vector lib] -> vector aggregator :6000

[pront]

Can you share you config (redacting any sensitive data) please?

If you can achieve it with HTTP there should not be much difference vs Vector --> Vector. But I am missing some context here.

Appendix

Using vector generate:

vector generate vector/remap/vector
data_dir: /var/lib/vector/
sources:
  source0:
    address: 0.0.0.0:6000
    type: vector
    acknowledgements: {}
transforms:
  transform0:
    inputs:
    - source0
    drop_on_abort: false
    drop_on_error: false
    metric_tag_values: single
    reroute_dropped: false
    runtime: ast
    type: remap
sinks:
  sink0:
    inputs:
    - transform0
    address: 127.0.0.1:6000
    compression: false
    type: vector
    batch: {}
    request:
      rate_limit_duration_secs: 1
      rate_limit_num: 9223372036854775807
      retry_attempts: 9223372036854775807
      retry_initial_backoff_secs: 1
      retry_jitter_mode: Full
      retry_max_duration_secs: 30
      timeout_secs: 60
      adaptive_concurrency:
        decrease_ratio: 0.9
        ewma_alpha: 0.4
        initial_concurrency: 1
        max_concurrency_limit: 200
        rtt_deviation_scale: 2.5
    healthcheck:
      enabled: true
      uri: null
    buffer:
      type: memory
      max_events: 500
      when_full: block

[rolandjitsu]

Can you share you config (redacting any sensitive data) please?

I can. But I think your 2nd statement is kind of answering my question or the reason why I was asking it.

If you can achieve it with HTTP there should not be much difference vs Vector --> Vector. But I am missing some context here.

I was under the impression that by not going with a vector 2 vector approach I'd not benefit from things like acknowledgements, buffering, etc. or whatever perf optimization may exist in that protocol (e.g. gRPC may have better perf over plain http).

Also, I wouldn't have to maintain the rust client for using the HTTP endpoint and I'd rely just on this lib.

[pront]

So the answer to the original question is yes, it's possible. And there are a number of ways to do it.

I was under the impression that by not going with a vector 2 vector approach I'd not benefit from things like acknowledgements, buffering, etc. or whatever perf optimization may exist in that protocol (e.g. gRPC may have better perf over plain http).

Right, are we talking about the following scenarios?

1. Custom Program -> Vector Source -> Vector Sink
2. Custom Program -> Vector Source (over a socket)