Use Windows System environment variable in host_key

[marketier]

I have several System environment variables defined on a Windows host. Can an environment variable be defined for host_key?

My source is:

[sources.winapplogs]
type = "file"
ignore_older_secs = 600
glob_minimum_cooldown_ms = 1_800
include = [ "L:/Logs/**/Logfile.log" ]
read_from = "beginning"
max_line_bytes = 102_400
max_read_bytes = 2_048
file_key = "file"
host_key = "host"

Is something like the following possible?

host_key = "HOST_ALIAS_ENV_VAR"

[neuronull]

Hi @marketier !

Yes, you should be able to use environment variables to configure the host_key.

(see https://vector.dev/docs/reference/configuration/#environment-variables)

Try:

host_key = "${HOST_ALIAS_ENV_VAR}"

[marketier]

Hi @neuronull ,

Thanks for your reply and pointing me to the doc. This prompted me to check out add_fields transform and I now have it working.

For anyone interested, I am sourcing logs on Windows and forwarding to New Relic. This is part of a pipeline in Octopus Deploy. #{Variable_Name} denotes variable substitution in Octopus Deploy.

[sources.worker_logs]
type = "file"
ignore_older_secs = 600
glob_minimum_cooldown_ms = 1_800
include = [ "L:/Logs/Windows/**/Logfile.log" ]
read_from = "beginning"
max_line_bytes = 102_400
max_read_bytes = 2_048
file_key = "file"
host_key = "host"
fingerprint.lines = 5
fingerprint.ignored_header_bytes = 0
fingerprint.strategy = "checksum"

[transforms.add_host]
type = "add_fields"
inputs = [ "worker_logs" ]

[transforms.add_host.fields]
hostname = "${HOST_ALIAS_ENV_VAR}"
environment = "#{Octopus.Environment.Name}"

[sinks.new_relic]
type = "new_relic"
inputs = [ "add_host" ]
license_key = "#{NewRelicLicenceKey}"
account_id = "#{NewRelicAccountID}"
region = "us"
compression = "none"
api = "logs"

I like Vector. There's an elegance to it.

[jszwedko]

Hi @marketier !

Just a small note that the add_fields transform is deprecated and will be going away in the next release. Instead, you can use remap to accomplish the same thing:

[transforms.add_host]
type = "remap"
inputs = [ "worker_logs" ]
source = """
.hostname = "${HOST_ALIAS_ENV_VAR}"
.environment = "#{Octopus.Environment.Name}"
"""

See vrl.dev for more details about the DSL (Vector Remap Language) used in the remap transform.

[jszwedko]

I am noticing add_fields is still used in that doc though 😅 . I'll fix that.

[marketier]

Thanks for the feedback. I've replaced add_fields with remap as you suggested and the logs are appearing in New Relic along with the additional fields.