Using all_metrics in log_to_metric transform?

[generate-me12]

Hi, I'm trying to use the log_to_metric transform but all_metrics. Basically I just vector to try to transform my logs to metrics if possible since all the logs will have different metric type counter, gauge, and different fields and metric kind so I won't be able to define them all as they are dynamic.

I've tried using it but all_metrics and from the documentation it says "notably the metrics field will be ignored" but it isn't ignored.

I tried converting host metrics to logs and then back to metrics but it just uses the fields defined in metrics instead of ignoring it.
This is my test configuration.

sources:
  host_metrics:
    type: host_metrics
    scrape_interval_secs: 3

transforms:
    internal_to_log:
      type: metric_to_log
      inputs: 
        - host_metrics
    log_to_metrics:
      type: log_to_metric
      inputs:
        - internal_to_log
      metrics:
        - type: counter
          field: namespace  # Use an actual field name
          name: "converted_metric"

sinks:
  my_sink_id:
    type: console
    inputs:
      - log_to_metrics
    encoding:
      codec:
        json

api:
  enabled: true

This is the output metrics:
{"name":"converted_metric","timestamp":"2025-01-30T19:31:24.821371Z","kind":"incremental","counter":{"value":1.0}}

[pront]

Hi @generate-me12, I don't see an all_metrics: true in your test configuration. In general, if you want all metrics, then the config should look like:

transforms:
    t0:
      type: log_to_metric
      inputs:
        - internal_to_log
      all_metrics: true

[generate-me12]

My bad, I was doing testing and forgot to include it. I do see that the metrics field is ignored now. So when we do the configuration, we will have to include the metrics field and supply dummy values in there to use all_metrics: true? Since if I omit it it says:

Configuration error. error=missing field `metrics`

Last question, is there any way to know what metrics are dropped by all_metrics if it doesn't match the format?

[pront]

Ah, it should be one or the other. This error you are seeing is probably a bug. Can you please https://github.com/vectordotdev/vector/issues/new?template=bug.yml?

Last question, is there any way to know what metrics are dropped by all_metrics if it doesn't match the format?

Yes, you can use component_errors_total with the tag error_type: parser_failed.