Confusion regarding DataDog Agent vs Vector

[julealgon]

I started with the following problem at hand: DataDog doesn't provide an OpenTelemetry Collector exporter for logs:

• [exporter/datadog] Log exporter support open-telemetry/opentelemetry-collector-contrib#2651

Since I want to keep vendor code away from my apps, I started looking into potential solutions.

The first thing that came to my mind was routing otel-collector logs to a DataDog Agent that would then push the logs into DD itself. However, after a quick search, it appears that the DD Agent doesn't support logs at all...

Then I came back to the original issue and found that one poster was using a tool called "vector" in combination with the otel-collector for processing logs into DataDog to overcome the limitation of lack of an exporter in otel-collector.

I proceeded to search for vector and then found this..... which is..... "by DataDog"?! I'm very confused now. Can someone clarify please? Are all of these true?

• DD has not implemented an otel-collector-compatible log exporter
• DD has an agent called "DataDog Agent", that also doesn't support processing logs in open telemetry protocol format
• DD has.... this project, which is another agent, similar to the purpose of both the DataDog Agent as well as the otel-collector, that does support consuming and pushing logs in otel format

Why does the DataDog Agent exist at all at this point, if this is supposedly a full-blown solution that does everything the DD Agent does and is done by the same company?

[binarylogic]

Hey @julealgon, happy to help, and agree it is a little bit confusing right now. I know we (Datadog) are working to expand Otel support in the Datadog Agent. I'm not sure what the timeline is for logs exactly, but we did just recently announce trace support

https://www.datadoghq.com/blog/ingest-opentelemetry-traces-metrics-with-datadog-agent/

Vector is prioritizing logs and trace support, we're hoping to release this in Q3 of this year.

this project, which is another agent

Vector is a very limited agent and focuses on simple data collection (i.e., tailing files, basic metrics collection, etc) where the Datadog Agent is much more comprehensive (500+ integrations, RUM, profiling, tight Datadog platform integration, etc). Vector is focused more on the control aspect of the data, giving you robust tools to process and route data. Where the Datadog Agent is all about simple, turnkey integration with the Datadog platform.

In your case, I would consider sending Otel logs in JSON format to Datadog. If you're writing your Otel logs to a file, the Datadog agent should be able to pick these up pretty easily.

[julealgon]

Vector is prioritizing logs and trace support, we're hoping to release this in Q3 of this year.

That's again confusing. All the documentation (and the comments in the issue I linked) indicate that Vector already fully supports logs. I did see that traces functionality is currently marked as "under construction" though. Can you clarify please? Is there any current limitation or known issue with flowing logs through Vector that is not documented?

In your case, I would consider sending Otel logs in JSON format to Datadog. If you're writing your Otel logs to a file, the Datadog agent should be able to pick these up pretty easily.

Are you saying that the DataDog agent doesn't accept logs in OTEL format being pushed to it, but it accepts logs in OTEL format when pulling from a file? That's.... a bit unexpected, but if that's the case, would you then suggest I install the DD agent alongside my otel-collector, output the logs to a file in the otel-collector, and then setup DD agent to pick from that file location?

I was outputting the data to a file for testing purposes, but if this is a recommended path instead of piping through Vector like the poster said in the thread, we could look into it. Honestly, I just want the simplest possible solution to this issue while DataDog works on the log exporter. The less maintenance overhead for us the better.

[jszwedko]

Hey @julealgon !

I can definitely understand there still being some confusion here given the overlap of these tools. Some responses in line:

That's again confusing. All the documentation (and the comments in the issue I linked) indicate that Vector already fully supports logs. I did see that traces functionality is currently marked as "under construction" though. Can you clarify please? Is there any current limitation or known issue with flowing logs through Vector that is not documented?

Vector does fully support logs as a data type, but we don't support ingesting logs via the OpenTelemetry protocol yet . There is an open PR working on adding this #13320

Are you saying that the DataDog agent doesn't accept logs in OTEL format being pushed to it, but it accepts logs in OTEL format when pulling from a file? That's.... a bit unexpected, but if that's the case, would you then suggest I install the DD agent alongside my otel-collector, output the logs to a file in the otel-collector, and then setup DD agent to pick from that file location?

The Datadog Agent just collects and forwards the data as raw text so, if you write JSON OTel data to a file, you could have the Datadog Agent (or Vector) read it and forward it to Datadog.

I was outputting the data to a file for testing purposes, but if this is a recommended path instead of piping through Vector like the poster said in the thread, we could look into it. Honestly, I just want the simplest possible solution to this issue while DataDog works on the log exporter. The less maintenance overhead for us the better.

Yeah, agreed, hopefully that happens soon.