Enrich file source with docker metadata

[OrjanPettersen]

Hi

Docker is using json-file log driver and are writing logs to the file system. Was using Filebeat earlier and there it was possible to enrich these log files with Docker metadata like container id and container name. Is there a way of doing the same in Vector using source: file?

Or is there maybe a different way of getting docker logs with the possibility to see which container they belong to?

Orjanp

[jszwedko]

Hi @OrjanPettersen !

Not currently, but would you want to open a feature request? Was this the logstash plugin you were using: https://github.com/itzg/logstash-filter-docker_container ? Or another one?

[jszwedko]

As a workaround, you could probably do it with the lua transform which embeds a full Lua runtime.

[OrjanPettersen]

Hi

Sorry for the late reply.

Filebeat has an option to add docker metadata before sending it to logstash. This is what we were using. I don't know what's under the hood though.
https://www.elastic.co/guide/en/beats/filebeat/7.17/add-docker-metadata.html

I'll make a feature request.

Thanks for the reply

Orjanp

[OrjanPettersen]

I can see now that docker_logs are an option as a source. Not sure if it was gone for a while, or I just missed it in the documentation. Saw that a docker source existed before, but it was gone and I didn't see an alternative when I looked at it earlier.

I'll have a look at it.

[spencergilbert]

The docker_logs source actually hits the Docker daemon directly for logs, rather than the file written out by json-file logger. I think it's still worth opening a feature request for a transform that can enrich logs, that could be used with a greater number of logging backends for Docker.