all: Limit amounts of logs from the agent itself #5
Description
Background
Excessive amounts of logs from the agent could be triggered by I/O error (ex: disk quota) or broken input logs. In the worst case, we could assume one malformed input log would at least trigger one warning level log from the agent. This is unacceptable as it could easily cause TBs of logs in a few minutes.
What to do
Set up log limit per component and apply it to the smallest unit possible: a TCP connection, a pipelines (not sub-pipelines) or a set of metric keys. The limit should be configurable, for example:
- max_logging_count: 500
- max_logging_duration: 1h (reset counter afterwards)