-
Notifications
You must be signed in to change notification settings - Fork 35
Open
Labels
bugSomething isn't workingSomething isn't workingdetectorsAbout nex or existing detectorsAbout nex or existing detectors
Description
What is the module?
otel-collector_kubernetes-common
What is the detector?
pod_phase_status
Describe the bug
pod_phase_status alerts are still active even when the pod does not exist anymore
To Reproduce
Steps to reproduce the behavior:
- Nominal "ok" state of the detector
- The pod is in a failed state the alert is raised
- The pod is automaticaly recreated by k8s
- The alert is still active unless we do a manual clear
Expected behavior
The alert should clear itself when the pod does not exist anymore or if the pod just pop-up and dies quickly this detector should not triggers at all
Screenshots
Additional context
A local solution has been found :
Add .fill(2,duration='1s') in the line :
signal = data('k8s.pod.phase', filter=base_filtering and filter('env', 'prod') and filter('sfx_monitored', 'true')).fill(2,duration='1s').publish('signal')
Pull request should come up soon.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingdetectorsAbout nex or existing detectorsAbout nex or existing detectors