docker-compose elastalert not working on restart

[vibarco]

Hello,

when using elasalert2 from docker-compose elastalert is not resuming alerts,

after changing a rule i do:

docker-compose stop elastalert
docker-compose start elastalert

and elastalert is not resuming until past hours, no activity in alert_status_status

my config file

es_host: elasticsearch
es_port: 9200

writeback_index: elastalert_status

rules_folder: rules

run_every:
   minutes: 1

buffer_time:
   minutes: 30

logging:
 version: 1
 incremental: false
 disable_existing_loggers: false
 formatters:
   logline:
     format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'

loggers:
   elastalert:
     level: DEBUG
     handlers: []
     propagate: true

elasticsearch:
     level: DEBUG
     handlers: []
     propagate: true

rule config

name: Test rule

 run_every:
    seconds: 20

 type: any

 timeframe:
   minutes: 120

 scan_entire_timeframe: true

 filter:
  - query:
      query_string:
        query: "\"string_to_find\""

 index: "cqg*"
 is_enable: true

 alert:
   - email:
      from_addr: ""
      email: ""
 alert_subject: "Test"
 smtp_host: ""
 smtp_port:
 smtp_ssl: true
 smtp_auth_file: "/opt/elastalert/smtp-auth.yml"

What i am doing wrong?

Best regards

[jertel]

I don't understand this statement:

and elastalert is not resuming until past hours, no activity in alert_status_status

ElastAlert 2 will not re-alert on something it's already alerted on. However, if you change the name of the rule, then it will re-alert on everything again, since it has no stored knowledge of already having executed searches and alerts for that new rule name.

[vibarco]

Thank you @jertel. I am not getting anything distinct to this in logs with this logging section conf. No errors

logging:
 version: 1
 incremental: false
 disable_existing_loggers: false
 formatters:
   logline:
     format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'
handlers:
  console:
     class: logging.StreamHandler
     formater: logline
     level: DEBUG
     stream: ext://sys.stderr
loggers:
   elastalert:
     level: DEBUG
     handlers: []
     propagate: true
elasticsearch:
     level: DEBUG
     handlers: []
     propagate: true
'':
     level: DEBUG
     handlers:
        - console
     propagate: false

docker-compose logs -f elastalert

[jertel]

The elasticsearch and '' loggers don't look like they're indented correctly. It should look like this:

logging:
  version: 1
  incremental: false
  disable_existing_loggers: false
  formatters:
    logline:
      format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'

  handlers:
    console:
      class: logging.StreamHandler
      formatter: logline
      level: DEBUG
      stream: ext://sys.stderr

  loggers:
    elastalert:
      level: DEBUG
      handlers: []
      propagate: true

    elasticsearch:
      level: DEBUG
      handlers: []
      propagate: true

    elasticsearch.trace:
      level: DEBUG
      handlers: []
      propagate: true

    '':  # root logger
      level: DEBUG
      handlers:
        - console
      propagate: false

[vibarco]

Hello again!,

i am still stuck with logging, no logs when running docker elastalert image from docker-compose. Looking inside the container. Elastalert is running launching run.sh

maybe the problem is how is elastalert is launching

elastalert --config /opt/elastalert/config.yaml "$@"

Should it be instead?

elastalert --verbose --config /opt/elastalert/config.yaml "$@"

I am not getting logs on docker-compose logs -f elastalert ( see my screenshot before )

Best

[jertel]

No that isn't necessary. How about this: Specify an entrypoint of /bin/bash in your Docker compose file for the ElastAlert container. Then once the container is running, Docker exec into it and manually execute run.sh. Let's see what you get printed to the console after doing that. While you're inside the container you can also cat /opt/elastalert/config.yaml just to make sure the right config file with your logging changes is being mounted into the container.

[vibarco]

Thanks @jertel,

seems everytging is correct, changes inmmediatly appear in the container. Sure i am not understanding correcty elastalert. If i test the rule from intereactive docker exec inside the container

elastalert-test-rule --config /opt/elastalert/config.yaml /opt/elastalert/rules/rule-2.yaml

output show logs and elastalert is executing the rule correctly

But the INFO logs that elastalert-test-rule shows are not saved into elastalert_status_status index immediatly, never appearing when execute run.sh and of course when running docker-compose logs -f elastalert either.

I don't know what i am missing

Best