Skip to content

executing plugin "did not finish" in 30s − possibilty to increase the timeout? #2027

@JeanFred

Description

@JeanFred

I have a Docker container which, on start-up, renders a configuration file (for a datadog-agent to monitor some data stores). The file is rendered using exec mode:

/usr/local/bin/consul-template \
  -consul-addr="${REAL_CONSUL_ADDR}" \
  -template="/etc/datadog-agent/datadog.yaml.ctmpl:/etc/datadog-agent/datadog.yaml" \
  -template="/etc/datadog-agent/conf.d/postgres.d/postgres.yaml.ctmpl:/etc/datadog-agent/conf.d/postgres.d/conf.yaml.kms:/bin/bash -c '/usr/local/bin/gruntkms decrypt --aws-region ${REGION} < /etc/datadog-agent/conf.d/postgres.d/conf.yaml.kms > /etc/datadog-agent/conf.d/postgres.d/conf.yaml'" \
  -template="/etc/datadog-agent/conf.d/redisdb.d/redisdb.yaml.ctmpl:/etc/datadog-agent/conf.d/redisdb.d/conf.yaml.kms:/bin/bash -c '/usr/local/bin/gruntkms decrypt --aws-region ${REGION} < /etc/datadog-agent/conf.d/redisdb.d/conf.yaml.kms > /etc/datadog-agent/conf.d/redisdb.d/conf.yaml'" \
  -consul-retry \
  -max-stale=0 \
  -exec "/usr/bin/datadog-agent run"

One of the template also invokes a 400+ lines Python script which does all kinds of things (including iterating over all DBs, running CREATE OR REPLACE FUNCTION statements for each, etc )

{{ $path := ( print "/config/infrastructure/datadog/files/postgres" ) }}
{{ key $path | plugin "updatepgconfig.py" }}

This used to work really well ; but since recently, presumably as we have more and more data-stores, this does not terminate in time and it crashes with

2025-03-05T14:58:53.760Z [ERR] (cli) /etc/datadog-agent/conf.d/postgres.d/postgres.yaml.ctmpl: execute: template: :2:15: executing "" at <plugin "/updatepgconfig.py">: error calling plugin: exec "updatepgconfig.py": did not finish in 30s

This was set to 30s following #818.

Now, I realize that this setup is perhaps abusing consul-template a little bit too much, and that this is simply not a use-case that consul-template should support, and we are considering changing that setup. Meanwhile though, I was wondering whether it would be an option to make that timeout configurable (via environment variable or something)? While no long-term fix, it would solve the immediate issue I am currently having. But as said, I would totally understand if this is declined :)

consul-template version: 0.39.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions