docker: shlink-task-worker-x shows 100% CPU in htop

[michabbb]

hi,

i am using the docker image shlink:stable (v2.6.2) with mysql.
while watching htop (on the host) i noticed:

and I am surprised because the server only has 200 visits a day.
so: is this "normal" that these workers take so much CPU ???

thanks for any help!

[acelaya]

No, that shouldn't be the case. Have you tried restarting Shlink?

[acelaya]

Thanks!

I'm not sure what could be the reason. The task workers only geolocate visits, which is usually super fast.

They also download updated versions of the GeoLite db. Did you provide your own license key?

[michabbb]

yes, i do ;)

[paolobasso99]

I just checked and it happened to me as well. After restart everything is ok.

I am using shlink inside docker on an ubuntu vps and I did provide my own license key as well.

[acelaya]

I would need to know after how much time do you notice this starts to happen, and what is the config you provide to the container.

[paolobasso99]

I am running a single host shlink container using docker-compose and keeping almost all configuration to default. The instance was running for abount 2 months and I do not know when the issue started, if it happens again I will inform you.

This is the service defined in my docker-compose.yml:

shlink_api:
    image: shlinkio/shlink:stable
    container_name: shlink_api
    depends_on:
      - shlink_db
    environment:
      - SHORT_DOMAIN_HOST=${SHLINK_API_SUBDOMAIN}.${SHLINK_BASEDOMAIN}
      - SHORT_DOMAIN_SCHEMA=https
      - GEOLITE_LICENSE_KEY=${SHLINK_GEOLITE_LICENSE_KEY}
      - DEFAULT_BASE_URL_REDIRECT=https://${SHLINK_BASEDOMAIN}
      - DB_DRIVER=postgres
      - DB_NAME=shlink
      - DB_USER=shlink
      - DB_PASSWORD=${SHLINK_DB_PASSWORD}
      - DB_HOST=shlink_db
    restart: unless-stopped
    labels:
      - traefik proxy labels ...
    networks:
      - web_proxy
      - shlink

The server is on version 2.10.1, I will now update it and report back if the issue reapears.

[djsime1]

This has been happening intermittently over the course of the past year or so. I've always kept Shlink updated to the latest version but it keeps cropping up after running for a few weeks straight. My config is virtually identical to the one above this comment.

[djsime1]

Thanks. I'll report back in a few weeks or a month with any updates.

[djsime1]

I can confirm this issue is still present in the roadrunner version. Just like before, the worker process is maxing out a core.

[acelaya]

Thanks for the heads-up.

I have created an issue to track this.

[djsime1]

I don't know if I would call it a memory leak since as far as I could tell memory usage was normal for all processes in the container. Most of the time only one worker process started using 100% CPU usage on one core, but there was one or two times where I saw multiple (2-3) using 100% CPU after I hadn't checked my server's performance metrics for a long time. What I'm saying is that each worker process has a chance of entering this weird 100% CPU issue after a few weeks of running.

[acelaya]

Ah, sorry, you are right. This is a CPU problem, not a memory one 😅

[djsime1]

@michabbb Out of curiosity, what sort of machine is Shlink running on? Is it a cloud/VPS machine, or a dedicated/bare metal? Also which provider if applicable? I'm running on a 'VPS M' from Contabo.

[djsime1]

Ah, that might be the issue since I still use the built-in updater. I'll try disabling it and see if that's the magical fix.

[acelaya]

My suspicion is also that it is related with the Geolite DB update

[acelaya]

@djsime1 I have started to investigate this. Did you notice this issue again since you removed the GeoLite key?

[djsime1]

So far, no issues.

[acelaya]

Thanks!

[acelaya]

I have found reports of other people suffering from 100% utilization when guzzle is involved (guzzle is the HTTP client used by Shlink to, among other things, download GeoLite databases), usually when using curl_multi_exec, due to how they handle calls internally with an infinite loop.

• https://snelling.io/curl-multi-high-cpu-usage/
• https://www.drupal.org/project/drupal/issues/2595927
• 100% of CPU usage on curl_multi_exec guzzle/guzzle#756

I have changed it to use curl_exec instead. It's slightly less performant when running multiple requests, but that should not be a problem for Shlink.

Will get this released with Shlink 3.6.0