Attaching to Celery - is it possible?

[zacho112]

Hi there,

We have a python application running in k8s, and along with it a Celery worker to manage tasks etc. I'm trying attach to the instance, which seems to work, but not catching any breakpoints :(

It's just working out of the box in VSCode, and usually I'm a native Vim user, but vimspector does not support Celery, because of multiprocessing. However, I've been searching this forum, and are a bit uncertain if I'm supposed to get it working or not, and I hope you are able to help me out :)

Adapter config:

dap.adapters = {
        pythonWorker = {
                type = 'server';
                host = 'localhost';
                port = 6968;
         }
}

Configuration:

dap.configurations.python = {
        {
                type = 'pythonWorker';
                request = 'attach';
                name = "Python: Worker";
                pathMappings = {
                        {
                                localRoot = "${workspaceFolder}",
                                remoteRoot = "/opt/myapp"
                        }
                }
        },
}

Above is kinda the same config which just works in VSCode:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
      {
        "name": "Worker Remote Attach",
        "type": "python",
        "request": "attach",
        "connect": {
          "host": "localhost",
          "port": 6968
        },
        "pathMappings": [
          {
            "localRoot": "${workspaceFolder}",
            "remoteRoot": "/opt/myapp"
          }
        ]
      },
    ]
 }

And the log from nvim-dap:

  body = {
    connect = {
      host = "127.0.0.1",
      port = 6969
    },
    name = "Subprocess 74",
    pathMappings = { {
        localRoot = "/Users/user/Code/myapp",
        remoteRoot = "/opt/myapp"
      } },
    request = "attach",
    subProcessId = 74,
    type = "pythonWorker"
  },
  event = "debugpyAttach",
  seq = 12,
  type = "event"
}
[ WARN ] 2023-01-19T22:15:15Z+0100 ] /Users/user/.vim/plugged/nvim-dap/lua/dap/session.lua:927 ]	"No event handler for "	{
  body = {
    connect = {
      host = "127.0.0.1",
      port = 6969
    },
    name = "Subprocess 80",
    pathMappings = { {
        localRoot = "/Users/user/Code/myapp",
        remoteRoot = "/opt/myapp"
      } },
    request = "attach",
    subProcessId = 80,
    type = "pythonWorker"
  },
  event = "debugpyAttach",
  seq = 13,
  type = "event"
}

Any suggestions?

[mfussenegger]

vimspector does not support Celery, because of multiprocessing

It's the same for nvim-dap. debugpy uses a custom debugpyAttach event that the client needs to support to debug multiprocessing.
I had started with a PR in nvim-dap-python to support it (mfussenegger/nvim-dap-python#21) but abandoned it in favor of the startDebugging reverse request (microsoft/debug-adapter-protocol#344).

But this needs to be implemented in debugpy first (microsoft/debugpy#1074) and nvim-dap. I have some early prototype for that as well, but would need at least one debug-adapter implementation to wrap up it.

[zacho112]

Thanks for clarifying. I'll use vscode untill debugpy is ready, if I need to debug these kind of applications.

Keep up the good work, and thanks for a quick answer!

[mfussenegger]

Both nvim-dap master and a debugpy installed from git now have startDebugging support and I tested celery debugging with a configuration like the following and it worked.

        {
            "name": "Celery",
            "type": "python",
            "request": "launch",
            "module": "celery",
            "args": [
                "-A",
                "app_name",
                "worker",
                "--pool=gevent",
                "-l",
                "INFO",
                "--concurrency",
                "1"
            ],
            "gevent": true,
            "console": "integratedTerminal"
        }

So I assume your scenario should now work as well.