Consider watchfiles and other minor changes to tasks.json

[blakeNaccarato]

I stumbled upon your .vscode/tasks.json implementation while searching for creative uses of $python as a base problemMatcher. It's a pretty cool use-case, I didn't know you could use the problem matcher that way! Upon trying a variation of it in one of my projects I noticed the following:

• ptw stumbles on valid syntax in pyproject.toml via its use of configparser. Swapping pytest-watch for watchfiles fixes this
• Setting owner to external in problemMatcher entries causes reported issues to persist/stale in the VSCode Problems pane, even after fixing the problem. Removing "owner": "external" fixes this

I figured I'd contribute the changes back to a fork/branch of your repo, in case you're interested in a PR. I didn't want to actually submit the PR unsolicited in case your existing solution is working fine in your application, or if there's an edge-case I'm missing, so I figured I'd open a discussion about it.

See the diff at main...blakeNaccarato:django-ninja-crudl:task.

[jhassine]

@blakeNaccarato thanks for the interest for the projects.

Yeah, the ptw runner is kind of mess at the moment, did not pay attention too much yet. I have not decided to whether keep that ptw bg task there at all. It might also conflict the VScode's own test runner with the coverage info...

I'll have a look at your diff.

[blakeNaccarato]

Thanks! Yeah, I always like when I stumble on creative uses of tooling in the wild, and the problemMatcher customization in this project really opens up possibilities. My change is a bit rough-and-ready, e.g. the beginsPattern and endsPattern will likely need some adjustment if incorporating it as an actual PR, and I didn't run it through your infra e.g. Docker/CI. Feel free to pull it into a Draft PR if you want to poke around (not sure if that will give you edit permissions automatically...).

The upside to not rolling your own VSCode extension is that the command in your tasks.json could be pushed down to a justfile or other VSCode-agnostic runner, and allow the same command to run a watcher even outside of VSCode. So I think it's still good to have these kinds of tasks.

I think your problemMatcher customization of $python with file watching and minimal test suite re-run on changes still does quite a bit more than VSCode's "Testing" pane, even with their latest incorporation of running tests with coverage and the coverage gutter. See the VSCode docs on "Task Integration" where they provide the following note:

Note

Currently, tasks don't have special integration into VS Code's testing functionality, so running tests in a task won't update the user interface. Individual testing extensions can provide specific test automation functionality that integrates in the UI.

So, aside from rolling your own VSCode extension, e.g. hooking into the Test Coverage API, or if Python tests eventually get "continuous run" support (see April 2023, January 2023, November 2024 release notes), then maybe this will finally put to bed the use of custom problemMatchers.

There's also a newly proposed task problem matcher status API, that may be relevant.

[jhassine]

Thanks for the pointers!

That justfile looks interesting and better alternative to Makefile in this type of context, gonna check if adding that to the toolings.

I tried that watchfiles approach and it works but what I noticed that it does not abort and re-run the command immediately after a new change is done while the previous command is still running, it runs the existing command until it is finished. Not sure if this matters a lot but sometimes one would want to immediately run the tests with the new code when modifying the code and not to wait the previous run to finish.

I noticed also that my problemMatcher.background.endsPattern in tasks.json was not taking into account all cases. No surprise though, I originally did not put too much time to take account all cases.

[blakeNaccarato]

Yeah, if ptw has more sensible defaults and is responsive to rapid file edits and is working in this setup, that's sufficient. I think coming from Windows and using certain mis-parsed patterns in my pyproject.toml was the only reason I stirred up some errors in using ptw.

There may be some combination of arguments to watchfiles to make it more responsive, possibly the --debounce parameter.

Then at a deeper level it has an async implementation and can run arbitrary Python code on change detection, so there's probably some way to get watchfiles as immediately responsive as ptw, but at what cost of time and complexity to figure out the incantation? 😅