Surprisingly, `format.json` problem matchers aren't working as intended!

[DavisVaughan]

Consider the ruff-action test.yml, which has a purposefully failing test that generates Annotations using Problem Matchers

Notably, note that the Would reformat annotations are on #L1 for line 1, and show up both in the Actions menu and in the Pull Request files view (the latter is where it's actually useful)
https://github.com/astral-sh/ruff-action/actions/runs/14531202637/job/40771137207
https://github.com/astral-sh/ruff-action/pull/123/files

Here's the bonkers thing. If you actually try and use astral-sh/ruff-action@v3 in a minimal repo, then the Would reformat file annotation is attributed to #L0 (???) and only shows up in the Actions menu, but doesn't show up in the Pull Request files menu! Presumably this is because line 0 doesn't actually exist in the file, so it has nowhere to put the annotation in the Pull Request files view.

DavisVaughan/ruff-action-test#1

I have to imagine this is some kind of weird GitHub bug somehow. It should probably either be line 0 in both cases, or line 1 in both cases.

So, IIUC, the formatting problem matcher added in #26 hasn't actually been triggering (I could be wrong here, but I can't get it to trigger)!

Two possible solutions:

• Maybe you can specify line in the problem matcher and force it to line 1, but I'm not sure if you can do that

• Ruff itself gets a change, and a new ruff format --check --output-format=github would directly emit github workflow commands. I think in theory you wouldn't even need the problem matcher then, and you could specify line=1 there.

[eifinger]

Very strange indeed. Can you please run again with debug logging enabled? I would assume that forcing the correct line is something GitHub does because the problem matcher does not define a line property

[DavisVaughan]

I think I even figured out roughly where github does the translation to annotation:

https://github.com/actions/runner/blob/1c319b4d4225d4a74ac92f1974ca86b1dac965dc/src/Runner.Worker/ExecutionContext.cs#L519-L525

https://github.com/actions/runner/blob/1c319b4d4225d4a74ac92f1974ca86b1dac965dc/src/Sdk/RSWebApi/Contracts/IssueExtensions.cs#L20

It looks roughly like it is intended to be 0 if no line is provided, so I'm not entirely sure there the #L1 comes from. I imagine it has something to do with the fact that its testing an action hosted inside the repo itself and maybe goes through some other code path, but that's just a guess.

---

Relevantish section from debug logs https://github.com/DavisVaughan/ruff-action-test/actions/runs/14654324515/job/41159696827

##[add-matcher]/home/runner/work/_actions/astral-sh/ruff-action/v3/.github/matchers/check.json
##[debug]Added matchers: 'ruff-check'. Problem matchers scan action output for known warning or error strings and report these inline.
##[add-matcher]/home/runner/work/_actions/astral-sh/ruff-action/v3/.github/matchers/format.json
##[debug]Added matchers: 'ruff-format'. Problem matchers scan action output for known warning or error strings and report these inline.
Successfully installed ruff version 0.11.7
/opt/hostedtoolcache/ruff/0.11.7/x86_64/ruff format --check /home/runner/work/ruff-action-test/ruff-action-test
Error: Would reformat: test.py
1 file would be reformatted
Error: The process '/opt/hostedtoolcache/ruff/0.11.7/x86_64/ruff' failed with exit code 1
##[debug]Node Action run completed with exit code 1
##[debug]RUFF_OUTPUT_FORMAT='github'
##[debug]Set output ruff-version = 0.11.7
##[debug]Finishing: Format check

So it does add the matchers, but doesn't report each match it finds, which would have been nice.

[DavisVaughan]

The other relevant bit is this ConvertToIssue() helper, which should give some kind of debug log if it finds a match but has to throw it out, and I don't see any of those (which makes sense, the match does show up in the action run's Annotations view):
https://github.com/actions/runner/blob/1c319b4d4225d4a74ac92f1974ca86b1dac965dc/src/Runner.Worker/Handlers/OutputManager.cs#L200

From what I understand the conversion github does is regex match -> Issue -> Annotation