R: Test wrappers not detected by Test Explorer

[samclark2015]

System details:

Positron and OS details:

Positron Version: 2025.04.0 (Universal) build 250
Code - OSS Version: 1.98.0
Commit: 627b3d723ee0b3a5bd86731a75902d419cafe22d
Date: 2025-04-03T16:31:38.728Z
Electron: 34.2.0
Chromium: 132.0.6834.196
Node.js: 20.18.2
V8: 13.2.152.36-electron.0
OS: Darwin arm64 24.3.0

Interpreter details:

R 4.4.0, but affects any version

Describe the issue:

The R test explorer does not display tests that are generated using wrappers of testthat, as described in #7210.

The below screenshot shows a testing file which leverages a targets test, and implements a custom test wrapper customtest, neither of which are displayed in the test explorer sidebar.

Expected or desired behavior:

All tests, declared directly using testthat or by using another wrapper, are displayed in the test explorer.

Were there any error messages in the UI, Output panel, or Developer Tools console?

From R Language Pack output channel:

2025-04-10 11:40:39.772 [debug] Received test data: {"type":"start_reporter"}
2025-04-10 11:40:39.772 [debug] Received test data: {"type":"start_file","filename":"test-rename-me.R"}
2025-04-10 11:40:39.772 [info] Setting hostFile to test-rename-me.R
2025-04-10 11:40:39.775 [debug] Received test data: {"type":"start_test","test":"test_tar"}
2025-04-10 11:40:39.775 [debug] Looking for test with id test-rename-me.R&test_tar
2025-04-10 11:40:39.775 [error] Test with id test-rename-me.R&test_tar could not be found. Please report this.
2025-04-10 11:40:39.775 [debug] Received test data: {"type":"add_result","context":{},"test":"test_tar","result":"skip","location":"test-rename-me.R:4:1","filename":"test-rename-me.R","message":"Reason: empty test"}
2025-04-10 11:40:39.775 [debug] Looking for test with id test-rename-me.R&test_tar
2025-04-10 11:40:39.775 [error] Test with id test-rename-me.R&test_tar could not be found. Please report this.
2025-04-10 11:40:39.776 [debug] Received test data: {"type":"end_test","test":"test_tar"}
2025-04-10 11:40:39.777 [debug] Received test data: {"type":"start_test","test":"multiplication works"}
2025-04-10 11:40:39.777 [debug] Looking for test with id test-rename-me.R&multiplication works
2025-04-10 11:40:39.778 [debug] Received test data: {"type":"add_result","context":{},"test":"multiplication works","result":"success","location":"test-rename-me.R:9:3","filename":"test-rename-me.R"}
2025-04-10 11:40:39.778 [debug] Looking for test with id test-rename-me.R&multiplication works
2025-04-10 11:40:39.781 [debug] Received test data: {"type":"end_test","test":"multiplication works"}
2025-04-10 11:40:39.784 [debug] Received test data: {"type":"end_file","filename":"test-rename-me.R"}
2025-04-10 11:40:39.784 [debug] Received test data: {"type":"end_reporter"}

[jennybc]

Yeah, this is a deficiency I have thought about. Happy to have an issue about it.

In my life, it's cli::test_that_cli() where this comes up.

We need a way to register test_that()-like functions in the "after market".

For test_that() and describe(), this is done in tree-sitter queries kept in these files:

https://github.com/posit-dev/positron/tree/main/extensions/positron-r/resources/testing

The query for test_that() looks like so:

(call
	function: [
		(identifier) @function
		(namespace_operator
			lhs: (identifier)
			rhs: (identifier) @function
		)
	] (#eq? @function "test_that")
	arguments: (arguments
		(argument
			value: (string) @desc
		)
	)
) @call

So the entry level solution is to register an alternative string (an alternative to "test_that" here) that we should treat in exactly the same way. Things get more complicated if the signature of the target test_that() wrapper departs from that of test_that().

Open questions:

• Who does the registering? The developer of a package that offers a test_that()-like function? The package developer who has their own homegrown test_that()-like function?
• Is registration done in a file that lives in the source of the package-under-test? Or does this go in a section of .vscode/settings.json that targets positron-r?

[lionel-]

The developer of a package that offers a test_that()-like function?

That'd be nice but realistically only possible in the LSP once we have more sophisticated symbol analysis. So we should start with user repo configuration.

Is registration done in a file that lives in the source of the package-under-test?

We'd need access to this in the LSP (see posit-dev/ark#856) so it should be in the source. I'm thinking about a .scm TS query file that we could run on contents.

[lionel-]

That'd be nice but realistically only possible in the LSP once we have more sophisticated symbol analysis. So we should start with user repo configuration.

oh but we could just look at the dependencies in DESCRIPTION and just import all defs. I'm not too worried about clashes. So the .scm file would live in the package exporting the test function, somewhere in inst/.

[jennybc]

I also don't know that we'd necessarily require a full .scm file. I think the first step would be to support something that is close enough to test_that() that we could interpolate into the existing .scm. Like, step one might be just to handle a wrapper that has a different name, but is otherwise approachable with the built-in query.

Note: I'm thinking about this from the test explorer POV and haven't fully thought through it all from the LSP perspective.