feat(debug): support debugging with browser mode #603

nCastle1 · 2025-03-21T22:34:25Z

Fixes #474

This PR improves support for debugging in browser scenarios by automatically setting up an additional launch configuration to run after starting debugging. This enables connection to the browser running the tests, which enables hitting breakpoints. This is done only when browser mode is enabled.

Currently debugging support is limited to chrome via Playwright and webdriverio; additional setup needed for webdriverio.

Implementation details: after the debug session starts, if vitest's resolved config includes a browser entry, a second debug session is launched as a child of the first. Stopping the parent debug session or the child session will stop the other. The CLI args to start the vitest debug run are manipulated as needed to make debugging work, and this is different for playwright and webdriverio.

Vitest fails if called with --inspect-brk when using webdriverio. A future area of inquiry might involve updating the inspect-brk implementation to work seamlessly with webdriverio.

I have included sample projects for a few scenarios impacted by the changes in this PR.

This is my first time contributing to this repo. I checked for contribution guidelines but didn't find anything beyond the code of conduct; please let me know if you have any other guidance.

This commit improves support for debugging in non-headless browser scenarios by adding the option to define an additional launch configuration to run after starting debugging. This enables connection to the browser running the tests, which enables setting breakpoints. This addresses vitest-dev#474. Implementation details: after the debug session starts, if configured, a second debug session is launched as a child of the first. I confirmed by inspection of VS Code source that child debug sessions are stopped when the parent is stopped, so I don't think additional clean up is needed. ([Evidence](https://github.com/microsoft/vscode/blob/3286791a7e920c9b61b84fc2dbba136da912bb64/src/vs/workbench/contrib/debug/browser/debugSession.ts#L426)) I have included a sample project that demonstrates how to configure headful browser tests; it is copy-pasted version of the browser sample, with some config changes. I have also updated the readme to document this feature.

This is useful because sometimes you want to use different arguments to vitest when debugging. In particular, you don't want to pass inspect-brk when not debugging, but you need to pass that for browser-based debugging.

Automatically stop the debugging session when the secondary session is stopped; otherwise user needs to press stop button twice.

sheremet-va · 2025-03-25T05:48:08Z

Thank you for PR! Looks very good, but I don't think we should require any new additional config options. We can infer them from the config ourselves.

nCastle1 · 2025-03-25T19:06:35Z

Thank you for that feedback, I will re-work the PR to infer as much as possible from the existing config.

- Removes existing implementation using explicit config options - Updates the rpc/worker to support reading vite's resolved browser configuration. This information is used to determine when the secondary attach config should be started. This information is also used to set the inspect-brk and browser flags on the vitest CLI when starting the debug session. This enables zero-config debugging when the user has started a test debug without interfering with running without debug. inspect-brk is used to guarantee that breakpoints early in the test run are hit (otherwise it is possible tests run before the debugger is attached). - Adds samples which verify that this feature works correctly when there are multiple instances configured and when webdriverio is being used (note: debugging is not supported for webdriverio, this just makes sure the feature doesn't break test running). Only chromium with playwright is supported by the current implementation.

Not a great solution because it relies on additional configuration being done at the project level, including setting up custom command to communicate with webdriverio (this is that project's promoted pattern).

nCastle1 · 2025-03-29T05:29:28Z

src/debug.ts

@@ -183,6 +217,24 @@ async function getRuntimeOptions(pkg: VitestPackage) {
  }
 }

+function getAttachConfigForBrowser(config: ReturnType<typeof getConfig>, pkg: VitestPackage): vscode.DebugConfiguration {


I split this out originally with intent to support multiple browsers, but I'm not sure how feasible that is. Vitest only docs playwright with chromium for the vs code/IDE-integrated debugging. Firefox doesn't support inspect-brk flag, and the VS Code integration has some limitations that will make integration challenging - if the browser instance isn't running at attach time, the attach config fails immediately instead of waiting like the built-in chrome attach implementation.

I'd say if it doesn't work in Vitest CLI, the extension cannot add support for it, and waiting for it is a bit useless

It's better to throw an error, maybe? Or a warning?

nCastle1 · 2025-03-29T05:30:48Z

src/worker/worker.ts

@@ -146,6 +146,10 @@ export class ExtensionWorker implements ExtensionWorkerTransport {
    return files.map(([project, spec]) => [project.config.name || '', spec])
  }

+  public getResolvedBrowserOptions(): ResolvedBrowserOptions | undefined {
+    return { ...this.ctx.config?.browser, commands: undefined }


This needs to be reworked, complex configuration (e.g. custom commands) will break the extension currently.

I think you can just rename this to getBrowserDebugOptions and return only needed fields

nCastle1 · 2025-03-29T05:34:13Z

I've updated the PR to take a zero configuration approach, reading the browser config as resolved by vitest. I have it working with playwright and chromium; this seems pretty solid. I'm working towards a solution for webdriverio, but I have zero experience with that technology so it is slow going.

I've noted a few key missing pieces, and I don't think this is in a mergable state, so I will keep as a draft. I will plan to revisit and add tests soon (pending some competing deadlines). Any feedback is very welcome in the meantime.

Still not quite ideal, because the test has to be specially configured to enable debugging, but it does at least show it working.

nCastle1 · 2025-04-04T22:22:36Z

I've made some changes in response to feedback. Have a few more scenarios to test before requesting review and merge, specifically want to investigate multi-project setups with mix of browser and non-browser setups, among others

Updates seamlessly support debugging in scenarios with multiple projects, mixing browser and non-browser tests. When any of the included tests are in a browser project, the attach configuration is run. The test request is inspected before debugging to check which projects are included. Existing tests verify that browser mode debugging still works in single-project scenarios.

# Conflicts: # src/debug.ts

nCastle1 · 2025-04-17T23:26:31Z

Thank you for the feedback so far and patience with the lingering PR.

I have updated the PR to work correctly in multi-project scenarios where some tests are browser mode and some are not. Debugging with playwright is working consistently (chromium only). Debugging with webdriverio is also sort-of working; I think pretty close to as well as it can work without enhancements to vitest.

I believe this PR is ready for another review.

sheremet-va

Looks very promising! Just a few suggestions. Thank you for your work on this 🙏🏻

sheremet-va · 2025-04-29T18:30:44Z

samples/browser-webdriverio/test/.gitignore

@@ -0,0 +1 @@
+__snapshots__


Can you also add screenshots to gitignore?

sheremet-va · 2025-04-29T18:34:17Z

src/worker/worker.ts

@@ -146,6 +146,12 @@ export class ExtensionWorker implements ExtensionWorkerTransport {
    return files.map(([project, spec]) => [project.config.name || '', spec])
  }

+  public getBrowserDebugOptions(): (Pick<ResolvedBrowserOptions, 'enabled' | 'provider'> & { project: string })[] | undefined {


I think it might be better to just return this in

vscode/src/worker/index.ts

Line 51 in f0fa52b

emitter.ready(configs, workspaceSource)

Then you can store it in

vscode/src/api.ts

Line 398 in 8fbf0ea

export interface ResolvedMeta {

And get the values at any point

sheremet-va · 2025-04-29T18:37:29Z

src/debug.ts

+  return []
+}
+
+function getBrowserModeLaunchArgs(isPlaywright: boolean, config: any): string {


Can we use an actual type instead of the any?

sheremet-va · 2025-04-29T18:38:13Z

src/debug.ts

+}
+
+function getProjectsFromTests(item: vscode.TestItem | undefined, api: VitestFolderAPI, tree: TestTree): string[] {
+  const items = getTestProjectsInFolder(item?.uri?.fsPath ?? '', api, tree)


It's quite hard to read a lot of ?., ??, | undefined everywhere. Can this be simplified to only accept the value if it's there?

sheremet-va · 2025-04-29T18:39:19Z

src/debug.ts

+  // Later, after debugging session starts, a secondary debug session is started; that session attaches to the launched browser instance.
+  const { browserModeProjects, isPlaywright } = await api.getBrowserModeInfo()
+  const testProjects = request.include?.filter(inc => inc.uri?.fsPath != null).flatMap(inc => getProjectsFromTests(inc, api, tree)) ?? []
+  const needsBrowserMode = !!browserModeProjects?.length && testProjects.some(project => browserModeProjects?.includes(project))


Should we add a warning that non-chromium webdriverio tests are not supported?

sheremet-va · 2025-04-29T18:43:02Z

src/debug.ts

+}
+
+function getBrowserModeLaunchArgs(isPlaywright: boolean, config: any): string {
+  const browser = isPlaywright ? 'chromium' : 'chrome'


I like that we provide the browser configuration ourselves. I think at one point there should be a UI in the extension to select the browser, but it's not related to these changes

sheremet-va · 2025-04-29T18:46:47Z

src/debug.ts

+  // Only playwright provider supports --inspect-brk currently
+  const inspectBrk = isPlaywright ? `--inspect-brk=localhost:${config.debuggerPort ?? '9229'}` : ''
+  // regardless of user config, some properties need to be set when debugging with browser mode enabled
+  return `vitest ${config.cliArguments ?? ''} ${inspectBrk} --browser=${browser}`


What happens if cliArguments already have these options? They will be converted into arrays here:

vscode/src/worker/init.ts

Line 21 in f0fa52b

? vitestModule.parseCLI(meta.arguments, {

Maybe we should normalize them afterwards just in case?

# Conflicts: # src/debug.ts # src/extension.ts

nCastle1 added 2 commits March 21, 2025 15:24

Merge branch 'main' into feat/enable-debugging-with-headed-browser

29fb33a

nCastle1 marked this pull request as draft March 24, 2025 17:20

nCastle1 added 3 commits March 24, 2025 18:22

feat: add setting for debug-specific CLI arguments

edaa90d

This is useful because sometimes you want to use different arguments to vitest when debugging. In particular, you don't want to pass inspect-brk when not debugging, but you need to pass that for browser-based debugging.

fix: Stop main session when child stopped

843eccc

Automatically stop the debugging session when the secondary session is stopped; otherwise user needs to press stop button twice.

doc: add arg to readme, remove redundant sample

218a495

nCastle1 added 2 commits March 28, 2025 21:10

fix: get webdriverio working

2fc7d89

Not a great solution because it relies on additional configuration being done at the project level, including setting up custom command to communicate with webdriverio (this is that project's promoted pattern).

nCastle1 commented Mar 29, 2025

View reviewed changes

nCastle1 added 3 commits April 4, 2025 13:46

fix: simplify config handling

26d5895

fix: improve webdriverio sample

ca72517

Still not quite ideal, because the test has to be specially configured to enable debugging, but it does at least show it working.

improve config handling

36074be

nCastle1 changed the title ~~feat(debug): support debugging with headed browser~~ feat(debug): support debugging with browser mode Apr 4, 2025

nCastle1 added 5 commits April 17, 2025 15:50

fix(sample): clarify webdriverio limitation

bf555a7

fix: remove outdated changes

a8a427f

fix: readability enhancement

7068700

Merge branch 'main' into feat/enable-debugging-with-headed-browser

f7c5a30

# Conflicts: # src/debug.ts

nCastle1 requested a review from sheremet-va April 17, 2025 23:26

nCastle1 marked this pull request as ready for review April 17, 2025 23:26

sheremet-va requested changes Apr 29, 2025

View reviewed changes

nCastle1 added 2 commits June 3, 2025 18:53

fix: ignore screenshots

f73759d

Merge branch 'main' into feat/enable-debugging-with-headed-browser

ff7709a

# Conflicts: # src/debug.ts # src/extension.ts

		@@ -0,0 +1 @@
		__snapshots__

Uh oh!

feat(debug): support debugging with browser mode #603

Are you sure you want to change the base?

feat(debug): support debugging with browser mode #603

Conversation

nCastle1 commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sheremet-va commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nCastle1 commented Mar 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nCastle1 commented Mar 29, 2025

Uh oh!

nCastle1 commented Apr 4, 2025

Uh oh!

nCastle1 commented Apr 17, 2025

Uh oh!

sheremet-va left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nCastle1 commented Mar 21, 2025 •

edited

Loading

sheremet-va commented Mar 25, 2025 •

edited

Loading