Skip to content

Commit 0d293b8

Browse files
authored
feat(coverage): Basic test coverage support (#366)
This commit lays the foundation for displaying coverage results from `bazel coverage`. Currently, the functionality is only exposed through the user-defined tasks in the `tasks.json`. It is thereby a bit hard to discover. But this is fine for the time being, because coverage still has a couple of rough edges anyway. As soon as it is more stable, we should add builtin commands and expose coverage runs also in the "Bazel Build Target" tree. Changes in this commit: * Bumps the VS Code version to 1.88, i.e. the first VS Code version which supports the test coverage API. * Upgrades to ES2022. I wanted to use `replaceAll` which was introduced in ES2021. VS Code 1.88 is based on Node 18 which in turn is based on V8 10.1. V8 10.18 supports ECMA-262 also known as ES2023. However, ES2023 is not yet available a target language in the `tsconfig.json`. Furthermore, Firefox does not fully support ES2023, yet. While web browsers are currently not relevant, they might become so in the future if we want to turn this into a browser-enabled VSCode extension. An upgrade to ES2021 would have been sufficient, but I went directly to ES2022 because it might have some of the new features might also turn out useful. * Introduces a custom LCOV parser. I could not find any other high-quality open-source parser. E.g., most other parser don't properly parse function names with `:` and / or `,` in them. * Introduces test cases for that custom LCOV parser. * Add the test cases to GitHub Actions. I followed the instructions from https://code.visualstudio.com/api/working-with-extensions/continuous-integration. Future work: * Support for branch coverage * Demangling of function names * Builtin commands to trigger coverage runs & offer them in the "Bazel Build Tree" Tested with: Java, C++, Go, Rust Untested: Python, Swift, Kotlin, Scala and many more This is the first step towards #362
1 parent 8972b0e commit 0d293b8

File tree

19 files changed

+9351
-772
lines changed

19 files changed

+9351
-772
lines changed

.github/workflows/build.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,9 @@ jobs:
4040
- name: Lint
4141
run: npm run check-lint
4242

43+
- run: xvfb-run -a npm test
44+
if: runner.os == 'Linux'
45+
4346
- name: Package VS Code extension
4447
run: npm run package
4548

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
out
22
node_modules
3+
.vscode-test
34
*.vsix
45

56
# Ignore the generated .d.ts and .js files for protos that end up in the src/

.vscode-test.js

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
const { defineConfig } = require("@vscode/test-cli");
2+
3+
module.exports = defineConfig({
4+
files: "out/test/**/*.test.js",
5+
mocha: { ui: "bdd" },
6+
});

.vscodeignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ out/test/**
44
scripts/**
55
src/**
66
test/**
7+
.vscode-test/
78

89
**/*.map
910

README.md

Lines changed: 39 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ This extension provides support for Bazel in Visual Studio.
1414
- **Buildifier** integration to lint and format your Bazel files (requires that
1515
[Buildifier](https://github.com/bazelbuild/buildtools/releases) be installed)
1616
- **Bazel Task** definitions for `tasks.json`
17+
- **Coverage Support** showing coverage results from `bazel coverage` directly
18+
in VS Code.
1719
- Debug Starlark code in your `.bzl` files during a build (set breakpoints, step
1820
through code, inspect variables, etc.)
1921

@@ -77,7 +79,7 @@ We can't currently make any recommendation between these two. Both are under act
7779

7880
Bazel tasks can be configured from the `tasks.json` using the following structure:
7981

80-
```json
82+
```jsonc
8183
{
8284
// See https://go.microsoft.com/fwlink/?LinkId=733558
8385
// for the documentation about the tasks.json format
@@ -88,8 +90,8 @@ Bazel tasks can be configured from the `tasks.json` using the following structur
8890
"type": "bazel",
8991
"command": "test",
9092
"targets": ["${input:pickFlakyTest}"],
91-
"options": ["--runs_per_test=9"]
92-
}
93+
"options": ["--runs_per_test=9"],
94+
},
9395
],
9496
"inputs": [
9597
{
@@ -98,13 +100,43 @@ Bazel tasks can be configured from the `tasks.json` using the following structur
98100
"command": "bazel.pickTarget",
99101
"args": {
100102
"query": "kind('.*_test', //...:*)",
101-
"placeHolder": "Which test to check for flakyness?"
102-
}
103-
}
104-
]
103+
"placeHolder": "Which test to check for flakyness?",
104+
},
105+
},
106+
],
107+
}
108+
```
109+
110+
## Coverage support (Experimental)
111+
112+
For all `coverage` tasks, the coverage results are automatically loaded into VS
113+
Code upon completion of the task. E.g., you could define your own task to
114+
display the coverage provided by your integration tests using the following task
115+
definition:
116+
117+
```jsonc
118+
{
119+
"label": "Show test coverage from integration test",
120+
"type": "bazel",
121+
"command": "coverage",
122+
"targets": ["//test/integration/...", "//cpp/test/integration/..."],
123+
"options": ["--instrumentation_filter=.*"],
105124
}
106125
```
107126

127+
You might need additional Bazel `options` to get the intended coverage results.
128+
In particular if are using remote builds, you might need to use the
129+
`--experimental_split_coverage_postprocessing` and `--experimental_fetch_all_coverage_outputs`
130+
options. See the documentation on [Code Coverage with Bazel](https://bazel.build/configure/coverage)
131+
for more details.
132+
133+
Code coverage support in this extension is still rather fresh and might still
134+
have rough edges. It was tested with the Java, C++, Go and Rust rules.
135+
In case you are using the code coverage integration with any other language
136+
(Python, Swift, Kotlin, Scala, ...), please let us know how things are going in
137+
bazelbuild/vscode-bazel#367. Please share both positive and negative experiences
138+
you might have.
139+
108140
## Contributing
109141

110142
If you would like to contribute to the Bazel Visual Studio extension, please

eslint.config.js

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -175,12 +175,13 @@ module.exports = tseslint.config(
175175
},
176176
},
177177
{
178-
files: ["eslint.config.js"],
179-
// `@eslint/js` is currnetly missing type information.
178+
// `@eslint/js` is currently missing type information.
180179
// Re-enable the type checks as soon as we have type infos.
180+
// For vscode-test.js, we also don't use TypeScript, yet.
181+
files: ["eslint.config.js", ".vscode-test.js"],
181182
extends: [tseslint.configs.disableTypeChecked],
182183
rules: {
183-
// Re-enable as soon as we are using ES modules for this config file.
184+
// Re-enable as soon as we are using ES modules for the config files.
184185
"@typescript-eslint/no-var-requires": "off",
185186
},
186187
},

0 commit comments

Comments
 (0)