Is it possible to verify coverage when tests are sharded?

[donalmurtagh]

In vite.config.js I've defined minimum coverage thresholds

  test: {
    coverage: {
      include: ['src'],
      thresholds: {
        branches: 90,
        functions: 70,
        lines: 85,
        statements: 85
      }
    }
  }

When Vitest runs with the --coverage flag, the command will fail if coverage is below any of this thresholds. Currently, if I run Vitest on a single machine, the command passes.

If I run the tests over two shards, the command fails due to insufficient coverage. Presumably this is because the coverage is being measured per shard, rather than across all shards.

What I need is something like --merge-reports, but rather than merging the reports on each shard, I want to merge the coverage data on each shard, and fail only if the combined coverage is below the thresholds.

As far as I can tell no such thing exists. Is there a solution for this problem or is it impossible to have sharding and coverage verification?

[donalmurtagh]

It is possible. You need to run vitest on each shard with coverage enabled, then combine each shard report using --merge-reports. Here's an Azure Pipelines example

  - job: UnitTest
    displayName: 'Unit Test Shard'
    pool:
      vmImage: 'ubuntu-latest'

    strategy:
      parallel: 2

    steps:
      - task: UseNode@1
        inputs:
          version: '24.x'

      - script: yarn install --immutable
        displayName: Install Dependencies

      - script: |
          yarn vitest run \
          --shard=$(System.JobPositionInPhase)/$(System.TotalJobsInPhase) \
          --reporter=blob \
          --coverage
        displayName: Run Unit Tests
        env:
          VITEST_SHARD: $(System.JobPositionInPhase)

      - task: PublishPipelineArtifact@1
        displayName: 'Publish Vitest shard report'
        inputs:
          targetPath: .vitest-reports
          artifact: vitest-report-$(System.JobPositionInPhase)

  - job: MergeVitestReports
    displayName: 'Publish a merged Vitest report with coverage'
    dependsOn: UnitTest
    condition: always()
    pool:
      vmImage: 'ubuntu-latest'

    steps:
      - task: UseNode@1
        inputs:
          version: '24.x'

      - script: yarn install --immutable
        displayName: Install Dependencies

      # Download the Vitest shard reports
      - task: DownloadPipelineArtifact@2
        inputs:
          artifactName: vitest-report-1
          targetPath: .vitest-reports
      - task: DownloadPipelineArtifact@2
        inputs:
          artifactName: vitest-report-2
          targetPath: .vitest-reports

      - script: |
          yarn vitest --merge-reports \
          --coverage.enabled=true \
          --coverage.reportsDirectory=.vitest-reports \
          --coverage.reporter=lcov \
          --coverage.reporter=text \
          --coverage.reporter=text-summary
        displayName: 'Create a combined Vitest report by merging the shard reports'

      - task: PublishPipelineArtifact@1
        displayName: 'Upload the merged Vitest report'
        inputs:
          targetPath: .vitest-reports
          artifact: merged-vitest-report

The VITEST_SHARD variable is used in the Vitest configuration to disable coverage verification on individual shards

import { defineConfig } from 'vite'

export default defineConfig({
  test: {
    coverage: {
      provider: 'v8',
      include: ['src'],
      // Disable coverage for the test shards, because we want to measure coverage over the entire test suite, not
      // individual shards.
      thresholds: process.env.VITEST_SHARD
        ? { branches: 0, functions: 0, lines: 0, statements: 0 }
        : { branches: 90, functions: 70, lines: 85, statements: 85 }
    }
  }
})