Skip to content

fix: ensure debugging your ci failures message displays #31563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/cache-version.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Bump this version to force CI to re-create the cache from scratch.

4-17-2025-v1
4-22-2025
22 changes: 17 additions & 5 deletions .circleci/workflows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ mainBuildFilters: &mainBuildFilters
- /^release\/\d+\.\d+\.\d+$/
# use the following branch as well to ensure that v8 snapshot cache updates are fully tested
- 'update-v8-snapshot-cache-on-develop'
- 'ryanm/chore/full-snapshot-threaded'
- 'ryanm/fix/having-trouble-debugging-your-ci-failures-minimal'

# usually we don't build Mac app - it takes a long time
# but sometimes we want to really confirm we are doing the right thing
Expand All @@ -49,7 +49,11 @@ macWorkflowFilters: &darwin-workflow-filters
- equal: [ develop, << pipeline.git.branch >> ]
# use the following branch as well to ensure that v8 snapshot cache updates are fully tested
- equal: [ 'update-v8-snapshot-cache-on-develop', << pipeline.git.branch >> ]
- equal: [ 'ryanm/chore/full-snapshot-threaded', << pipeline.git.branch >> ]
- equal:
[
'ryanm/fix/having-trouble-debugging-your-ci-failures-minimal',
<< pipeline.git.branch >>
]
- matches:
pattern: /^release\/\d+\.\d+\.\d+$/
value: << pipeline.git.branch >>
Expand All @@ -60,7 +64,11 @@ linuxArm64WorkflowFilters: &linux-arm64-workflow-filters
- equal: [ develop, << pipeline.git.branch >> ]
# use the following branch as well to ensure that v8 snapshot cache updates are fully tested
- equal: [ 'update-v8-snapshot-cache-on-develop', << pipeline.git.branch >> ]
- equal: [ 'ryanm/chore/full-snapshot-threaded', << pipeline.git.branch >> ]
- equal:
[
'ryanm/fix/having-trouble-debugging-your-ci-failures-minimal',
<< pipeline.git.branch >>
]
- matches:
pattern: /^release\/\d+\.\d+\.\d+$/
value: << pipeline.git.branch >>
Expand All @@ -83,7 +91,11 @@ windowsWorkflowFilters: &windows-workflow-filters
- equal: [ develop, << pipeline.git.branch >> ]
# use the following branch as well to ensure that v8 snapshot cache updates are fully tested
- equal: [ 'update-v8-snapshot-cache-on-develop', << pipeline.git.branch >> ]
- equal: [ 'ryanm/chore/full-snapshot-threaded', << pipeline.git.branch >> ]
- equal:
[
'ryanm/fix/having-trouble-debugging-your-ci-failures-minimal',
<< pipeline.git.branch >>
]
- matches:
pattern: /^release\/\d+\.\d+\.\d+$/
value: << pipeline.git.branch >>
Expand Down Expand Up @@ -157,7 +169,7 @@ commands:
name: Set environment variable to determine whether or not to persist artifacts
command: |
echo "Setting SHOULD_PERSIST_ARTIFACTS variable"
echo 'if ! [[ "$CIRCLE_BRANCH" != "develop" && "$CIRCLE_BRANCH" != "release/"* && "$CIRCLE_BRANCH" != "ryanm/chore/full-snapshot-threaded" ]]; then
echo 'if ! [[ "$CIRCLE_BRANCH" != "develop" && "$CIRCLE_BRANCH" != "release/"* && "$CIRCLE_BRANCH" != "ryanm/fix/having-trouble-debugging-your-ci-failures-minimal" ]]; then
export SHOULD_PERSIST_ARTIFACTS=true
fi' >> "$BASH_ENV"
# You must run `setup_should_persist_artifacts` command and be using bash before running this command
Expand Down
1 change: 1 addition & 0 deletions cli/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ _Released 4/22/2025 (PENDING)_
**Bugfixes:**

- Fixed an issue where auto scroll in the Cypress Command Log was not scrolling correctly. Fixes [#31530](https://github.com/cypress-io/cypress/issues/31530).
- Fixed an issue where a message pointing users to the Cypress Cloud was not displaying on runs with failures in CI. Fixes [#31550](https://github.com/cypress-io/cypress/issues/31550).

## 14.3.1

Expand Down
7 changes: 7 additions & 0 deletions cli/vue2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# @cypress/vue2

Mount Vue 2 components in the open source [Cypress.io](https://www.cypress.io/) test runner

> **Note:** This package is bundled with the `cypress` package and should not need to be installed separately. See the [Vue Component Testing Docs](https://docs.cypress.io/guides/component-testing/vue/overview) for mounting Vue components. Installing and importing `mount` from `@cypress/vue2` should only be done for advanced use-cases.

## [Changelog](./CHANGELOG.md)
53 changes: 53 additions & 0 deletions cli/vue2/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"name": "@cypress/vue2",
"version": "0.0.0-development",
"description": "Browser-based Component Testing for Vue.js@2 with Cypress.io ✌️🌲",
"main": "dist/cypress-vue2.cjs.js",
"scripts": {
"build": "rimraf dist && yarn rollup -c rollup.config.mjs",
"postbuild": "node ../../scripts/sync-exported-npm-with-cli.js",
"build-prod": "yarn build",
"lint": "eslint --ext .js,.jsx,.ts,.tsx,.json,.vue .",
"test": "echo \"Tests for @cypress/vue2 are run from system-tests\"",
"test-ci": "node ../../scripts/run-ct-examples.js --examplesList=./examples.env",
"typecheck": "tsc --noEmit",
"watch": "yarn build --watch --watch.exclude ./dist/**/*"
},
"devDependencies": {
"@cypress/mount-utils": "0.0.0-development",
"@rollup/plugin-json": "^4.1.0",
"@rollup/plugin-replace": "^2.3.1",
"@vue/test-utils": "^1.3.1",
"tslib": "^2.1.0",
"typescript": "^4.7.4",
"vue": "2.6.12"
},
"peerDependencies": {
"cypress": ">=4.5.0",
"vue": "^2.0.0"
},
"files": [
"dist/**/*",
"src/**/*.js"
],
"engines": {
"node": ">=8"
},
"types": "dist/index.d.ts",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/cypress-io/cypress.git"
},
"homepage": "https://github.com/cypress-io/cypress/blob/develop/npm/vue/#readme",
"bugs": "https://github.com/cypress-io/cypress/issues/new?assignees=&labels=npm%3A%20%40cypress%2Fvue&template=1-bug-report.md&title=",
"keywords": [
"cypress",
"vue"
],
"unpkg": "dist/cypress-vue2.browser.js",
"module": "dist/cypress-vue2.esm-bundler.js",
"publishConfig": {
"access": "public"
}
}
6 changes: 3 additions & 3 deletions packages/server/__snapshots__/cypress_spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -412,10 +412,10 @@ exports['CLOUD_RECOMMENDATION_MESSAGE'] = `

----------------------------------------------------------------------------------------------------

Having trouble debugging your CI failures?
Debug faster with full visibility.

Record your runs to Cypress Cloud to watch video recordings for each test,
debug failing and flaky tests, and integrate with your favorite tools.
Record to Cypress Cloud and get instant access to full test details and replays.
Inspect the DOM, network events, and console logs exactly as they ran in CI.

>> https://on.cypress.io/cloud-get-started

Expand Down
6 changes: 3 additions & 3 deletions packages/server/lib/util/print-run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ type Screenshot = {
}

export const cloudRecommendationMessage = `
Having trouble debugging your CI failures?
Debug faster with full visibility.

Record your runs to Cypress Cloud to watch video recordings for each test,
debug failing and flaky tests, and integrate with your favorite tools.
Record to Cypress Cloud and get instant access to full test details and replays.
Inspect the DOM, network events, and console logs exactly as they ran in CI.
`

function color (val: any, c: string) {
Expand Down
51 changes: 51 additions & 0 deletions system-tests/__snapshots__/ci_failure_spec_ts.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
exports['CI failure fails and displays the message that points users to the cloud 1'] = `
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 1.2.3 │
│ Browser: FooBrowser 88 │
│ Specs: 1 found (simple_failing.cy.js) │
│ Searched: cypress/e2e/simple_failing.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────

Running: simple_failing.cy.js (1 of 1)
simple failing spec
1) fails1
2) fails2
0 passing
2 failing
1) simple failing spec
fails1:
AssertionError: Timed out retrying after 100ms: expected true to be false
[stack trace lines]
2) simple failing spec
fails2:
Error: fails2
[stack trace lines]
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 0 │
│ Failing: 2 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: X seconds │
│ Spec Ran: simple_failing.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✖ simple_failing.cy.js XX:XX 2 - 2 - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✖ 1 of 1 failed (100%) XX:XX 2 - 2 - -
----------------------------------------------------------------------------------------------------
Debug faster with full visibility.
Record to Cypress Cloud and get instant access to full test details and replays.
Inspect the DOM, network events, and console logs exactly as they ran in CI.
>> https://on.cypress.io/cloud-get-started
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ryanthemanuel This snapshot doesn't have the line spacing I would expect here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the newlines stripped out of the snapshot? I don't remember that being the case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's passing for me locally?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. Something is fishy.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that should fix it. My guess is my IDE auto formatted something. Weird that it was passing for me locally though...

----------------------------------------------------------------------------------------------------
`
21 changes: 21 additions & 0 deletions system-tests/test/ci_failure_spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import systemTests from '../lib/system-tests'

describe('CI failure', () => {
systemTests.setup()

it('fails and displays the message that points users to the cloud', function () {
return systemTests.exec(this, {
browser: 'electron',
spec: 'simple_failing.cy.js',
processEnv: {
CI: '1',
CYPRESS_COMMERCIAL_RECOMMENDATIONS: '1',
},
expectedExitCode: 2,
snapshot: true,
config: {
screenshotOnRunFailure: false,
},
})
})
})
3 changes: 1 addition & 2 deletions tooling/v8-snapshot/cache/darwin/snapshot-meta.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"norewrite": [
"./ci-info/index.js",
"./evil-dns/evil-dns.js",
"./get-stream/buffer-stream.js",
"./graceful-fs/polyfills.js",
Expand Down Expand Up @@ -70,10 +71,8 @@
"./packages/server/lib/util/process_profiler.ts",
"./packages/server/lib/util/suppress_warnings.js",
"./packages/server/node_modules/axios/lib/adapters/http.js",
"./packages/server/node_modules/ci-info/index.js",
"./packages/server/node_modules/glob/node_modules/minimatch/minimatch.js",
"./packages/server/node_modules/graceful-fs/polyfills.js",
"./packages/server/node_modules/is-ci/index.js",
"./packages/server/node_modules/mocha/node_modules/debug/src/node.js",
"./process-nextick-args/index.js",
"./signal-exit/index.js",
Expand Down
3 changes: 1 addition & 2 deletions tooling/v8-snapshot/cache/linux/snapshot-meta.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"norewrite": [
"./ci-info/index.js",
"./evil-dns/evil-dns.js",
"./get-stream/buffer-stream.js",
"./graceful-fs/polyfills.js",
Expand Down Expand Up @@ -70,10 +71,8 @@
"./packages/server/lib/util/process_profiler.ts",
"./packages/server/lib/util/suppress_warnings.js",
"./packages/server/node_modules/axios/lib/adapters/http.js",
"./packages/server/node_modules/ci-info/index.js",
"./packages/server/node_modules/glob/node_modules/minimatch/minimatch.js",
"./packages/server/node_modules/graceful-fs/polyfills.js",
"./packages/server/node_modules/is-ci/index.js",
"./packages/server/node_modules/mocha/node_modules/debug/src/node.js",
"./process-nextick-args/index.js",
"./signal-exit/index.js",
Expand Down
3 changes: 1 addition & 2 deletions tooling/v8-snapshot/cache/win32/snapshot-meta.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"norewrite": [
"./ci-info/index.js",
"./evil-dns/evil-dns.js",
"./get-stream/buffer-stream.js",
"./graceful-fs/polyfills.js",
Expand Down Expand Up @@ -70,10 +71,8 @@
"./packages/server/lib/util/process_profiler.ts",
"./packages/server/lib/util/suppress_warnings.js",
"./packages/server/node_modules/axios/lib/adapters/http.js",
"./packages/server/node_modules/ci-info/index.js",
"./packages/server/node_modules/glob/node_modules/minimatch/minimatch.js",
"./packages/server/node_modules/graceful-fs/polyfills.js",
"./packages/server/node_modules/is-ci/index.js",
"./packages/server/node_modules/mocha/node_modules/debug/src/node.js",
"./process-nextick-args/index.js",
"./signal-exit/index.js",
Expand Down
1 change: 1 addition & 0 deletions tooling/v8-snapshot/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
},
"devDependencies": {
"@tooling/system-tests": "0.0.0-development",
"chai-as-promised": "7.1.1",
"cpr": "^3.0.1",
"mocha": "7.0.1",
"snap-shot-it": "7.9.10",
Expand Down
3 changes: 1 addition & 2 deletions tooling/v8-snapshot/src/setup/force-no-rewrite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ export default [
'node_modules/prettier/parser-meriyah.js',
'node_modules/prettier/parser-typescript.js',
'node_modules/prettier/third-party.js',
'packages/server/node_modules/is-ci/index.js',
'packages/server/node_modules/ci-info/index.js',
'ci-info/index.js',
'node_modules/@babel/traverse/lib/index.js',
'node_modules/@babel/types/lib/definitions/index.js',
'packages/server/node_modules/axios/lib/adapters/http.js',
Expand Down
Loading