ci: open version update PR as draft if tests fail

[thephez]

Updates the workflow to always open a PR when a new version of the SDK is available. If the tests don't pass, it sets the PR to draft and indicates which test set failed. Also pins the GitHub action versions by hash

Summary by CodeRabbit

• Chores
  • Enhanced CI/CD workflow with improved test result reporting, displaying detailed outcomes of automated tests in pull requests for better visibility.
  • Implemented conditional draft pull request creation based on test results to prevent accidental merging when tests fail.
  • Optimized workflow configuration with updated action references and added direct links to workflow runs for easier debugging and monitoring.

[coderabbitai]

Walkthrough

Updated GitHub Actions workflow for SDK version management with conditional PR draft creation based on test outcomes. Modified action references to use fixed commit SHAs, added test result tracking, and enhanced PR body with test result summaries.

Changes

Cohort / File(s)	Summary
Workflow configuration and test result tracking \..github/workflows/update-evo-sdk.yml	Renamed workflow, updated action references to fixed commit SHAs, added smoke_tests and query_tests step IDs with continue-on-error enabled, made PR draft status conditional on test outcomes, enhanced PR body to display per-test results with success/failure indicators and conditional messaging based on test status

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

• Verify that test step IDs (smoke_tests, query_tests) are correctly referenced in conditional logic
• Confirm the draft PR condition correctly evaluates test outcomes
• Check PR body template rendering, particularly the conditional success message and workflow run link formatting
• Validate action reference commit SHAs are appropriate and match intended versions

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "ci: open version update PR as draft if tests fail" accurately describes a significant aspect of the changeset—the conditional logic that marks pull requests as drafts when tests fail. The title is specific, clear, and directly relates to the workflow's new behavior of conditionally setting the PR status based on test outcomes. While the title doesn't explicitly mention that the PR is now always opened regardless of test status (just with conditional draft marking), it captures a core functional change and is not vague, generic, or misleading about what the PR implements.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ci-auto-pr-sdk-update

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

.github/workflows/update-evo-sdk.yml (1)

117-130: Complex conditional expression in PR body may have rendering issues.

The multiline string literals within the GitHub Actions expression (lines 120–126) could potentially cause parsing or rendering problems. The nested format() call is structurally sound, but the overall expression mixes multiple operators (&&, ||) and spans multiple lines, which may be fragile.

-            **Test Results:**
-            - Smoke tests: ${{ steps.smoke_tests.outcome == 'success' && '✅ Passed' || '❌ Failed' }}
-            - Query tests: ${{ steps.query_tests.outcome == 'success' && '✅ Passed' || '❌ Failed' }}
-            ${{ steps.smoke_tests.outcome == 'success' && steps.query_tests.outcome == 'success' && '
-
-            ✅ Smoke and query tests passed!' || format('
-
-            ⚠️ **This PR is in draft because tests failed.**
-
-            Review the test failures in the [workflow run]({0}/{1}/actions/runs/{2}) and update the site code to support the new SDK version.', github.server_url, github.repository, github.run_id) }}
+            **Test Results:**
+            - Smoke tests: ${{ steps.smoke_tests.outcome == 'success' && '✅ Passed' || '❌ Failed' }}
+            - Query tests: ${{ steps.query_tests.outcome == 'success' && '✅ Passed' || '❌ Failed' }}
+            
+            ${{ steps.smoke_tests.outcome == 'success' && steps.query_tests.outcome == 'success' && '✅ Smoke and query tests passed!' || format('⚠️ **This PR is in draft because tests failed.**\n\nReview the test failures in the [workflow run]({0}/{1}/actions/runs/{2}) and update the site code to support the new SDK version.', github.server_url, github.repository, github.run_id) }}

Alternatively, use a clearer ternary with escaped newlines:

+            ${{ steps.smoke_tests.outcome == 'success' && steps.query_tests.outcome == 'success' ? '✅ Smoke and query tests passed!' : format('⚠️ **This PR is in draft because tests failed.**\n\nReview the test failures in the [workflow run]({0}/{1}/actions/runs/{2}) and update the site code to support the new SDK version.', github.server_url, github.repository, github.run_id) }}

Could you verify that the PR body renders correctly with the current multiline string approach, particularly in the GitHub UI? If there are any rendering issues, the refactored version above should be more reliable.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 736c936 and 022d44f.

📒 Files selected for processing (1)

• .github/workflows/update-evo-sdk.yml (4 hunks)

🔇 Additional comments (4)

.github/workflows/update-evo-sdk.yml (4)

1-16: Workflow restructuring looks solid.

The rename from "Update evo-sdk version on npm release" to "Check/Update SDK version" better reflects the scheduled nature, and the concurrency policy correctly prevents overlapping runs. Workflow permissions are appropriately scoped for PR creation.

---

24-24: Pinned action versions with commit SHAs improve reproducibility and security.

Using fixed commit hashes instead of major version tags (e.g., v5 → 08c6903cd8c0fde910a37f88322edcfb5dd907a8) prevents supply chain attacks and ensures consistent behavior across workflow runs. Comments preserving the semantic version are helpful for maintainability.

Also applies to: 27-27, 105-105

---

86-102: Test isolation with step IDs and continue-on-error is correct.

Setting id on each test step and using continue-on-error: true allows both tests to run independently and later steps to inspect outcomes via steps.<id>.outcome. This is the right pattern for capturing test results and making decisions based on them.

---

107-107: Draft PR logic correctly ties to test outcomes.

The condition draft: ${{ steps.smoke_tests.outcome != 'success' || steps.query_tests.outcome != 'success' }} properly marks a PR as draft if either test fails, achieving the objective of always opening a PR but flagging failures for review. Branch naming and title are appropriately versioned.

Also applies to: 108-110