Skip to content

Test example notebooks against latest code #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 25, 2025

Conversation

gabrielfeo
Copy link
Owner

@gabrielfeo gabrielfeo commented Jul 24, 2025

Adds a new test suite that verifies running the example Kotlin notebooks against the current library code, i.e. a SNAPSHOT artifact compiled just before the tests and published to Maven local.

This test suite aims to prevent regressions such as #426.

  • Creates an examplesTest test suite
  • Creates an unsigned SNAPSHOT publication as an input to examplesTest
  • Wires a properly cacheable task to download the pip packages required by the notebooks before the test suite. Only pip install runs as part of test setup, but restricted to not download anything.
  • Adds a test which executes the example notebooks in a Python venv with any dependency on the library replaced with one on a SNAPSHOT in Maven local and verifies the main cells' outputs, i.e. that the API fetch succeeded and that Dataframe and Kandy cells had outputs
  • Implements a notebook preprocessor to support replacing the dependency declarations in the notebooks source .ipynb/JSON file

Also ensures every test task considers DEVELOCITY_API_ variables as inputs, since they might intentionally or unintentionally be used in the test suite, potentially changing its outcomes.

@gabrielfeo gabrielfeo force-pushed the gf/example-testing-test-suite branch from 6a11a69 to a8b9338 Compare July 25, 2025 10:44
@gabrielfeo gabrielfeo marked this pull request as ready for review July 25, 2025 10:47
@gabrielfeo gabrielfeo requested a review from Copilot July 25, 2025 10:48
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds a new test suite that verifies example Kotlin notebooks can run against the current library code by using a SNAPSHOT artifact published to Maven local. This prevents regressions in notebook compatibility.

  • Creates an examplesTest source set with utilities for executing Jupyter notebooks in Python virtual environments
  • Implements a notebook preprocessor to replace dependency declarations with SNAPSHOT versions
  • Adds test validation for notebook execution outputs including dataframes and visualizations

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
library/src/examplesTest/resources/preprocessors.py Python preprocessor for replacing notebook cell content while preserving magic commands
library/src/examplesTest/kotlin/com/gabrielfeo/develocity/api/example/notebook/*.kt Kotlin utilities for managing Python venvs, executing notebooks, and parsing JSON outputs
library/src/examplesTest/kotlin/com/gabrielfeo/develocity/api/example/*.kt Support utilities for shell execution, resource copying, and JSON parsing
library/build.gradle.kts Adds examples test suite configuration and SNAPSHOT publication setup
build-logic/src/main/kotlin/com/gabrielfeo/*.gradle.kts Build script plugins for test suite configuration
examples/example-notebooks/MostFrequentBuilds.ipynb Minor output message update and formatting cleanup
examples/build.gradle.kts Removes notebook execution tasks (moved to test suite)
Comments suppressed due to low confidence (1)

library/src/examplesTest/kotlin/com/gabrielfeo/develocity/api/example/Shell.kt:32

  • The error message only shows the exit code but not the actual stderr output that would help diagnose the failure. Consider including the stderr content in the error message since it's already being captured.
    check(exitCode == 0) { "Exit code '$exitCode' for command: $command" }

@gabrielfeo gabrielfeo enabled auto-merge (squash) July 25, 2025 14:08
@gabrielfeo gabrielfeo merged commit b53ecab into main Jul 25, 2025
7 checks passed
@gabrielfeo gabrielfeo deleted the gf/example-testing-test-suite branch July 25, 2025 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant