Create lifecycle plugin

[d-ryan-ashcraft]

Overview

The standard practice when working with Python projects is to individually run a series of commands to perform tasks like creating virtual environments, testing, packing, publishing, versioning, etc. We are strong advocates of understanding all these various steps but have also found that it can be advantageous to a single command that runs through a standard and/or tailored lifecycle. This allows anyone building a project only to learn a small set of commands to build any supported project while consistently getting the results they desire. While there are likely a few lifecycles worth supporting, this ticket will focus on what is traditionally called the build lifecycle.

What is a Python Build Lifecycle?

When discussing this concept, it is easy to get wrapped around the point that "Python is interpreted; it DOESN'T need a build as there is nothing to compile!". Believe it or not, we understand this. There is much more to a build than compilation (it's also worth noting that you can compile Python, even though it isn't widely done).

Background

Okay, like what? We take a nod from venerable build tools like Maven and Gradle. Let's consider the standard default lifecycle from Maven:

• validate - ✅ validate the project is correct, and all necessary information is available
• compile- ❓compile the source code of the project
• test - ✅ test the compiled source code using a suitable unit testing framework. These tests should not require the code to be packaged or deployed
• package - ✅ take the compiled code and package it in its distributable format, such as a JAR.
• verify - ✅ run any checks on the results of integration tests to ensure quality criteria are met
• install - ❓install the package into the local repository for use as a dependency in other projects locally
• deploy - ✅ done in the build environment, copies the final package to the remote repository for sharing with other developers and projects.

As we can see from the check marks, most of these lifecycle steps still apply. But there is more to the story as this just represents the high-level phases. When we look a bit deeper, we find even more applicable steps (see bolded steps):

• validate - ✅ validate the project is correct, and all necessary information is available
• initialize - ✅ initialize build state, e.g., set properties or create directories.
• generate-sources - ✅ generate any source code for inclusion in compilation.
• process-sources - ✅ process the source code, for example, to filter any values.
• generate-resources - ✅ generate resources for inclusion in the package.
• process-resources - ✅ copy and process the resources into the destination directory, ready for packaging.
• compile- ❓compile the source code of the project
• process-classes - ❓post-process the generated files from compilation, for example, to do bytecode enhancement.
• generate-test-sources - ✅ generate any test source code for inclusion in compilation.
• process-test-sources - ✅ process the test source code, for example, to filter any values.
• generate-test-resources - ✅ create resources for testing.
• process-test-resources - ✅ copy and process the resources into the test destination directory
• test-compile - ❓compile the test source code into the test destination directory.
• process-test-classes - ❓post-process the generated files from test compilation, for example, to do bytecode enhancement
• test - ✅ test the compiled source code using a suitable unit testing framework. These tests should not require the code to be packaged or deployed
• prepare-package - perform any operations necessary to prepare a package before the actual packaging. This often results in an unpacked, processed version of the package.
• package - ✅ take the compiled code and package it in its distributable format, such as a JAR.
• pre-integration-test - ✅ perform actions required before executing integration tests. This may involve things such as setting up the required environment.
• integration-test - ✅ process and deploy the package, if necessary, into an environment where integration tests can be run.
• post-integration-test - ✅ perform actions required after executing integration tests. This may include cleaning up the environment.
• verify - ✅ run any checks on the results of integration tests to ensure quality criteria are met
• install - ❓install the package into the local repository for use as a dependency in other projects locally
• deploy - ✅ done in the build environment, copies the final package to the remote repository for sharing with other developers and projects.

Applying a Build Lifecycle

Many of these steps are not used in any specific lifecycle execution. However, having them available provides a consistent framework for binding actions. For instance,

• Validating installation of correct tools as well as the versions of those tools - often done in the validate step
• Apply licensing information - often done in the process-sources step

But What If I Want to Run My Commands Manually?

That's cool too! We certainly don't want to suggest that a lifecycle is the only way. We often mix and match lifecycle and individual commands as we develop. We also like understanding the basic lifecycle quickly through a standard build lifecycle definition. It provides a set of well-known rocks to look under for everyday tasks, as well as pre-defined hooks to use when you need to hang a new feature.

In the big picture, having a consistent lifecycle that lets us pull and build a new-to-us project without prior knowledge is incredibly valuable. We also like the consistency of running these lifecycles locally or on CI. We think you'll find that helpful, too, but ultimately, work the way that is best for you.

Definition of Done

We will work incrementally to create this type of lifecycle with this plugin. This ticket will be a start - not the entire lifecycle. The initial lifecycle will focus on orchestrating existing Pants goals:

• validate: validate
• dependencies: generate-lockfiles
• process: format, lint
• process-test: format, lint
• test: test
• package: package
• publish: publish

[meliz19]

For awareness, currently within the CI build, the package step is run via the command pants uv-build :: instead of pants package and the publish step is run via the command uv publish instead of pants publish. All other commands within the CI build are using the Pants native tooling. As we continue the pants/uv integration effort, we will investigate the best way to integrate the custom uv goals into the build lifecycle whether that means using the uv custom goals themselves or by integrating the custom goals into the already available Pants goals.