Skip to content

Update plugins documentation #6033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
c487aaa
Update plugins docs
christopher-hakkaart Apr 24, 2025
a046331
Add run command example
christopher-hakkaart Apr 27, 2025
989af21
Update links
christopher-hakkaart Apr 28, 2025
5471abc
Add repo link
christopher-hakkaart Apr 28, 2025
3dfb945
Add Trace observers note
christopher-hakkaart Apr 28, 2025
e254b63
Revise defined registry
christopher-hakkaart May 5, 2025
5a6e4d9
Add plugin create documentation
christopher-hakkaart May 6, 2025
0a5f013
Merge branch 'master' into pr/christopher-hakkaart/6033
bentsherman May 6, 2025
9053ca9
Fix plugin config scope example
bentsherman May 6, 2025
ce87361
Only support registry
christopher-hakkaart May 7, 2025
d68a8d7
Merge branch 'docs-plugins' of https://github.com/christopher-hakkaar…
christopher-hakkaart May 7, 2025
2bb3960
cleanup
bentsherman May 7, 2025
e153d96
Remove registry URL
bentsherman May 7, 2025
1be0336
cleanup migration guide
bentsherman May 7, 2025
7bfc78d
cleanup gradle plugin guide
bentsherman May 7, 2025
93691b8
Merge branch 'master' into docs-plugins
christopher-hakkaart May 7, 2025
fa2dc0f
Unlist h3 headings
christopher-hakkaart May 7, 2025
ea585de
Merge branch 'master' into pr/christopher-hakkaart/6033
bentsherman May 9, 2025
aa87a22
Replace nf-hello example with plugin template
bentsherman May 9, 2025
758b818
Cleanup private beta notes
bentsherman May 9, 2025
e0866bc
Add transition plan for legacy index -> registry
bentsherman May 9, 2025
5ab28f6
Merge branch 'master' into pr/christopher-hakkaart/6033
bentsherman May 10, 2025
de7aaa1
Apply suggestions from code review
bentsherman May 10, 2025
d011970
Unlist timeline subheadings
bentsherman May 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
541 changes: 0 additions & 541 deletions docs/developer/plugins.md

This file was deleted.

85 changes: 85 additions & 0 deletions docs/gradle-plugin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
(gradle-plugin-page)=

# Using the Gradle plugin

The [Gradle plugin for Nextflow plugins](https://github.com/nextflow-io/nextflow-plugin-gradle) simplifies plugin development by configuring default dependencies needed for Nextflow integration and incorporates custom Gradle tasks that streamline building, testing, and publishing Nextflow plugins. The [`nf-plugin-template`](https://github.com/nextflow-io/nf-plugin-template/) is a scaffold for plugin development and incorporates the Gradle plugin by default. You can use the `nextflow plugin create` sub-command to create plugins scaffolds with the [`nf-plugin-template`](https://github.com/nextflow-io/nf-plugin-template/) scaffold.

:::{note}
Nextflow Plugins can be developed without the Gradle plugin. However, this approach is only suggested if you are an advanced developer and your project is incompatible with the Gradle plugin.
:::

(gradle-plugin-create)=

## Creating a plugin

:::{versionadded} 25.04.0
:::

To create a Nextflow plugin with the Gradle plugin:

1. Run `nextflow plugin create`.
- When prompted `Enter plugin name:`, enter your plugin name.
- When prompted `Enter organization:`, enter your organization.
- When prompted `Enter project path:`, enter the project path in your local file system.
- When prompted `All good, are you OK to continue [y/N]?`, enter `y`.
2. Develop your plugin extension points. See {ref}`dev-plugins-extension` for descriptions and examples.
3. In the plugin root directory, run `make assemble`.

(gradle-plugin-install)=

## Installing a plugin

Plugins can be installed locally without being packaged, uploaded, and published.

To install a plugin locally:

1. In the plugin root directory, run `make install`.

:::{note}
Running `make install` will add your plugin to your `$HOME/.nextflow/plugins` directory.
:::

2. Run your pipeline:

```bash
nextflow run main.nf -plugins <PLUGIN_NAME>@<VERSION>
```

:::{note}
Plugins can also be configured via nextflow configuration files. See {ref}`using-plugins-page` for more information.
:::


(gradle-plugin-unit-test)=

## Unit testing a plugin

Unit tests are small, focused tests designed to verify the behavior of individual plugin components and are an important part of software development.

To run unit tests:

1. Develop your unit tests. See [MyObserverTest.groovy](https://github.com/nextflow-io/nf-plugin-template/blob/main/src/test/groovy/acme/plugin/MyObserverTest.groovy) in [nf-plugin-template](https://github.com/nextflow-io/nf-plugin-template/tree/main) for unit test examples.
2. In the plugin root directory, run `make test`.

(gradle-plugin-package)=

## Packaging, uploading, and publishing a plugin

The Gradle plugin for Nextflow plugins simplifies publishing your plugin to the Nextflow Plugin Registry.

:::{note}
The Nextflow Plugin Registry is currently available as private beta technology. Contact [info@nextflow.io](mailto:info@nextflow.io) to learn how to get access.
:::

To package, upload, and publish your plugin to the Nextflow Plugin Registry:

1. Create a file named `$HOME/.gradle/gradle.properties`, where `$HOME` is your home directory.
2. Add the following properties:

```bash
pluginRegistry.accessToken=<REGISTRY_ACCESS_TOKEN>
```

Replace <REGISTRY_ACCESS_TOKEN> with your plugin registry access token.

3. Run `make release`.
15 changes: 13 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ config
executor
cache-and-resume
reports
plugins
```

```{toctree}
Expand Down Expand Up @@ -146,14 +145,26 @@ migrations/index
developer/index
developer/diagram
developer/packages
developer/plugins
```

```{toctree}
:hidden:
:caption: Plugins
:maxdepth: 1

plugins/plugins
plugins/using-plugins
plugins/developing-plugins
plugins/example-nf-hello
```

```{toctree}
:hidden:
:caption: Guides
:maxdepth: 1

gradle-plugin
migrating-gradle-plugin
updating-spot-retries
metrics
flux
Expand Down
133 changes: 133 additions & 0 deletions docs/migrating-gradle-plugin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
(migrating-plugin-page)=

# Migrating to the Gradle plugin for Nextflow plugins

This page introduces the Gradle plugin for Nextflow plugins, the Nextflow plugin registry, and how to migrate to the new plugin framework.


## Improvements to the plugin framework

The Nextflow plugin ecosystem is evolving to support a more robust and user-friendly experience by simplifying plugin development, streamlining publishing and discovery, and improving how plugins are loaded into workflows. These improvements make plugins more accessible, maintainable, and interoperable with Nextflow.

### Gradle plugin for Nextflow plugins

The Gradle plugin for Nextflow plugins simplifies and standardizes the development of Nextflow plugins. It configures default dependencies required for Nextflow integration and introduces custom Gradle tasks to streamline building, testing, packaging, and publishing plugins.

The Gradle plugin is versioned and published to the [Gradle Plugin Portal](https://plugins.gradle.org/), allowing developers to manage it like any other dependency. As the plugin ecosystem evolves, this Gradle plugin will enable easier maintenance and adoption of ongoing improvements to the Nextflow plugin framework.

### Nextflow Plugin Registry

The Nextflow plugin registry is a centralized repository of assembled plugins. It hosts an index of plugin metadata that supports plugin discovery, accessibility, and version tracking. The registry is integrated with the Nextflow runtime. Nextflow will automatically locate and download configured plugins.

:::{note}
The Nextflow Plugin Registry is currently available as private beta technology. Contact [info@nextflow.io](mailto:info@nextflow.io) to learn how to get access.
:::

## Impact on users and developers

The impact of the Gradle plugin for Nextflow plugins differs for plugin users and developers.

### Plugin Users

If you are a plugin user, no immediate actions are required. The plugin configuration has not changed.

### Plugin developers

Developers are encouraged to migrate to the Gradle plugin for Nextflow plugins and benefit from features that simplify plugin development and integration with the wider plugin ecosystem.

To migrate an existing Nextflow plugin:

1. Remove the following files and folders:
- `buildSrc/`
- `nextflow.config`
- `launch.sh`
- `plugins/build.gradle`
2. If your plugin uses a `plugins` directory, move the `src` directory to the project root.

:::{note}
Plugin sources should be in `src/main/groovy` or `src/main/java`.
:::

3. Replace the contents of `settings.gradle` with the following:

```groovy
rootProject.name = '<PLUGIN_NAME>'
```

Replace `PLUGIN_NAME` with your plugin name.

4. In the project root, create a new `build.gradle` file with the following configuration:

```groovy
// Plugins
plugins {
id 'io.nextflow.nextflow-plugin' version '0.0.1-alpha4'
}

// Dependencies (optional)
dependencies {
<DEPENDENCY>
}

// Plugin version
version = '<PLUGIN_VERSION>'

nextflowPlugin {
// Minimum Nextflow version
nextflowVersion = '<MINIMUM_NEXTFLOW_VERSION>'

// Plugin metadata
provider = '<PROVIDER>'
className = '<CLASS_NAME>'
extensionPoints = [
'<EXTENSION_POINT>'
]

publishing {
registry {
url = 'https://nf-plugins-registry.dev-tower.net/api'
authToken = project.findProperty('pluginRegistry.accessToken')
}
}
}
```

Replace the following:

- `DEPENDENCY`: (Optional) Your plugins dependency libraries—for example, `commons-io:commons-io:2.18.0`.
- `PLUGIN_VERSION:` Your plugin version—for example, `0.5.0`.
- `MINIMUM_NEXTFLOW_VERSION`: The minimum Nextflow version required to run your plugin—for example, `25.03.0-edge`.
- `PROVIDER`: Your name or organization—for example, `acme`.
- `CLASS_NAME`: Your plugin class name—for example, `acme.plugin.MyPlugin`.
- `EXTENSION_POINT`: Your extension point identifiers that the plugin will implement or expose—for example, `acme.plugin.MyFactory`.

5. Replace the contents of `Makefile` with the following:

```
# Build the plugin
assemble:
./gradlew assemble

clean:
rm -rf .nextflow*
rm -rf work
rm -rf build
./gradlew clean

# Run plugin unit tests
test:
./gradlew test

# Install the plugin into local nextflow plugins dir
install:
./gradlew install

# Publish the plugin
release:
./gradlew releasePlugin
```

6. Update `README.md` with information about the structure of your plugin.
7. In the plugin root directory, run `make assemble`.

The Gradle plugin for Nextflow plugins also supports publishing plugins. See {ref}`gradle-plugin-package` for more information.
6 changes: 6 additions & 0 deletions docs/migrations/25-04.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ The third preview of workflow outputs introduces the following breaking changes

See {ref}`workflow-output-def` to learn more about the workflow output definition.

<h3>Creating plugins</h3>

The `nextflow plugin create` sub-command creates a scaffold for a Nextflow plugin using [`nf-plugin-template`](https://github.com/nextflow-io/nf-plugin-template/).

See {ref}`gradle-plugin-create` for details.

## Enhancements

<h3>Improved <code>inspect</code> command</h3>
Expand Down
63 changes: 0 additions & 63 deletions docs/plugins.md

This file was deleted.

Loading