You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cli.md
+30Lines changed: 30 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -243,6 +243,36 @@ $ nextflow run <pipeline> --files "*.fasta"
243
243
```
244
244
:::
245
245
246
+
Parameters specified on the command line can be also specified in a params file using the `-params-file` option.
247
+
248
+
```bash
249
+
nextflow run main.nf -params-file pipeline_params.yml
250
+
```
251
+
252
+
The `-params-file` option loads parameters for your Nextflow pipeline from a JSON or YAML file. Parameters defined in the file are equivalent to specifying them directly on the command line. For example, instead of specifying parameters on the command line:
253
+
254
+
```bash
255
+
nextflow run main.nf --alpha 1 --beta foo
256
+
```
257
+
258
+
Parameters can be represented in YAML format:
259
+
260
+
```yaml
261
+
alpha: 1
262
+
beta: 'foo'
263
+
```
264
+
265
+
Or in JSON format:
266
+
267
+
```json
268
+
{
269
+
"alpha": 1,
270
+
"beta": "foo"
271
+
}
272
+
```
273
+
274
+
The parameters specified in a params file are merged with the resolved configuration. The values provided via a params file overwrite those of the same name in the Nextflow configuration file, but not those specified on the command line.
275
+
246
276
## Managing projects
247
277
248
278
Nextflow seamlessly integrates with popular Git providers, including [BitBucket](http://bitbucket.org/), [GitHub](http://github.com), and [GitLab](http://gitlab.com) for managing Nextflow pipelines as version-controlled Git repositories.
Copy file name to clipboardExpand all lines: docs/conda.md
+69-23Lines changed: 69 additions & 23 deletions
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@
6
6
7
7
Nextflow has built-in support for Conda that allows the configuration of workflow dependencies using Conda recipes and environment files.
8
8
9
-
This allows Nextflow applications to use popular tool collections such as [Bioconda](https://bioconda.github.io) whilst taking advantage of the configuration flexibility provided by Nextflow.
9
+
This allows Nextflow applications to use popular tool collections such as [Bioconda](https://bioconda.github.io)and the [Python Package index](https://pypi.org/), whilst taking advantage of the configuration flexibility provided by Nextflow.
10
10
11
11
## Prerequisites
12
12
@@ -22,7 +22,7 @@ Dependencies are specified by using the {ref}`process-conda` directive, providin
22
22
Conda environments are stored on the file system. By default, Nextflow instructs Conda to save the required environments in the pipeline work directory. The same environment may be created/saved multiple times across multiple executions when using different work directories.
23
23
:::
24
24
25
-
You can specify the directory where the Conda environments are stored using the `conda.cacheDir` configuration property. When using a computing cluster, make sure to use a shared file system path accessible from all compute nodes. See the {ref}`configuration page <config-conda>` for details about Conda configuration.
25
+
You can specify the directory where the Conda environments are stored using the `conda.cacheDir` configuration property. When using a computing cluster, make sure to use a shared file system path accessible from all compute nodes. See the {ref}`configuration page <config-conda>` for details about Conda configuration.
26
26
27
27
:::{warning}
28
28
The Conda environment feature is not supported by executors that use remote object storage as a work directory. For example, AWS Batch.
@@ -49,9 +49,10 @@ Conda package names can specified using the `conda` directive. Multiple package
49
49
process foo {
50
50
conda 'bwa samtools multiqc'
51
51
52
-
'''
52
+
script:
53
+
"""
53
54
your_command --here
54
-
'''
55
+
"""
55
56
}
56
57
```
57
58
@@ -61,6 +62,7 @@ The usual Conda package syntax and naming conventions can be used. The version o
61
62
62
63
The name of the channel where a package is located can be specified prefixing the package with the channel name as shown here `bioconda::bwa=0.7.15`.
63
64
65
+
(conda-env-files)=
64
66
### Use Conda environment files
65
67
66
68
Conda environments can also be defined using one or more Conda environment files. This is a file that lists the required packages and channels structured using the YAML format. For example:
@@ -76,20 +78,6 @@ dependencies:
76
78
- bwa=0.7.15
77
79
```
78
80
79
-
This other example shows how to leverage a Conda environment file to install Python packages from the [PyPI repository](https://pypi.org/)), through the `pip` package manager (which must also be explicitly listed as a required package):
80
-
81
-
```yaml
82
-
name: my-env-2
83
-
channels:
84
-
- defaults
85
-
dependencies:
86
-
- pip
87
-
- pip:
88
-
- numpy
89
-
- pandas
90
-
- matplotlib
91
-
```
92
-
93
81
Read the Conda documentation for more details about how to create [environment files](https://conda.io/docs/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually).
94
82
95
83
The path of an environment file can be specified using the `conda` directive:
@@ -98,17 +86,37 @@ The path of an environment file can be specified using the `conda` directive:
98
86
process foo {
99
87
conda '/some/path/my-env.yaml'
100
88
101
-
'''
89
+
script:
90
+
"""
102
91
your_command --here
103
-
'''
92
+
"""
104
93
}
105
94
```
106
95
107
96
:::{warning}
108
97
The environment file name **must** have a `.yml` or `.yaml` extension or else it won't be properly recognised.
109
98
:::
110
99
111
-
Alternatively, it is possible to provide the dependencies using a plain text file, just listing each package name as a separate line. For example:
100
+
(conda-pypi)=
101
+
### Python Packages from PyPI
102
+
103
+
Conda environment files can also be used to install Python packages from the [PyPI repository](https://pypi.org/), through the `pip` package manager (which must also be explicitly listed as a required package):
104
+
105
+
```yaml
106
+
name: my-env-2
107
+
channels:
108
+
- defaults
109
+
dependencies:
110
+
- pip
111
+
- pip:
112
+
- numpy
113
+
- pandas
114
+
- matplotlib
115
+
```
116
+
117
+
### Conda text files
118
+
119
+
It is possible to provide dependencies by listing each package name as a separate line in a plain text file. For example:
112
120
113
121
```
114
122
bioconda::star=2.5.4a
@@ -120,6 +128,43 @@ bioconda::multiqc=1.4
120
128
Like before, the extension matters. Make sure the dependencies file has a `.txt` extension.
121
129
:::
122
130
131
+
### Conda lock files
132
+
133
+
The final way to provide packages to Conda is with [Conda lock files](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#identical-conda-envs).
134
+
135
+
These are generated from existing Conda environments using the following command:
136
+
137
+
```bash
138
+
conda list --explicit > spec-file.txt
139
+
```
140
+
141
+
or if using Mamba / Micromamba:
142
+
143
+
```bash
144
+
micromamba env export --explicit > spec-file.txt
145
+
```
146
+
147
+
Conda lock files can also be downloaded from [Wave](https://seqera.io/wave/) build pages.
148
+
149
+
These files include every package and their dependencies. As such, no Conda environment resolution step is needed. This is faster and more reproducible.
150
+
151
+
The files contain package URLs and an optional md5hash for each download to confirm identity:
152
+
153
+
```
154
+
# micromamba env export --explicit
155
+
# This file may be used to create an environment using:
To use with Nextflow, simply set the `conda` directive to the lock file path.
167
+
123
168
### Use existing Conda environments
124
169
125
170
If you already have a local Conda environment, you can use it in your workflow specifying the installation directory of such environment by using the `conda` directive:
@@ -128,9 +173,10 @@ If you already have a local Conda environment, you can use it in your workflow s
0 commit comments