Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .docusaurus_site/src/theme/prism-nextflow.js
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ export default function(Prism) {
],
// Nextflow-specific directives and keywords
'nextflow-directive': {
pattern: /\b(?:accelerator|afterScript|arch|array|beforeScript|cache|clusterOptions|conda|container|containerOptions|cpus|debug|disk|echo|errorStrategy|executor|ext|fair|label|machineType|maxErrors|maxForks|maxRetries|memory|module|penv|pod|publishDir|queue|resourceLabels|resourceLimits|scratch|secret|spack|stageInMode|stageOutMode|storeDir|tag|time)\b/,
pattern: /\b(?:accelerator|afterScript|arch|array|beforeScript|cache|clusterOptions|conda|container|containerOptions|cpus|debug|disk|echo|errorStrategy|executor|ext|fair|hints|label|machineType|maxErrors|maxForks|maxRetries|maxSubmitAwait|memory|module|penv|pod|publishDir|queue|resourceLabels|resourceLimits|scratch|secret|spack|stageInMode|stageOutMode|storeDir|tag|time)\b/,
alias: 'property'
},
'nextflow-keyword': {
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/process.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Reference for all task properties, input and output qualifiers, and

# Process reference

This section describes the task properties, input/output qualifiers, and directives available in [process][process-page] definitions.
[Process][process-page] definitions support the following task properties, input and output qualifiers, and directives:

| Page | Description |
| ---- | ----------- |
Expand Down
6 changes: 3 additions & 3 deletions docs/reference/process/directives.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,17 @@ description: Reference for the directives that control the execution of process

Directives are optional settings that affect the execution of the current process. They are declared at the top of a process definition, before any input, output, or script section.

By default, directives are evaluated when the process is defined. However, if the value is a dynamic string or closure, it will be evaluated separately for each task. See [Dynamic directives][dynamic-directives] for more information.
By default, directives are evaluated when the process is defined. However, if the value is a dynamic string or closure, it is evaluated separately for each task. See [Dynamic directives][dynamic-directives] for more information.

Directives can also be specified in a configuration file using the `process` config scope, which is the recommended way to configure a pipeline for a particular compute environment. See [Process configuration][config-process-scope] for more information.
You can also specify directives in a configuration file using the `process` config scope. This is the recommended way to configure a pipeline for a particular compute environment. See [Process configuration][config-process-scope] for more information.

Some directives are only supported by specific executors. See [Executors][executor-page] for more information about each executor.

The following directives are available:

| Directive | Description |
| --------- | ----------- |
| [`accelerator`][accelerator] | The number of hardware accelerators (e.g. GPUs) required by each task. |
| [`accelerator`][accelerator] | The number of hardware accelerators (for example, GPUs) required by each task. |
| [`afterScript`][after-script] | A Bash snippet to execute after the task script. |
| [`arch`][arch] | The CPU architecture to build the software required by each task. |
| [`array`][array] | Submit tasks as job arrays, for executors that support it. |
Expand Down
22 changes: 14 additions & 8 deletions docs/reference/process/directives/accelerator.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ description: Reference for the `accelerator` process directive.

# `accelerator`

The `accelerator` directive defines the number of hardware accelerators (e.g. GPUs) required by each task execution. For example:
The `accelerator` directive defines the number of hardware accelerators (for example, GPUs) required by each task execution.

## Usage

For example:

```nextflow
process hello {
Expand All @@ -18,23 +22,21 @@ process hello {
}
```

The above example requests 4 GPUs of type `nvidia-tesla-k80` for each task.
This example requests 4 GPUs of type `nvidia-tesla-k80` for each task.

:::note
This directive is only used by certain executors. Refer to the [Executors][executor-page] page to see which executors support this directive.
Additional options may be required to use accelerators. When using containers with GPUs, you must pass the GPU drivers through to the container. For Docker, this requires the option `--gpus all` in the `docker run` command. For Apptainer/Singularity, this requires the option `--nv`. Implementation details depend on the accelerator and container type.
:::

:::note
Additional options may be required to fully enable the use of accelerators. When using containers with GPUs, you must pass the GPU drivers through to the container. For Docker, this requires the option `--gpus all` in the `docker run` command. For Apptainer/Singularity, this requires the option `--nv`. The specific implementation details depend on the accelerator and container type being used.
:::
## Options

The following options are available:

##### `request: Integer`

The number of requested accelerators.

Specifying this directive with a number (e.g., `accelerator 4`) is equivalent to the `request` option (e.g., `accelerator request: 4`).
Specifying this directive with a number (for example, `accelerator 4`) is equivalent to the `request` option (for example, `accelerator request: 4`).

##### `type: String`

Expand All @@ -45,6 +47,10 @@ The meaning of this option depends on the target execution platform. See the pla
- [Google Cloud](https://cloud.google.com/compute/docs/gpus/)
- [Kubernetes](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#clusters-containing-different-types-of-gpus)

This option is not supported for AWS Batch. You can control the accelerator type indirectly through the allowed instance types in your Compute Environment. See the [AWS Batch FAQs](https://aws.amazon.com/batch/faqs/?#GPU_Scheduling_) for more information.
This option is not supported for AWS Batch. You can control the accelerator type indirectly through the allowed instance types in your compute environment. See the [AWS Batch FAQs](https://aws.amazon.com/batch/faqs/?#GPU_Scheduling_) for more information.

## Executor support

This directive is only used by certain executors. See [Executors][executor-page] for the executors that support this directive.

[executor-page]: ../../../executor
8 changes: 6 additions & 2 deletions docs/reference/process/directives/after-script.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ description: Reference for the `afterScript` process directive.

# `afterScript`

The `afterScript` directive executes a custom (Bash) snippet immediately *after* the main process has run. This may be useful to clean up your staging area.
The `afterScript` directive executes a custom Bash snippet immediately *after* the main process has run.

When combined with the [container](./container) directive, the `afterScript` is executed outside the specified container. In other words, the `afterScript` is always executed in the host environment.
## Usage

Use it to clean up your staging area.

The `afterScript` is always executed in the host environment. When combined with the [container](./container) directive, the `afterScript` is executed outside the specified container.
21 changes: 13 additions & 8 deletions docs/reference/process/directives/arch.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ description: Reference for the `arch` process directive.

# `arch`

The `arch` directive defines the CPU architecture to build the software in use by the process' task. For example:
The `arch` directive defines the CPU architecture used to build the software required by each task.

## Usage

For example:

```nextflow
process blast {
Expand All @@ -19,19 +23,20 @@ process blast {
}
```

The example above declares that the CPU generic architecture is `linux/x86_64` (X86 64 bit), and more specifically that the microarchitecture is `cascadelake` (a specific generation of Intel CPUs).
This example declares that the generic CPU architecture is `linux/x86_64` (X86 64 bit), and more specifically that the microarchitecture is `cascadelake` (a specific generation of Intel CPUs).

This directive is used by the following Nextflow features:

This directive is currently used by the following Nextflow functionalities:
- The [spack](./spack) directive, to build microarchitecture-optimized applications.
- The [Wave][wave-page] service, to build containers for one of the generic families of CPU architectures.
- The `spack` strategy within [Wave][wave-page], to optimize the container builds for specific CPU microarchitectures.

- by the [spack](./spack) directive, to build microarchitecture-optimized applications;
- by the [Wave][wave-page] service, to build containers for one of the generic families of CPU architectures (see below);
- by the `spack` strategy within [Wave][wave-page], to optimize the container builds for specific CPU microarchitectures.
The following values are allowed, grouped by equivalent family (multiple choices are available for compatibility):

Allowed values for the `arch` directive are as follows, grouped by equivalent family (choices available for the sake of compatibility):
- X86 64 bit: `linux/x86_64`, `x86_64`, `linux/amd64`, `amd64`
- ARM 64 bit: `linux/aarch64`, `aarch64`, `linux/arm64`, `arm64`, `linux/arm64/v8`
- ARM 64 bit, older generation: `linux/arm64/v7`

Examples of values for the architecture `target` option are `cascadelake`, `icelake`, `zen2` and `zen3`. See the [Spack documentation](https://spack.readthedocs.io/en/latest/basic_usage.html#support-for-specific-microarchitectures) for the full and up-to-date list of meaningful targets.
Examples of values for the architecture `target` option are `cascadelake`, `icelake`, `zen2`, and `zen3`. See the [Spack documentation](https://spack.readthedocs.io/en/latest/basic_usage.html#support-for-specific-microarchitectures) for the full list of targets.

[wave-page]: ../../../wave
32 changes: 18 additions & 14 deletions docs/reference/process/directives/array.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ description: Reference for the `array` process directive.

The `array` directive submits tasks as *job arrays* for executors that support it.

A job array is a collection of jobs with the same resource requirements and the same script (parameterized by an index). Job arrays incur significantly less scheduling overhead compared to individual jobs, and as a result they are preferred by HPC schedulers where possible.
A job array is a collection of jobs with the same resource requirements and the same script (parameterized by an index). Job arrays incur less scheduling overhead than individual jobs and are preferred by HPC schedulers where possible.

The directive should be specified with a given array size, along with an executor that supports job arrays. For example:
## Usage

Specify the directive with an array size, along with an executor that supports job arrays. For example:

```nextflow
process hello {
Expand All @@ -25,19 +27,9 @@ process hello {
}
```

Nextflow currently supports job arrays for the following executors:

- [AWS Batch][awsbatch-executor]
- [Google Cloud Batch][google-batch-executor]
- [LSF][lsf-executor]
- [PBS][pbs-executor]
- [PBS Pro][pbspro-executor]
- [SGE][sge-executor]
- [SLURM][slurm-executor]

A process using job arrays collects tasks and submits each batch as a job array when it is ready. Any "leftover" tasks are submitted as a partial job array.

Once a job array is submitted, each "child" task is executed as an independent job. Any tasks that fail (and can be retried) are retried without interfering with the tasks that succeeded. Retried tasks are submitted individually rather than through a job array, in order to allow for the use of [dynamic resources][dynamic-task-resources].
After a job array is submitted, each "child" task is executed as an independent job. Any tasks that fail (and can be retried) are retried without interfering with the tasks that succeeded. Retried tasks are submitted individually rather than through a job array, to allow the use of [dynamic resources][dynamic-task-resources].

The following directives must be uniform across all tasks in a process that uses job arrays, because these directives are specified once for the entire job array:

Expand All @@ -52,7 +44,7 @@ The following directives must be uniform across all tasks in a process that uses
- [resourceLimits](./resource-limits)
- [time](./time)

For cloud-based executors like AWS Batch, or when using Fusion with any executor, the following additional directives must be uniform:
For cloud-based executors such as AWS Batch, or when using Fusion with any executor, the following additional directives must be uniform:

- [container](./container)
- [containerOptions](./container-options)
Expand All @@ -61,6 +53,18 @@ When using Wave, the following additional directives must be uniform:

- [conda](./conda)

## Executor support

Nextflow supports job arrays for the following executors:

- [AWS Batch][awsbatch-executor]
- [Google Cloud Batch][google-batch-executor]
- [LSF][lsf-executor]
- [PBS][pbs-executor]
- [PBS Pro][pbspro-executor]
- [SGE][sge-executor]
- [SLURM][slurm-executor]

[awsbatch-executor]: ../../../executor/aws-batch
[dynamic-task-resources]: ../../../process#dynamic-task-resources
[google-batch-executor]: ../../../executor/google-batch
Expand Down
8 changes: 5 additions & 3 deletions docs/reference/process/directives/before-script.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ description: Reference for the `beforeScript` process directive.

# `beforeScript`

The `beforeScript` directive executes a custom (Bash) snippet *before* the main process script is run. This may be useful to initialize the underlying cluster environment or for other custom initialization.
The `beforeScript` directive executes a custom Bash snippet *before* the main process script is run.

For example:
## Usage

Use it to initialize the underlying cluster environment or perform other custom initialization. For example:

```nextflow
process hello {
Expand All @@ -20,4 +22,4 @@ process hello {
}
```

When the process is containerized (using the [container](./container) directive), the `beforeScript` is executed in the container only if the executor is *container-native* (e.g. cloud batch executors, Kubernetes). Otherwise, the `beforeScript` is executed outside the container.
When the process is containerized (using the [container](./container) directive), the `beforeScript` is executed in the container only if the executor is *container-native* (for example, cloud batch executors and Kubernetes). Otherwise, the `beforeScript` is executed outside the container.
12 changes: 8 additions & 4 deletions docs/reference/process/directives/cache.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ description: Reference for the `cache` process directive.

The `cache` directive controls whether and how task executions are cached.

By default, cached task executions are re-used when the pipeline is launched with the [resume][getstarted-resume] option. The `cache` directive can be used to disable caching for a specific process:
## Usage

By default, cached task executions are re-used when the pipeline is launched with the [resume][getstarted-resume] option. Use the `cache` directive to disable caching for a specific process:

```nextflow
process hello {
Expand All @@ -19,23 +21,25 @@ process hello {

See [Cache and resume][cache-resume-page] for more information.

The following options are available:
## Options

The following values are available:

##### `false`

Disable caching.

##### `true` (default)

Enable caching. Input file metadata (name, size, last updated timestamp) are included in the cache keys.
Enable caching. Input file metadata (name, size, last updated timestamp) is included in the cache keys.

##### `'deep'`

Enable caching. Input file content is included in the cache keys.

##### `'lenient'`

Enable caching. Minimal input file metadata (name and size only) are included in the cache keys.
Enable caching. Minimal input file metadata (name and size only) is included in the cache keys.

This strategy provides a workaround for incorrect caching invalidation observed on shared file systems due to inconsistent file timestamps.

Expand Down
16 changes: 9 additions & 7 deletions docs/reference/process/directives/cluster-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ description: Reference for the `clusterOptions` process directive.

The `clusterOptions` directive specifies additional submission options for grid executors. You can use it to specify options for your cluster that are not supported directly by other process directives.

## Usage

The cluster options can be a string:

```nextflow
Expand All @@ -18,7 +20,7 @@ process hello {
```

<ChangedInVersion version="24.04">
Prior to this version, grid executors that require each option to be on a separate line in the job script would attempt to split multiple options using a variety of different conventions. Multiple options can now be specified more clearly using a string list as shown below.
Prior to this version, grid executors that require each option to be on a separate line in the job script attempted to split multiple options using different conventions. Multiple options can now be specified more clearly using a string list, as shown in the following example.
</ChangedInVersion>

The cluster options can also be a string list:
Expand All @@ -31,14 +33,14 @@ process hello {
}
```

Grid executors that require one option per line will write each option to a separate line, while grid executors that allow multiple options per line will write all options to a single line, the same as with a string. This form is useful to control how the options are split across lines when it is required by the scheduler.

:::note
This directive is only used by grid executors. Refer to the [Executors][executor-page] page to see which executors support this directive.
:::
Grid executors that require one option per line write each option to a separate line. Grid executors that allow multiple options per line write all options to a single line, the same as with a string. This form is useful to control how the options are split across lines when the scheduler requires it.

:::warning
While you can use the `clusterOptions` directive to specify options that are supported as process directives (`queue`, `memory`, `time`, etc), you should not use both at the same time, as it will cause undefined behavior. Most HPC schedulers will either fail or simply ignore one or the other.
Do not use the `clusterOptions` directive to specify options that are supported as process directives, such as `queue`, `memory`, and `time`. Using both at the same time causes undefined behavior. Most HPC schedulers either fail or ignore one or the other.
:::

## Executor support

This directive is only used by grid executors. See [Executors][executor-page] for the executors that support this directive.

[executor-page]: ../../../executor
10 changes: 7 additions & 3 deletions docs/reference/process/directives/conda.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ description: Reference for the `conda` process directive.

# `conda`

The `conda` directive defines the set of [Conda](https://conda.io) packages required by each task. For example:
The `conda` directive defines the set of [Conda](https://conda.io) packages required by each task.

## Usage

For example:

```nextflow
process hello {
Expand All @@ -20,8 +24,8 @@ process hello {

Nextflow automatically creates an environment for each unique set of Conda packages.

The name of the desired channel for a specific package can be specified using the standard Conda notation, e.g. `bioconda::bwa=0.7.15`. Multiple packages can be specified separating them with a blank space, e.g. `bwa=0.7.15 fastqc=0.11.5`.
Specify the channel for a package using the standard Conda notation, for example, `bioconda::bwa=0.7.15`. Specify multiple packages by separating them with a space, for example, `bwa=0.7.15 fastqc=0.11.5`.

The `conda` directive can also accept a Conda environment file path or the path of an existing Conda environment. See [Conda][conda-page] for more information.
The `conda` directive also accepts a Conda environment file path or the path of an existing Conda environment. See [Conda][conda-page] for more information.

[conda-page]: ../../../conda
Loading
Loading