From 9e14ab31198e4461f01d63d23c93391a2c3a51c9 Mon Sep 17 00:00:00 2001 From: piplus2 Date: Tue, 15 Sep 2026 11:01:46 +0200 Subject: [PATCH 1/3] port module preprocess_transcripts_fasta_gencode to nf-core structure Add environment.yml, meta.yml, a stub, task.ext.prefix support and nf-tests to the local PREPROCESS_TRANSCRIPTS_FASTA_GENCODE module. It moves to the seqera community container shared with GUNZIP and CAT_FASTQ (coreutils, gzip, sed) and reports the coreutils version of cut, which does the work, instead of sed, which only parsed the version string. The module now emits [ meta, fasta ] instead of a bare path. PREPARE_GENOME already treats ch_transcript_fasta as that tuple everywhere else, so --gencode with --transcript_fasta aborted at the first `.map { _meta, tr -> tr }` with `Invalid method invocation 'call' with arguments: ... UnixPath`. The tests need no test-datasets upload: the setup block writes a two record GENCODE style fasta, plain and gzipped, as the strand_junctions tests and the same module in nf-core/rnaseq do. Three cases: plain, gzipped and stub, asserting the cut headers explicitly on top of the snapshot. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01MKJushdvSrzFfFCrKk29Yr --- CHANGELOG.md | 2 + .../environment.yml | 12 +++ .../main.nf | 28 ++++-- .../meta.yml | 72 ++++++++++++++ .../tests/main.nf.test | 97 +++++++++++++++++++ .../tests/main.nf.test.snap | 95 ++++++++++++++++++ 6 files changed, 296 insertions(+), 10 deletions(-) create mode 100644 modules/local/preprocess_transcripts_fasta_gencode/environment.yml create mode 100644 modules/local/preprocess_transcripts_fasta_gencode/meta.yml create mode 100644 modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test create mode 100644 modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test.snap diff --git a/CHANGELOG.md b/CHANGELOG.md index 88e8f786..e13af181 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - #270 - Move the `LEAFCUTTER` subworkflow to the nf-core subworkflow template. Its `juncs` output is now the per sample `[ meta, junc ]` tuples, instead of a single list flattening the meta maps in with the paths (by @piplus2) - #272 - `--leafcutter` now works with `--source genome_bam`, whatever the strandedness, since the junction strand no longer has to come from the alignment. For unstranded libraries the clusters differ slightly from LeafCutter's documented STAR route, see the LeafCutter section of `docs/usage.md` (by @piplus2) - #281 - Refactor `ISOFORMSWITCHANALYZER` to the nf-core module template and update `IsoformSwitchAnalyzeR` 2.2.0 -> 2.12.0 (R 4.3 -> 4.5). `bin/run_isoformswitchanalyzer.R` is now a module template (by @piplus2) +- #284 - Refactor `PREPROCESS_TRANSCRIPTS_FASTA_GENCODE` to the nf-core module template, with `environment.yml`, `meta.yml`, a stub and nf-tests. It reports the `coreutils` version of `cut`, which does the work, instead of `sed` (by @piplus2) ### Fixed @@ -87,6 +88,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - #258 - Fix `ISOFORMSWITCHANALYZER` receiving the `.tar.gz` archives instead of the extracted Salmon directories with `--source salmon_results`. `TX2GENE_TXIMPORT` extracts them and now emits them (by @piplus2) - #261 - Fix the paired rMATS sample order when a sample spans several samplesheet rows (by @piplus2) - #261 - Fix single condition rMATS runs, which aborted before producing any output (by @piplus2) +- #284 - Fix `--gencode` with `--transcript_fasta` aborting in `PREPARE_GENOME`, as `PREPROCESS_TRANSCRIPTS_FASTA_GENCODE` dropped the meta map the downstream channel expects (by @piplus2) - #261 - Fix samples split over several samplesheet rows, whose fastq files were passed on without being concatenated (by @piplus2) - #263 - **Breaking change**: `DEXSEQ_COUNT` no longer counts BAM input as forward stranded, it follows the samplesheet `strandedness`, so DEXSeq results change for BAM samplesheets that do not set the column (reported by @albamasmalavila, fix by @piplus2) - #264 - Fix the `DEXSEQ_DTU` stub, which wrote file names the process outputs did not match, so a stub run of the DTU path failed (by @piplus2) diff --git a/modules/local/preprocess_transcripts_fasta_gencode/environment.yml b/modules/local/preprocess_transcripts_fasta_gencode/environment.yml new file mode 100644 index 00000000..9b926b1f --- /dev/null +++ b/modules/local/preprocess_transcripts_fasta_gencode/environment.yml @@ -0,0 +1,12 @@ +--- +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json +channels: + - conda-forge + - bioconda +dependencies: + - conda-forge::coreutils=9.5 + - conda-forge::grep=3.11 + - conda-forge::gzip=1.13 + - conda-forge::lbzip2=2.5 + - conda-forge::sed=4.8 + - conda-forge::tar=1.34 diff --git a/modules/local/preprocess_transcripts_fasta_gencode/main.nf b/modules/local/preprocess_transcripts_fasta_gencode/main.nf index 1ffa1f4e..5e2a7ec2 100644 --- a/modules/local/preprocess_transcripts_fasta_gencode/main.nf +++ b/modules/local/preprocess_transcripts_fasta_gencode/main.nf @@ -1,26 +1,34 @@ process PREPROCESS_TRANSCRIPTS_FASTA_GENCODE { - tag "$fasta" + tag "${meta.id ?: fasta.baseName}" + label 'process_single' - conda "conda-forge::sed=4.7" - container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/ubuntu:20.04' : - 'nf-core/ubuntu:20.04' }" + conda "${moduleDir}/environment.yml" + container "${workflow.containerEngine in ['singularity', 'apptainer'] && !task.ext.singularity_pull_docker_container + ? 'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/52/52ccce28d2ab928ab862e25aae26314d69c8e38bd41ca9431c67ef05221348aa/data' + : 'community.wave.seqera.io/library/coreutils_grep_gzip_lbzip2_pruned:838ba80435a629f8'}" input: tuple val(meta), path(fasta) output: - path "*.fa" , emit: fasta - tuple val("${task.process}"), val('sed'), eval('sed --version 2>&1 | head -1 | sed "s/sed (GNU sed) //g"'), topic: versions, emit: versions_sed + tuple val(meta), path("*.fixed.fa"), emit: fasta + tuple val("${task.process}"), val('cut'), eval('cut --version 2>&1 | head -1 | sed "s/^.*coreutils) //"'), topic: versions, emit: versions_cut when: task.ext.when == null || task.ext.when script: - def gzipped = fasta.toString().endsWith('.gz') - def outfile = gzipped ? file(fasta.baseName).baseName : fasta.baseName + def gzipped = fasta.extension == 'gz' + def prefix = task.ext.prefix ?: (gzipped ? file(fasta.baseName).baseName : fasta.baseName) def command = gzipped ? 'zcat' : 'cat' """ - ${command} ${fasta} | cut -d "|" -f1 > ${outfile}.fixed.fa + ${command} ${fasta} | cut -d "|" -f1 > ${prefix}.fixed.fa + """ + + stub: + def gzipped = fasta.extension == 'gz' + def prefix = task.ext.prefix ?: (gzipped ? file(fasta.baseName).baseName : fasta.baseName) + """ + touch ${prefix}.fixed.fa """ } diff --git a/modules/local/preprocess_transcripts_fasta_gencode/meta.yml b/modules/local/preprocess_transcripts_fasta_gencode/meta.yml new file mode 100644 index 00000000..e6c77c9f --- /dev/null +++ b/modules/local/preprocess_transcripts_fasta_gencode/meta.yml @@ -0,0 +1,72 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/meta-schema.json +name: "preprocess_transcripts_fasta_gencode" +description: Strip the GENCODE transcript FASTA headers down to the transcript id, + so that they match the transcript ids of the GTF +keywords: + - fasta + - gencode + - transcriptome + - header +tools: + - "cut": + description: "GNU coreutils cut, which removes sections from each line of a file" + homepage: "https://www.gnu.org/software/coreutils/" + documentation: "https://www.gnu.org/software/coreutils/manual/html_node/cut-invocation.html" + licence: ["GPL-3.0-or-later"] + identifier: "" + +input: + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - fasta: + type: file + description: GENCODE transcript FASTA, whose headers hold the transcript id, + gene id, names, length and biotype separated by `|`, optionally gzipped + pattern: "*.{fa,fasta,fa.gz,fasta.gz}" + ontologies: + - edam: "http://edamontology.org/format_1929" # FASTA + +output: + fasta: + - - meta: + type: map + description: | + Groovy Map containing sample information + e.g. [ id:'test' ] + - "*.fixed.fa": + type: file + description: Uncompressed transcript FASTA with the headers cut down to + the transcript id, the first `|` separated field + pattern: "*.fixed.fa" + ontologies: + - edam: "http://edamontology.org/format_1929" # FASTA + versions_cut: + - - ${task.process}: + type: string + description: The process the versions were collected from + - cut: + type: string + description: The tool name + - cut --version 2>&1 | head -1 | sed "s/^.*coreutils) //": + type: eval + description: The expression to obtain the version of the tool + +topics: + versions: + - - ${task.process}: + type: string + description: The process the versions were collected from + - cut: + type: string + description: The tool name + - cut --version 2>&1 | head -1 | sed "s/^.*coreutils) //": + type: eval + description: The expression to obtain the version of the tool +authors: + - "@drpatelh" + - "@piplus2" +maintainers: + - "@piplus2" diff --git a/modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test b/modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test new file mode 100644 index 00000000..b9dccb23 --- /dev/null +++ b/modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test @@ -0,0 +1,97 @@ +// nf-core modules test preprocess_transcripts_fasta_gencode +nextflow_process { + + name "Test Process PREPROCESS_TRANSCRIPTS_FASTA_GENCODE" + script "../main.nf" + process "PREPROCESS_TRANSCRIPTS_FASTA_GENCODE" + + tag "modules" + tag "modules_local" + tag "preprocess_transcripts_fasta_gencode" + + // Two records with GENCODE headers, whose `|` separated fields the module cuts + // down to the first one, written plain and gzipped + setup { + def records = [ + '>ENST00000456328.2|ENSG00000290825.1|-|-|DDX11L2-202|DDX11L2|1657|lncRNA|', + 'GTGGAAAACTTTTCAGTGGCTGTGACAGGTTACAGAAGAAGATTGGAGGGGGGAAGGGGAG', + '>ENST00000450305.2|ENSG00000223972.6|OTTHUMG00000000961.2|OTTHUMT00000002844.2|DDX11L1-201|DDX11L1|632|transcribed_unprocessed_pseudogene|', + 'ACTCAAGATAATGATGAGTAAAGAATATATTTCTAACAACAAAAAGGAAATTTGATAGTA', + ] + def fasta = records.join('\n') + '\n' + file("${outputDir}/gencode.transcripts.fa").text = fasta + new java.util.zip.GZIPOutputStream(file("${outputDir}/gencode.transcripts.fa.gz").newOutputStream()).withStream { out -> + out.write(fasta.bytes) + } + } + + test("gencode fasta") { + + when { + process { + """ + input[0] = [ [ id:'test' ], file("${outputDir}/gencode.transcripts.fa", checkIfExists: true) ] + """ + } + } + + then { + def headers = path(process.out.fasta[0][1]).readLines().findAll { line -> line.startsWith('>') } + assertAll( + { assert process.success }, + { assert headers == [ '>ENST00000456328.2', '>ENST00000450305.2' ] }, + { assert snapshot( + process.out.fasta, + process.out.findAll { key, _val -> key.startsWith('versions') } + ).match() } + ) + } + + } + + test("gencode fasta - gzipped") { + + when { + process { + """ + input[0] = [ [ id:'test' ], file("${outputDir}/gencode.transcripts.fa.gz", checkIfExists: true) ] + """ + } + } + + then { + def headers = path(process.out.fasta[0][1]).readLines().findAll { line -> line.startsWith('>') } + assertAll( + { assert process.success }, + { assert headers == [ '>ENST00000456328.2', '>ENST00000450305.2' ] }, + { assert snapshot( + process.out.fasta, + process.out.findAll { key, _val -> key.startsWith('versions') } + ).match() } + ) + } + + } + + test("gencode fasta - stub") { + + options "-stub" + + when { + process { + """ + input[0] = [ [ id:'test' ], file("${outputDir}/gencode.transcripts.fa.gz", checkIfExists: true) ] + """ + } + } + + then { + assertAll( + { assert process.success }, + { assert snapshot(process.out).match() } + ) + } + + } + +} diff --git a/modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test.snap b/modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test.snap new file mode 100644 index 00000000..e7efae5a --- /dev/null +++ b/modules/local/preprocess_transcripts_fasta_gencode/tests/main.nf.test.snap @@ -0,0 +1,95 @@ +{ + "gencode fasta - gzipped": { + "content": [ + [ + [ + { + "id": "test" + }, + "gencode.transcripts.fixed.fa:md5,f1330752bc0e3db2d2dc5a7b4a51f7f5" + ] + ], + { + "versions_cut": [ + [ + "PREPROCESS_TRANSCRIPTS_FASTA_GENCODE", + "cut", + "9.5" + ] + ] + } + ], + "timestamp": "2026-09-15T10:15:57.964340955", + "meta": { + "nf-test": "0.9.5", + "nextflow": "26.04.4" + } + }, + "gencode fasta": { + "content": [ + [ + [ + { + "id": "test" + }, + "gencode.transcripts.fixed.fa:md5,f1330752bc0e3db2d2dc5a7b4a51f7f5" + ] + ], + { + "versions_cut": [ + [ + "PREPROCESS_TRANSCRIPTS_FASTA_GENCODE", + "cut", + "9.5" + ] + ] + } + ], + "timestamp": "2026-09-15T10:15:52.861538513", + "meta": { + "nf-test": "0.9.5", + "nextflow": "26.04.4" + } + }, + "gencode fasta - stub": { + "content": [ + { + "0": [ + [ + { + "id": "test" + }, + "gencode.transcripts.fixed.fa:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "1": [ + [ + "PREPROCESS_TRANSCRIPTS_FASTA_GENCODE", + "cut", + "9.5" + ] + ], + "fasta": [ + [ + { + "id": "test" + }, + "gencode.transcripts.fixed.fa:md5,d41d8cd98f00b204e9800998ecf8427e" + ] + ], + "versions_cut": [ + [ + "PREPROCESS_TRANSCRIPTS_FASTA_GENCODE", + "cut", + "9.5" + ] + ] + } + ], + "timestamp": "2026-09-15T10:16:03.016288191", + "meta": { + "nf-test": "0.9.5", + "nextflow": "26.04.4" + } + } +} \ No newline at end of file From 4f4fa7705bd50233a67d5faaaeb66dee20714a90 Mon Sep 17 00:00:00 2001 From: piplus2 Date: Tue, 15 Sep 2026 11:05:20 +0200 Subject: [PATCH 2/3] add agents md --- AGENTS.md | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..64d298a4 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,128 @@ +# nf-core: agents + +This is the main AI context file for nf-core pipelines. All AI agents and coding assistants **MUST** follow the rules contained in this document. + +## Natural language +All comments and documentation **MUST** be written in English with British spelling. Documentation files **SHOULD** additionally follow the style guide at https://nf-co.re/docs/developing/documentation/style-guide. +Never use emdashes in prose text, be succinct and to the point. Avoid telltale LLM phrasing such as "Not X, but Y", and excessive use of bold formatting. + +## Key nf-core terms +- Module: a single process that achieves a single, well defined task (e.g. aligning reads to a genome) +- Subworkflow: a sequence of chained modules that achieve a specific objective (e.g. FASTQ cleanup and quality check) +- Workflow: a complete sequence of modules and subworkflows that performs a specific analysis (e.g. bulk RNA-seq analysis) +- Pipeline: a complete, executable Nextflow project that defines workflow logic, input handling, and output publishing + +## Nextflow pitfalls +- Nextflow supports 2 ways to publish files to the output directory: workflow outputs (modern) and `publishDir` configuration directives in modules.config (legacy). You **SHOULD** publish output consistently with the existing code. +- nf-core tools commands may fail. If that happens, ask the user for help. You **MUST NOT** generate any file that is supposed to be generated by nf-core tools. +- Certain very old pipelines might be using Nextflow DSL1 syntax (with the entire workflow in a single file and channel from/to keywords). This syntax is now deprecated. You **MUST NOT** attempt to work on those pipelines. + +## nf-core template structure +The directory you are working on was created with the nf-core pipeline template. Key features of the template are demonstrated below: +``` +. + +├── conf // directory containing Nextflow configurations for the pipeline (see "Configuration files" below) +├── main.nf // core Nextflow script, may need editing if input structure changes +├── modules // Nextflow DSL2 modules +│ ├── local // local modules (see "Modules" below) +| | └── mymodule // each module must be in a separate directory +│ └── nf-core // nf-core modules (see "Modules" below) +├── nextflow_schema.json // JSON schema describing pipeline parameters +├── subworkflows // Nextflow subworkflows (see "Subworkflows" below) +│ ├── local // local subworkflows +| | └── myswf // each subworkflow must be in a separate directory +│ └── nf-core // nf-core subworkflows +├── tests // nf-test end-to-end tests for the pipeline +│ └── default.nf.test // main test script, must exist +└── workflows // do not add files + └── {pipeline-name}.nf // Nextflow file containing main pipeline logic +``` + +The pipeline also contains other files and directories. If a file does not follow the treemap above, you **MUST** verify with the user before editing it. + +## Modules +- You **SHOULD** use existing nf-core modules for the tools you need, where available. +- You can find available modules and install modules with nf-core tools (see "nf-core tools" section below). +- You **SHOULD NOT** edit nf-core modules in the pipeline modules directory. If unavoidable, you **MAY** edit their `main.nf` if necessary, and if target pipeline logic cannot be achieved with the existing module code. If you do it, you **MUST** run `nf-core modules patch {name}` afterwards, and flag that a PR will be needed to upstream the change. +- The pipeline has a local modules directory. If a script is only useful within this pipeline, you **MAY** create a local module for it. +- Use nf-core tools (see below) to create local module boilerplate and then edit the files. +- Use `ext.args` to pass any command-line arguments (except input files) to the underlying tool. +- Use `ext.prefix` to customize the name of the output files. To include runtime variables in those arguments, use Groovy-style closures, for example: `ext.prefix = { "${meta.id}_filtered" }`, usually through the modules.config file. + +## Subworkflows +- You **SHOULD** use nf-core subworkflows that are relevant to the pipeline tasks. +- If none is applicable, create a local subworkflow when it thematically makes sense. + +## Pipeline structure +An nf-core pipeline contains 3 main parts called by the root `workflow` block in `main.nf`: +- initialisation workflow (defined in `subworkflows/local/utils_nfcore_{name}_pipeline/main.nf`): handles input processing and validation +- main workflow (defined in `workflows/{name}.nf`): contains the main analysis, including generation of all output files (some nf-core pipelines contain more than one workflow) +- completion workflow (defined in `subworkflows/local/utils_nfcore_{name}_pipeline/main.nf`): handles sending completion notifications + +## Configuration files +- You **MUST NOT** edit `base.config`, `igenomes.config`, and `igenomes_ignored.config` +- Set `ext.args` and `ext.prefix` for modules in `modules.config`, using `withName` blocks +- The test in `test.config` **SHOULD** take a few minutes and only test the basic functionality with minimal input +- The test in `test_full.config` **SHOULD** use input and parameters that trigger all pipeline functionality + +## Meta map +The meta map is a Nextflow map passed along with each file that contains sample-specific information. The map is created during input processing and passed through modules. + +- The meta map **MUST** contain an `id` field with a unique identifier. +- nf-core modules and subworkflows may **only** access `id` and `single_end` fields. +- Local modules, subworkflows, and workflows may create and access any meta fields that are useful for the pipeline. +- Channel operations **MUST** preserve the meta map when present; they **MAY** add, remove, or modify specific keys as required. + +## nf-core tools +nf-core provides a CLI toolkit for working with the nf-core template. The core command is `nf-core`. You **SHOULD** always use the tools instead of creating files manually. + +Use `nf-core --help` to obtain information about nf-core commands. You can also use `--help` for subcommands. + +Write the names of subtool modules in commands with a slash, like `samtools/sort`. + +## nf-test and testing +- Each pipeline **MUST** have at least 1 test case. +- Tests have a standardized syntax, with setup (optional), input ("when"), and assertion ("then") sections. +- Tests at a path can be executed with `nf-test test {path}`. +- Most tests create at least 1 snapshot file. You **MUST NOT** edit snapshots manually. +- If you expect the output to change (e.g. after a tool update), update the snapshot with `nf-test test --profile +{docker|singularity|conda} --update-snapshot`. Only regenerate snapshots on the same CPU architecture as CI. +- If a new output file has unstable content, add it to `.nftignore`. + +## git and branch policy +This repository has at least 3 git branches: `main` (or `master`), `dev`, and `TEMPLATE`. +- You **MUST NOT** switch or write to the TEMPLATE branch. +- You **MUST NOT** write any code to `main`; use a pull request instead. +- Always create a new branch with a meaningful name for each feature, then open a pull request to `dev`. +- You **MUST NOT** commit feature work directly to `dev`, even on a fork. +- If you work on multiple features in parallel, you **SHOULD** use a separate worktree for each task to prevent clobber. + +## Commit rules and routine +- Each commit **SHOULD** contain one logical change. +- Commit title **SHOULD** be concise and written in imperative mood. +- If the commit consists only of installing or updating an nf-core module or subworkflow, limit the commit title to `Install/update nf-core module/subworkflow {name}`. +- Before each commit, you **MUST** stage changes and then run `prek`. Resolve all errors and all possible warnings. Repeat until there are no solvable outstanding issues. + +## Push routine +- You should only push to GitHub after implementing some meaningful changes and if the code is working. +- Before pushing, you **MUST** run `nf-core pipelines lint`, resolve all errors and all possible warnings. Repeat until there are no solvable outstanding issues. +- If you are preparing a release (PR to main), use `nf-core pipelines lint --release` instead. +- You **MUST** also run `nf-test test tests/`. If the pipeline fails, resolve the underlying issues. If the test fails due to mismatching snapshots, update them if permitted (see "nf-test and testing" above). Otherwise, fix the issue that caused the unexpected change. +- If you know the code will cause issues or you intend to push more changes, you **SHOULD** add `[skip ci]` at the end of the commit title. You **SHOULD** omit this tag for final review-ready commits. + +## PR procedure +- A PR **SHOULD** contain a single feature. +- You **SHOULD** add a line in the relevant section in CHANGELOG.md, listing contributors and the expected PR number. +- The PR **MUST** use and follow the nf-core PR template, including the checklist. +- The PR message **SHOULD** start with a brief explanation of the changes made and the motivation. +- Each PR requires reviews (1 for dev, 2 for main) and passing CI before merging. +- A human can request PR reviews on Slack. + +## Agent self-disclosure +- If you generated a majority of the code in a commit, you **MUST** add "Generated by {your name}" at the end of the commit message body. +- If you open a PR autonomously, you **MUST** add "Generated by {your name}" at the end of the PR message (above the checklist). + +## References +- Nextflow documentation: https://docs.seqera.io/nextflow +- nf-core tools documentation: https://nf-co.re/docs/nf-core-tools/ +- nf-test documentation: https://www.nf-test.com/docs/getting-started/ From a8fc0887f1c50c2137e4b303b3e03c9890f6c102 Mon Sep 17 00:00:00 2001 From: piplus2 Date: Tue, 15 Sep 2026 11:06:51 +0200 Subject: [PATCH 3/3] fix with prek --- AGENTS.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 64d298a4..a161db96 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -3,22 +3,27 @@ This is the main AI context file for nf-core pipelines. All AI agents and coding assistants **MUST** follow the rules contained in this document. ## Natural language + All comments and documentation **MUST** be written in English with British spelling. Documentation files **SHOULD** additionally follow the style guide at https://nf-co.re/docs/developing/documentation/style-guide. Never use emdashes in prose text, be succinct and to the point. Avoid telltale LLM phrasing such as "Not X, but Y", and excessive use of bold formatting. ## Key nf-core terms + - Module: a single process that achieves a single, well defined task (e.g. aligning reads to a genome) - Subworkflow: a sequence of chained modules that achieve a specific objective (e.g. FASTQ cleanup and quality check) - Workflow: a complete sequence of modules and subworkflows that performs a specific analysis (e.g. bulk RNA-seq analysis) - Pipeline: a complete, executable Nextflow project that defines workflow logic, input handling, and output publishing ## Nextflow pitfalls + - Nextflow supports 2 ways to publish files to the output directory: workflow outputs (modern) and `publishDir` configuration directives in modules.config (legacy). You **SHOULD** publish output consistently with the existing code. - nf-core tools commands may fail. If that happens, ask the user for help. You **MUST NOT** generate any file that is supposed to be generated by nf-core tools. - Certain very old pipelines might be using Nextflow DSL1 syntax (with the entire workflow in a single file and channel from/to keywords). This syntax is now deprecated. You **MUST NOT** attempt to work on those pipelines. ## nf-core template structure + The directory you are working on was created with the nf-core pipeline template. Key features of the template are demonstrated below: + ``` . @@ -42,6 +47,7 @@ The directory you are working on was created with the nf-core pipeline template. The pipeline also contains other files and directories. If a file does not follow the treemap above, you **MUST** verify with the user before editing it. ## Modules + - You **SHOULD** use existing nf-core modules for the tools you need, where available. - You can find available modules and install modules with nf-core tools (see "nf-core tools" section below). - You **SHOULD NOT** edit nf-core modules in the pipeline modules directory. If unavoidable, you **MAY** edit their `main.nf` if necessary, and if target pipeline logic cannot be achieved with the existing module code. If you do it, you **MUST** run `nf-core modules patch {name}` afterwards, and flag that a PR will be needed to upstream the change. @@ -51,22 +57,27 @@ The pipeline also contains other files and directories. If a file does not follo - Use `ext.prefix` to customize the name of the output files. To include runtime variables in those arguments, use Groovy-style closures, for example: `ext.prefix = { "${meta.id}_filtered" }`, usually through the modules.config file. ## Subworkflows + - You **SHOULD** use nf-core subworkflows that are relevant to the pipeline tasks. - If none is applicable, create a local subworkflow when it thematically makes sense. ## Pipeline structure + An nf-core pipeline contains 3 main parts called by the root `workflow` block in `main.nf`: + - initialisation workflow (defined in `subworkflows/local/utils_nfcore_{name}_pipeline/main.nf`): handles input processing and validation - main workflow (defined in `workflows/{name}.nf`): contains the main analysis, including generation of all output files (some nf-core pipelines contain more than one workflow) - completion workflow (defined in `subworkflows/local/utils_nfcore_{name}_pipeline/main.nf`): handles sending completion notifications ## Configuration files + - You **MUST NOT** edit `base.config`, `igenomes.config`, and `igenomes_ignored.config` - Set `ext.args` and `ext.prefix` for modules in `modules.config`, using `withName` blocks - The test in `test.config` **SHOULD** take a few minutes and only test the basic functionality with minimal input - The test in `test_full.config` **SHOULD** use input and parameters that trigger all pipeline functionality ## Meta map + The meta map is a Nextflow map passed along with each file that contains sample-specific information. The map is created during input processing and passed through modules. - The meta map **MUST** contain an `id` field with a unique identifier. @@ -75,6 +86,7 @@ The meta map is a Nextflow map passed along with each file that contains sample- - Channel operations **MUST** preserve the meta map when present; they **MAY** add, remove, or modify specific keys as required. ## nf-core tools + nf-core provides a CLI toolkit for working with the nf-core template. The core command is `nf-core`. You **SHOULD** always use the tools instead of creating files manually. Use `nf-core --help` to obtain information about nf-core commands. You can also use `--help` for subcommands. @@ -82,6 +94,7 @@ Use `nf-core --help` to obtain information about nf-core commands. You can also Write the names of subtool modules in commands with a slash, like `samtools/sort`. ## nf-test and testing + - Each pipeline **MUST** have at least 1 test case. - Tests have a standardized syntax, with setup (optional), input ("when"), and assertion ("then") sections. - Tests at a path can be executed with `nf-test test {path}`. @@ -90,7 +103,9 @@ Write the names of subtool modules in commands with a slash, like `samtools/sort - If a new output file has unstable content, add it to `.nftignore`. ## git and branch policy + This repository has at least 3 git branches: `main` (or `master`), `dev`, and `TEMPLATE`. + - You **MUST NOT** switch or write to the TEMPLATE branch. - You **MUST NOT** write any code to `main`; use a pull request instead. - Always create a new branch with a meaningful name for each feature, then open a pull request to `dev`. @@ -98,12 +113,14 @@ This repository has at least 3 git branches: `main` (or `master`), `dev`, and `T - If you work on multiple features in parallel, you **SHOULD** use a separate worktree for each task to prevent clobber. ## Commit rules and routine + - Each commit **SHOULD** contain one logical change. - Commit title **SHOULD** be concise and written in imperative mood. - If the commit consists only of installing or updating an nf-core module or subworkflow, limit the commit title to `Install/update nf-core module/subworkflow {name}`. - Before each commit, you **MUST** stage changes and then run `prek`. Resolve all errors and all possible warnings. Repeat until there are no solvable outstanding issues. ## Push routine + - You should only push to GitHub after implementing some meaningful changes and if the code is working. - Before pushing, you **MUST** run `nf-core pipelines lint`, resolve all errors and all possible warnings. Repeat until there are no solvable outstanding issues. - If you are preparing a release (PR to main), use `nf-core pipelines lint --release` instead. @@ -111,6 +128,7 @@ This repository has at least 3 git branches: `main` (or `master`), `dev`, and `T - If you know the code will cause issues or you intend to push more changes, you **SHOULD** add `[skip ci]` at the end of the commit title. You **SHOULD** omit this tag for final review-ready commits. ## PR procedure + - A PR **SHOULD** contain a single feature. - You **SHOULD** add a line in the relevant section in CHANGELOG.md, listing contributors and the expected PR number. - The PR **MUST** use and follow the nf-core PR template, including the checklist. @@ -119,10 +137,12 @@ This repository has at least 3 git branches: `main` (or `master`), `dev`, and `T - A human can request PR reviews on Slack. ## Agent self-disclosure + - If you generated a majority of the code in a commit, you **MUST** add "Generated by {your name}" at the end of the commit message body. - If you open a PR autonomously, you **MUST** add "Generated by {your name}" at the end of the PR message (above the checklist). ## References + - Nextflow documentation: https://docs.seqera.io/nextflow - nf-core tools documentation: https://nf-co.re/docs/nf-core-tools/ - nf-test documentation: https://www.nf-test.com/docs/getting-started/