Skip to content

Add New Parameters for User-input Filters#35

Open
yz533cb wants to merge 2 commits into
devfrom
new_parameters
Open

Add New Parameters for User-input Filters#35
yz533cb wants to merge 2 commits into
devfrom
new_parameters

Conversation

@yz533cb
Copy link
Copy Markdown
Collaborator

@yz533cb yz533cb commented May 31, 2026

PR checklist

Closes issue#23

  • This comment contains a description of changes (with reason).
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • CHANGELOG.md is updated.

@sanger-tolsoft
Copy link
Copy Markdown
Contributor

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.5.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@yz533cb yz533cb self-assigned this May 31, 2026
@github-actions
Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit f993f1a

+| ✅ 166 tests passed       |+
#| ❔  28 tests were ignored |#
!| ❗   2 tests had warnings |!
Details

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 0.2.0
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline

❔ Tests ignored:

  • files_exist - File is ignored: conf/igenomes.config
  • files_exist - File is ignored: conf/igenomes_ignored.config
  • files_exist - File is ignored: conf/igenomes.config
  • files_exist - File is ignored: conf/igenomes_ignored.config
  • files_exist - File is ignored: CODE_OF_CONDUCT.md
  • files_exist - File is ignored: assets/nf-core-variantcomposition_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-variantcomposition_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-variantcomposition_logo_dark.png
  • files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • nextflow_config - Config variable ignored: manifest.name
  • nextflow_config - Config variable ignored: manifest.homePage
  • files_unchanged - File ignored due to lint config: .gitattributes
  • files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
  • files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
  • files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
  • files_unchanged - File ignored due to lint config: assets/nf-core-variantcomposition_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-variantcomposition_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-variantcomposition_logo_dark.png
  • files_unchanged - File ignored due to lint config: docs/README.md
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/variantcomposition/variantcomposition/.github/workflows/awstest.yml

✅ Tests passed:

Run details

  • nf-core/tools version 3.5.2
  • Run at 2026-05-31 10:00:02

Copy link
Copy Markdown
Member

@muffato muffato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so you've introduced distinct parameters for all steps. I wasn't expecting that. I was expecting a single vcf_filter that would be reused across all steps.
BUT I have to admit I didn't think of it, didn't check if that was possible in vcftools, etc. Let's check.

Remember when we talked to Faculty they mentioned the pipeline shouldn't be called on all variants. Some filtering should happen. And the filtering is sort-of specific to the dataset, so can't be universally set in the pipeline, hence having some visible filter parameters. I don't think they said that the same filter was to be applied to all analyses. That may be me mis-assuming things. Do you recall ?

My idea was to allow someone adding a filter in the lines of "exclude chromosome 4", "only select calls with quality 20 or higher", etc, once, and it'd be applied by all variantcomposition analyses.
First of all: is it even possible ? Are such filters applicable to all the vcftools commands ? If each command takes different arguments, then yeah, need 1 pipeline parameter per analysis.

And then, taking an alternative point of view, if someone wants all steps of the pipeline to filter the VCF the same way, they might as well pre-filter it themselves once for all (which is also more efficient !)
And in that case, having 1 parameter per step is also the right move.

Comment thread CHANGELOG.md
Comment on lines +20 to +24
| --input | |
| --include_positions | |
| --exclude_positions | |
| --snp_density_window | |
| --roh_threshold | |
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These parameters still exist

Suggested change
| --input | |
| --include_positions | |
| --exclude_positions | |
| --snp_density_window | |
| --roh_threshold | |

(+ prettier to reformat the table)

Comment thread docs/usage.md
- To BCFtools RoH via the `--roh_filter` option

Note that you will need to add a leading whitespace in front of `--`,
otherwise the pipeline's own parameter validation will consider it a sanger-tol/variantcalling option.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
otherwise the pipeline's own parameter validation will consider it a sanger-tol/variantcalling option.
otherwise the pipeline's own parameter validation will consider it a sanger-tol/variantcomposition option.

Comment thread conf/modules.config

withName: '.*:FEATURES:VCFTOOLS_HET' {
ext.args = '--het'
ext.args = { "--het ${params.het_filter ?: ''}" }
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if you don't put ?: '' ? Do you get null inserted ? If so, what about setting the default values to '' (and marking them as required in nextflow_schema.json to disallow null) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose filtering options

3 participants