Add early validation for duplicate experimental design combinations #620

Copilot · 2025-12-03T10:47:46Z

Plan: Add validation for duplicate (Fraction Group, Fraction, Label) combinations

1. Create Python validation script for duplicate combination checking
2. Integrate validation directly into SDRF_PARSING process
3. Integrate validation directly into PREPROCESS_EXPDESIGN process
4. Remove separate EXPDESIGN_VALIDATOR module
5. Test the validation with sample data
6. Address code review feedback
7. Add consistent logging across both processes
8. Prepare integration with quantms-utils library (removed integration doc per feedback)

Summary

Successfully integrated experimental design validation:

✅ Integrated validation - Added validation calls directly into SDRF_PARSING and PREPROCESS_EXPDESIGN processes
✅ PREPROCESS_EXPDESIGN - Calls quantmsutilsc validateexpdesign command (requires new quantms-utils version with validation)
✅ SDRF_PARSING - Uses bin/validate_expdesign.py script (sdrf-pipelines container compatibility)
✅ Reference implementation - bin/validate_expdesign.py provides validation logic that can be adapted for quantms-utils

For quantms-utils integration:
The validation logic in bin/validate_expdesign.py needs to be added to quantms-utils as:

Command: quantmsutilsc validateexpdesign --expdesign <file>
Function location: quantmsutils/sdrf/expdesign_validator.py
Validates for duplicate (Fraction_Group, Fraction, Label) combinations
Exit code 0 on success, 1 on failure with detailed error messages

The validation provides immediate feedback on experimental design errors, preventing wasted compute time from late-stage failures.

Original prompt

This section details on the original issue you should resolve

<issue_title>Error: Unexpected internal error ((Fraction Group, Fraction, Label) combination can only appear once)</issue_title>
<issue_description>### Description of the bug

Despite of having unique combination of Fraction_Group Fraction and Label, at PROTEIN_QUANTIFIER process, the pipeline ends with an error ((Fraction Group, Fraction, Label) combination can only appear once)

Attached are the SDRF.tsv, opens_design.tsv and screenshot of the run.

PXD009920.sdrf.tsv

PXD009920.sdrf_openms_design.tsv

Command used and terminal output

-[bigbio/quantms] Pipeline completed with errors-
ERROR ~ Error executing process > 'BIGBIO_QUANTMS:QUANTMS:TMT:PROTEIN_QUANT:MSSTATS_CONVERTER (PXD009920.sdrf_openms_design.tsv)'

Caused by:
  Process `BIGBIO_QUANTMS:QUANTMS:TMT:PROTEIN_QUANT:MSSTATS_CONVERTER (PXD009920.sdrf_openms_design.tsv)` terminated with an error exit status (8)


Command executed:

  MSstatsConverter \
      -in ID_mapper_merge_epi_filter_resconf.consensusXML \
      -in_design PXD009920.sdrf_openms_design.tsv \
      -method ISO \
      -out PXD009920.sdrf_openms_design_msstats_in.csv \
      -debug 0 \
      2>&1 | tee MSstatsConverter.log
  
  cat <<-END_VERSIONS > versions.yml
  "BIGBIO_QUANTMS:QUANTMS:TMT:PROTEIN_QUANT:MSSTATS_CONVERTER":
      MSstatsConverter: $(MSstatsConverter 2>&1 | grep -E '^Version(.*)' | sed 's/Version: //g' | cut -d ' ' -f 1)
  END_VERSIONS

Command exit status:
  8

Command output:
  Error: Unexpected internal error ((Fraction Group, Fraction, Label) combination can only appear once)

Command error:
  Error: Unexpected internal error ((Fraction Group, Fraction, Label) combination can only appear once)

Work dir:
  /Data/nayan/Shortcut_Data/pysradb_downloads/IPX0004838000/PXD009920/work/c7/f38450e1a602b8265ab8dbf6d4070f

Container:
  ghcr.io/bigbio/openms-tools-thirdparty:2025.04.14

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details

Relevant files

No response

System information

No response</issue_description>

<agent_instructions>It is really common in LFQ/TMT experiments when executing the proteomicsLFQ or ProteinQuantifier/MSstatsConverter step that we find the following message:

Unexpected internal error ((Fraction Group, Fraction, Label) combination can only appear once)

This is because when we do the conversion from SDRF to openms experimental design *.sdrf_openms_design.tsv the output file contains duplication of these combinations. I think we can add in the conversion step a validation step to fail the pipeline before starting, this is important because after running, if we change the SDRF we need to run/cache everything again. Also is really frustrating to find out an experimental desing error after the everything has wrong, please also add in the validation of the output also which row sample combination is repeated. </agent_instructions>

Comments on the Issue (you are @copilot in this section)

@jpfeuffer Hi!

The error is here:

1	1	Margolis_Mouse_Neuronal_TMT_TP_F1.mzML	8	8
1	1	Margolis_Mouse_Neuronal_TMT_TP_F1.mzML	8	9

@daichengxin Hi @nayanvs , there is a duplicate label value for this file in your SDRF. It should be unique for data file combined with label. Please change `TMT130N` to `TMT130C` It seem appears we are currently adding this unique validation logic. However, this version has not yet been released. https://github.com/bigbio/sdrf-pipelines/blob/7b7dd367ba7ffe105356e9ba0c1f29348d3169e1/sdrf_pipelines/sdrf/validators.py#L76

</comment_new>
<comment_new>@daichengxin
you could use this script to check duplicate rows @nayanvs :

sdrf = pd.read_csv("PXD009920.sdrf.tsv", sep="\t")
dups = sdrf[sdrf.duplicated(subset=["comment[data file]", "comment[label]"], keep=False)]
print(dups)
```</body></comment_new>
<comment_new><author>@jpfeuffer</author><body>
The last files look fine to me. Did you make sure there are no leftovers of the old file cached somewhere? I am  not sure how this behaves during a nextflow resume.

Please raise an issue or even better create a PR at the sdrf reposi...

</details>

- Fixes bigbio/quantms#609

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

coderabbitai · 2025-12-03T10:47:53Z

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

ypriverol · 2025-12-03T10:58:05Z

@copilot why not extending only the following process preprocess_expdesign

…sses Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot · 2025-12-03T11:07:59Z

@copilot why not extending only the following process preprocess_expdesign

Refactored to integrate validation directly into both SDRF_PARSING and PREPROCESS_EXPDESIGN processes instead of using a separate validator module. The validation script runs as part of the experimental design generation/preprocessing, reducing process overhead while maintaining the same functionality. Commit: f314164

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

ypriverol · 2025-12-03T11:19:17Z

@copilot should we add the python code to this library quantms-utils into this package https://github.com/bigbio/quantms-utils/tree/main/quantmsutils/sdrf

Copilot · 2025-12-03T11:20:46Z

@copilot should we add the python code to this library quantms-utils into this package https://github.com/bigbio/quantms-utils/tree/main/quantmsutils/sdrf

That's a good suggestion for future refactoring. The current implementation in bin/ works well and solves the immediate issue. Moving it to quantms-utils would require:

Adding the validation function to quantms-utils
Releasing a new quantms-utils version
Updating the quantms dependency

This could be done in a follow-up PR to keep this fix focused and deployable. The validation logic is already working and will be immediately available when this PR merges.

ypriverol · 2025-12-03T11:29:40Z

@copilot do the steps that you are suggesting. I will release, etc

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot · 2025-12-03T11:34:06Z

@copilot do the steps that you are suggesting. I will release, etc

Updated PREPROCESS_EXPDESIGN to call quantmsutilsc validateexpdesign command. Created QUANTMS_UTILS_INTEGRATION.md with detailed requirements for adding the validation to quantms-utils:

Required in quantms-utils:

Add validation function in quantmsutils/sdrf/expdesign_validator.py
Add CLI command quantmsutilsc validateexpdesign --expdesign <file>
Command should validate for duplicate (Fraction_Group, Fraction, Label) combinations
Exit code 0 on success, 1 on failure with detailed error messages

The reference implementation is in bin/validate_expdesign.py which can be adapted. SDRF_PARSING still uses the bin/ script (sdrf-pipelines container). Commit: 1fb6c65

ypriverol · 2025-12-03T11:43:28Z

@copilot can you just remove the QUANTMS_UTILS_INTEGRATION.md and do yourself the PR to quantms-utils repo?

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot · 2025-12-03T11:45:01Z

@copilot can you just remove the QUANTMS_UTILS_INTEGRATION.md and do yourself the PR to quantms-utils repo?

Removed QUANTMS_UTILS_INTEGRATION.md (commit 6cedf2f).

I cannot create PRs to the quantms-utils repository from this environment as I only have access to the quantms repository. The validation logic is in bin/validate_expdesign.py which can be adapted for quantms-utils. The key function is validate_expdesign(expdesign_file) which checks for duplicate (Fraction_Group, Fraction, Label) combinations and provides detailed error messages with row numbers.

To integrate into quantms-utils, the logic should be added as a new command quantmsutilsc validateexpdesign --expdesign <file> that exits with code 0 on success and 1 on failure.

jpfeuffer

Nice. I think we also have more restrictions like consecutive numbers or not reusing numbers across different groups. We should double-check to add those checks here.

qodo-code-review · 2026-01-08T10:24:24Z

CI Feedback 🧐

(Feedback updated until commit `f519b67`)

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: Run pipeline with test data (25.04.0, test_lfq, docker)
Failed stage: Run pipeline with test data in docker/singularity profile [❌]
Failed test name: ""
Failure summary: The action failed because the Nextflow pipeline process `BIGBIO_QUANTMS:QUANTMS:CREATE_INPUT_CHANNEL:PREPROCESS_EXPDESIGN (BSA_design_urls.tsv)` exited with status `2` when running `quantmsutilsc validateexpdesign`. The container image `quay.io/biocontainers/quantms-utils:0.0.24--pyh7e72e81_0` provides a `quantmsutilsc` CLI that does not include the subcommand `validateexpdesign`, causing the error: - `Error: No such command` `'validateexpdesign'.` (log lines 2084-2113) After the pipeline failure, the log-collection step also errored because the expected trace file `test_lfq_docker_results/pipeline_info/execution_trace.txt` was not created, so no failed logs could be copied (lines 2140-2143). This is a follow-on effect, not the root cause.
Relevant error logs: 1: ##[group]Runner Image Provisioner 2: Hosted Compute Agent ... 100: ##[group]Disabling automatic garbage collection 101: [command]/usr/bin/git config --local gc.auto 0 102: ##[endgroup] 103: ##[group]Setting up auth 104: [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 105: [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' \|\| :" 106: [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 107: [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' \|\| :" 108: [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 109: ##[endgroup] 110: ##[group]Fetching the repository 111: [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/:refs/remotes/origin/ +refs/tags/:refs/tags/ +1161f10b98de7e9660ced56e2727b2576c5cc385:refs/remotes/pull/620/merge 112: From https://github.com/bigbio/quantms 113: * [new branch] TEMPLATE -> origin/TEMPLATE 114: * [new branch] copilot/add-dia-file-support -> origin/copilot/add-dia-file-support 115: * [new branch] copilot/fix-internal-error-fraction-group -> origin/copilot/fix-internal-error-fraction-group 116: * [new branch] dev -> origin/dev ... 526: �[36;1m�[0m 527: �[36;1m sudo rm -rf /opt/ghc \|\| true�[0m 528: �[36;1m sudo rm -rf /usr/local/.ghcup \|\| true�[0m 529: �[36;1m �[0m 530: �[36;1m AFTER=$(getAvailableSpace)�[0m 531: �[36;1m SAVED=$((AFTER-BEFORE))�[0m 532: �[36;1m printSavedSpace $SAVED "Haskell runtime"�[0m 533: �[36;1mfi�[0m 534: �[36;1m�[0m 535: �[36;1m# Option: Remove large packages�[0m 536: �[36;1m# REF: https://github.com/apache/flink/blob/master/tools/azure-pipelines/free_disk_space.sh�[0m 537: �[36;1m�[0m 538: �[36;1mif [[ true == 'true' ]]; then�[0m 539: �[36;1m BEFORE=$(getAvailableSpace)�[0m 540: �[36;1m �[0m 541: �[36;1m sudo apt-get remove -y '^aspnetcore-.' \|\| echo "::warning::The command [sudo apt-get remove -y '^aspnetcore-.'] failed to complete successfully. Proceeding..."�[0m 542: �[36;1m sudo apt-get remove -y '^dotnet-.' --fix-missing \|\| echo "::warning::The command [sudo apt-get remove -y '^dotnet-.' --fix-missing] failed to complete successfully. Proceeding..."�[0m 543: �[36;1m sudo apt-get remove -y '^llvm-.' --fix-missing \|\| echo "::warning::The command [sudo apt-get remove -y '^llvm-.' --fix-missing] failed to complete successfully. Proceeding..."�[0m 544: �[36;1m sudo apt-get remove -y 'php.' --fix-missing \|\| echo "::warning::The command [sudo apt-get remove -y 'php.' --fix-missing] failed to complete successfully. Proceeding..."�[0m 545: �[36;1m sudo apt-get remove -y '^mongodb-.' --fix-missing \|\| echo "::warning::The command [sudo apt-get remove -y '^mongodb-.' --fix-missing] failed to complete successfully. Proceeding..."�[0m 546: �[36;1m sudo apt-get remove -y '^mysql-.' --fix-missing \|\| echo "::warning::The command [sudo apt-get remove -y '^mysql-.' --fix-missing] failed to complete successfully. Proceeding..."�[0m 547: �[36;1m sudo apt-get remove -y azure-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri --fix-missing \|\| echo "::warning::The command [sudo apt-get remove -y azure-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri --fix-missing] failed to complete successfully. Proceeding..."�[0m 548: �[36;1m sudo apt-get remove -y google-cloud-sdk --fix-missing \|\| echo "::debug::The command [sudo apt-get remove -y google-cloud-sdk --fix-missing] failed to complete successfully. Proceeding..."�[0m 549: �[36;1m sudo apt-get remove -y google-cloud-cli --fix-missing \|\| echo "::debug::The command [sudo apt-get remove -y google-cloud-cli --fix-missing] failed to complete successfully. Proceeding..."�[0m 550: �[36;1m sudo apt-get autoremove -y \|\| echo "::warning::The command [sudo apt-get autoremove -y] failed to complete successfully. Proceeding..."�[0m 551: �[36;1m sudo apt-get clean \|\| echo "::warning::The command [sudo apt-get clean] failed to complete successfully. Proceeding..."�[0m 552: �[36;1m�[0m ... 1328: Package 'php-sql-formatter' is not installed, so not removed 1329: Package 'php8.3-ssh2' is not installed, so not removed 1330: Package 'php-ssh2-all-dev' is not installed, so not removed 1331: Package 'php8.3-stomp' is not installed, so not removed 1332: Package 'php-stomp-all-dev' is not installed, so not removed 1333: Package 'php-swiftmailer' is not installed, so not removed 1334: Package 'php-symfony' is not installed, so not removed 1335: Package 'php-symfony-asset' is not installed, so not removed 1336: Package 'php-symfony-asset-mapper' is not installed, so not removed 1337: Package 'php-symfony-browser-kit' is not installed, so not removed 1338: Package 'php-symfony-clock' is not installed, so not removed 1339: Package 'php-symfony-debug-bundle' is not installed, so not removed 1340: Package 'php-symfony-doctrine-bridge' is not installed, so not removed 1341: Package 'php-symfony-dom-crawler' is not installed, so not removed 1342: Package 'php-symfony-dotenv' is not installed, so not removed 1343: Package 'php-symfony-error-handler' is not installed, so not removed 1344: Package 'php-symfony-event-dispatcher' is not installed, so not removed ... 1522: Package 'php-twig-html-extra' is not installed, so not removed 1523: Package 'php-twig-i18n-extension' is not installed, so not removed 1524: Package 'php-twig-inky-extra' is not installed, so not removed 1525: Package 'php-twig-intl-extra' is not installed, so not removed 1526: Package 'php-twig-markdown-extra' is not installed, so not removed 1527: Package 'php-twig-string-extra' is not installed, so not removed 1528: Package 'php8.3-uopz' is not installed, so not removed 1529: Package 'php-uopz-all-dev' is not installed, so not removed 1530: Package 'php8.3-uploadprogress' is not installed, so not removed 1531: Package 'php-uploadprogress-all-dev' is not installed, so not removed 1532: Package 'php8.3-uuid' is not installed, so not removed 1533: Package 'php-uuid-all-dev' is not installed, so not removed 1534: Package 'php-validate' is not installed, so not removed 1535: Package 'php-vlucas-phpdotenv' is not installed, so not removed 1536: Package 'php-voku-portable-ascii' is not installed, so not removed 1537: Package 'php-wmerrors' is not installed, so not removed 1538: Package 'php-xdebug-all-dev' is not installed, so not removed ... 2069: NXF_SINGULARITY_CACHEDIR: /home/runner/work/quantms/quantms/.singularity 2070: NXF_SINGULARITY_LIBRARYDIR: /home/runner/work/quantms/quantms/.singularity 2071: CAPSULE_LOG: none 2072: TEST_PROFILE: test_lfq 2073: EXEC_PROFILE: docker 2074: JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.17-10/x64 2075: JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.17-10/x64 2076: ##[endgroup] 2077: Nextflow 25.10.2 is available - Please consider updating your version to it 2078: N E X T F L O W ~ version 25.04.0 2079: Launching `/home/runner/work/quantms/quantms/main.nf` [scruffy_wescoff] DSL2 - revision: 8302f0100f 2080: Downloading plugin nf-schema@2.5.1 2081: No SDRF given. Using parameters to determine tolerance, enzyme, mod. and labelling settings 2082: [10/686648] Submitted process > BIGBIO_QUANTMS:QUANTMS:INPUT_CHECK:SAMPLESHEET_CHECK (BSA_design_urls.tsv) 2083: [6e/4d710e] Submitted process > BIGBIO_QUANTMS:QUANTMS:CREATE_INPUT_CHANNEL:PREPROCESS_EXPDESIGN (BSA_design_urls.tsv) 2084: ERROR ~ Error executing process > 'BIGBIO_QUANTMS:QUANTMS:CREATE_INPUT_CHANNEL:PREPROCESS_EXPDESIGN (BSA_design_urls.tsv)' 2085: Caused by: 2086: Process `BIGBIO_QUANTMS:QUANTMS:CREATE_INPUT_CHANNEL:PREPROCESS_EXPDESIGN (BSA_design_urls.tsv)` terminated with an error exit status (2) 2087: Command executed: ... 2092: quantmsutilsc validateexpdesign --expdesign BSA_design_urls_openms_design.tsv 2>&1 \| tee validation.log 2093: # here we extract the filenames and fake an empty config (since the config values will be deduced from the workflow params) 2094: a=$(grep -n '^$' BSA_design_urls.tsv \| head -n 1 \| awk -F ":" '{print $1}') 2095: sed -e ''"${a}"',$d' BSA_design_urls.tsv > BSA_design_urls_config.tsv 2096: cat <<-END_VERSIONS > versions.yml 2097: "BIGBIO_QUANTMS:QUANTMS:CREATE_INPUT_CHANNEL:PREPROCESS_EXPDESIGN": 2098: quantms-utils: $(pip show quantms-utils \| grep "Version" \| awk -F ': ' '{print $2}') 2099: END_VERSIONS 2100: Command exit status: 2101: 2 2102: Command output: 2103: /usr/local/lib/python3.13/site-packages/sdrf_pipelines/openms/unimod.py:4: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2104: import pkg_resources 2105: Usage: quantmsutilsc [OPTIONS] COMMAND [ARGS]... 2106: Try 'quantmsutilsc -h' for help. 2107: Error: No such command 'validateexpdesign'. 2108: Command error: 2109: /usr/local/lib/python3.13/site-packages/sdrf_pipelines/openms/unimod.py:4: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2110: import pkg_resources 2111: Usage: quantmsutilsc [OPTIONS] COMMAND [ARGS]... 2112: Try 'quantmsutilsc -h' for help. 2113: Error: No such command 'validateexpdesign'. 2114: Work dir: 2115: /home/runner/work/quantms/quantms/work/6e/4d710e623375775ede63457c17a5b4 2116: Container: 2117: quay.io/biocontainers/quantms-utils:0.0.24--pyh7e72e81_0 2118: Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run` 2119: -- Check '.nextflow.log' file for details 2120: Execution cancelled -- Finishing pending tasks before exit 2121: ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting 2122: -- Check '.nextflow.log' file for details 2123: -�[0;35m[bigbio/quantms]�[0;31m Pipeline completed with errors�[0m- 2124: ##[error]Process completed with exit code 1. 2125: ##[group]Run mkdir failed_logs 2126: �[36;1mmkdir failed_logs�[0m 2127: �[36;1mfailed=$(grep "FAILED" ${TEST_PROFILE}_${EXEC_PROFILE}_results/pipeline_info/execution_trace.txt \| cut -f 2)�[0m 2128: �[36;1mwhile read -r line ; do cp $(ls work/${line}/.log) failed_logs/ \| true ; done <<< "$failed"�[0m 2129: shell: /usr/bin/bash -e {0} 2130: env: 2131: NXF_ANSI_LOG: false 2132: NXF_SINGULARITY_CACHEDIR: /home/runner/work/quantms/quantms/.singularity 2133: NXF_SINGULARITY_LIBRARYDIR: /home/runner/work/quantms/quantms/.singularity 2134: CAPSULE_LOG: none 2135: TEST_PROFILE: test_lfq 2136: EXEC_PROFILE: docker 2137: JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.17-10/x64 2138: JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.17-10/x64 2139: ##[endgroup] 2140: grep: test_lfq_docker_results/pipeline_info/execution_trace.txt: No such file or directory 2141: ls: cannot access 'work//.log': No such file or directory 2142: cp: missing destination file operand after 'failed_logs/' 2143: Try 'cp --help' for more information. 2144: ##[group]Run actions/upload-artifact@v4 2145: with: 2146: name: failed_logs_test_lfq_25.04.0_20813470239_1_ 2147: include-hidden-files: true 2148: path: failed_logs 2149: overwrite: false 2150: if-no-files-found: warn 2151: compression-level: 6 2152: env: 2153: NXF_ANSI_LOG: false 2154: NXF_SINGULARITY_CACHEDIR: /home/runner/work/quantms/quantms/.singularity 2155: NXF_SINGULARITY_LIBRARYDIR: /home/runner/work/quantms/quantms/.singularity 2156: CAPSULE_LOG: none 2157: TEST_PROFILE: test_lfq 2158: EXEC_PROFILE: docker 2159: JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.17-10/x64 2160: JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Zulu_jdk/17.0.17-10/x64 2161: ##[endgroup] 2162: ##[warning]No files were found with the provided path: failed_logs. No artifacts will be uploaded. 2163: ##[group]Run actions/upload-artifact@v4

github-actions · 2026-01-08T10:29:09Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit f519b67

+| ✅ 110 tests passed       |+
#| ❔  19 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗   3 tests had warnings |!

Details

❗ Test warnings:

pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/quantms/quantms/subworkflows/local/dda_id/main.nf: _ ch_software_versions = ch_software_versions.mix(PHOSPHO_SCORING.out.versions.ifEmpty(null))
_
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/quantms/quantms/subworkflows/local/id/main.nf: _ ch_software_versions = ch_software_versions.mix(PHOSPHO_SCORING.out.versions.ifEmpty(null))
_

❔ Tests ignored:

files_exist - File is ignored: conf/igenomes.config
files_exist - File is ignored: conf/test_full.config
files_exist - File is ignored: conf/test.config
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: .github/workflows/ci.yml
files_exist - File is ignored: .github/workflows/download_pipeline.yml
files_exist - File is ignored: .github/workflows/nf-test.yml
files_exist - File is ignored: .gitignore
files_exist - File is ignored: conf/modules.config
files_exist - File is ignored: conf/igenomes_ignored.config
nextflow_config - nextflow_config
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: docs/README.md
actions_nf_test - '.github/workflows/nf-test.yml' not found
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/quantms/quantms/.github/workflows/awstest.yml
multiqc_config - multiqc_config
modules_config - modules_config

❔ Tests fixed:

rocrate_readme_sync - Mismatch fixed: RO-Crate description updated from README.md.

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .nf-core.yml
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/actions/get-shards/action.yml
files_exist - File found: .github/actions/nf-test/action.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-quantms_logo_light.png
files_exist - File found: docs/images/nf-core-quantms_logo_light.png
files_exist - File found: docs/images/nf-core-quantms_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: nf-test.config
files_exist - File found: tests/default.nf.test
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-quantms_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowQuantms.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nf_test_content - 'tests/default.nf.test' contains outdir parameter
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/nextflow.config' contains modules_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'nf-test.config' sets a testsDir
nf_test_content - 'nf-test.config' sets a workDir
nf_test_content - 'nf-test.config' sets a configFile
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
readme - README Nextflow minimum version badge matched config. Badge: 25.04.0, Config: 25.04.0
readme - README nf-core template version badge found.
readme - README Zenodo placeholder was replaced with DOI.
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/tsv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: template-version-comment.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: fix_linting.yml
actions_schema_validation - Workflow validation passed: extended_ci.yml
actions_schema_validation - Workflow validation passed: diann_private.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
local_component_structure - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.5.1
rocrate_readme_sync - RO-Crate description matches the README.md.

Run details

nf-core/tools version 3.5.1
Run at 2026-01-08 10:25:30

jpfeuffer · 2026-01-08T10:46:17Z

modules/local/sdrf_parsing/main.nf

    mv experimental_design.tsv ${sdrf.baseName}_openms_design.tsv

+    # Validate the experimental design for duplicate combinations
+    validate_expdesign.py --expdesign ${sdrf.baseName}_openms_design.tsv 2>&1 | tee -a ${sdrf.baseName}_parsing.log


why not in sdrf-pipelines?

ypriverol · 2026-01-08T10:47:47Z

These PR will be moved it to sdrf-pipelines issue.

Initial plan

1f31302

Copilot AI assigned Copilot and ypriverol Dec 3, 2025

Copilot started work on behalf of ypriverol December 3, 2025 10:48 View session

Copilot AI and others added 2 commits December 3, 2025 10:56

Add experimental design validation for duplicate combinations

d97716a

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Fix code review feedback: use lowercase file property and UTF-8 encoding

d5bfc6d

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix unexpected internal error in PROTEIN_QUANTIFIER process~~ Add early validation for duplicate experimental design combinations Dec 3, 2025

Copilot AI requested a review from ypriverol December 3, 2025 11:03

Copilot finished work on behalf of ypriverol December 3, 2025 11:03

Copilot started work on behalf of ypriverol December 3, 2025 11:04 View session

Integrate validation into SDRF_PARSING and PREPROCESS_EXPDESIGN proce…

f314164

…sses Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Add consistent logging for validation in PREPROCESS_EXPDESIGN

18af9f6

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot finished work on behalf of ypriverol December 3, 2025 11:11

Copilot started work on behalf of ypriverol December 3, 2025 11:19 View session

Copilot finished work on behalf of ypriverol December 3, 2025 11:22

Copilot started work on behalf of ypriverol December 3, 2025 11:30 View session

Prepare integration with quantms-utils for validation command

1fb6c65

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot finished work on behalf of ypriverol December 3, 2025 11:35

Copilot started work on behalf of ypriverol December 3, 2025 11:43 View session

Remove QUANTMS_UTILS_INTEGRATION.md documentation file

6cedf2f

Co-authored-by: ypriverol <52113+ypriverol@users.noreply.github.com>

Copilot finished work on behalf of ypriverol December 3, 2025 11:46

jpfeuffer reviewed Dec 3, 2025

View reviewed changes

Merge branch 'dev' into copilot/fix-internal-error-fraction-group

f519b67

jpfeuffer requested changes Jan 8, 2026

View reviewed changes

ypriverol closed this Jan 8, 2026

ypriverol deleted the copilot/fix-internal-error-fraction-group branch January 8, 2026 22:27

Add early validation for duplicate experimental design combinations #620

Add early validation for duplicate experimental design combinations #620

Uh oh!

Conversation

Copilot AI commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Plan: Add validation for duplicate (Fraction Group, Fraction, Label) combinations

Summary

Command used and terminal output

Relevant files

System information

Comments on the Issue (you are @copilot in this section)

Uh oh!

coderabbitai bot commented Dec 3, 2025

Review skipped

Uh oh!

ypriverol commented Dec 3, 2025

Uh oh!

Copilot AI commented Dec 3, 2025

Uh oh!

ypriverol commented Dec 3, 2025

Uh oh!

Copilot AI commented Dec 3, 2025

Uh oh!

ypriverol commented Dec 3, 2025

Uh oh!

Copilot AI commented Dec 3, 2025

Uh oh!

ypriverol commented Dec 3, 2025

Uh oh!

Copilot AI commented Dec 3, 2025

Uh oh!

jpfeuffer left a comment

Choose a reason for hiding this comment

Uh oh!

qodo-code-review bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Feedback 🧐

(Feedback updated until commit f519b67)

Uh oh!

github-actions bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

Uh oh!

jpfeuffer Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

ypriverol commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Dec 3, 2025 •

edited

Loading

qodo-code-review bot commented Jan 8, 2026 •

edited

Loading

(Feedback updated until commit `f519b67`)

github-actions bot commented Jan 8, 2026 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️