[FEAT] MEDIC distortion correction via warpkit#541
Open
vanandrew wants to merge 48 commits into
Open
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #541 +/- ##
==========================================
+ Coverage 84.09% 84.41% +0.31%
==========================================
Files 30 33 +3
Lines 2880 3086 +206
Branches 383 424 +41
==========================================
+ Hits 2422 2605 +183
- Misses 382 408 +26
+ Partials 76 73 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
vanandrew
force-pushed
the
feat/medic-warpkit
branch
from
57ccc47 to
21ab1c6
Compare
vanandrew
changed the title
vanandrew
force-pushed
the
feat/medic-warpkit
branch
6 times, most recently
from
bcaf4ad to
9ecfbae
Compare
vanandrew
added a commit
to vanandrew/sdcflows
that referenced
this pull request
May 14, 2026
Three additions targeting the codecov/patch failure on nipreps#541: * sdcflows/interfaces/tests/test_warpkit.py (new) — instantiates every warpkit-backed interface so its spec class body is hit at import (was 0% on codecov despite running locally; likely an xdist worker-merge artefact). Also covers ``_as_str_list``, the ``_pkg`` invariant, and the ``border_filt=(1, 5)`` traits default regression. * sdcflows/workflows/tests/test_outputs.py (new) — direct construction test for ``init_fmap_derivatives_wf``, exercising both the default and ``write_dynamic=True`` paths. The MEDIC dynamic sinks branch was only reached transitively from the ``test_fmap_wf`` slow test, hence 0% patch coverage on outputs.py. * sdcflows/workflows/fit/tests/test_medic.py — fill the remaining helper gaps: empty-metadata rejection, ``sloppy=True`` zooms_min, ``_first``, and ``_temporal_mean`` for both 3D and 4D inputs. The ``_run_interface`` method bodies in interfaces/warpkit.py remain uncovered in the fast/slow envs since they require warpkit to import; the existing ``veryslow`` MEDIC fixtures exercise them when the optional dependency is installed.
Author
|
I have no idea why test (3.13, pre, fast) is failing on the cache step. It looks like an out-of-space error on the runner (but the other jobs with the same steps succeeded, so very confused): |
Contributor
|
@vanandrew I think someone must have rerun that failing job and now it's passing. |
tsalo
requested changes
May 15, 2026
tsalo
left a comment
tsalo
left a comment
Contributor
There was a problem hiding this comment.
Thanks for this! I made a quick pass through the code.
tsalo
reviewed
May 15, 2026
Author
|
Pushed four commits addressing your feedback (noise_frames, ctor inputs, nitransforms switch + interface cleanup, unified fmap output). |
Author
|
I can't tell if these test failures are something wrong with my dataset additions or if it's just another one-off error: |
vanandrew
force-pushed
the
feat/medic-warpkit
branch
from
a4c4781 to
b070878
Compare
Author
|
Looks like the CI keeps failing because it's running out of space (I added two datasets for tests so that might have pushed it over the edge). I pushed 6f5cfff2 adding a - name: Free disk space
uses: jlumbroso/free-disk-space@main
with:
tool-cache: false
android: true
dotnet: true
haskell: true
large-packages: false
swap-storage: falseThis gets rid of some unneeded development tools on the github runner to free room for data. |
Author
|
@tsalo Alright. I made changes based on our convo today:
|
tsalo
reviewed
Jun 2, 2026
tsalo
reviewed
Jun 2, 2026
Comment on lines
+82
to
+83
| # DO NOT add it to the `all` alias below — the default install must remain | ||
| # clean-Apache. |
Contributor
There was a problem hiding this comment.
I'm not sure this is necessary. A number of SDCFlows' dependencies do not allow commercial use. I think we need to review and document restrictively licensed dependencies rather than put them in extra groups. @effigies wdyt?
Contributor
There was a problem hiding this comment.
I agree, IMO it's fine to include in the normal dependencies - though up to @vanandrew how available they would like it
Author
There was a problem hiding this comment.
Moved warpkit into the core dependencies (0092e43) rather than an extras group, per the consensus here. The non-commercial WUSTL license is now documented in the init_medic_wf module docstring instead of the dependency list.
vanandrew
added a commit
to vanandrew/sdcflows
that referenced
this pull request
Jun 2, 2026
Three additions targeting the codecov/patch failure on nipreps#541: * sdcflows/interfaces/tests/test_warpkit.py (new) — instantiates every warpkit-backed interface so its spec class body is hit at import (was 0% on codecov despite running locally; likely an xdist worker-merge artefact). Also covers ``_as_str_list``, the ``_pkg`` invariant, and the ``border_filt=(1, 5)`` traits default regression. * sdcflows/workflows/tests/test_outputs.py (new) — direct construction test for ``init_fmap_derivatives_wf``, exercising both the default and ``write_dynamic=True`` paths. The MEDIC dynamic sinks branch was only reached transitively from the ``test_fmap_wf`` slow test, hence 0% patch coverage on outputs.py. * sdcflows/workflows/fit/tests/test_medic.py — fill the remaining helper gaps: empty-metadata rejection, ``sloppy=True`` zooms_min, ``_first``, and ``_temporal_mean`` for both 3D and 4D inputs. The ``_run_interface`` method bodies in interfaces/warpkit.py remain uncovered in the fast/slow envs since they require warpkit to import; the existing ``veryslow`` MEDIC fixtures exercise them when the optional dependency is installed.
vanandrew
force-pushed
the
feat/medic-warpkit
branch
from
da9d07b to
c55efd0
Compare
A revival of nipreps#435. Wraps warpkit (>=1.2.1) — which now ships pre-compiled PyPI wheels — instead of reimplementing MEDIC in pure Python. - sdcflows/interfaces/warpkit.py: seven SimpleInterface classes wrapping warpkit.api (MEDIC, UnwrapPhase, ComputeFieldmap, ApplyWarp, ConvertWarp, ConvertFieldmap, ComputeJacobian). - sdcflows/fieldmaps.py: new EstimatorType.MEDIC with auto-detection in FieldmapEstimation.__attrs_post_init__ for bold/epi/sbref sources tagged with the BIDS part-{phase,mag} entity. PEPOLAR branch gated on UNKNOWN method to avoid clobbering MEDIC. - pyproject.toml: warpkit added as an optional extra (sdcflows[warpkit]).
for more information, see https://pre-commit.ci
- New sdcflows/workflows/fit/medic.py: init_medic_wf using the two-stage warpkit Python API (UnwrapPhase -> ComputeFieldmap) so per-frame masks surface as a real output. Module-load pure: warpkit is only resolved at run time via the LibraryBaseInterface guard. - Outputs: static (fmap/fmap_ref/fmap_mask/fmap_coeff) for compatibility with init_unwarp_wf, plus dynamic (fmap_dynamic/fmap_dynamic_ref/ fmap_dynamic_mask) carrying the 4D per-frame data for a future MEDIC-aware apply path. fmap_coeff is documented as a structural shim (init_unwarp_wf only consumes B-spline coefficients); the spline fit adds no scientific value for MEDIC since the field is already on the EPI grid and warpkit's SVD filter already smooths. - Wrangler picks up multi-echo phase BOLDs via IntendedFor as MEDIC estimators; single-PE EPI fallback also accepts the 'bold' suffix. Drops the implicit part=mag base filter so phase data surfaces. - fieldmaps.py: get_workflow() now dispatches EstimatorType.MEDIC to init_medic_wf; sources are sorted by EchoTime before wiring. - workflows/base.py: registers MEDIC in the INPUT_FIELDS dispatch dict. - pyproject.toml: comment block on the warpkit extra spelling out the WUSTL non-commercial license terms; keeps the extra out of the `all` alias. - Tests: workflow construction smoke + metadata-helper unit tests always run; slow integration test guarded by pytest.importorskip on warpkit. MEDIC parametrize entries added to test_wrangler.
Auto-discover MEDIC estimators from complex multi-echo BOLD even when neither ``IntendedFor`` nor ``B0FieldIdentifier`` is set, gated behind an opt-in ``force_medic`` flag for public datasets that ship the required mag+phase echoes without the metadata the default path needs.
Three additions targeting the codecov/patch failure on nipreps#541: * sdcflows/interfaces/tests/test_warpkit.py (new) — instantiates every warpkit-backed interface so its spec class body is hit at import (was 0% on codecov despite running locally; likely an xdist worker-merge artefact). Also covers ``_as_str_list``, the ``_pkg`` invariant, and the ``border_filt=(1, 5)`` traits default regression. * sdcflows/workflows/tests/test_outputs.py (new) — direct construction test for ``init_fmap_derivatives_wf``, exercising both the default and ``write_dynamic=True`` paths. The MEDIC dynamic sinks branch was only reached transitively from the ``test_fmap_wf`` slow test, hence 0% patch coverage on outputs.py. * sdcflows/workflows/fit/tests/test_medic.py — fill the remaining helper gaps: empty-metadata rejection, ``sloppy=True`` zooms_min, ``_first``, and ``_temporal_mean`` for both 3D and 4D inputs. The ``_run_interface`` method bodies in interfaces/warpkit.py remain uncovered in the fast/slow envs since they require warpkit to import; the existing ``veryslow`` MEDIC fixtures exercise them when the optional dependency is installed.
BIDS keeps trailing noise frames in separate noRF files, so SDCFlows shouldn't carry an interface input for them — let warpkit's default of zero apply.
Passing n_cpus and debug through the interface constructor keeps node construction in one place and removes the post-init mutation pattern.
…sforms The dynamic-fieldmap apply workflow was the only consumer of ``ApplyWarp``/``ConvertFieldmap`` in warpkit; rewrite it on top of ``sdcflows.transform.apply_dynamic_unwarp``, a per-frame extension of the existing scipy/nitransforms-backed resampling that powers the static path. With that internal consumer gone, the warpkit interface module shrinks to just the two stages SDCFlows actually runs (``UnwrapPhase``, ``ComputeFieldmap``); the unused ``MEDIC``, ``ApplyWarp``, ``ConvertWarp``, ``ConvertFieldmap`` and ``ComputeJacobian`` wrappers, along with the helpers that only served them, are removed.
``init_medic_wf`` previously emitted both a 3D static ``fmap`` (a B-spline-smoothed temporal mean of the dynamic field) and a separate 4D ``fmap_dynamic``, alongside parallel ``fmap_dynamic_ref`` and ``fmap_dynamic_mask`` plumbing in the base preproc workflow and a ``write_dynamic`` branch in the derivatives sinks. Per review, collapse all of that to a single ``fmap`` output whose dimensionality (3D for static estimators, 4D for MEDIC) tells consumers which apply path to use. The B-spline shim is gone (MEDIC's field is already on the EPI grid), the derivatives sink uses ``MergeSeries(allow_4D=True)`` to pass 3D and 4D fmaps through the same path, and the per-volume apply workflow renames its input from ``fmap_dynamic`` to ``fmap`` to match. Downstream tools that don't yet handle 4D field maps must block MEDIC-based estimators until they do.
The static init_magnitude_wf averaged across time before brain-extracting, which collapsed MEDIC's per-volume magnitude into one 3D ref/mask. The new init_dynamic_magnitude_wf splits the 4D first-echo magnitude, runs the IntensityClip/N4/BrainExtraction chain per frame, and re-concatenates so fmap_ref and fmap_mask are 4D and track the per-volume fieldmap.
…nt accepted kwargs Adds three regression tests around the MEDIC guard clauses in ``FieldmapEstimation.__attrs_post_init__`` (single-part input, single-echo pair, mismatched mag/phase counts), and documents the two ``init_medic_wf`` parameters (``use_metadata_estimates``, ``fallback_total_readout_time``) that are accepted via ``init_fmap_preproc_wf`` for parity but currently ignored.
…ation.is_dynamic Replaces the ``estimator.method == EstimatorType.MEDIC`` check in ``init_fmap_preproc_wf`` with a new ``FieldmapEstimation.is_dynamic`` property backed by a module-level ``_DYNAMIC_METHODS`` set in ``sdcflows.fieldmaps``. Future per-volume estimators (DOCMA, TOAST, ...) only need to be added to ``_DYNAMIC_METHODS`` — the shared workflow no longer carries per-method branching.
Adds ``test_wrangler_medic_trigger`` covering the three modes of MEDIC discovery against synthetic BIDS skeletons in a single parametrized test: 1. ``default-IntendedFor`` — sidecars carry ``IntendedFor`` and the default discovery path picks them up. 2. ``force_medic`` — sidecars carry no metadata; the explicit flag short-circuits the default ``IntendedFor`` gate. 3. ``baseline-no-trigger`` — no metadata, no override, no fmapless; MEDIC must refuse to fire so runs without expected metadata are not silently picked up. The existing ``test_wrangler_force_medic_without_intended_for`` covers cases 2 and 3 implicitly via ``medic_no_intended_for``; this new test pins all three side-by-side and was validated against a real ds006926/sub-a01 layout before being formalized here.
The bumped data-cache-v3 (now includes ds006926 + ds007637) tips the 3.10/min/fast lane over the 14 GB root-disk limit during cache restore. Free ~20 GB up front by purging the Android SDK, .NET, and Haskell tool-caches; keep the tool-cache, swap, and apt large-packages alone so later steps (apt-cache restore, conda, uv) aren't disturbed.
…arpkit masks The MEDIC workflow now exposes the first-echo magnitude series untouched as ``fmap_ref`` (the ``pick_mag1`` output) and routes warpkit's per-frame ``UnwrapPhase`` masks straight to ``fmap_mask``, instead of running a per-frame N4 + skull-strip pass to synthesize both. ``init_dynamic_magnitude_wf`` was only used by MEDIC, so it is removed from ``fit/fieldmap.py`` entirely. The expensive per-volume MapNode work (N4, intensity clip, brain extraction over every frame) is gone with it. Updates ``test_medic_construct`` to drop the removed ``magnitude_wf`` node.
…DIC-first ordering Make MEDIC discovery follow the BIDS fieldmap-intent model rather than file structure: * MEDIC is now discovered only from declared intent metadata -- a complex multi-echo BOLD is picked up via its ``B0FieldIdentifier`` (the self-referential pattern BIDS endorses for images that estimate their own B0 field, as in pepolar) through the existing Step 1 path, or via legacy ``IntendedFor`` in the dedicated MEDIC block. The structure-only auto-discovery (and the ``force_medic`` flag that enabled it) is removed: part-mag/part-phase alone no longer triggers MEDIC. * Add ``no_medic`` to disable MEDIC discovery via either route (kwarg + ``--no-medic`` CLI flag + ``config.workflow.no_medic``); it also skips a MEDIC-shaped ``B0FieldIdentifier`` group in Step 1. Return estimators in a deterministic order: ``estimators.sort(key=lambda e: (not e.is_dynamic, e.bids_id))``. Step 1 iterates a ``set`` of ``B0FieldIdentifier``s, so the prior order was hash-seed dependent. The sort also encodes one intentional, documented priority -- dynamic (MEDIC) estimators come first, so a consumer selecting the first applicable estimator per target prefers MEDIC over a coexisting static fieldmap. Fieldmap-less ANAT estimators are appended afterwards and stay last. Tests: cover the B0FieldIdentifier and IntendedFor MEDIC routes, a guard that structure alone does not fire MEDIC, and that MEDIC sorts ahead of a coexisting PEPOLAR estimator.
vanandrew
force-pushed
the
feat/medic-warpkit
branch
from
c55efd0 to
b0a6904
Compare
Author
|
@tsalo I did a rebase to the latest main, hopefully to quash the 3.13 test error that happened in the last CI run: https://github.com/nipreps/sdcflows/actions/runs/26792754492/job/79155816998#step:22:320 |
mgxd
reviewed
Jun 10, 2026
mgxd
left a comment
mgxd
left a comment
Contributor
There was a problem hiding this comment.
Thanks for pushing on this @vanandrew! I gave it a first pass and left some comments / questions, but overall the implementation looks reasonable.
While waiting for this to get more feedback, we should probably coordinate with nipreps/fmriprep#3645 (cc @DVSneuro) to sync up and use this branch, so we can have a branch that emulates how integration will actually look.
Comment on lines
+601
to
+746
| async def _dynamic_unwarp_parallel( | ||
| fulldataset: np.ndarray, | ||
| coordinates: np.ndarray, | ||
| fmap_dynamic: np.ndarray, | ||
| pe_info: Sequence[tuple[int, float]], | ||
| jacobian: bool, | ||
| order: int = 3, | ||
| mode: str = 'constant', | ||
| cval: float = 0.0, | ||
| prefilter: bool = True, | ||
| output_dtype: str | np.dtype | None = None, | ||
| max_concurrent: int = min(os.cpu_count(), 12), | ||
| ) -> np.ndarray: | ||
| """Per-volume unwarp where each EPI frame uses its matching fmap frame.""" | ||
| semaphore = asyncio.Semaphore(max_concurrent) | ||
| if fulldataset.ndim == 3: | ||
| fulldataset = fulldataset[..., np.newaxis] | ||
|
|
||
| tasks = [] | ||
| for volid, volume in enumerate(np.rollaxis(fulldataset, -1, 0)): | ||
| func = partial( | ||
| _sdc_unwarp, | ||
| jacobian=jacobian, | ||
| fmap_hz=fmap_dynamic[..., volid], | ||
| output_dtype=output_dtype, | ||
| order=order, | ||
| mode=mode, | ||
| cval=cval, | ||
| prefilter=prefilter, | ||
| ) | ||
| tasks.append( | ||
| asyncio.create_task( | ||
| worker( | ||
| volume, | ||
| coordinates.copy(), | ||
| pe_info[volid], | ||
| None, | ||
| func, | ||
| semaphore, | ||
| ) | ||
| ) | ||
| ) | ||
|
|
||
| await asyncio.gather(*tasks) | ||
| return np.stack([t.result() for t in tasks], -1) | ||
|
|
||
|
|
||
| def apply_dynamic_unwarp( | ||
| moving, | ||
| fmap_dynamic, | ||
| pe_dir, | ||
| ro_time, | ||
| jacobian: bool = True, | ||
| order: int = 3, | ||
| mode: str = 'constant', | ||
| cval: float = 0.0, | ||
| prefilter: bool = True, | ||
| output_dtype: str | np.dtype | None = None, | ||
| num_threads: int | None = None, | ||
| allow_negative: bool = False, | ||
| ): | ||
| r"""Apply a per-frame 4D Hz fieldmap to unwarp a 4D EPI series. | ||
|
|
||
| Unlike :class:`B0FieldTransform`, the fieldmap is assumed to already be on | ||
| the EPI grid (one Hz volume per EPI volume), so no B-spline reconstruction | ||
| or coregistration takes place. Each EPI volume is resampled through its | ||
| matching fieldmap frame using the same scipy-backed primitives that the | ||
| static apply path uses (:func:`_sdc_unwarp`). | ||
|
|
||
| Parameters | ||
| ---------- | ||
| moving : :obj:`str` or :class:`~nibabel.spatialimages.SpatialImage` | ||
| 4D EPI image to unwarp. | ||
| fmap_dynamic : :obj:`str` or :class:`~nibabel.spatialimages.SpatialImage` | ||
| 4D Hz fieldmap, one volume per ``moving`` frame, on ``moving``'s grid. | ||
| pe_dir : :obj:`str` or list of :obj:`str` | ||
| ``PhaseEncodingDirection`` metadata value(s). A scalar is broadcast | ||
| across frames. | ||
| ro_time : :obj:`float` or list of :obj:`float` | ||
| Total readout time(s) in seconds. A scalar is broadcast across frames. | ||
| jacobian : :obj:`bool` | ||
| Apply Jacobian determinant correction after resampling. | ||
| num_threads : :obj:`int`, optional | ||
| Cap on parallel volume resamplings. | ||
| """ | ||
| if isinstance(moving, (str, bytes, Path)): | ||
| moving = nb.load(moving) | ||
| if isinstance(fmap_dynamic, (str, bytes, Path)): | ||
| fmap_dynamic = nb.load(fmap_dynamic) | ||
|
|
||
| moving, axcodes = ensure_positive_cosines(moving) | ||
| fmap_dynamic, _ = ensure_positive_cosines(fmap_dynamic) | ||
|
|
||
| newshape = moving.shape[:3] + tuple(d for d in moving.shape[3:] if d > 1) | ||
| data = np.asarray(nb.arrayproxy.reshape_dataobj(moving.dataobj, newshape)) | ||
| n_volumes = data.shape[3] if data.ndim == 4 else 1 | ||
| output_dtype = output_dtype or moving.header.get_data_dtype() | ||
|
|
||
| fmap_data = np.asanyarray(fmap_dynamic.dataobj, dtype='float32') | ||
| if fmap_data.ndim == 3: | ||
| fmap_data = fmap_data[..., np.newaxis] | ||
| if fmap_data.shape[-1] != n_volumes: | ||
| raise ValueError( | ||
| f'Dynamic fieldmap frame count ({fmap_data.shape[-1]}) does not match ' | ||
| f'EPI volumes ({n_volumes}).' | ||
| ) | ||
|
|
||
| if isinstance(pe_dir, str): | ||
| pe_dir = [pe_dir] * n_volumes | ||
| if isinstance(ro_time, (int, float)): | ||
| ro_time = [float(ro_time)] * n_volumes | ||
|
|
||
| pe_info = [] | ||
| for vol_pe_dir, vol_ro_time in zip(pe_dir, ro_time, strict=False): | ||
| pe_axis = 'ijk'.index(vol_pe_dir[0]) | ||
| flip = (axcodes[pe_axis] in 'LPI') ^ vol_pe_dir.endswith('-') | ||
| pe_info.append((pe_axis, -vol_ro_time if flip else vol_ro_time)) | ||
|
|
||
| voxcoords = ( | ||
| nt.linear.Affine(reference=moving) | ||
| .reference.ndindex.T.reshape((3, *data.shape[:3])) | ||
| .astype('float32') | ||
| ) | ||
|
|
||
| resampled = asyncio.run( | ||
| _dynamic_unwarp_parallel( | ||
| data, | ||
| voxcoords, | ||
| fmap_data, | ||
| pe_info, | ||
| jacobian=jacobian, | ||
| output_dtype='float32', | ||
| order=order, | ||
| mode=mode, | ||
| cval=cval, | ||
| prefilter=prefilter, | ||
| max_concurrent=num_threads or min(os.cpu_count(), 12), | ||
| ) | ||
| ) | ||
|
|
||
| if not allow_negative: | ||
| resampled[resampled < 0] = cval | ||
|
|
||
| moved = moving.__class__(resampled, moving.affine, moving.header) | ||
| moved.header.set_data_dtype(output_dtype) | ||
| return reorient_image(moved, axcodes) |
Contributor
There was a problem hiding this comment.
there's too much overlap here with unwarp_parallel and apply - can we unify these with the new 4D fieldmap methods?
Author
There was a problem hiding this comment.
This is a bigger change. I'll take a look and see what I can merge.
Author
There was a problem hiding this comment.
Unified in e94f285. I made one path for both 3D/4D fields.
unwarp_parallel takes a 3D or 4D field. A 3D (static) field is np.broadcast_to-viewed across frames, so per-frame selection is a single branchless fmap_hz[..., volid]. The 3D case is now just the degenerate 4D case.
B0FieldTransform is where it gets a little messy, since MEDIC field maps don't use bspline coefficients. I modified the class so that it can be constructed from a pre-gridded field (mapped=) while maintaining initialization from B-spline coeffs. apply() dispatches on provenance (coeffs → fit(); pre-gridded → use as-is), not on shape, then both routes go through one shared _resample_with_fieldmap helper.
Address review: replace the fmt: off/on pair around workflow.connect with an inline # fmt:skip, and use pathlib.Path over os.path for the corrected output. Also normalize the warpkit module/test copyright headers to the unversioned NiPreps form.
Address review: collapse the unused use_metadata_estimates and fallback_total_readout_time parity args into **kwargs (dropping the now-dead del statement and their docstring entries), and inline the single-use _MEDIC_DESC string at the workflow.__desc__ assignment.
Address review: move the duplicated _MEDIC_TEST_VOLUMES, _truncate_to_volumes helper, and MEDIC_FIXTURES list out of the fit/apply test modules into shared conftest fixtures (medic_test_volumes, truncate_to_volumes, and a parametrized medic_fixture). Both end-to-end tests now consume the fixtures, removing the keep-in-sync duplication. Also normalize remaining copyright headers.
Move warpkit into the core dependencies (keeping the python_version >= '3.11' marker so 3.10 installs still resolve) rather than gating it behind an opt-in extra. Drop the [warpkit] optional-dependencies group and the now-redundant veryslow: warpkit tox extra. The non-commercial WUSTL license is documented in the init_medic_wf module docstring.
Address review: the helper recomputed fmap_hz * ro_time internally, duplicating the VSM already computed in _sdc_unwarp. Change the signature to accept the VSM and reuse it at the call site, keeping the 3D/4D-capable helper for downstream use.
nipype prunes a node's working directory to the files referenced by its string-valued outputs. _dynamic_unwarp returned a PosixPath, which the pruning did not recognize, so corrected.nii.gz was deleted before the downstream average node could read it. Return str() of the path.
Collapse the parallel dynamic apply stack into the static machinery: - unwarp_parallel accepts a 3D or 4D field; a 3D (shared) field is np.broadcast_to-viewed across frames so per-frame selection is a single branchless fmap_hz[..., volid]. The 3D case is the degenerate 4D case. - B0FieldTransform can be constructed from a pre-gridded field (mapped=) in addition to B-spline coeffs; apply() dispatches on provenance (coeffs -> fit(); pre-gridded -> use as-is), then both routes share one _resample_with_fieldmap helper (the formerly duplicated tail). Guard added for the empty (no coeffs, no mapped) case. - Drop _dynamic_unwarp_parallel and the apply_dynamic_unwarp wrapper; the MEDIC apply node constructs B0FieldTransform(mapped=...).apply(...) directly, mirroring how ApplyCoeffsField drives the static path with coeffs.
vanandrew
force-pushed
the
feat/medic-warpkit
branch
from
adcd22c to
9993c5a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds multi-echo dynamic distortion correction (MEDIC) to sdcflows, backed by warpkit. MEDIC estimates a per-volume B0 fieldmap directly from a multi-echo, mag+phase BOLD series, capturing breathing- and motion-driven field changes that a single static fieldmap can't.
Revives #435 / #438 with the simpler pure-Python warpkit (post vanandrew/warpkit#16, no Julia/C++ setup required). Closes #36.
What's new
Workflows
init_medic_wf(sdcflows/workflows/fit/medic.py) — multi-echo phase + magnitude → 4D Hz fieldmap (one volume per timepoint, on the EPI grid) + brain-extracted reference + brain mask. Two-stage warpkit call (UnwrapPhase→ComputeFieldmap) so the per-frame masks stay accessible. Singlefmapoutput is 4D for MEDIC, leaving the 3D-vs-4D dispatch to the apply consumer.init_dynamic_unwarp_wf(sdcflows/workflows/apply/dynamic.py) — per-volume apply path built onsdcflows.transform.apply_dynamic_unwarp, a per-frame extension of the same scipy/nitransforms-backed resampling that powers the staticinit_unwarp_wf. No warpkit needed at runtime. Includes Jacobian determinant intensity correction (jacobian=Truedefault) sharingtransform.fieldmap_jacobianwith the static path.Interfaces
sdcflows/interfaces/warpkit.py— thinLibraryBaseInterfacewrappers around the two MEDIC stages SDCFlows actually drives:UnwrapPhase(ROMEO) andComputeFieldmap. Lazy import — sdcflows imports warpkit only when these interfaces actually run.Detection / dispatch
EstimatorType.MEDICadded.FieldmapEstimation.__attrs_post_init__detects MEDIC inputs (part-{phase,mag}onbold/epi/sbrefsources), enforces matched echo cardinality (≥2 phase, equal mag count), and rejects partial or mixed part sets.get_workflowinstantiatesinit_medic_wfwithEchoTime-sorted lists (BIDS doesn't guaranteeechoentity == numeric order).part='phase'andpart='mag'queries (some datasets carryIntendedForonly on one side); dedup walks the full sibling set, no reliance on pybids ordering.force_medicopt-in onfind_estimators— auto-discover MEDIC estimators from complex multi-echo BOLD even when neitherIntendedFornorB0FieldIdentifieris set. Pairing is unambiguous because the part-mag/part-phase echoes of the same run are MEDIC sources by construction. Intended for public datasets that ship the required echoes without the metadata the default discovery path needs.Plumbing
init_fmap_preproc_wfskipsfmap_coefffor MEDIC (the fieldmap is on the EPI grid by construction, no B-spline rep).init_fmap_derivatives_wfusesMergeSeries(allow_4D=True)so MEDIC's 4Dfmappasses through the sameds_fieldmapsink as the 3D static fmaps.Packaging
warpkitextra inpyproject.toml, explicitly excluded from[all]because warpkit ships under a non-commercial WUSTL license. Defaultpip install sdcflowsstays Apache-clean.tox.ini: theveryslowenv pulls the warpkit extra so MEDIC end-to-end tests only run there.v3).Validation
echo=Query.REQUIRED),FieldmapEstimationcardinality check (len(phase_files) < 2), and_unpack_metadataruntime guard insideinit_medic_wf.test_apply_dynamic_unwarp_matches_staticpinsapply_dynamic_unwarpto the same Hz→VSM + scipy.ndimage convention as the static_sdc_unwarppath — catches drift in sign / pe_info handling.test_wrangler_filter/test_wrangler_URIsparametrized with a 3-session × 3-echo × {mag,phase} BIDS skeleton.Known compromises
IntendedFor/B0FieldIdentifier— same constraint as the existing single-PE EPI branch. Datasets missing both can opt into discovery viaforce_medic=Trueonfind_estimators(see Detection / dispatch above).fmapoutput is 4D for MEDIC. Downstream tools that expect 3D field maps need to either dispatch on dimensionality or block MEDIC-based estimators until they do.Test plan
pytest sdcflows/utils/tests/test_wrangler.py— wrangler MEDIC detection paths (includingforce_medic)pytest sdcflows/workflows/fit/tests/test_medic.py—init_medic_wfconstruction +_unpack_metadataguardspytest sdcflows/workflows/apply/tests/test_dynamic.py—init_dynamic_unwarp_wfconstruction + jacobian flag + per-frame resampling vs. static pathpytest -m veryslowwithpip install sdcflows[warpkit]and ds006926 / ds007637 fixtures present — full MEDIC fit + dynamic apply