Skip to content

Fix wavefront boundary with dynamic sequence length constraints#27

Open
unavailable-2374 wants to merge 3 commits into
thegenemyers:mainfrom
unavailable-2374:fix-wavefront-dynamic-bounds
Open

Fix wavefront boundary with dynamic sequence length constraints#27
unavailable-2374 wants to merge 3 commits into
thegenemyers:mainfrom
unavailable-2374:fix-wavefront-dynamic-bounds

Conversation

@unavailable-2374
Copy link
Copy Markdown

Summary

This PR fixes out-of-bounds access in the wavefront alignment algorithm by adding dynamic sequence length constraints during wave expansion.

Problem

Under certain conditions, the wavefront can expand beyond valid coordinate bounds, causing errors like:

Subrange 86,138 out of bounds (Get_Contig_Piece)

The static boundary constraints applied at the start of alignment functions are insufficient because the valid diagonal range changes as the anti-diagonal
progresses.

Solution

During each wavefront expansion step, calculate the effective diagonal range based on the current anti-diagonal position:

// Forward wave: anti-diagonal increases
int curr_anti = mida + 2*(dif+1);

// Reverse wave: anti-diagonal decreases
int curr_anti = mida - 2*(dif+1);

// Sequence length constraints
int seq_minp = curr_anti - 2*blen;  // ensures y <= blen
int seq_maxp = 2*alen - curr_anti;  // ensures x <= alen

// Take stricter constraint
int eff_minp = max(minp, seq_minp);
int eff_maxp = min(maxp, seq_maxp);

Changes

Applied dynamic bounds checking to all 6 wavefront functions:
- forward_wave, reverse_wave
- forward_wrap, reverse_wrap
- forward_extend, reverse_extend

Also restored simple boundary check: if (x < 0 || x < k)

Testing

Tested with the same assemblies that previously caused the 'Subrange out of bounds' error, and it now passes.

unavailable-2374 and others added 3 commits January 28, 2026 18:23
Root cause analysis:
- Wave algorithm uses diagonal coordinate system where k = x - y
- For valid coordinates: x = (anti+k)/2 >= 0 and y = (anti-k)/2 >= 0
- This requires k to be in range [-anti, anti]
- Original code only checked hgh <= anti (y >= 0), not low >= -anti (x >= 0)

Fixes applied:

1. Entry point validation (Local_Alignment, Wrap_Around_Alignment, Find_Extension):
   - Add check: while (((anti+low)>>1) < 0) low += 1
   - Constrain minp to -anti (prevents wave expansion into x < 0 region)
   - Constrain maxp to anti (prevents wave expansion into y < 0 region)
   - Return error if no valid diagonals remain (low > hgh)

2. Internal bounds checking (all 6 wave functions):
   - Add check: if (x < 0 || x < k) mark diagonal as dead
   - Applied to both 0-wave initialization and main wave loops

3. Safety checks (forward_wave, reverse_wave):
   - Add check: if (avail == 0) return error
   - Catches case where all diagonals fail bounds checking

4. Early termination (all wave expansion loops):
   - Add check: if (hgh < low) break
   - Prevents infinite loops when all diagonals are trimmed

This multi-layer fix prevents invalid coordinates from being computed,
rather than just detecting and skipping them after the fact.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Skip alignments where coordinates exceed sequence bounds:
- path->abpos < 0 or path->aepos > aln->alen
- path->bbpos < 0 or path->bepos > aln->blen
- start >= end for either sequence

This handles edge cases where wave algorithm produces out-of-bounds
coordinates, preventing "Subrange out of bounds" errors in Get_Contig_Piece.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
During each wavefront expansion step, calculate the effective diagonal
range based on the current anti-diagonal position:

Forward wave: curr_anti = mida + 2*(dif+1)
Reverse wave: curr_anti = mida - 2*(dif+1)

Sequence length constraints:
- seq_minp = curr_anti - 2*blen  (ensures y <= blen)
- seq_maxp = 2*alen - curr_anti  (ensures x <= alen)

Take the stricter of user constraints and sequence constraints:
- eff_minp = max(minp, seq_minp)
- eff_maxp = min(maxp, seq_maxp)

Applied to all 6 wavefront functions:
- forward_wave, reverse_wave
- forward_wrap, reverse_wrap
- forward_extend, reverse_extend

Also restored simple boundary check: if (x < 0 || x < k)

This dynamic approach is more precise than static constraints because
the valid diagonal range changes as the wavefront expands.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant