fix(pipelines): guard against ZeroDivisionError in chunk_and_embed_incremental when chunk list is empty

[Kunal-Somani]

Summary

Fixes #163

chunk_and_embed_incremental in pipelines/incremental-pipeline.py has the same ZeroDivisionError crash as kubeflow-pipeline.py, fixed in PR #149 (issue #148) — but the incremental pipeline was not addressed by that fix.

Root Cause

The incremental pipeline applies identical aggressive regex cleaning (Hugo frontmatter, HTML tags, URLs). A document can pass the < 50 char content guard but produce zero chunks after splitting. The print statement on the next line divides by len(chunks) == 0, crashing the entire incremental KFP run.

Affected Line

print(f"File: {file_data['path']} -> {len(chunks)} chunks (avg: {sum(len(c) for c in chunks)/len(chunks):.0f} chars)")

Fix

Added the same empty-chunk guard already applied in PR #149:

if not chunks:
    print(f"Skipping file after chunking (no chunks produced): {file_data['path']}")
    continue

Checklist

• Commits are signed off (DCO)
• Fixes bug(pipelines): ZeroDivisionError in chunk_and_embed_incremental when text splitter returns empty chunks #163
• Consistent with fix already applied in PR fix(pipelines): guard against ZeroDivisionError when chunk list is empty #149
• No regressions to incremental pipeline logic

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign franciscojavierarceo for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment