Skip to content

Fix release bug#508

Open
stafot wants to merge 1 commit intomasterfrom
CLD-9505-cp-cannot-stat-cr-index-index-yaml-no-such-file-or-directory
Open

Fix release bug#508
stafot wants to merge 1 commit intomasterfrom
CLD-9505-cp-cannot-stat-cr-index-index-yaml-no-such-file-or-directory

Conversation

@stafot
Copy link
Copy Markdown
Contributor

@stafot stafot commented Aug 26, 2025

Summary

Fix chart-releaser index update failure when no new releases are detected

Problem

The Helm chart release pipeline fails when running on commits that don't introduce new chart versions, with the following error:

Index .cr-index/index.yaml did not change
The flag will be removed with the next major release.Switched to a new branch 'gh-pages'
branch 'gh-pages' set up to track 'origin/gh-pages'.
cp: cannot stat '.cr-index/index.yaml': No such file or directory
make: *** [Makefile:51: release] Error 1
Error: Process completed with exit code 2.

Root Cause Analysis

The issue occurs in the update_index() function in scripts/release.sh due to the interaction between Git branch switching and untracked directories:

The Failing Sequence

  1. Chart-releaser runs: cr index command creates .cr-index/index.yaml regardless of whether changes are detected
  2. Directory exists check passes: if [[ ! -d "$CR_INDEX_DIR" ]] - script continues because .cr-index/ exists
  3. Branch switch occurs: git checkout gh-pages switches working directory from master to gh-pages
  4. Directory disappears: Git removes .cr-index/ during branch switch because:
    • It's explicitly listed in .gitignore (untracked by Git)
    • Git doesn't preserve untracked directories across branch switches
    • The gh-pages branch has a completely different file structure (5 files vs. 50+ on master)
  5. Copy command fails: cp --force .cr-index/index.yaml index.yaml - source file no longer exists

Why This Worked Previously

This is a conditional failure that only manifests when:

  • Chart-releaser detects no new releases (Index .cr-index/index.yaml did not change)
  • The script attempts to process existing charts without changes

Previous successful runs had actual new chart versions to release, where chart-releaser's behavior may have differed in preserving the directory through the process.

Evidence from Logs

Successful run (4 days ago):

Updating index .cr-index/index.yaml          # ← New changes detected
Switched to a new branch 'gh-pages'
[gh-pages da9b9be] Update index.yaml        # ← Copy succeeded

Failed run (today):

Index .cr-index/index.yaml did not change   # ← No changes detected
Switched to a new branch 'gh-pages' 
cp: cannot stat '.cr-index/index.yaml'      # ← File missing after branch switch

Solution

Preserve the index.yaml file before the Git branch switch to eliminate dependency on the .cr-index directory surviving Git operations:

Code Changes

# OLD (broken):
git checkout gh-pages
cp --force .cr-index/index.yaml index.yaml

# NEW (fixed):  
cp --force .cr-index/index.yaml ./new-index.yaml  # Copy before branch switch
git checkout gh-pages                              # Switch branches  
cp --force ./new-index.yaml index.yaml           # Copy from preserved location
rm -f ./new-index.yaml                           # Clean up temporary file

Complete Function Update

update_index() {
    docker run --rm -u "$(id -u):$(id -g)" \
        -v "$(pwd):/src" \
        -w /src \
        "${DOCKER_IMAGE_CR}" \
       index -o mattermost -r mattermost-helm -c https://helm.mattermost.com
    if [[ ! -d "$CR_INDEX_DIR" ]]; then
        echo "No changes found for the index.yaml"
        exit 0
    fi

    # Copy the index.yaml before switching branches
    cp --force .cr-index/index.yaml ./new-index.yaml
    git checkout gh-pages
    cp --force ./new-index.yaml index.yaml
    rm -f ./new-index.yaml
    git add index.yaml
    git commit --message="Update index.yaml" --signoff
    git push
}

Why This Fix Works

  1. Eliminates timing dependency: File is preserved regardless of when Git operations occur
  2. Handles both scenarios:
    • ✅ Charts with new releases (worked before, still works)
    • ✅ Charts with no changes (was broken, now fixed)
  3. Minimal risk: Uses temporary file pattern, no logic changes
  4. Git-agnostic: Works regardless of Git's untracked directory handling

Testing

The fix addresses the exact failure condition:

  • ✅ Syntax validation: bash -n scripts/release.sh passes
  • ✅ Logic verification: Temporary file approach eliminates race condition
  • ✅ Backwards compatibility: No changes to existing successful workflows

Historical Context

This issue was introduced by the interaction between:

  • Commit 04ae8b7 (Aug 2022): Migrated from CircleCI to GitHub Actions
  • Commit 3944a73 (Oct 2022): Added --skip-existing flag and directory check
  • The --skip-existing flag changes chart-releaser behavior for no-change scenarios

The previous fix in 3944a73 only addressed the case where .cr-index is never created, but missed the case where it's created then destroyed by Git operations.

Impact

  • Fixes: Release pipeline failures when no new chart versions are present
  • Maintains: All existing functionality for actual releases
  • Improves: Pipeline reliability for routine commits without chart changes

Files Changed

  • scripts/release.sh: Updated update_index() function to preserve index file before branch operations

Ticket Link

https://mattermost.atlassian.net/browse/CLD-9505

Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>
@mattermost-build
Copy link
Copy Markdown
Contributor

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants