feat: cleanup_duplicates.py + Flex Query period warning

[flowcool]

Summary

• cleanup_duplicates.py — outil de nettoyage des doublons créés lors du passage de la Flex Query IBKR d'une période courte à Last 365 Calendar Days. Patche les entrées manuelles avec leur IBKR#{tradeID} et supprime les copies dupliquées. Dry-run par défaut, log de sécurité avant toute mutation, probe read-only de l'endpoint avant d'agir.
• README.md — avertissement critique sur le paramètre période de la Flex Query, description des deux classes d'échec silencieux (ventes skippées, réimports dupliqués), et pointeur vers cleanup_duplicates.py comme chemin de recovery.

Contexte

Bug découvert en prod : Flex Query configurée sur "Last Month" au lieu de "Last 365 Calendar Days". Conséquences : ventes récentes absentes du XML IBKR, et à la correction du paramètre, 90 activités réimportées en doublon avec les entrées manuelles existantes. Le script de cleanup a résolu les 26 doublons trades identifiés (correspondance symbol+type+qty+prix+date ±2j).

Test plan

• Dry-run validé : 0 paires détectées après cleanup (dédup propre)
• Sync manuel post-cleanup : New trade activities: 0, duplicates skipped: 104
• Endpoint probe read-only vérifié avant toute mutation

🤖 Generated with Claude Code

https://claude.ai/code/session_0125mJBtEmStCj8rFFkrtGxb

Summary by CodeRabbit

• New Features

  • Added manual trigger option for Docker image builds.
  • Introduced utility to clean up duplicate activities from IBKR imports.

• Documentation

  • Added critical warning: use minimum 365-day period in Flex Query setup to prevent missed transactions and duplicates.
  • Documented recovery procedure for handling duplicate imports.
  • Updated CI/deployment and merge strategy documentation.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23e9ad68-9052-425e-804d-3c5ab6317379

📥 Commits

Reviewing files that changed from the base of the PR and between e4bc0b9 and 9eb0334.

📒 Files selected for processing (4)

• .github/workflows/docker-publish.yml
• CLAUDE.md
• README.md
• cleanup_duplicates.py

---

📝 Walkthrough

Walkthrough

Adds cleanup_duplicates.py, a new Python CLI that matches IBKR#-tagged Ghostfolio activities to their manual duplicates, patches manual entry comments with the IBKR trade ID, and deletes the IBKR# copies. The README gains a critical warning about Flex Query date ranges. The Docker publish workflow gains a manual dispatch trigger and action version upgrades, with matching CLAUDE.md documentation.

Changes

Duplicate Activity Cleanup Script

Layer / File(s)	Summary
Config, auth, date utils, and API mutation helpers cleanup_duplicates.py	Establishes DATE_TOLERANCE, environment-backed config/header loading, bulk activity fetch from Ghostfolio, parse_date, a read-only endpoint verification probe, put_comment, delete_activity, and symbol_of helpers.
Main CLI matching logic and apply/dry-run execution cleanup_duplicates.py	Implements main() with --apply flag parsing, activity partitioning into IBKR# vs manual sets, symbol/type/quantity/unitPrice matching with date tolerance, dry-run report output, JSON safety snapshot write, and the mutation loop with patched/deleted/error counters and sys.exit(1) on failure.
README critical warning and recovery instructions README.md	Inserts a bold Critical note in Flex Query setup steps warning against periods shorter than 365 days, describes missing-sell and silent-duplicate failure modes, and documents the dry-run then --apply recovery path via cleanup_duplicates.py.

Docker Publish Workflow and CI Docs Updates

Layer / File(s)	Summary
workflow_dispatch trigger and action version upgrades .github/workflows/docker-publish.yml	Adds workflow_dispatch with a ref input defaulting to staging, and upgrades all five Docker-related GitHub Actions to newer major versions.
CLAUDE.md merge strategy and CI deployment docs CLAUDE.md	Documents the merge-commits-only Git strategy and expands the CI/deployment section to describe main (push latest + Portainer redeploy) and staging (push staging + manual NAS test) branch workflows.

Sequence Diagram

sequenceDiagram
  participant User as User (CLI)
  participant Script as cleanup_duplicates.py
  participant GF as Ghostfolio API

  User->>Script: python cleanup_duplicates.py [--apply]
  Script->>GF: GET /api/v1/activities (verify + fetch all)
  GF-->>Script: full activities list
  Script->>GF: GET /api/v1/activities/{id} (single probe)
  GF-->>Script: single activity (verify OK)
  Script->>Script: partition into IBKR# entries and manual entries
  Script->>Script: match by symbol/type/qty/price + DATE_TOLERANCE
  Script-->>User: dry-run report (if no --apply)

  rect rgba(255, 140, 0, 0.5)
    Note over Script,GF: --apply mode only
    Script->>Script: write JSON safety snapshot
    Script->>GF: GET /api/v1/activities/{manual_id} (re-fetch for PUT payload)
    GF-->>Script: current manual activity
    Script->>GF: PUT /api/v1/activities/{manual_id} (patch comment to IBKR#{trade_id})
    GF-->>Script: 200 OK
    Script->>GF: DELETE /api/v1/activities/{ibkr_id}
    GF-->>Script: 200 OK
  end

  Script-->>User: summary (patched/deleted/errors), exit 1 on errors

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 A bunny found duplicates hiding in rows,
With IBKR# stamps and manual woes.
I matched them by symbol, by price, and by date,
Then patched and deleted — oh, wasn't that great!
A snapshot for safety, a dry-run to check,
Now Ghostfolio's tidy — no duplicate wreck! ✨

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch staging

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}