Deploy Gitea GitHub Mirror for Disaster Recovery

[morey-tech]

GitHub Issue #66 - Updated Description with NFS Storage

Executive Summary

This issue documents the comprehensive implementation plan for deploying Gitea as a GitHub repository mirror on the ocp-home OpenShift cluster. The primary objective is disaster recovery and backup of all morey-tech organization repositories (public and private) with automated hourly synchronization.

📝 Update 2026-01-25: Storage architecture updated to use NFS-backed storage for repositories, following the proven pattern from immich and paperless deployments. See detailed plan.

Key Objectives

• Mirror ALL morey-tech GitHub repositories (public and private) to local Gitea instance
• Automated hourly sync via Kubernetes CronJob
• Disaster recovery and backup capability
• Leverage existing ocp-home infrastructure (CloudNativePG, Velero, External Secrets, NFS)

Resource Requirements

• Storage: 120Gi total (100Gi repositories on NFS + 20Gi database on LVMS)
• Compute: 250m CPU, 640Mi memory

---

Infrastructure Analysis

Current ocp-home Cluster Capabilities

GitOps Architecture:

• ArgoCD ApplicationSets with app-of-apps pattern
• Automatic discovery from kubernetes/ocp-home/applications/ and kubernetes/ocp-home/system/
• Kustomize + Helm chart deployment pattern (see immich, paperless)
• Automated sync, prune, and self-heal enabled

Available Operators:

• CloudNativePG (v1.28.0): PostgreSQL database management
• External Secrets: Bitwarden integration for secret management
• Velero: Backup and restore with NFS backend
• Cert-Manager: Let's Encrypt certificates with Cloudflare DNS-01
• External-DNS: Cloudflare DNS automation
• LVM Storage: NVMe-backed persistent volumes

Storage:

• NFS: 192.168.6.20:/storage-mass/ocp-home/ for large data volumes (immich, paperless)
• LVMS: lvms-lvm-nvme1n1-vg (NVMe-backed, thin provisioning, XFS) for performance-critical workloads
• CSI snapshot support via VolumeSnapshotClass

Networking:

• OpenShift Routes with wildcard TLS: *.apps.ocp-home.rh-lab.morey.tech
• MetalLB with BGP peering for load balancing
• External-DNS for automatic Cloudflare record creation

---

Deployment Architecture

Storage Strategy

Hybrid NFS + LVMS Approach:

Component	Storage Type	Size	Access Mode	Rationale
Repository Data	NFS	100Gi	ReadWriteMany	Consistent with immich/paperless; supports scaling; network-accessible DR
PostgreSQL Database	LVMS	20Gi	ReadWriteOnce	Low-latency I/O for transactional workloads

NFS Configuration:

• Server: 192.168.6.20
• Path: /storage-mass/ocp-home/gitea
• Protocol: NFSv4 with mount options: hard, intr
• Pattern: Follows immich and paperless

Components

1. Gitea Application

• Source: Official Gitea Helm chart v12.x from https://dl.gitea.com/charts/
• Deployment Pattern: Kustomize wrapper around Helm chart (follow immich pattern)
• Configuration:
  • Rootless image for OpenShift SCC compatibility
  • HTTPS-only access (SSH disabled for simplicity)
  • LFS support enabled
  • In-memory cache (Valkey disabled for single instance)
  • External PostgreSQL connection
• Resource Limits: 100m-1000m CPU, 256Mi-1Gi memory

2. PostgreSQL Database (CloudNativePG)

• Pattern: Follow immich CloudNativePG cluster configuration
• Specification:
  • Single instance (sufficient for homelab)
  • Standard PostgreSQL 16 (no special extensions needed)
  • 20Gi storage on LVMS NVMe
  • Service endpoint: gitea-postgresql-rw:5432
  • Managed role: gitea user with superuser privileges
• Reference: kubernetes/ocp-home/applications/immich/postgresql-cluster.yaml

3. Repository Storage (Updated)

PersistentVolume (NFS-backed):

apiVersion: v1
kind: PersistentVolume
metadata:
  name: gitea-repositories
  labels:
    type: nfs
    app: gitea
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  mountOptions:
    - nfsvers=4
    - hard
    - intr
  nfs:
    server: 192.168.6.20
    path: /storage-mass/ocp-home/gitea

PersistentVolumeClaim:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gitea-repositories
  labels:
    app: gitea
    backup.velero.io/schedule: daily
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  volumeName: gitea-repositories
  storageClassName: ""

References: immich/pv-data.yaml, paperless/pv.yaml

4. Automated Mirror Sync

• Implementation: Kubernetes Deployment using ghcr.io/raylabshq/gitea-mirror:latest
• Schedule: Internal scheduler with 8-hour sync interval
• Functionality:
  • Queries GitHub API for all morey-tech org repos
  • Automatically creates mirrors in Gitea for new repos
  • Syncs existing mirrors (pull updates)
  • Supports both public and private repositories
  • Metadata mirroring: Issues, PRs, labels, milestones, wiki
  • Git LFS support: Large file mirroring
  • Auto-cleanup: Archives deleted repos
• Environment Variables:
  • GITHUB_TOKEN: Organization token from ExternalSecret
  • GITHUB_USER: morey-tech
  • GITEA_URL: https://git.apps.ocp-home.rh-lab.morey.tech
  • GITEA_TOKEN: Admin user API token
  • AUTO_IMPORT_REPOS: true
  • CLEANUP_DELETE_IF_NOT_IN_GITHUB: archive
• Reference: https://github.com/RayLabsHQ/gitea-mirror

5. Ingress/TLS

• Route: git.apps.ocp-home.rh-lab.morey.tech
• Pattern: OpenShift Ingress with ingressClassName: openshift-default
• TLS: Edge termination with wildcard certificate (automatic)
• DNS: External-DNS creates Cloudflare A record automatically
• Reference: kubernetes/ocp-home/applications/home-assistant/ingresses.yaml

6. Secret Management

Three ExternalSecrets pulling from Bitwarden ClusterSecretStores:

a) GitHub Organization Token:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: github-org-token
spec:
  target:
    name: github-org-token
  data:
    - secretKey: token
      sourceRef:
        storeRef:
          name: bitwarden-fields
          kind: ClusterSecretStore
      remoteRef:
        key: <bitwarden-uuid>
        property: github-token

b) Gitea Admin Credentials:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: gitea-admin-credentials
spec:
  target:
    name: gitea-admin-credentials
  data:
    - secretKey: username
      sourceRef:
        storeRef:
          name: bitwarden-login
          kind: ClusterSecretStore
      remoteRef:
        key: <bitwarden-uuid>
        property: username
    - secretKey: password
      sourceRef:
        storeRef:
          name: bitwarden-login
          kind: ClusterSecretStore
      remoteRef:
        key: <bitwarden-uuid>
        property: password

c) PostgreSQL Credentials:

• Follow pattern from kubernetes/ocp-home/applications/immich/postgresql-credentials-secret.yaml

7. Backup Strategy (Updated)

Dual-Layer Backup Approach:

Layer 1 - Velero Kubernetes Backup:

• Provider: Velero with daily schedule
• Namespace Label: backup.velero.io/schedule: daily
• PVC Label: backup.velero.io/schedule: daily
• Retention: 7 days (168h) per existing daily schedule
• Backup Targets:
  • All namespace resources (Gitea deployment, services, ingress)
  • Repository PVC metadata (NFS mount point)
  • PostgreSQL data with CSI snapshots
• Reference: kubernetes/ocp-home/system/velero/schedule-daily.yaml

Layer 2 - NFS-Level Backup:

• Advantages:
  • File-level restore capability
  • Independent from Kubernetes cluster
  • Efficient deduplication/compression
  • Survives cluster failures
• Recommended Methods:
  • ZFS snapshots (if NFS backend supports)
  • rsync-based incremental backups
  • Velero Restic for file-level backup
• Retention: 30 days (configurable on NFS server)

---

Directory Structure

kubernetes/ocp-home/applications/gitea/
├── kustomization.yaml                    # Main Kustomize config with Helm chart
├── namespace.yaml                        # Namespace with backup labels
├── pv-repositories.yaml                  # NFS PersistentVolume
├── pvc-repositories.yaml                 # PVC with static binding
├── postgresql-cluster.yaml               # CloudNativePG cluster (LVMS)
├── postgresql-credentials-secret.yaml    # ExternalSecret for DB credentials
├── gitea-admin-secret.yaml              # ExternalSecret for admin credentials
├── github-org-token-secret.yaml         # ExternalSecret for GitHub token
├── ingress.yaml                         # OpenShift route configuration
├── deployment-scc-fix.yaml              # Security context patches for OpenShift
├── gitea-mirror-deployment.yaml         # RayLabsHQ gitea-mirror Deployment
├── gitea-mirror-secret.yaml             # ExternalSecret for mirror auth
└── README.md                            # Documentation and maintenance notes

---

Implementation Details

Helm Chart Configuration

Key values in kustomization.yaml valuesInline section:

valuesInline:
  # Use rootless image (OpenShift compatible)
  image:
    rootless: true
    pullPolicy: IfNotPresent
  
  # Disable built-in PostgreSQL (use CloudNativePG)
  postgresql-ha:
    enabled: false
  postgresql:
    enabled: false
  
  # Disable Valkey for single-instance
  valkey-cluster:
    enabled: false
  
  # External PostgreSQL configuration
  gitea:
    config:
      database:
        DB_TYPE: postgres
        HOST: gitea-postgresql-rw:5432
        NAME: gitea
        USER: gitea
        PASSWD: # From secret
        SCHEMA: public
        SSL_MODE: disable
      
      session:
        PROVIDER: memory  # Use memory for single instance
      
      cache:
        ADAPTER: memory
      
      repository:
        ROOT: /data/git/repositories
      
      server:
        DOMAIN: git.apps.ocp-home.rh-lab.morey.tech
        ROOT_URL: https://git.apps.ocp-home.rh-lab.morey.tech
        DISABLE_SSH: true  # Use HTTPS only (simpler for homelab)
        LFS_START_SERVER: true  # Enable LFS support
  
  # Persistence configuration (NFS-backed)
  persistence:
    enabled: true
    existingClaim: gitea-repositories
    size: 100Gi
    storageClass: ""  # Static binding
    accessModes:
      - ReadWriteMany  # NFS supports RWX
  
  # Ingress disabled (use separate OpenShift Ingress resource)
  ingress:
    enabled: false
  
  # Service configuration
  service:
    http:
      type: ClusterIP
      port: 3000
  
  # Resource limits
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 1000m
      memory: 1Gi

OpenShift SCC Compatibility

Security Context Patch (deployment-scc-fix.yaml):

- op: remove
  path: /spec/template/spec/securityContext/fsGroup

Rationale:

• OpenShift restricted SCC assigns random UID (100000+)
• Helm chart defaults include fsGroup: 1000 which conflicts
• Rootless Gitea image handles permissions correctly without fsGroup
• Pattern follows kubernetes/ocp-home/applications/paperless/deployment-patch.yaml

Mirror Automation Deployment

Deployment Specification (gitea-mirror-deployment.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitea-mirror
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitea-mirror
  template:
    metadata:
      labels:
        app: gitea-mirror
    spec:
      containers:
      - name: gitea-mirror
        image: ghcr.io/raylabshq/gitea-mirror:latest
        env:
        - name: GITHUB_TOKEN
          valueFrom:
            secretKeyRef:
              name: github-org-token
              key: token
        - name: GITHUB_USER
          value: "morey-tech"
        - name: GITEA_URL
          value: "https://git.apps.ocp-home.rh-lab.morey.tech"
        - name: GITEA_TOKEN
          valueFrom:
            secretKeyRef:
              name: gitea-admin-credentials
              key: api-token
        - name: AUTO_IMPORT_REPOS
          value: "true"
        - name: CLEANUP_DELETE_IF_NOT_IN_GITHUB
          value: "archive"
        - name: GITEA_MIRROR_INTERVAL
          value: "8h"
        - name: SCHEDULE_ENABLED
          value: "true"
        resources:
          requests:
            cpu: 50m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 512Mi

---

Pre-Deployment Steps

1. NFS Server Setup

Create NFS Export:

# On NFS server (192.168.6.20)
sudo mkdir -p /storage-mass/ocp-home/gitea
sudo chown -R nobody:nogroup /storage-mass/ocp-home/gitea
sudo chmod 777 /storage-mass/ocp-home/gitea

# Add to /etc/exports
echo "/storage-mass/ocp-home/gitea *(rw,sync,no_subtree_check,no_root_squash)" | sudo tee -a /etc/exports

# Reload NFS exports
sudo exportfs -ra

# Verify export
showmount -e 192.168.6.20

Verify NFS Mount:

# From any ocp-home cluster node
sudo mount -t nfs4 -o hard,intr 192.168.6.20:/storage-mass/ocp-home/gitea /mnt
ls -la /mnt
sudo umount /mnt

2. Bitwarden Setup

Create Gitea Admin Credentials Entry:

• Username: admin
• Password: [Generate strong password]
• Note the Bitwarden UUID for ExternalSecret reference

Create GitHub Fine-Grained Personal Access Token:

• Navigate to GitHub Settings → Developer settings → Fine-grained tokens
• Repository access: All repositories
• Organization permissions:
  • Metadata: Read-only
  • Contents: Read-only
• Expiration: 1 year (set calendar reminder)
• Store token in Bitwarden
• Note the Bitwarden UUID for ExternalSecret reference

3. Storage Verification

Check NFS Available Capacity:

# On NFS server
df -h /storage-mass
# Verify at least 100Gi available

Check LVMS Available Capacity:

oc get lvmcluster -n openshift-storage -o yaml
# Verify /dev/nvme1n1 has at least 20Gi available

Query GitHub API for Repo Sizes:

gh api /orgs/morey-tech/repos --paginate | jq '[.[].size] | add / 1024' 
# Returns total size in MB

4. DNS Verification

Verify External-DNS is Running:

oc get pods -n external-dns-system

Verify Cloudflare API Access:

oc logs -n external-dns-system deployment/external-dns | grep -i cloudflare

---

Critical Files to Reference

Implementation should follow these existing patterns:

1. Helm + Kustomize Pattern:

   • kubernetes/ocp-home/applications/immich/kustomization.yaml

2. NFS PersistentVolume:

   • kubernetes/ocp-home/applications/immich/pv-data.yaml
   • kubernetes/ocp-home/applications/paperless/pv.yaml

3. NFS PersistentVolumeClaim:

   • kubernetes/ocp-home/applications/immich/pvc-data.yaml
   • kubernetes/ocp-home/applications/paperless/pvc.yaml

4. CloudNativePG PostgreSQL:

   • kubernetes/ocp-home/applications/immich/postgresql-cluster.yaml

5. ExternalSecret Pattern:

   • kubernetes/ocp-home/applications/immich/postgresql-credentials-secret.yaml

6. OpenShift SCC Fixes:

   • kubernetes/ocp-home/applications/paperless/deployment-patch.yaml

7. OpenShift Route/Ingress:

   • kubernetes/ocp-home/applications/home-assistant/ingresses.yaml

8. Velero Backup Schedule:

   • kubernetes/ocp-home/system/velero/schedule-daily.yaml

---

Post-Deployment Validation

• NFS export accessible from cluster nodes
• PV bound to PVC: oc get pv gitea-repositories
• NFS mount verified in pod: oc exec -n gitea <pod> -- df -h
• Gitea UI accessible at https://git.apps.ocp-home.rh-lab.morey.tech
• Admin login successful with Bitwarden credentials
• PostgreSQL connection healthy (check Gitea admin dashboard)
• First mirror sync completes successfully
• All repos from morey-tech org mirrored to Gitea
• Mirrors marked as read-only in Gitea UI
• Metadata mirrored (issues, PRs, labels, milestones)
• Velero backup discovers gitea namespace
• DNS record created in Cloudflare for git.apps.ocp-home.rh-lab.morey.tech
• TLS certificate valid (Let's Encrypt via wildcard cert)
• Mirror sync logs show no errors
• Test manual trigger of mirror sync
• NFS-level backup configured

---

Troubleshooting Guide

NFS-Specific Issues

1. PVC Stuck in Pending

• Symptom: oc get pvc gitea-repositories shows Pending
• Solution:

  # Check PV exists and is Available
  oc get pv gitea-repositories
  
  # Verify PV and PVC names match exactly
  oc describe pvc -n gitea gitea-repositories | grep -A 3 "Volume:"
  
  # Check PV isn't already bound
  oc get pv gitea-repositories -o yaml | grep claimRef

2. NFS Mount Fails in Pod

• Symptom: Pod fails with mount.nfs: access denied or timeout
• Solution:

  # Verify NFS export from cluster node
  ssh <ocp-node> "showmount -e 192.168.6.20"
  
  # Test manual mount
  ssh <ocp-node> "sudo mount -t nfs4 192.168.6.20:/storage-mass/ocp-home/gitea /mnt"
  
  # Check network connectivity
  oc debug node/<node-name>
  chroot /host
  ping 192.168.6.20
  telnet 192.168.6.20 2049

3. Permission Denied Errors

• Symptom: Gitea pod logs show "permission denied" when writing to /data/git/repositories
• Solution:

  # On NFS server - verify directory permissions
  ls -la /storage-mass/ocp-home/gitea/
  
  # Check pod security context UID
  oc get pod -n gitea <pod> -o yaml | grep -A 3 securityContext
  
  # Fix NFS permissions (if needed)
  sudo chown -R 1000:1000 /storage-mass/ocp-home/gitea

4. Slow Performance

• Symptom: Git operations are slow, Gitea UI sluggish
• Solution:

  # Check NFS server load
  # On NFS server
  nfsstat -s
  
  # Verify network latency
  oc debug node/<node-name>
  chroot /host
  ping -c 10 192.168.6.20
  
  # Test I/O performance
  oc exec -n gitea deployment/gitea -- dd if=/dev/zero of=/data/git/test bs=1M count=100

Common Issues

1. SCC Permission Errors

• Symptom: Pod fails to start with "unable to validate against any security context constraint"
• Solution: Verify fsGroup removed from Deployment securityContext
• Check: oc get pod <pod> -o yaml | grep -A 5 securityContext

2. Database Connection Failed

• Symptom: Gitea logs show "database connection failed"
• Solution:
  • Verify CloudNativePG cluster is ready: oc get cluster gitea-postgresql
  • Check credentials in ExternalSecret match PostgreSQL cluster
  • Confirm service name is gitea-postgresql-rw:5432

3. Mirror Sync Failures

• Symptom: Mirror deployment fails or repos not appearing in Gitea
• Solution:
  • Verify GitHub token has not expired
  • Check token has correct scopes (repo read access)
  • Review deployment logs: oc logs -n gitea deployment/gitea-mirror
  • Confirm GitHub API rate limits not exceeded

4. Storage Full

• Symptom: "no space left on device" errors
• Solution:

  # Check NFS server capacity
  # On NFS server
  df -h /storage-mass
  
  # Expand PVC size if needed
  oc edit pvc gitea-repositories -n gitea
  
  # Clean up old/archived mirrors if needed

---

Security Considerations

NFS Security

Network Security:

• ✅ Internal Network: NFS server on isolated network (192.168.6.20)
• ⚠️ NFSv4 Kerberos: Consider enabling for authentication (advanced)
• ✅ Export Restrictions: Limit NFS exports to OpenShift cluster IPs

File Permissions:

• ⚠️ no_root_squash: Required for OpenShift, but increases security risk
• ✅ Alternative: Use all_squash with specific anonuid/anongid
• ✅ Directory Permissions: Set minimal required permissions

Data at Rest:

• ⚠️ NFS Encryption: Not enabled by default
• ✅ Storage-Level Encryption: If NFS backend supports it (e.g., LUKS, ZFS encryption)

Application Security

• GitHub Token: Read-only organization access (minimal privilege principle)
• Storage: All tokens in Bitwarden, pulled via ExternalSecrets
• Rotation: GitHub fine-grained tokens support max 1-year expiration
• Audit: No secrets in Git repository (verified via .gitignore)

Network Security

• HTTPS Only: SSH service disabled (reduced attack surface)
• TLS Termination: At OpenShift router with Let's Encrypt certificate
• Internal Communication: PostgreSQL connection not exposed externally
• Ingress: Single controlled entry point via OpenShift Route

Access Control

• Gitea Admin: Via Bitwarden-stored credentials only
• User Registration: Consider disabling (backup-only use case)
• Mirror Repositories: Read-only (prevents accidental local commits)
• API Access: Gitea admin token for mirror automation only

---

Resource Estimates

Storage Resources

Component	Storage Type	Size	Access Mode	IOPS Profile
Gitea Repositories	NFS	100Gi	RWX	Low (batch sync)
Git LFS Objects	NFS	(included)	RWX	Low (occasional)
PostgreSQL Data	LVMS (NVMe)	20Gi	RWO	High (transactional)
Total NFS	-	100Gi	-	-
Total LVMS	-	20Gi	-	-

Compute Resources

Component	CPU Request	CPU Limit	Memory Request	Memory Limit
Gitea	100m	1000m	256Mi	1Gi
PostgreSQL	100m	500m	256Mi	512Mi
Mirror (RayLabsHQ)	50m	500m	128Mi	512Mi
Total	250m	2000m	640Mi	2Gi

Network Resources

• Bandwidth: Initial sync will download all repos (estimate based on GitHub API query)
• Ongoing: Incremental updates only (minimal bandwidth)
• Ingress: Standard HTTPS traffic for Gitea UI access

---

Monitoring and Maintenance

Regular Tasks

Weekly:

• Monitor NFS repository storage growth: df -h /storage-mass/ocp-home/gitea
• Monitor database storage usage: oc exec -n gitea gitea-postgresql-1 -- psql -U gitea -c "SELECT pg_size_pretty(pg_database_size('gitea'));"
• Review mirror sync success rate: oc logs -n gitea deployment/gitea-mirror --tail=100

Monthly:

• Check for Gitea Helm chart updates: helm repo update gitea-charts && helm search repo gitea
• Update Gitea version if stable release available
• Review mirror sync logs for patterns
• Verify NFS backup completion

Quarterly:

• Test Velero backup restoration to temporary namespace
• Test NFS-level backup restoration
• Verify all critical repos are mirrored
• Review and clean up archived/deleted repo mirrors
• Check PostgreSQL performance (may need tuning)

Annually:

• Rotate GitHub fine-grained token (1-year expiration)
• Review storage capacity planning
• Update documentation

Metrics to Track

Metric	Command	Target
Mirrored Repos Count	oc exec -n gitea <pod> -- gitea admin repo list | wc -l	Match morey-tech repo count
Repository Storage Used	oc exec -n gitea <pod> -- df -h /data/git/repositories	< 80% of 100Gi
NFS Server Storage	df -h /storage-mass/ocp-home/gitea	< 80% of 100Gi
Database Size	oc exec -n gitea gitea-postgresql-1 -- psql -U gitea -c "SELECT pg_size_pretty(pg_database_size('gitea'));"	< 15Gi of 20Gi
Mirror Sync Success	oc logs -n gitea deployment/gitea-mirror --tail=50	No errors
Backup Completion	velero backup get --selector backup.velero.io/schedule=daily | grep gitea	Daily completion

---

Alternative Approaches Considered

1. Pure LVMS Approach (Original Plan)

Approach: Use LVMS for both repository and database storage
Pros: Higher performance, simpler configuration, lower network dependency
Cons:

• Inconsistent with existing infrastructure (immich, paperless use NFS)
• Limited capacity (constrained by NVMe size)
• ReadWriteOnce limitation (no multi-pod scaling)
• Single point of failure (host-dependent)
  Decision: ❌ Rejected - Inconsistent pattern, limited scalability

2. All-NFS Approach

Approach: Use NFS for both repository and database storage
Pros: Consistent storage pattern, maximum capacity, simple backup
Cons:

• Poor database performance (network latency)
• Not recommended by CloudNativePG operator
• IOPS bottleneck for transactional workloads
  Decision: ❌ Rejected - Database performance critical

3. Hybrid NFS + LVMS (Selected)

Approach: NFS for repositories (100Gi RWX), LVMS for database (20Gi RWO)
Pros:

• ✅ Leverages strengths of both storage types
• ✅ Consistent with existing infrastructure
• ✅ Optimal performance for each workload
• ✅ Maximum capacity for repository data
• ✅ Supports future scaling (RWX)
  Cons:
• More complex architecture (two storage backends)
• Requires NFS server setup
  Decision: ✅ Selected - Best balance of performance, capacity, and consistency

4. Manual Mirror Setup

Approach: Use Gitea UI to manually create mirrors for each repo
Pros: Simple, no additional automation complexity
Cons:

• Doesn't scale for all morey-tech repos
• Manual effort to keep in sync
• Risk of missing new repos
  Decision: ❌ Rejected - automation required for all repos

5. jaedle/mirror-to-gitea

Approach: Use jaedle/mirror-to-gitea container for automation
Pros: Simpler configuration
Cons:

• Only mirrors Git repositories (no issues, PRs, labels)
• No auto-discovery or cleanup features
• Less configuration flexibility
  Decision: ❌ Rejected - RayLabsHQ/gitea-mirror provides metadata mirroring

---

Success Criteria

Must Have (MVP)

• NFS export configured on 192.168.6.20
• NFS PV and PVC created and bound
• Gitea deployed and accessible via HTTPS
• PostgreSQL on LVMS healthy and connected
• All public morey-tech repos mirrored
• All private morey-tech repos mirrored
• RayLabsHQ/gitea-mirror auto-discovery functioning
• Metadata mirrored (issues, PRs, labels)
• Velero backups capturing gitea namespace
• NFS-level backup configured

Should Have

• Documentation complete in README.md
• NFS mount performance benchmarked
• Monitoring dashboard or metrics collection
• Mirror sync failure alerting
• Quarterly backup restoration test procedure documented

Nice to Have

• Prometheus metrics integration
• Grafana dashboard for Gitea and storage metrics
• Slack/webhook notifications on mirror sync failures
• Historical storage growth tracking
• NFSv4 Kerberos authentication

---

References

Official Documentation

• Gitea Helm Chart Repository
• Gitea Helm Chart Values
• Gitea Configuration Cheat Sheet
• Repository Mirror Documentation
• CloudNativePG Documentation
• CloudNativePG Cluster API
• Kubernetes NFS Persistent Volumes
• OpenShift NFS Storage Guide

Third-Party Tools

• RayLabsHQ/gitea-mirror GitHub Repository
• RayLabsHQ/gitea-mirror Container Registry

OpenShift Resources

• Security Context Constraints Guide
• OpenShift Security Context Constraints Blog

Community Resources

• Gitea Helm Chart v11 Release Blog
• Database Performance Discussion
• Gitea OpenShift Deployments

---

Next Steps

1. Review and Approve: Review this updated plan for completeness and accuracy
2. Assign: Assign to appropriate team member or self-assign
3. Milestone: Add to appropriate milestone/sprint
4. Branch: Create feature branch feat/gitea-mirror-nfs-storage
5. Phase 1: Configure NFS server export
6. Phase 2: Create Kubernetes resources (PV, PVC, namespace)
7. Phase 3: Deploy Gitea application
8. Phase 4: Deploy mirror automation
9. Phase 5: Validation and testing
10. Documentation: Update homelab documentation
11. Closure: Close issue with summary of deployment results

---

Related Documentation

• Detailed NFS Storage Plan: plans/gitea-nfs-storage-update.md
• Original Issue: This issue (Deploy Gitea GitHub Mirror for Disaster Recovery #66)

---

This issue was generated as part of the homelab infrastructure improvement initiative. Updated 2026-01-25 to use NFS-backed storage for repositories, following proven patterns from immich and paperless deployments.

[morey-tech]

Updated Recommendation: RayLabsHQ/gitea-mirror

After further research, I recommend using RayLabsHQ/gitea-mirror instead of jaedle/mirror-to-gitea for the automated mirroring component.

Key Advantages

Enhanced Mirroring Capabilities:

• ✅ Metadata mirroring: Issues, pull requests (as issues), labels, milestones, and wiki content
• ✅ Git LFS support: Built-in support for large file mirroring
• ✅ Auto-discovery (v3.4.0+): Automatically imports new repositories from GitHub
• ✅ Auto-cleanup: Removes deleted repos with configurable actions (archive or delete)
• ✅ Flexible strategies: Preserve structure, single organization, flat user, or mixed mode
• ✅ Token encryption: AES-256-GCM encryption for API credentials
• ✅ Active development: Recent updates indicate ongoing maintenance

vs. jaedle/mirror-to-gitea:

• jaedle tool only mirrors Git repositories (no issues, PRs, labels, etc.)
• No auto-discovery or cleanup features
• Less configuration flexibility

Updated CronJob Configuration

Container Image:

image: ghcr.io/raylabshq/gitea-mirror:latest

Environment Variables:

env:
  # GitHub Configuration
  - name: GITHUB_TOKEN
    valueFrom:
      secretKeyRef:
        name: github-org-token
        key: token
  
  # Gitea Configuration
  - name: GITEA_URL
    value: "https://git.apps.ocp-home.rh-lab.morey.tech"
  - name: GITEA_TOKEN
    valueFrom:
      secretKeyRef:
        name: gitea-admin-credentials
        key: api-token
  
  # Mirroring Configuration
  - name: AUTO_IMPORT_REPOS
    value: "true"  # Auto-discover new repos
  - name: CLEANUP_DELETE_IF_NOT_IN_GITHUB
    value: "archive"  # Archive deleted repos instead of deleting
  - name: GITEA_MIRROR_INTERVAL
    value: "8h"  # Sync every 8 hours (or "1d" for daily)
  - name: SCHEDULE_ENABLED
    value: "true"
  - name: SCHEDULE_INTERVAL
    value: "0 */8 * * *"  # Every 8 hours
  
  # Security
  - name: BETTER_AUTH_SECRET
    valueFrom:
      secretKeyRef:
        name: gitea-mirror-secret
        key: auth-secret

Implementation Notes

Deployment Options:

1. Option A: CronJob (Original Approach)

   • Run gitea-mirror as a Kubernetes CronJob
   • Executes sync operation on schedule
   • Container exits after sync completes

2. Option B: Deployment with Internal Scheduler (Recommended)

   • Deploy gitea-mirror as a regular Deployment (not CronJob)
   • Built-in scheduler handles sync intervals
   • Includes web UI for monitoring and configuration (optional)
   • More efficient - container stays running, no cold starts

Recommended Approach: Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitea-mirror
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitea-mirror
  template:
    metadata:
      labels:
        app: gitea-mirror
    spec:
      containers:
      - name: gitea-mirror
        image: ghcr.io/raylabshq/gitea-mirror:latest
        ports:
        - containerPort: 3000  # Web UI (optional)
        env:
          # Environment variables as above
        resources:
          requests:
            cpu: 50m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 512Mi

Additional Benefits for Disaster Recovery

Since your primary use case is disaster recovery, the metadata mirroring is particularly valuable:

• Complete backup: Not just code, but issues, PRs, labels, milestones, wiki
• True mirror: If GitHub goes down, you have all project context
• LFS support: Large binary files are backed up automatically
• Auto-cleanup: Deleted repos are archived, preserving history

Updated Directory Structure

Add to kubernetes/ocp-home/applications/gitea/:

├── gitea-mirror-deployment.yaml         # Gitea mirror deployment (replaces cronjob)
├── gitea-mirror-secret.yaml            # ExternalSecret for auth secret
└── gitea-mirror-service.yaml           # Service for web UI (optional)

Migration Path

If you've already implemented with jaedle/mirror-to-gitea:

1. Deploy RayLabsHQ/gitea-mirror alongside existing CronJob
2. Verify it discovers and syncs all repos correctly
3. Confirm metadata (issues, PRs) is being mirrored
4. Remove jaedle CronJob once validated
5. Enjoy enhanced mirroring capabilities

References

• GitHub Repository: https://github.com/RayLabsHQ/gitea-mirror
• Container Registry: ghcr.io/raylabshq/gitea-mirror
• License: AGPL-3.0

This tool provides a significantly more complete disaster recovery solution for the homelab infrastructure.