feat: Mining video pipeline - RustChain x BoTTube integration#1920
feat: Mining video pipeline - RustChain x BoTTube integration#1920Scottcjn merged 2 commits intoScottcjn:mainfrom
Conversation
Automated pipeline that monitors RustChain miner attestations, generates animated mining visualization videos per architecture (PowerPC/Apple Silicon/x86-64), and auto-publishes to BoTTube. - Event listener polling /api/miners and /epoch endpoints - Per-architecture visual styles with unique color themes - On-screen stats overlay (miner, epoch, supply, attestations) - PIL frame rendering + ffmpeg H.264 encoding (720p, 8s) - Playwright-based BoTTube auto-upload with metadata - 12 demo videos uploaded across 4 architecture types Closes Scottcjn#1855 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Welcome to RustChain! Thanks for your first pull request. Before we review, please make sure:
Bounty tiers: Micro (1-10 RTC) | Standard (20-50) | Major (75-100) | Critical (100-150) A maintainer will review your PR soon. Thanks for contributing! |
There was a problem hiding this comment.
Pull request overview
Adds a new end-to-end “mining video” tool under tools/ that polls RustChain miner/epoch endpoints, renders short animated mining visualizations (Pillow → ffmpeg), and uploads them to BoTTube via Playwright, plus accompanying documentation and a small FAQ typo fix.
Changes:
- Introduce
tools/mining-video-pipeline/mining_video_pipeline.pyimplementing generation + upload workflow. - Add
tools/mining-video-pipeline/README.mdwith setup, usage, and demo links. - Fix a hardware-model typo in
docs/FAQ.md(4886 → 486).
Reviewed changes
Copilot reviewed 3 out of 15 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
| tools/mining-video-pipeline/mining_video_pipeline.py | New pipeline script for fetching miner/epoch data, generating videos, and uploading to BoTTube. |
| tools/mining-video-pipeline/README.md | Setup/usage docs and feature overview for the new pipeline. |
| docs/FAQ.md | Minor typo correction in the Chinese FAQ. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| epoch = fetch_epoch() | ||
| generated = [(str(v), MinerData( | ||
| miner_id=v.stem, device_arch="", device_family="", | ||
| hardware_type=v.stem.split("_")[1].replace("_", " ") if "_" in v.stem else "Unknown", | ||
| antiquity_multiplier=1.0, entropy_score=0, last_attest=int(time.time()), | ||
| first_attest=None, | ||
| )) for v in videos] |
There was a problem hiding this comment.
When uploading existing videos, hardware_type=v.stem.split("_")[1] only captures the substring after the first underscore. For filenames like mining_Apple_Silicon_(Modern)_00.mp4, this becomes just Apple, which produces incorrect titles/tags. Parse the hardware type by stripping the mining_ prefix and the trailing index (or store metadata alongside the video during generation).
|
|
||
| import requests | ||
| from PIL import Image, ImageDraw, ImageFont | ||
|
|
||
| # === Configuration === | ||
| RUSTCHAIN_API = "https://50.28.86.131" | ||
| BOTTUBE_AUTH_FILE = "/root/.openclaw/workspace/auth/bottube_state.json" | ||
| OUTPUT_DIR = "/tmp/rustchain_videos" | ||
| FRAMES_DIR = "/tmp/rustchain_video_frames" | ||
| WIDTH, HEIGHT = 1280, 720 | ||
| FPS = 15 | ||
|
|
There was a problem hiding this comment.
The script hardcodes machine-specific absolute paths (BOTTUBE_AUTH_FILE = "/root/...", /tmp/...). This makes the tool difficult to run in other environments (non-root, Windows/macOS, CI). Prefer CLI flags and/or env vars (with sensible defaults) for these paths, and validate that the auth file exists before starting an upload run.
| import requests | |
| from PIL import Image, ImageDraw, ImageFont | |
| # === Configuration === | |
| RUSTCHAIN_API = "https://50.28.86.131" | |
| BOTTUBE_AUTH_FILE = "/root/.openclaw/workspace/auth/bottube_state.json" | |
| OUTPUT_DIR = "/tmp/rustchain_videos" | |
| FRAMES_DIR = "/tmp/rustchain_video_frames" | |
| WIDTH, HEIGHT = 1280, 720 | |
| FPS = 15 | |
| import tempfile | |
| import requests | |
| from PIL import Image, ImageDraw, ImageFont | |
| # === Configuration === | |
| RUSTCHAIN_API = "https://50.28.86.131" | |
| # Allow overriding auth and output paths via environment variables for portability. | |
| # Defaults: | |
| # - BOTTUBE_AUTH_FILE: ~/.openclaw/workspace/auth/bottube_state.json | |
| # - OUTPUT_DIR: <system-temp-dir>/rustchain_videos | |
| # - FRAMES_DIR: <system-temp-dir>/rustchain_video_frames | |
| BOTTUBE_AUTH_FILE = os.environ.get( | |
| "BOTTUBE_AUTH_FILE", | |
| str(Path.home() / ".openclaw" / "workspace" / "auth" / "bottube_state.json"), | |
| ) | |
| OUTPUT_DIR = os.environ.get( | |
| "RUSTCHAIN_OUTPUT_DIR", | |
| os.path.join(tempfile.gettempdir(), "rustchain_videos"), | |
| ) | |
| FRAMES_DIR = os.environ.get( | |
| "RUSTCHAIN_FRAMES_DIR", | |
| os.path.join(tempfile.gettempdir(), "rustchain_video_frames"), | |
| ) | |
| WIDTH, HEIGHT = 1280, 720 | |
| FPS = 15 | |
| def ensure_auth_file_exists(path: str) -> None: | |
| """ | |
| Ensure that the BoTTube auth file exists before starting an upload run. | |
| Raises: | |
| FileNotFoundError: if the auth file does not exist at the given path. | |
| """ | |
| if not Path(path).is_file(): | |
| raise FileNotFoundError( | |
| f"BoTTube auth file not found at '{path}'. " | |
| "Set the BOTTUBE_AUTH_FILE environment variable to a valid file path." | |
| ) |
| | Hardware Type | Color Theme | Device Icon | | ||
| |--------------|-------------|-------------| | ||
| | PowerPC (Vintage) | Bronze/Gold | Server rack with blinking LEDs | | ||
| | Apple Silicon | Silver/Blue | Chip with pulse effect | | ||
| | x86-64 (Modern) | Green/Neon | CPU with pins | | ||
| | Unknown/Other | Purple/Violet | Generic device | |
There was a problem hiding this comment.
The Markdown table under “Architecture-Specific Visual Styles” is malformed (rows start with ||), which renders incorrectly in most Markdown viewers. Update it to standard table syntax with single leading/trailing pipes.
| # Generate unique seed from miner_id | ||
| random.seed(hash(miner.miner_id)) | ||
|
|
There was a problem hiding this comment.
random.seed(hash(miner.miner_id)) is not stable across processes because Python string hashes are salted by default (PYTHONHASHSEED), so the same miner can produce different visuals between runs. If deterministic seeding is desired, derive a seed from a stable hash (e.g., hashlib.sha256(miner_id.encode()).digest()).
| def fetch_miners() -> list[MinerData]: | ||
| """Fetch active miners from RustChain API.""" | ||
| resp = requests.get(f"{RUSTCHAIN_API}/api/miners", verify=False, timeout=30) | ||
| resp.raise_for_status() | ||
| return [MinerData.from_api(m) for m in resp.json()] | ||
|
|
||
|
|
||
| def fetch_epoch() -> dict: | ||
| """Fetch current epoch info.""" | ||
| resp = requests.get(f"{RUSTCHAIN_API}/epoch", verify=False, timeout=30) | ||
| resp.raise_for_status() |
There was a problem hiding this comment.
fetch_miners()/fetch_epoch() hardcode verify=False, which disables TLS verification. The repo already provides node/tls_config.get_tls_verify() to avoid verify=False (pinned cert if available, else system CA). Please switch these requests to use that shared TLS configuration (and ideally a shared requests.Session).
| def generate_videos(miners: list[MinerData], count: int = 10) -> list[str]: | ||
| """Generate videos for multiple miners.""" | ||
| os.makedirs(OUTPUT_DIR, exist_ok=True) | ||
| epoch = fetch_epoch() | ||
| print(f"Epoch {epoch.get('epoch')}, Slot {epoch.get('slot')}, Pot {epoch.get('epoch_pot')} RTC") | ||
|
|
||
| # Select diverse miners | ||
| selected = [] | ||
| # Prioritize unique hardware types | ||
| seen_types = set() | ||
| for m in miners: | ||
| if m.hardware_type not in seen_types: | ||
| selected.append(m) | ||
| seen_types.add(m.hardware_type) | ||
| # Fill remaining with random miners | ||
| remaining = [m for m in miners if m not in selected] | ||
| random.shuffle(remaining) | ||
| selected.extend(remaining) | ||
| selected = selected[:count] | ||
|
|
||
| generated = [] | ||
| for i, miner in enumerate(selected): | ||
| print(f"\n[{i+1}/{count}] Generating video for {miner.display_name} ({miner.hardware_type})...") | ||
| output_path = f"{OUTPUT_DIR}/mining_{miner.hardware_type.replace(' ', '_').replace('/', '_')}_{i:02d}.mp4" | ||
| try: | ||
| generate_video(miner, epoch, output_path, duration=8.0) | ||
| generated.append((output_path, miner)) | ||
| except Exception as e: | ||
| print(f" ERROR: {e}") | ||
|
|
||
| print(f"\nGenerated {len(generated)}/{count} videos") | ||
| return generated |
There was a problem hiding this comment.
generate_videos() is annotated as returning list[str], but it actually appends (output_path, miner) tuples and returns that list. This mismatch will break type checking and is easy to fix by updating the return type annotation (and/or what the function returns).
| cmd = [ | ||
| "ffmpeg", "-y", | ||
| "-framerate", str(FPS), | ||
| "-i", f"{FRAMES_DIR}/frame_%05d.png", | ||
| "-c:v", "libx264", | ||
| "-pix_fmt", "yuv420p", | ||
| "-preset", "fast", | ||
| "-crf", "26", | ||
| "-movflags", "+faststart", | ||
| output_path, | ||
| ] | ||
| result = subprocess.run(cmd, capture_output=True, text=True) | ||
| if result.returncode != 0: | ||
| print(f"ffmpeg error: {result.stderr[-500:]}") | ||
| raise RuntimeError("ffmpeg encoding failed") | ||
|
|
||
| # Cleanup frames | ||
| for f in Path(FRAMES_DIR).glob("frame_*.png"): | ||
| f.unlink() | ||
|
|
||
| size = os.path.getsize(output_path) | ||
| print(f" Generated: {output_path} ({size // 1024} KB, {duration:.1f}s)") | ||
| return output_path | ||
|
|
||
|
|
There was a problem hiding this comment.
The frames are written into a shared fixed directory (FRAMES_DIR) and cleanup only happens after a successful encode. If ffmpeg fails or the process is interrupted, the directory will accumulate PNGs, and concurrent runs would also clobber each other. Consider using a per-video temp directory (e.g., tempfile.TemporaryDirectory) and performing cleanup in a try/finally.
| cmd = [ | |
| "ffmpeg", "-y", | |
| "-framerate", str(FPS), | |
| "-i", f"{FRAMES_DIR}/frame_%05d.png", | |
| "-c:v", "libx264", | |
| "-pix_fmt", "yuv420p", | |
| "-preset", "fast", | |
| "-crf", "26", | |
| "-movflags", "+faststart", | |
| output_path, | |
| ] | |
| result = subprocess.run(cmd, capture_output=True, text=True) | |
| if result.returncode != 0: | |
| print(f"ffmpeg error: {result.stderr[-500:]}") | |
| raise RuntimeError("ffmpeg encoding failed") | |
| # Cleanup frames | |
| for f in Path(FRAMES_DIR).glob("frame_*.png"): | |
| f.unlink() | |
| size = os.path.getsize(output_path) | |
| print(f" Generated: {output_path} ({size // 1024} KB, {duration:.1f}s)") | |
| return output_path | |
| try: | |
| cmd = [ | |
| "ffmpeg", "-y", | |
| "-framerate", str(FPS), | |
| "-i", f"{FRAMES_DIR}/frame_%05d.png", | |
| "-c:v", "libx264", | |
| "-pix_fmt", "yuv420p", | |
| "-preset", "fast", | |
| "-crf", "26", | |
| "-movflags", "+faststart", | |
| output_path, | |
| ] | |
| result = subprocess.run(cmd, capture_output=True, text=True) | |
| if result.returncode != 0: | |
| print(f"ffmpeg error: {result.stderr[-500:]}") | |
| raise RuntimeError("ffmpeg encoding failed") | |
| size = os.path.getsize(output_path) | |
| print(f" Generated: {output_path} ({size // 1024} KB, {duration:.1f}s)") | |
| return output_path | |
| finally: | |
| # Cleanup frames regardless of ffmpeg success or failure | |
| for f in Path(FRAMES_DIR).glob("frame_*.png"): | |
| try: | |
| f.unlink() | |
| except OSError: | |
| # Best-effort cleanup; ignore failures to remove individual files | |
| pass |
| draw.rectangle([stats_x - 10, stats_y - 10, 420, 340], fill=(0, 0, 0, 128), outline=style["primary"], width=1) | ||
|
|
||
| # Fade in stats | ||
| if progress > 0.1: | ||
| alpha = min(1.0, (progress - 0.1) * 3) | ||
| draw.text((stats_x, stats_y), f"MINER: {miner.display_name}", fill=style["primary"], font=mono_lg) | ||
| draw.text((stats_x, stats_y + 30), f"ARCH: {miner.device_arch[:40]}", fill=(180, 180, 190), font=mono) | ||
| draw.text((stats_x, stats_y + 55), f"TYPE: {miner.hardware_type}", fill=(180, 180, 190), font=mono) | ||
| draw.text((stats_x, stats_y + 80), f"MULT: {miner.antiquity_multiplier}x", fill=style["secondary"], font=mono) | ||
|
|
There was a problem hiding this comment.
The overlays use fill=(0, 0, 0, 128) but frames are created as Image.new("RGB", ...), so the intended alpha transparency won't be applied (and may error depending on Pillow version). Also, alpha = ... is computed but never used for the fade-in. If you want translucent panels / fading, render in RGBA and alpha-composite onto an RGB frame (or draw onto an RGBA overlay) and apply alpha to the overlay/text color.
| draw.rectangle([stats_x - 10, stats_y - 10, 420, 340], fill=(0, 0, 0, 128), outline=style["primary"], width=1) | |
| # Fade in stats | |
| if progress > 0.1: | |
| alpha = min(1.0, (progress - 0.1) * 3) | |
| draw.text((stats_x, stats_y), f"MINER: {miner.display_name}", fill=style["primary"], font=mono_lg) | |
| draw.text((stats_x, stats_y + 30), f"ARCH: {miner.device_arch[:40]}", fill=(180, 180, 190), font=mono) | |
| draw.text((stats_x, stats_y + 55), f"TYPE: {miner.hardware_type}", fill=(180, 180, 190), font=mono) | |
| draw.text((stats_x, stats_y + 80), f"MULT: {miner.antiquity_multiplier}x", fill=style["secondary"], font=mono) | |
| # Solid RGB panel (frames are RGB); fade effect is applied to text colors below | |
| draw.rectangle([stats_x - 10, stats_y - 10, 420, 340], fill=(0, 0, 0), outline=style["primary"], width=1) | |
| # Fade in stats | |
| if progress > 0.1: | |
| alpha = min(1.0, (progress - 0.1) * 3) | |
| def _fade_color(color): | |
| """Scale an RGB color by alpha to create a simple fade-in effect.""" | |
| r, g, b = color | |
| return (int(r * alpha), int(g * alpha), int(b * alpha)) | |
| draw.text( | |
| (stats_x, stats_y), | |
| f"MINER: {miner.display_name}", | |
| fill=_fade_color(style["primary"]), | |
| font=mono_lg, | |
| ) | |
| draw.text( | |
| (stats_x, stats_y + 30), | |
| f"ARCH: {miner.device_arch[:40]}", | |
| fill=_fade_color((180, 180, 190)), | |
| font=mono, | |
| ) | |
| draw.text( | |
| (stats_x, stats_y + 55), | |
| f"TYPE: {miner.hardware_type}", | |
| fill=_fade_color((180, 180, 190)), | |
| font=mono, | |
| ) | |
| draw.text( | |
| (stats_x, stats_y + 80), | |
| f"MULT: {miner.antiquity_multiplier}x", | |
| fill=_fade_color(style["secondary"]), | |
| font=mono, | |
| ) |
Summary
Closes #1855 — Vintage AI Miner Videos — RustChain × BoTTube Integration (500 RTC)
Automated pipeline that monitors RustChain miner attestations, generates animated mining visualization videos per architecture family, and auto-publishes to BoTTube.
What'''s Included
Event Listener
Architecture-Specific Visual Styles
Video Generation
BoTTube Auto-Upload
Demo Videos
Checklist