Skip to content

GEMINI.md

Latest commit

 

History

History
86 lines (63 loc) · 3.76 KB

GEMINI.md

File metadata and controls

86 lines (63 loc) · 3.76 KB

WebCodec Player

Project Overview

webcodec-player is a lightweight, browser-based video player designed to demonstrate low-latency video streaming using the modern WebCodecs API. It supports FLV streams delivered over WebSockets or HTTP (HTTP-FLV), decoding H.264 video content and rendering it to a Canvas.

The project emphasizes performance by offloading the intensive tasks of demuxing and decoding to a Web Worker, ensuring the main thread remains responsive.

Key Technologies

  • WebCodecs API: Used for hardware-accelerated video decoding (VideoDecoder).
  • Web Workers: Handles stream fetching, FLV demuxing, and decoding in a background thread.
  • OffscreenCanvas: Allows the Web Worker to render decoded frames directly without blocking the DOM.
  • WebSocket: Used for receiving the live video stream.
  • FLV (Flash Video): The supported container format for the video stream.

Architecture

The player is structured into three main layers:

  1. UI & Controller (index.html, player.js):

    • index.html provides the controls (Play, Pause, Destroy) and the container for the video.
    • player.js defines the Player class. It initializes the Canvas, spawns the Web Worker, and manages the communication (messaging) between the UI and the worker.

  2. Worker Layer (demux_decode_worker.js):

    • Acts as the engine of the player.
    • Receives the OffscreenCanvas from the main thread.
    • Initializes the FLVDemuxer and VideoDecoder.
    • Draws decoded frames onto the canvas.
    • Calculates and renders stats like FPS and bitrate.

  3. Data Processing (flv_demuxer.js):

    • Responsible for parsing the incoming FLV binary stream.
    • Extracts H.264 video packets and configuration (AVCC) for the decoder.

Building and Running

This project consists of static files (HTML, JS) but uses ES Modules and Web Workers, which means it must be served via an HTTP server. It cannot be run by simply opening index.html as a file.

Prerequisites

  • A modern browser with WebCodecs support (e.g., Chrome 94+).
  • A method to serve static files (e.g., Python, Node.js, VS Code Live Server).

Running the Player

  1. Start a local server. If you have Python installed:

    python3 -m http.server 8000

    Or using Node.js http-server:

    npx http-server .

  2. Access the player. Open your browser and navigate to: http://localhost:8000

  3. Test Stream: The player requires a WebSocket URL serving an FLV stream.

    • Enter a valid WebSocket URL in the input box (e.g., ws://your-server/live/stream.flv).
    • Click 播放 (Play).

Testing Streams (Backend)

The README.md suggests using ffmpeg to simulate a stream if you have an RTSP source:

ffmpeg -re -stream_loop -1 -i test.mp4 -c:v copy -rtsp_transport tcp -f rtsp rtsp://<server_ip>:8554/live/test121

Development Conventions

  • Modularization: Code is split into logical modules. common.js contains shared utilities (inferred).
  • Worker Communication: All control commands (play, destroy, showVbps) are sent to the worker via postMessage.
  • Error Handling: Basic error logging is present in the VideoDecoder configuration (error: e => console.error(e)).
  • Canvas Transfer: The transferControlToOffscreen() pattern is strictly used to decouple rendering from the main thread.

Future Improvements (TODO)

According to the documentation, the following features are planned:

  • Fullscreen support & player event callbacks.
  • Automatic reconnection logic on disconnect.
  • H.265 (HEVC) hardware decoding support.
  • HTTP-FLV protocol support.
  • Code packaging/bundling.