Skip to content

Implement backgroundBlur capability for video input devices #69

@General-Fault

Description

@General-Fault

Summary

Background blur is reported as a \�ackgroundBlur\ capability in \MediaTrackCapabilities\ per the W3C Media Capture spec. It is currently always empty (not reported) because libwebrtc does not implement background blur — Chromium implements it via a separate ML-based video effects pipeline (\BackgroundBlurDenoiser) running outside of libwebrtc.

Required work

To report \�ackgroundBlur: [true, false]\ we need a working implementation consumers can actually enable. On Windows this means one of:

  1. *WinRT \Windows.Media.Effects* — integrate a WinRT video effects processor that applies a background blur pass after the libwebrtc capture pipeline.
  2. ONNX Runtime / Windows ML — run an ML segmentation model (e.g., MediaPipe Selfie Segmentation) via \Microsoft.ML.OnnxRuntime\ or \Windows.AI.MachineLearning.
  3. DirectML — lower-level GPU inference.

Option 1 is the most Windows-idiomatic path for C++/CLI integration.

Architecture notes

  • \VideoCapabilityQuery::Query()\ is where \�ackgroundBlur\ should be set; it currently always passes
    ullptr.
  • The constraint application path (processing a \getUserMedia\ call with \�ackgroundBlur: true) is also unimplemented and is blocked on having an effects pipeline.
  • Background blur is a video track post-processor, not a hardware device capability, so reporting it in capabilities implies we own its implementation.

References

  • W3C def-constraint-backgroundBlur
  • Chromium reference: \ hird_party/blink/renderer/modules/mediastream/input_device_info.cc\
  • \WebRtcInterop/Media/VideoCapabilityQuery.cpp\ — add \�ackgroundBlur\ here when implementation exists

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions