Audio and Video combined test

[yanj-github]

From the latest Fraunhofer in-factory testing, it was observed that none of the devices achieved a complete pass across all audio and video combined test observations.

• Device 1-2 passed all observations except for AV synchronization, where it failed.

  • FAIL: The allowed AV sync tolerance is [40, -120] ms. The starting tolerance is 1000ms and the ending tolerance is 1000ms. The required pass rate is 95%. Total failure count is 493, with 93 failures within the start and end tolerance. AV Sync was checked from 21000ms to 29000.0ms, and 18.86% was in sync. The AV Sync offset range is [131.88, 133.96] ms.

• Devices 2-1 and 2-2 passed the AV sync check; however, they failed in several other observations.

  • Devices 2-1
    • FAIL: Audio segments failed at the following timestamps: 20000ms 20020ms 20040ms 20060ms 20080ms 20100ms 20120ms 20140ms 20160ms 20180ms 20200ms 20220ms 20240ms 20260ms 20280ms 20300ms 20320ms 20340ms 20360ms 20380ms ...too many failures, reporting truncated. Found 27 segments are missing. 25 start segment failed, tolerance is 4. 2 mid segment failed, tolerance is 10.
    • FAIL: The earliest video and audio sample alignment tolerance is 80 ms. The earliest video sample presentation time is 20000.0 ms while the earliest audio sample presentation time is 20500 ms. There is a 500.0 ms time difference between video and audio sample presentation times.
  • Devices 2-2
    • FAIL: First frame found is 503, expected to start from 501. First frame number tolerance is 1. Mid frame number tolerance is 10. Total of missing frame count is 2.
    • FAIL: Audio segments failed at the following timestamps: 20000ms 20020ms 20040ms 20060ms 20080ms 20100ms 20120ms 29980ms Found 8 segments are missing. 7 start segment failed, tolerance is 4. 1 end segment failed, tolerance is 4.

This issue has been raised to track ongoing communication and to collect ideas and findings for further debugging and root-cause analysis.
Also related to previous issue #128