Skip to content

[BUG] Composite egress clipped at the end (audio-only, segmented) #1101

@decafdennis

Description

@decafdennis

Describe the bug

Some amount of audio is missing at the end of our composite egress output. We know this for certain because we have track egress running as well, and the missing audio is present in the track egress output.

I found a few resolved GH issues related to improper draining at the end of tracks, but I still experienced this today.

Egress Version

Everything is LiveKit-hosted.

Egress Request

How we configure the room that we dispatch the agent into:

  const s3Output = {
    case: "s3" as const,
    value: new S3Upload({
      accessKey: egressCredentials.AccessKeyId,
      secret: egressCredentials.SecretAccessKey,
      sessionToken: egressCredentials.SessionToken,
      region: LIVEKIT_EGRESS_BUCKET_REGION,
      bucket: LIVEKIT_EGRESS_BUCKET_NAME,
    }),
  };

  return new RoomEgress({
    // This gets us a mix of all audio tracks that is properly synchronized
    room: new RoomCompositeEgressRequest({
      audioOnly: true,
      // Dual channel doesn't seem to work, so we'll continue to do track egress just so we have a way to separate them in the future
      // audioMixing: AudioMixing.DUAL_CHANNEL_AGENT,
      // Segmented output gets us an m3u8 playlist with accurate wall clock timestamps
      segmentOutputs: [
        new SegmentedFileOutput({
          filenamePrefix: `${egressKeyPrefix}/composite/`,
          // The default is 4 seconds, which is around 100 KB, but that's a bit short.
          // 10 seconds is around 250 KB, which gives us 360 segments for an hour.
          segmentDuration: 10,
          output: s3Output,
        }),
      ],
    }),
    // This gets us raw, separate audio tracks in case we ever want to separate them
    tracks: new AutoTrackEgress({
      filepath: `${egressKeyPrefix}/tracks/${livekitEgressFilenamePattern}`,
      output: s3Output,
    }),
  });

Additional context

Reached out to LiveKit support, but haven't heard back in a while.

Example rooms where this occurred, if it helps:
RM_mVGuagRYKcuH (few weeks ago)
RM_TZyPtq8qfm2b (today)

How this problem manifests itself for us:

  • AI agent asks user if they're OK being transferred
  • User says "yes"
  • AI agent uses a tool that ends the phone call on the Twilio side, which in turn causes the SIP participant to disconnect and agent session to close itself
  • We transcribe the composite egress audio for quality assurance, and the "yes" will be missing from the composite egress (but it's there in the track egress)

Logs

n/a (LiveKit-hosted)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions