Skip to content

fix(skills): assemble voice clips at the audio file's duration, not the frame's - #5720

Closed
dennisofficial wants to merge 1 commit into
heygen-com:mainfrom
dennisofficial:fix/assemble-voice-clip-duration
Closed

dennisofficial wants to merge 1 commit into
heygen-com:mainfrom
dennisofficial:fix/assemble-voice-clip-duration

Conversation

@dennisofficial

Copy link
Copy Markdown

Problem

skills/faceless-explainer/scripts/assemble-index.mjs mounted each per-frame voice <audio> clip with data-duration="<frame duration>". The storyboard frame duration covers the frame's whole beat — content plus inter-frame padding — so the assembled voice clip's window was longer than the audio file. The runtime treats a data-duration longer than the media as an extended clip window, rendering the speech and then padding the remainder with silence: dead air before the next frame's line.

Reproduced in a real assembled project: resizing a voice clip in the Studio editor snapped it shorter than its placed duration, revealing the window had never matched the media. After hand-correcting the assembled index.html, each voice clip's data-duration equals its .wav media length (e.g. 6.91 for a 6.912s file, 15.45 for 15.445s).

Fix

Measure the voice file with ffprobe — the same invocation ensureBgmCovers already uses for BGM — and emit the measured duration as the clip's data-duration. When the probe is unavailable or unreadable, assembly falls back to the frame duration with an anomaly note rather than hard-failing on missing audio tooling, matching the script's existing degrade-gracefully stance.

The other clip kinds were checked and do not share the bug:

  • BGM is loop-extended to the full video length (ensureBgmCovers) before being mounted at data-duration=TOTAL — deliberate.
  • SFX carries its own cue.duration_s authored in audio_meta.json, never the frame duration.
  • transitions.mjs explicitly never touches audio elements (if (name.toLowerCase() === "audio") return tag;).

Verification

  • node --test skills/faceless-explainer/scripts/assemble-index.test.mjs — 6/6 pass: the 4 existing tests plus two new regression tests:
    • a 1.5s voice file under a 3s storyboard frame assembles with data-duration≈1.5, not 3;
    • an unprobeable voice file falls back to the frame duration and surfaces a could not probe anomaly.
  • Cross-checked against a hand-corrected production assembly: the fix's probed durations match the corrected values to probe rounding.
  • bunx oxfmt --check and bunx oxlint clean on both changed files.
  • skills-manifest.json regenerated with bun run --cwd packages/cli gen:skills-manifest (only the faceless-explainer hash drifted, as expected).

…he frame's

assemble-index.mjs mounted each per-frame voice <audio> with
data-duration=<frame duration>, but the storyboard frame duration covers
the frame's whole beat including inter-frame padding. A clip window longer
than the media renders the speech and then pads the remainder with
silence, producing dead air between frames.

Measure the file with ffprobe (the same invocation ensureBgmCovers already
uses) and emit that as data-duration; fall back to the frame duration with
an anomaly note when the probe is unavailable or unreadable, so assembly
still never hard-fails on missing audio tooling. BGM and SFX clip kinds
were checked and do not share the bug: BGM is loop-extended to TOTAL
before mounting, and SFX carries its own cue duration_s.
@jrusso1020

Copy link
Copy Markdown
Collaborator

Reviewed at 2fc708e4. This one can be closed. #5651 (merged 2026-10-21) retired the workflow skills, including faceless-explainer. skills/faceless-explainer/scripts/assemble-index.mjs and its test no longer exist on main, and none of the skills that remain assembles per-frame voice clips the same way, so the fix has nowhere to land.

Thanks for tracking this down carefully, especially for checking that BGM and SFX don't share the problem.

Need: no, #5651 removed the code · Size: ok · Reuse: n/a · Simpler: n/a

— Rames

@jrusso1020 jrusso1020 closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants