[wonjangcloud9]

What

transcribe now refuses an .srt or .vtt file that is not UTF-8 (or UTF-16LE with a BOM). Before, it imported or exported such a file as replacement characters. The error names the likely encodings and the iconv command that converts the file.

Why

Subtitle files were decoded as UTF-8 with replacement, so a file in a legacy East Asian encoding "succeeded" as U+FFFD text. That is the common case for Korean subtitles from older tools, which save CP949 (EUC-KR). Shift_JIS and GBK files hit the same path. Three-cue Korean SRT, converted with iconv -f UTF-8 -t CP949:

Before (main):

$ hyperframes transcribe cp949.srt --to vtt -o cp949.vtt
◇  Exported 3 words to VTT → …/cp949.vtt        (exit 0)

00:00:00.500 --> 00:00:02.000
�ȳ��ϼ���, ������.

Importing it instead (hyperframes transcribe cp949.srt) goes through the same loadTranscript and would put that text into transcript.json and the patched caption HTML.

After:

$ hyperframes transcribe cp949.srt --to vtt -o cp949.vtt
…/cp949.srt is not UTF-8 text. Subtitles saved by older Korean, Japanese or Chinese tools are often CP949 (EUC-KR), Shift_JIS or GBK: convert the file to UTF-8 first, e.g. iconv -f CP949 -t UTF-8 "…/cp949.srt" > subtitles.utf8.srt
(exit 1, no file written)

The same file as UTF-8, UTF-8 with a BOM, or UTF-16LE with a BOM exports 안녕하세요, 여러분. exactly as before.

Related work

None found: I searched issues and PRs for encoding, CP949, EUC-KR, Shift_JIS, mojibake and "not UTF-8". #5843 and #6845 fixed nearby UTF-8 and BOM reads for project files and JSON flags.

How

loadTranscript reads SRT and VTT through a new readSubtitleText. It decodes UTF-16LE when the file starts with its BOM, as readJsonFileText did, and otherwise uses TextDecoder("utf-8", { fatal: true }), which still drops a UTF-8 BOM. On invalid UTF-8 it throws, and readTranscript already turns that into an invalid_transcript error.

Whisper JSON keeps the lenient decoder on purpose. whisper.cpp writes token texts that split a multi-byte character, and those arrive as U+FFFD (the transcript guide in skills/media-use lists the { "text": "�" } tokens). A strict read there would reject real transcripts.

I chose to refuse rather than guess an encoding. A CP949 byte sequence is often also valid in another legacy encoding, so a fallback could silently pick the wrong one.

Test plan

• normalize.test.ts: a CP949 .srt and .vtt (the bytes of 안녕하세요) now throw the encoding error. Both new cases fail on main, where they load as U+FFFD text. A Korean SRT saved as UTF-8 with a BOM and as UTF-16LE with a BOM still loads, on main and here. 87/87 pass.
• Manual: built the CLI and ran transcribe <file> --to vtt on the CP949, UTF-8, UTF-8+BOM and UTF-16LE files, plus the CP949 file after iconv (outputs above).
• vitest --project unit src/whisper src/api: everything passes except 3 sherpa.test.ts runtime-install cases, which fail the same way with main's normalize.ts on this machine. oxlint, oxfmt, tsc --noEmit, comment ratchet and fallow audit --base origin/main are clean.
• Documentation updated: docs/packages/cli.mdx (transcribe section).
• Comments follow CONTRIBUTING.md "Comments".

🤖 Generated with Claude Code

[mintlify]

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 88aaa1e.

Approve preview