Darkroom NotesSeason 5STIL.NOTEPart 5 of 6
Listening twice — when English broke the transcript
The Japanese play-by-play came through fine. But the moment an English comment began, the transcript collapsed into "Wel actaltlysputure…" — a string that looks like English and isn't. A transcription model can only listen in one language at a time. So I decided to have two of them listen to the same audio.
Last time, not losing things. Today, the hearing itself.
The text broke partway through
It happened when I imported a recording of a rugby broadcast. The Japanese play-by-play was coming through fine. Then the Italian captain began speaking English, and the transcript turned into this:
"Wel actaltlysputure But litedito her…"
Like English, but not English. Fragments of words melted and run together. And not "couldn't make it out" — broken text, lined up confidently. It's an unsettling kind of failure to read.
A model listens in one language
The reason turned out to be plain. Transcription models are separate per language. The Japanese model knows only the sound patterns of Japanese. When English sounds flow in, it takes what it hears and forces it onto the patterns it knows. That was the melted string.
And there is no way — at least in today's API — to tell it "switching to English now" partway through a recording. One language at a time: that's the model's premise.
Real conversations don't work that way. English proper nouns drift into a Japanese meeting. A call with someone overseas moves back and forth between the two. The material doesn't arrange itself for the model's convenience.
Then let two of them listen
The answer, after some thought, was simple. Play the same audio, start to finish, to both the Japanese model and the English model.
The key is the "confidence" number that comes with a transcript. For each stretch of audio, the model reports how sure it is. Measured for real: in the Japanese stretches, the Japanese model's confidence runs high (0.84–0.94); in the English stretches, the English model's does (0.64–0.89). They trade places, cleanly.
So, stretch by stretch, take the transcription from whichever listener is confident. Where the Japanese model holds its head high, use its text; where the English model wins, use its. Stitch together only the sure parts of two listeners.
The broken English came back
Trying it on the same rugby broadcast, the captain's comment came back as proper English sentences. The hand-off from Japanese play-by-play to English comment ran seamless. The melted string stood back up as the words that had actually been spoken.
It costs double
But this method has a plain price: listening twice takes roughly twice the time and twice the battery.
So I narrowed where it applies. Importing recorded files, and re-running a transcript after the fact — situations with no clock running — are where listening twice is available. Real-time recording, where words become text as they're spoken, still uses a single listener. Rather than a fix applied everywhere, a fix applied exactly where it works. The battery is something to protect for as long as the recording runs.
Next time
The next post closes Season 5. From a conversation being recorded to the note leaving your hands — the whole run, start to finish.
STIL.NOTE is out on the App Store.
