What is speaker diarization?

How diarization works

A diarization system first finds the parts of the recording that contain speech and throws away silence, music and noise. It then cuts the speech into short windows, often a second or two long, and turns each window into a compact numeric fingerprint of the voice, sometimes called a voice embedding.

Windows whose fingerprints sit close together are assumed to come from the same person, so the system clusters them. Each cluster becomes a speaker label. Finally the labels are lined up with the transcript, so every sentence carries the label of whoever was talking at that moment.

Note what diarization does not do: it doesn't know anyone's name. It only knows that the voice at 03:12 sounds like the voice at 41:05. Putting a name on a cluster is a separate step, called speaker identification or voice recognition.

A concrete example

Take a 50-minute podcast interview with a host and one guest. Without diarization you get a single block of text. With it, you get alternating turns: Speaker 1 asks a question, Speaker 2 answers for three minutes, Speaker 1 follows up. Now you can pull every answer the guest gave, skip the host's ad reads, or search for a phrase and see who said it.

Diarization also makes long recordings easier to skim. A two-hour panel becomes a list of turns you can scan by speaker, and word counts per speaker show at a glance who dominated the conversation. For interviews, it lets you export only the subject's answers as a clean document for review or quotes.

Where diarization goes wrong

Diarization is useful but imperfect. The usual failure modes are predictable, so it helps to know them before you rely on the labels:

• Overlapping speech. When two people talk at once, most systems assign the overlap to one of them.
• Similar voices. Two speakers of the same age and accent can merge into one label.
• One voice split in two. A person who changes microphones, shouts or whispers may be split into two clusters.
• Short interjections. A quick “yeah” or “right” is too short to fingerprint reliably and often lands on the wrong speaker.
• Unknown speaker count. If the system has to guess how many people are present, it can guess wrong in either direction.

In MediaFind

MediaFind runs diarization on your computer, with no API key, as part of indexing. Transcripts come back with speaker labels, and you can search by speaker as well as by words. Each voice is also matched on-device against voices you've named, so a name you give once follows that voice into other recordings. Matches MediaFind isn't sure about stay unnamed rather than being guessed.

In Meetings mode (Pro), the same labels feed the written-up notes, so an action item carries the name of the person who said it. For a broader look at making spoken media searchable, see transcription and finding a spoken word in a video.

Frequently asked questions

Is speaker diarization the same as speaker recognition?

No. Diarization groups speech by voice and gives each group an anonymous label. Speaker recognition matches a voice to a known, named person. Many tools do diarization first and recognition second.

Does diarization need to know how many speakers there are?

Not always. Many systems estimate the number of speakers on their own, but telling them the count, when you know it, usually reduces merged or split speakers.

Can diarization run offline?

Yes. Diarization models are small enough to run on a normal laptop, so recordings never need to leave your computer.