🎤 Speech Recognition

The Human Transcriber Analogy

Imagine hiring a professional transcriber for a meeting:

• They listen to people talking
• They understand different accents and speaking styles
• They filter out background noise (coughing, papers shuffling)
• They know when one sentence ends and another begins
• They type out every word accurately

A great transcriber does all this in real-time, without asking people to repeat themselves.

Speech Recognition is an AI that does this job.

It converts spoken language into written text - enabling voice assistants, automatic subtitles, and hands-free computer control.

Why Speech Recognition Matters

Voice is the most natural human interface.

• Typing: 40 words per minute
• Speaking: 150 words per minute

That's 3-4x faster! Plus:

• You can talk while your hands are busy (driving, cooking)
• Accessibility for people who can't type
• Natural interface for smart devices

What it powers:

How It Works (Simplified)

The Challenge

Sound is just vibrations. To a computer, speech looks like this:

[waves of pressure changes over time]

How do you get from squiggly waves to "Hello, how are you?"

The Process

Step 1: Capture Audio

Microphone converts sound waves into digital signals.

Step 2: Break Into Pieces

Split the audio into small chunks (milliseconds long).

Step 3: Find Patterns

Identify speech sounds called "phonemes" - the building blocks of language:

"Hello" = /h/ + /e/ + /l/ + /o/

Step 4: Build Words

Match phoneme sequences to words in a dictionary.

Step 5: Understand Context

Use language understanding to pick the right words:

"I scream" vs "Ice cream"
(Same sounds, different words - context helps!)

Modern Approach: End-to-End

Many modern systems skip the separate steps:

Audio → Big Neural Network → Text

The model learns the entire process from examples.

The Hardest Problems

1. Accents and Dialects

"Water" sounds very different from:

• American: "wah-der"
• British: "waw-tuh"
• Australian: "waw-tah"

The system needs to handle many of them.

2. Background Noise

Conversations in cafes, cars, busy streets. Music playing. Multiple people talking.

3. Homophones

Words that sound the same:

• "their" / "there" / "they're"
• "to" / "too" / "two"
• "write" / "right"

Context is one of the main ways to tell them apart.

4. Fast or Mumbled Speech

People don't speak clearly like robots. We slur, skip sounds, talk fast.

5. Technical Terms and Names

"I met with Siobhan about the async API integration." (Good luck transcribing that correctly!)

Real-World Examples

Voice Assistants

You: "Hey Siri, set a timer"

1. "Hey Siri" - Wake word detection
2. "set a timer" - Speech recognition
3. Understanding intent - Natural language processing
4. Setting the timer - Executing command

Automatic Captions

YouTube processes millions of hours of video:

1. Extract audio track
2. Run speech recognition
3. Sync text with timestamps
4. Display as subtitles

Meeting Transcription

Services like Otter.ai or Zoom:

1. Record meeting audio
2. Identify different speakers
3. Transcribe in real-time
4. Create searchable notes

How Good Is It Today?

Modern speech recognition is remarkably accurate:

But context matters:

• Clean studio audio: Often very accurate
• Phone call: Good
• Noisy street: Degrades significantly
• Heavy accent: Variable

Common Mistakes to Understand

Why It Mishears Things

1. Rare words - Names, technical terms not in training data
2. Ambiguous sounds - "Fifteen" vs "Fifty" sound similar
3. Context missing - Can't see what you're looking at
4. Noise - Background sounds confuse the model

The "Oreo Problem"

Ask voice assistant: "Call my wife" It might hear: "Call Hawaii"

Same sounds, different interpretation!

FAQ

Q: What are modern ASR models?

They’re typically large neural networks (often transformer-based) trained on huge amounts of audio + text. Some are open-source, some are offered as cloud APIs.

Q: Real-time vs batch processing?

• Real-time: Transcribes as you speak (Siri, live captions)
• Batch: Processes recorded audio later (meeting transcripts)

Q: What is WER (Word Error Rate)?

The standard accuracy metric. If you said 100 words and 5 were wrong, that's 5% WER. Lower is better.

Q: Can it work offline?

Many modern phones have on-device speech recognition for privacy and speed. Quality slightly lower than cloud.

Q: How does it handle multiple languages?

Many modern models can detect the language automatically and transcribe it. Some systems can also translate.

Q: Why does my voice assistant still make mistakes?

Accents, background noise, unusual words, and context gaps. It keeps improving, but it can still struggle — especially with names and brand-new vocabulary.

Summary

Speech Recognition converts spoken language into text. It powers voice assistants, automatic subtitles, dictation, and accessibility features - making computers understand human speech.

Key Takeaways:

• Converts audio waves to text
• Biggest challenges: accents, noise, homophones
• Modern systems use large neural networks
• Near human-level accuracy in good conditions
• Powers Siri, Alexa, Google Assistant, Zoom captions
• Still struggles with names, accents, and noisy environments

Voice is our most natural communication. Speech recognition bridges that gap between humans and machines!