Acoustic Echo Cancellation (AEC) with LMS

Why does acoustic echo occur in voice communication systems?

Acoustic echo occurs because a loudspeaker's output couples back into a nearby microphone through air, enclosure vibration, and reflective surfaces, creating a delayed, attenuated copy of the far-end signal that the far-end talker hears as an echo of their own voice. In a hands-free phone, headset dock, or conferencing device, the speaker and microphone share the same acoustic enclosure, so isolation is never perfect no matter how good the mechanical design is.

Echo path and coupling mechanisms

The echo path is the transfer function between the loudspeaker output and the microphone input, and it includes three coupling mechanisms: direct acoustic coupling through air, structure-borne vibration through the device chassis, and room reverberation from walls and furniture. Direct coupling dominates in small devices (delay under 1 ms, strong amplitude), while room reflections add a long, decaying tail that can extend 200-400 ms in an untreated conference room. Each mechanism contributes a different impulse response segment, and the adaptive filter must model all of them simultaneously.

Near-end vs. far-end speaker scenarios

The far-end speaker is the remote party whose voice comes out of the local loudspeaker; the near-end speaker is the local person talking into the microphone. Echo cancellation specifically targets the far-end signal re-entering the near-end microphone — not near-end background noise, which is a separate problem handled by noise suppression or, in dual-microphone systems, by the kind of reference-subtraction architecture described in Adaptive Noise Cancellation (ANC) Architecture. Confusing the two problems is a common design mistake: an AEC (acoustic echo canceller) tuned to suppress near-end noise will also attenuate legitimate near-end speech.

Time-domain echo characteristics

In the time domain, echo appears as a scaled, delayed, and smeared version of the loudspeaker signal, with the smearing described by the room's impulse response length. A typical handset echo path is 8-16 ms long; a car cabin or conference room can require 64-200 ms of filter memory at 16 kHz sampling to capture the full tail. Filter length requirements drive both memory and CPU budgets, which is why echo path length is the first number you should measure before choosing an architecture.

How does the LMS algorithm adapt to changing acoustic environments?

The LMS (least mean squares) algorithm adapts by nudging each filter coefficient in the direction that reduces instantaneous squared error, using only the current input sample and error — no matrix inversion, no stored covariance. This makes it cheap enough to run on a Cortex-M-class microcontroller, which is why it remains the default starting point for embedded AEC despite the existence of more sophisticated methods.

Gradient descent and weight update rule

LMS performs stochastic gradient descent on the mean squared error surface. The weight update is:

w[n+1] = w[n] + mu * e[n] * x[n]

where w[n] is the coefficient vector at time n, x[n] is the vector of recent reference (far-end) samples, e[n] = d[n] - y[n] is the error between the microphone signal d[n] and the filter's echo estimate y[n] = w[n]^T x[n], and mu is the step size. Each sample produces one update; no averaging or blocking is required, which keeps latency low.

Step size (μ) selection and convergence trade-offs

The step size μ controls how fast the filter adapts and how much residual error remains at convergence, and choosing it wrong is the single most common cause of a "naive LMS diverges" bug report. Too large a μ causes the coefficients to overshoot and the filter output to blow up, often within a few hundred milliseconds of a loud far-end burst. Too small a μ makes the filter track slowly, leaving audible echo for seconds after someone picks up a laptop and changes the acoustic path. For stability, μ must satisfy 0 < mu < 2 / (N * Px), where N is the filter length and Px is the average power of the reference signal x[n]. Because Px varies with speech level by 20-30 dB in real calls, a fixed μ tuned for quiet speech will diverge on a loud burst — this is precisely the failure mode that motivates NLMS, covered below.

Real-time coefficient tracking

Because the echo path changes whenever someone moves, a door closes, or a Bluetooth speaker is picked up, the filter must keep adapting continuously rather than converging once and freezing. Production systems typically re-enable adaptation after every detected path change and disable it during double-talk (defined below) to avoid corrupting a good filter state with near-end speech energy.

What is the structure of an AEC system with LMS?

An LMS-based AEC system subtracts an adaptively estimated echo signal from the microphone signal, using the far-end playback signal as the reference input to the adaptive filter. The three signals in play are: the reference x[n] (what is being sent to the loudspeaker), the microphone signal d[n] (echo plus near-end speech plus noise), and the error e[n] (the cleaned signal sent to the far end).

Microphone signal path and reference signal

The reference signal must be tapped before the loudspeaker's analog or acoustic path, ideally the exact digital samples written to the DAC (digital-to-analog converter), so that the adaptive filter sees the same signal that will reappear as echo, modulo the acoustic transfer function. Any additional, uncompensated delay between the reference tap and the actual acoustic emission — from a Bluetooth codec, for instance — shows up as effective echo path length and must be included in the filter's memory span.

Adaptive filter architecture

The adaptive filter is a finite impulse response (FIR) structure of length N taps, producing the echo estimate y[n] as the dot product of its coefficients with the most recent N reference samples. N is chosen to cover the expected echo tail: for a 16 kHz sample rate and a 32 ms echo path, N = 512 taps.

Error signal feedback and stability constraints

The error e[n] = d[n] - y[n] is both the cancelled output and the training signal fed back into the weight update, which means any transient instability in the filter directly corrupts the audio the far end hears. This tight coupling is why stability constraints on μ are not optional tuning knobs but hard correctness requirements.

How do you implement LMS for embedded hardware?

Embedded LMS implementation centers on three constraints: a circular buffer for the reference history, a fixed-point arithmetic strategy that avoids overflow, and a per-sample compute budget that fits the audio frame interval. Getting any one of these wrong produces artifacts that look like algorithmic bugs but are actually implementation bugs.

Buffer management and circular memory

The reference history is stored in a circular buffer of length N so that each new sample overwrites the oldest one in O(1) time rather than shifting N samples. For N = 512 taps at 16-bit samples, this buffer costs 1 KB of RAM; the coefficient array costs another 1-2 KB depending on whether coefficients are stored as 16-bit or 32-bit values.

Fixed-point vs. floating-point trade-offs

Fixed-point implementations must guard against coefficient overflow when the reference signal is loud and μ is near its stability limit; a common technique is to clamp e[n] * x[n] * mu before accumulation.

Computational complexity and real-time constraints

LMS costs roughly 2N multiply-accumulate operations per sample for filtering plus 2N more for the weight update, so a 512-tap filter at 16 kHz needs about 2N216000 ≈ 32.8 million MACs per second. On a 100 MHz Cortex-M4F with a single-cycle MAC instruction, that consumes a meaningful fraction of the core budget, which is why frame sizes and interrupt latency must be checked against the vendor's DSP library documentation rather than assumed.

Double-talk detection requirements

Double-talk — when both near-end and far-end parties speak simultaneously — must freeze adaptation, because the near-end speech looks like uncancellable error and will drag the coefficients away from the correct echo path. A standard detector compares short-term energy of the microphone signal against the expected echo energy (the Geigel algorithm) or correlates e[n] with d[n]; when the ratio exceeds a threshold tuned in the 1.5-2.0 range, adaptation is paused until the ratio falls back below threshold for a hold-off period of 100-200 ms.

What practical challenges affect AEC performance?

The two dominant practical failures are divergence from non-stationary echo paths and misalignment error from excess step size; both manifest as audible echo returning after the filter had previously converged. Diagnosing which one is happening requires logging μ, the reference power estimate, and the double-talk flag alongside the audio.

Non-stationary acoustic paths and divergence

When the echo path changes abruptly — a phone lifted off a desk, a car window rolled down — the previously converged coefficients become wrong instantly, and if adaptation was frozen (e.g., mistaken for double-talk) the filter cannot re-converge, producing sustained echo until a reset or re-adaptation event occurs.

Misalignment error and excess MSE

Misalignment is the steady-state gap between the adaptive filter's coefficients and the true echo path, and it grows proportionally with μ: larger step sizes track faster but settle noisier, trading convergence speed for excess mean squared error (MSE). Production tuning typically starts μ conservatively low during the first few seconds of a call and only increases it once the reference signal's power estimate stabilizes.

Computational latency and delay sensitivity

Any processing delay between the reference tap and the microphone sample it corresponds to effectively lengthens the echo path the filter must model, so buffering, DMA (direct memory access) transfer latency, and codec delay all need to be accounted for in the tap count N, not just the physical room acoustics.

How does normalized LMS (NLMS) improve basic LMS performance?

NLMS fixes the step-size scaling problem by dividing μ by the instantaneous power of the reference signal, so the effective step size adapts automatically to loud and quiet speech segments. The update becomes:

w[n+1] = w[n] + (mu / (epsilon + ||x[n]||^2)) * e[n] * x[n]

where epsilon is a small positive constant preventing division by near-zero during silence.

Power normalization and input signal variance

Because speech power varies by 20-30 dB over the course of a call, a fixed-μ LMS filter that is stable during quiet passages will overshoot during loud ones. NLMS's power normalization keeps the effective step size within the stability bound regardless of input level, which is why nearly every shipping AEC product uses NLMS rather than plain LMS.

Faster convergence with variable step size

NLMS typically converges 2-5x faster than LMS tuned for the same steady-state misalignment, because its normalized step size can be set closer to the per-sample stability limit without risking divergence on loud bursts.

Robustness to reference signal scaling

NLMS is largely insensitive to the absolute amplitude of the reference signal, which matters in embedded systems where gain staging between the DAC, amplifier, and loudspeaker varies across hardware revisions and volume settings.

When should you choose frequency-domain methods over time-domain LMS?

Frequency-domain adaptive filtering becomes preferable once the echo path length exceeds roughly 100-200 ms (1600-3200 taps at 16 kHz), because time-domain LMS/NLMS cost scales linearly with tap count per sample while block-based frequency-domain methods amortize the cost using FFT (fast Fourier transform) blocks.

Long echo paths and computational efficiency

A time-domain NLMS filter at N = 4096 taps costs roughly 4x the MACs per second of the 512-tap example above, which can exceed the budget of a low-power MCU; a frequency-domain block LMS implementation processes the same tap count at a fraction of the multiply count by exploiting the FFT's O(N log N) complexity.

Partitioned convolution strategies

Partitioned-block frequency-domain filters split the long echo path into shorter segments, each transformed and processed independently, which bounds the algorithmic latency introduced by block processing — an important constraint since large FFT blocks otherwise add tens of milliseconds of delay before the first sample of a block can be processed.

Trade-offs between latency and filter length

Choose time-domain NLMS for short echo paths and hard real-time latency budgets below a few milliseconds; choose partitioned frequency-domain methods when the echo tail is long and the device has enough RAM for FFT buffers, as documented in the vendor's DSP library reference for exact cycle counts on the target part.

Related reading

• Fourier Transform: Definition and Properties — understanding frequency-domain AEC.
• Parseval's Theorem: Energy in Time = Energy in Frequency — relating residual error energy across domains.
• Memory Footprint Optimization — budgeting embedded delay lines and filter coefficients.