NDP120 / NDP200: Syntiant Neural Decision Processor

Always-on keyword spotting and sensor inference on a coin-cell budget rules out running a Cortex-M core at full tilt 24/7 — the MCU has to sleep, and something else has to listen. Syntiant's NDP (Neural Decision Processor) family is a class of purpose-built inference ASICs designed to sit in front of a host MCU/AP, continuously classify audio or sensor streams at tens to hundreds of microwatts, and wake the host only when something interesting happens. Understanding where these chips fit — and where they don't — matters when the power budget for "always listening" features is measured in µA, not mA.

What the NDP Family Actually Is

NDP120 and NDP200 are not microcontrollers running a neural-network library; they are dedicated neural-network inference accelerators with a fixed, small, deeply-pipelined array of multiply-accumulate (MAC) units optimized specifically for the layer types found in keyword spotting (KWS), voice activity detection (VAD), and simple sensor classifiers — mostly depthwise-separable convolutions, small dense layers, and GRU/RNN cells. They are not general-purpose vision or LLM accelerators.

Both parts embed a small Arm Cortex-M0 for supervisory tasks (model switching, control-plane housekeeping) alongside the dedicated inference array — the M0 is not what runs the neural network; it's the traffic cop.

The Core Idea: Always-On Front End, Not Application Processor

The system partition that makes this architecture make sense:

[Mic/IMU] → [NDP120/200: always-on inference] → wake/event → [Host MCU or AP: full app]
                (µW range, always running)         (mA-mW range, mostly asleep)

• The NDP chip runs continuously on a primary keyword/event model, drawing power in the hundreds-of-µW range.
• When the model detects a match above threshold (e.g., "Hey Device", or an anomalous vibration signature), it asserts an interrupt/GPIO to the host.
• The host wakes from deep sleep, handles the event (full ASR, cloud upload, UI action), and goes back to sleep.

This is the same "sensor hub" pattern used elsewhere in embedded design (e.g., an accelerometer's onboard tap-detect interrupt waking the SoC), just implemented as a full neural-network classifier rather than a hardwired threshold comparator. The advantage over running the same model on a Cortex-M4/M7 in a duty-cycled loop is both power and latency of response — the accelerator is never "off" waiting for a wake timer, so short transient events aren't missed between polling windows.

Why a Dedicated Array Beats a CPU Here

A general-purpose MCU running CMSIS-NN kernels has to fetch instructions, manage loop overhead, and move data through a von Neumann pipeline for every MAC. Syntiant's inference core avoids most of that overhead by using a fixed dataflow tailored to the small set of layer types it supports, with weights resident in tightly-coupled on-chip SRAM rather than streamed from flash — the same reason DSP/NPU accelerators in general beat CPUs on energy-per-MAC.

Rough order-of-magnitude comparison for a small KWS DS-CNN model (~30k parameters, running inference ~10×/second):

The exact numbers depend heavily on model size and clock, but the qualitative point holds: a fixed-function accelerator sized for exactly this workload wins by roughly an order of magnitude in energy per inference, and — more importantly for "always on" use cases — it wins even more decisively in average power because it never needs the MCU core awake at all.

Worked Example: Power Budget for a Wake-Word Product

Assume a battery-powered device needs "Hey Device" detection running 24/7 on a 200 mAh coin-cell-class battery, target life 1 year (8760 h).

Available average current budget:

I_avg = 200 mAh / 8760 h ≈ 22.8 µA

At a 3.0 V rail, that's a power budget of:

P_avg = 22.8 µA × 3.0 V ≈ 68 µW

for the entire system, not just the listening path — clearly too tight if the listening path alone draws 150–250 µW continuously. This is the real design tension: even a highly efficient always-on accelerator can dominate the power budget of an ultra-long-life product.

Reframing with a duty-cycled front end: many NDP-based designs instead use a lower-power VAD stage (or a coarser first-stage model) to gate the full KWS model, or accept a shorter target life. If the product target is relaxed to 90 days (2160 h) on the same 200 mAh cell:

I_avg = 200 mAh / 2160 h ≈ 92.6 µA → P_avg ≈ 278 µW

Now the ~150–250 µW NDP120 listening budget consumes roughly 55–90% of the total power budget, leaving headroom for host MCU sleep current, sensor bias, and leakage — a workable, if tight, budget.

Check: 278 µW total, ~200 µW to the NDP path, leaves ~78 µW for host MCU sleep + peripherals — consistent with typical Cortex-M sleep currents in the 1–5 µA range at 3 V (3–15 µW), plus margin for occasional wake events. The arithmetic closes: a 90-day always-listening product on a 200 mAh cell is realistic with this class of accelerator; a 1-year target is not, without further gating or a bigger cell.

Model Deployment Flow

Both parts use a proprietary toolchain (Syntiant's TENSAI/NDP tools) rather than running arbitrary TFLite Micro graphs directly on the array:

1. Train a supported topology (small CNN/DNN/GRU) in TensorFlow/Keras, subject to the layer and size constraints of the target NDP core.
2. Convert/quantize to the NDP-specific compiled format — weights are quantized (commonly to low-bit fixed point) and packed for the fixed dataflow array.
3. Simulate/validate accuracy against the quantized model before committing to silicon-specific packing.
4. Flash the packed model into the NDP's model memory (often multiple models resident simultaneously, selectable by the on-chip M0, e.g., one KWS model + one VAD model).
5. Integrate with the host over I²S (audio in) and I²C/SPI (control, interrupt status).

This is a narrower, more constrained pipeline than the TFLite Micro / CMSIS-NN flow used for general Cortex-M inference — the tradeoff for the power advantage is reduced model flexibility and vendor tool lock-in.

Practical Design Implications

• Model size ceilings are real: these arrays are sized for kilobyte-to-low-megabyte parameter counts, not for MobileNet-class vision models — check the datasheet's supported layer types and max parameter count before committing to an architecture.
• Multiple resident models: NDP120 typically supports storing several small models (e.g., multiple wake words, or KWS + VAD) and switching between them under M0 control without a host round-trip.
• I²S/PDM mic interface dominates BOM choice: since the chip listens continuously, using a PDM digital microphone avoids an always-on analog front end and ADC on the host side.
• False-wake budget drives system power just as much as the model's raw power draw: every false wake burns host energy; tightening the detection threshold trades false-accept rate against missed-wake rate, and that tradeoff should be characterized against real acoustic/vibration environments, not just clean lab data.
• NDP200's sensor-fusion role: for IMU + low-res-image or IMU + audio always-on classification (e.g., fall detection, gesture pre-trigger), NDP200 extends the same always-on-front-end pattern beyond pure audio.

Key Takeaways

• NDP120/NDP200 are dedicated, always-on neural-network inference ASICs, not general MCUs — they run a small class of layer types (DS-CNN, DNN, small GRU) at tens-to-hundreds of µW.
• They implement a sensor-hub-style architecture: continuous inference at the edge wakes a host MCU/AP only on a positive detection, cutting average system power far below a duty-cycled CPU-based classifier.
• Energy per inference is roughly an order of magnitude better than a Cortex-M4 running CMSIS-NN for equivalent small KWS models, because of a fixed dataflow array with on-chip weight storage vs. instruction-driven MAC loops.
• Power budgets must be computed system-wide (battery capacity / target life) — the "always-on" chip's own draw can dominate the entire budget, forcing either a shorter life target, a lower-power gating stage, or a larger battery.
• Deployment uses a proprietary quantize-and-pack toolchain rather than a stock TFLite Micro flow, trading model flexibility for the power/latency advantage of the dedicated array.