Edge vs. Fog vs. Cloud Processing: Latency Trade-off

Every IoT system has to answer one architectural question before anything else: where does the data get processed? Push everything to the cloud and you get elastic compute and simple devices, but you pay in round-trip latency, bandwidth cost, and a single point of failure if connectivity drops. Push everything to the device and you get microsecond response times, but you lose the aggregation, storage, and cross-site analytics that make IoT valuable. Edge, fog, and cloud processing are three points on that spectrum, and almost every real deployment splits work across all three. Getting the split wrong shows up directly as either unacceptable latency or unsustainable data/compute cost.

The Three Tiers Defined

• Edge: Processing on or immediately adjacent to the sensor/actuator — a microcontroller, a smart sensor with onboard DSP, or a single-board computer physically at the machine. Data never leaves the local device unless a decision requires it.
• Fog: Processing on a local gateway, industrial PC, or on-premises server that aggregates multiple edge devices — typically within the same building, plant, or LAN. Fog sits between edge and cloud, doing filtering, local rules, and short-term storage before selectively forwarding to the cloud.
• Cloud: Centralized compute in a data center, reachable over WAN, offering elastic storage, heavy analytics (ML training, cross-site correlation), and long-term historical data.

The terms aren't always used with hard boundaries — "edge computing" is sometimes used loosely to mean "not cloud," covering both edge and fog — but for architecture decisions the three-tier distinction is useful because each tier has a different latency budget, compute ceiling, and failure domain.

Latency Budget by Tier

These numbers assume: edge latency is dominated by the control loop execution time itself (interrupt to actuation), fog latency is dominated by local network hops (Ethernet/Wi-Fi, sub-millisecond to a few ms, plus gateway processing), and cloud latency is dominated by WAN transit — LTE/5G adds 30–70 ms one-way even before the internet hop, and satellite backhaul can add 500+ ms.

Rule of thumb: if a decision must be made faster than one round trip to the next tier up, it cannot live in that tier. This is a hard constraint, not a preference — a motor over-current trip needing a 2 ms response cannot wait on a 100 ms cloud round trip no matter how much compute the cloud offers.

Worked Example: Motor Protection on a Factory Floor

A CNC spindle has three control functions with very different time constants:

1. Over-current trip: must cut power within 2 ms of detecting a fault to protect the winding.
2. Vibration-based anomaly flag: should raise an alert within 1–2 seconds of a bearing wear signature appearing.
3. Fleet-wide predictive maintenance model: retrains weekly across 200 machines using months of vibration/temperature history.

Mapping each to a tier:

• Function 1 → Edge. A local ADC + comparator/MCU loop running at, say, 10 kHz sampling gives a 100 µs sample period; detection + gate-drive shutdown within 2 ms is achievable entirely on-device. Sending this over any network — even a 1 ms LAN hop — adds risk with zero benefit, since the actuation must happen locally regardless.
• Function 2 → Fog. The gateway on the shop floor collects vibration FFT features from 20 spindles, runs a lightweight anomaly classifier, and raises an alert within the LAN's few-millisecond latency plus ~100 ms of compute — well inside the 1–2 s budget, and it avoids sending raw high-rate vibration data (e.g. 5 kHz × 3 axes × 16-bit = 240 kbit/s = 30 kB/s per machine) over WAN.
• Function 3 → Cloud. Weekly retraining needs the storage and GPU/TPU compute that only the cloud offers economically, and a 500 ms round trip is irrelevant at a weekly cadence. The fog tier forwards only extracted features (kB/s, not hundreds of kB/s), which is what makes the WAN link and cloud storage bill sustainable.

Check: total raw data per spindle is 240 kB/s ≈ 20 GB/day. Across 200 machines that's ~4 TB/day if sent raw to the cloud — impractical over typical industrial WAN links and expensive to store. Fog-side feature extraction cutting this to, say, 1 kB/s per machine drops the WAN load to ~17 GB/day fleet-wide, a ~240× reduction, while the 2 ms trip function never touches the network at all. The split is what makes the numbers work; no single tier could satisfy all three constraints alone.

Why Not Just Use the Cloud for Everything

• Physical response time: control loops with millisecond deadlines cannot tolerate WAN jitter, which is often larger than the deadline itself.
• Bandwidth cost: raw sensor streams (audio, vibration, video) are orders of magnitude larger than the events derived from them; transmitting raw data to the cloud is usually the dominant recurring cost in a large deployment.
• Availability: a WAN or cloud outage should not stop a safety-critical local function. Edge/fog tiers must be able to operate autonomously (degraded mode) when the uplink is down.
• Privacy/regulatory: some data (e.g., health, video with faces) may be required to stay on-premises or be anonymized before leaving the site.

Why Not Just Use the Edge for Everything

• Compute ceiling: MCUs and even edge MPUs cannot run large ML models, long-horizon correlation across many devices, or store months of history.
• Fleet-wide visibility: a single machine's edge controller has no way to see patterns across 200 machines — that requires aggregation, which is fog or cloud's job.
• Update/maintenance cost: pushing analytics logic to thousands of distributed edge devices is harder to version and debug than centralizing it.

Design Implications

• Classify each function by its latency deadline first, then assign it to the lowest tier that can meet that deadline — not the most convenient tier to develop on. Pushing a function down a tier (edge instead of fog) tends to reduce latency and network dependency but increases per-device cost and update complexity; pushing it up a tier does the opposite.
• Fog is the traffic shaper. Its main architectural job is usually not "processing" in the sense of decisions, but data reduction — filtering, downsampling, feature extraction, and buffering during WAN outages — so the cloud only sees what it actually needs.
• Design for graceful degradation. Edge and fog tiers should define a defined offline behavior (e.g., last-known-good setpoint, local alarm buffering) for when the cloud is unreachable, rather than assuming connectivity.
• Re-evaluate the split as models and hardware evolve. Increasingly capable edge/fog hardware (NPUs on gateways, quantized ML on MCUs) is continuously pulling functions that used to require the cloud (e.g., simple anomaly detection) down to fog or edge — the boundary is not fixed, it shifts with silicon capability.

Key Takeaways

• Edge, fog, and cloud trade latency against compute/storage capacity: edge is fastest but weakest, cloud is slowest but most powerful, fog sits in between and typically does data reduction.
• Latency deadlines are a hard constraint: a function must live in a tier whose round-trip time is faster than its required response, regardless of available compute elsewhere.
• Bandwidth and cost, not just latency, drive the edge/fog split — raw high-rate sensor data is usually reduced to features or events before it reaches the WAN.
• Real systems split functions across all three tiers simultaneously based on each function's own deadline, not adopt one tier exclusively.
• The boundary moves over time as edge/fog hardware capability grows, so architecture reviews should periodically re-check which tier each function actually needs.