Every Copilot+ PC advertises the same magic number — “40+ TOPS NPU” — yet nobody at the store can tell you what a trillion operations per second feels like when you are captioning a meeting or blurring your video background. The number is real, the certification behind it is real, and the connection between the two and your experience is thinner than the marketing implies.

CPU shoppers learned gigahertz roughly track speed; NPU shoppers get TOPS, a number measured under conditions no real app reproduces.

This guide explains what TOPS measure, why real workloads land far below the peak, and how to judge an NPU by what it does for you rather than what it says on the box. Six sections, technician order, no marketing fog.

Why this happens

  • TOPS are measured on the friendliest possible math: The quoted figure comes from dense INT8 matrix multiplication running flat-out — the single operation NPUs do best — while real models mix precisions, activations, and memory traffic that all run slower.
  • Memory bandwidth is the real ceiling: An NPU can only compute as fast as weights arrive from memory; many laptop NPUs are memory-bound on large models, so doubling compute does nothing for them.
  • Apps rarely saturate the NPU: Live captions or background blur use a fraction of peak capacity; the NPU finishes each frame in milliseconds and idles, which is why a bigger TOPS number does not make captions noticeably faster.
  • Precision matters more than the headline: A model quantized to INT8 runs far faster than the same model at FP16, so two NPUs with identical TOPS can differ wildly depending on which precisions their toolchains support well.
  • The Copilot+ bar is a minimum, not a ranking: Microsoft’s 40+ TOPS requirement guarantees features will run, not that every certified NPU runs them equally — implementation quality varies between vendors.
  • Background system costs are invisible in TOPS: Virtualization-based security features can tax the whole system, which is why an NPU can look fine in a benchmark while the machine feels slow — see how VBS and HVCI slow ARM laptops for the full picture.

How to fix it

Work through these in order — diagnose first, then act.

Fix 1: Learn what an “operation” actually is

The first step to reading a TOPS number honestly is knowing what is being counted. An “operation” here is one tiny multiply-add on 8-bit integers — the atomic step of neural-network math — and a trillion of them per second is a throughput figure for a very narrow kind of work.

  1. Think of TOPS as horsepower measured on a dynamometer: real, standardized, and only loosely related to how the car drives in traffic.
  2. Understand that INT8 (8-bit integer) math is what the number assumes — the fastest, lowest-precision mode the NPU offers.
  3. Know that FP16 or FP32 work, which many models still use in parts, runs at a fraction of that rate — sometimes an eighth.
  4. Check your NPU vendor’s spec sheet for per-precision rates (Qualcomm, Intel, and AMD all publish them) instead of trusting the single headline figure.
  5. Remember this ratio whenever a bigger TOPS number tempts you: peak math is cheap to add; feeding the math units is the hard part.

Fix 2: Understand the memory wall

If TOPS are the engine, memory bandwidth is the fuel line. This is the single most important concept for why real-world NPU performance plateaus well below the headline number — and why spec-sheet comparisons mislead.

  1. Picture a large language model: every token generated requires streaming billions of weights through the NPU, so the chip spends much of its time waiting on memory, not computing.
  2. Look up your NPU’s memory bandwidth — on integrated AI-PC chips it shares system RAM bandwidth with the CPU and GPU, which caps effective throughput.
  3. Notice that small models (captions, blur, enhancement) fit in on-chip cache and run near peak efficiency, while large models (local LLMs) hit the wall hard.
  4. Compare this to why an external display can stutter through a weak link — the same bottleneck logic behind USB4 docks failing to drive ARM laptop displays applies inside the chip: the pipe, not the engine, decides.
  5. Judge NPUs by performance-per-watt on real models, not by TOPS, because memory efficiency is what actually separates good designs from big numbers.

Fix 3: Map TOPS to the features you actually use

Abstract numbers become useful the moment you attach them to features. Copilot+ experiences are small, quantized models — and for those, the 40-TOPS bar is genuinely comfortable headroom, not a constraint.

  1. List the NPU features you use daily: Live Captions, Studio Effects, Click to Do, Recall, Cocreator — all of them run compact INT8 models.
  2. Understand that each of these uses single-digit TOPS at most; the 40+ requirement exists so several can run simultaneously with margin.
  3. Test Live Captions under load yourself — if captions lag while system audio routes oddly, the problem is usually plumbing, as in Live Captions failing on system audio , not insufficient TOPS.
  4. Stop worrying about TOPS for these features entirely — once past the certification bar, driver quality and app integration matter more than the number.
  5. Reserve TOPS-comparison energy for heavy local workloads (LLMs, diffusion) where the number actually predicts something.

Fix 4: Compare NPUs on real benchmarks, not spec sheets

When you genuinely need to choose between NPUs — say, Snapdragon X Elite vs. Intel Core Ultra vs. AMD Ryzen AI — the honest comparison is measured inference on identical models, not the TOPS line on the box.

  1. Find benchmark results that name the exact model, precision, and runtime (ONNX Runtime, DirectML, Qualcomm AI Hub) — without those, the number is decoration.
  2. Compare tokens-per-second on a small LLM and seconds-per-image on a diffusion model; these two workloads expose memory behavior and compute behavior respectively.
  3. Check power draw alongside speed — an NPU that is 20% slower but uses half the watts wins on a laptop, where thermals cap sustained work.
  4. Weight the software stack: the best-supported NPU for your apps beats a theoretically faster one whose toolchain your apps do not use.
  5. Treat vendor-published “up to” figures as the dynamometer number from Fix 1 — independent, same-model measurements are the road test.

Fix 5: Stop optimizing the number and optimize the experience

For most owners, the productive move is not chasing TOPS at all — it is making sure the NPU you have is actually being used. A 45-TOPS NPU sitting idle while the CPU does the AI work is slower than a 20-TOPS one properly engaged.

  1. Open Task Manager during your AI workloads and confirm the NPU graph actually moves — flat NPU plus busy CPU means fallback, the real performance bug.
  2. Update NPU drivers from your laptop maker before judging speed; early-driver NPUs routinely left 30% of performance on the table.
  3. Keep Windows updated — Microsoft’s Windows support pages document steady NPU stack improvements in feature updates.
  4. Disable nothing in pursuit of TOPS; security features cost a little and protect a lot, and the NPU headroom absorbs them.
  5. Reframe the question from “how many TOPS do I have?” to “is the NPU doing the work?” — the second question has an actionable answer.

Fix 6: Know when TOPS genuinely predict the future

TOPS are not useless — they are a capacity reservation for workloads that do not exist yet. Understanding this turns the number from marketing into a longevity signal you can actually use when buying.

  1. Recognize that on-device models are growing: today’s comfortable INT8 captioning model is tomorrow’s baseline, and headroom is what lets a laptop stay capable for years.
  2. Treat 40+ TOPS as “certified for the Copilot+ feature set through its lifecycle” — that is the real promise behind the number.
  3. When buying, prefer the higher-TOPS option only if price, battery, and thermals are otherwise equal — never trade a better screen or keyboard for TOPS you cannot feel today.
  4. Watch precision support (INT4 is coming to NPUs) more than raw TOPS — a chip that runs tomorrow’s quantized models efficiently beats one with a bigger INT8 number.
  5. File TOPS where it belongs: a useful tiebreaker and a future-proofing hint, not a performance ranking.

Closing

TOPS measure the ceiling of ideal math, real apps live on the floor of memory bandwidth, precision, and driver quality — and for every Copilot+ feature you use daily, the certified 40+ is already plenty. Judge your NPU by whether it is engaged and how it handles real models, and the number on the box finally stops being a mystery.

Frequently asked questions

What does TOPS mean for an NPU?
TOPS stands for trillions of operations per second — the peak rate at which the NPU can do simple integer math. It is a ceiling measurement taken under ideal conditions, not the speed you will see in everyday apps.
Is 45 TOPS twice as fast as 22 TOPS?
Only on the exact workload used to measure it. Real apps mix operations, memory access, and different precisions, so the gap between two NPUs is usually much smaller than the TOPS ratio suggests.
Why do all Copilot+ PCs advertise 40+ TOPS?
Microsoft requires 40+ TOPS as the minimum for Copilot+ certification. It guarantees a baseline for on-device AI features, but it says little about how efficiently any particular NPU uses that capacity.
Does more TOPS mean better battery life?
Not directly. Higher TOPS at the same power draw means more efficient AI work, but an NPU with modest TOPS that finishes quickly and idles can outlast a hungrier one. Efficiency matters more than the headline number.