Running Local LLMs on an NPU Alone Is a Hard Choice

The NPU Question No One Else Is Asking Out Loud

There is a quiet experiment happening on desks in Tokyo right now, and it is not covered in English-language tech press. MINISFORUM, a Japanese mini PC maker with an appetite for aggressive hardware in small cases, has shipped a machine called the AI X1 Pro-470 built around AMD’s newest Ryzen AI 9 HX 470 processor. The chip packs a 44 TOPS neural processing unit alongside a Zen 5 CPU. The pitch, unspoken but unmistakable, is this: can you run useful local large language models on that NPU alone, without a discrete GPU?

That question matters because it sits at the center of the edge-AI hardware race. Cloud inference is expensive and latency-bound. Local inference on a GPU is powerful but expensive and power-hungry. The NPU is supposed to be the middle path — efficient enough for the desktop, dedicated enough to run models that would choke a CPU. Whether it actually is remains the question MINISFORUM’s latest machine is designed to answer.

What the Hardware Is Actually Saying

The AI X1 Pro-470 keeps the same chassis as its predecessor, the AI X1 Pro that carried the Ryzen AI 9 HX 370. Dimensions sit at roughly 195 by 195 by 47.5 millimeters. That is not tiny, but it is compact for what is inside. The real story is in the expansion architecture.

The back panel features dual 2.5GbE Ethernet ports using Realtek’s RTL8125 controller. Two independent network segments out of one box is unusual for a mini PC and signals that this machine was designed for people who run homelabs, lab networks, or local inference stacks that need separation from the home internet connection. It is a detail most reviewers will skip and the right people will notice.

Video outputs include HDMI 2.1 with FRL support, DisplayPort 2.0, and a USB4 port that supports DisplayPort alternate mode. There is also an OCuLink port. That port is the one worth focusing on. OCuLink carries raw PCIe signals directly, bypassing the USB controller entirely. It offers substantially more bandwidth than USB4 and far less latency overhead, which makes it the preferred route for external GPU enclosures and high-speed storage arrays. Hot-swapping is not supported, but the trade-off is almost certainly worth it if you plan to attach an eGPU later.

The machine also includes a fingerprint reader embedded in the top cover — Windows Hello biometric auth on a desktop form factor. That is not a spec people fight over, but it is the kind of detail that signals MINISFORUM understands its audience: users who want a desktop experience without treating their mini PC like a peripheral rig.

Where the Hardware Stumbles

It is not all strength. The built-in SD card slot on the right side is wired internally through a USB 2.0 card reader bridge, despite the slot being labeled for UHS-II cards. A ProGrade Digital SDXC UHS-II card tested in the machine pulled only 37.29 megabytes per second sequentially. That is USB 2.0 behavior, not UHS-II behavior. For anyone moving large photo or video files regularly, that bottleneck will be noticeable and frustrating. It is a modest cost saved at the PCB level that undersells the rest of the machine’s otherwise aggressive I/O layout.

The Real Test: NPU-Only Local LLM Inference

The hardware review is only half the story. The article’s third page is dedicated to running local LLM inference using the NPU alone and comparing those results against GPU-based inference. That is where the edge-AI debate gets concrete.

The Ryzen AI 9 HX 470’s NPU delivers 44 TOPS of peakt performance. For context, many current-generation discrete GPUs deliver between 30 and 130 TOPS depending on the model and quantization scheme. The NPU is not in the same league as a high-end GPU. But local LLM inference does not always require that kind of headroom, especially when models are quantized to 4-bit or 5-bit formats and run at modest context lengths.

The practical question is whether 44 TOPS on an NPU can sustain usable token generation for models in the 7B to 14B parameter range. Early data from similar platforms suggests that 4-bit quantized Llama 3.2 and Phi-4 class models can indeed run on NPU-only hardware, though typically at token rates that feel adequate for chat and code assistance rather than real-time streaming. The bottleneck is rarely raw compute — it is memory bandwidth and the efficiency of the inference runtime stacking on top of the NPU driver. AMD’s Ryzen AI software stack has been maturing quickly, and the X 470’s NPU is built on AMD’s Strix Point architecture, which includes improved quantization support and better integration with tools like llama.cpp and Ollama through their ROCm and DirectML backends.

Who Wins and Who Loses

If the NPU-only approach delivers acceptable performance for everyday local LLM workloads, the winners are obvious: laptop and mini PC buyers who do not want to carry an eGPU or maintain a desktop with a discrete graphics card. The energy profile is dramatically lower. A full desktop inference rig pulling 150 to 300 watts can be replaced by a machine drawing 30 to 60 watts under similar load. For home labs, remote offices, and anyone running local AI agents or personal copilot tools, that efficiency gap is the difference between a background service and a seasonal utility bill surprise.

The losers are the people who already own machines with capable GPUs and expect the NPU to replace them. It will not. An NPU is a specialization engine, not a general-purpose graphics accelerator. For anything beyond quantized inference — training, fine-tuning, multi-modal pipelines, or models larger than roughly 14B parameters — a discrete GPU remains necessary.

There is also a third group that loses without necessarily realizing it: cloud inference providers. Every local inference session that stays on-device is a session that does not hit a data center. The trend is small in absolute terms today but compounding. If consumer NPU inference reaches parity with cloud API calls for common use cases — summarization, coding assistance, basic reasoning — the economics of cloud LLM serving begin to erode at the low end.

What Happens Next

MINISFORUM is testing this live in Japan, and the results will matter beyond that market. The AI X1 Pro-470 is not a niche product — it is part of a broader shift toward consumer hardware that treats AI inference as a first-class workload, not an afterthought bolted onto gaming rigs.

AMD is pushing its Ryzen AI branding hard, and the X 470 chip is positioned as the company’s answer to the growing demand for efficient local AI. Intel is responding with its own NPU-equipped processors, and Qualcomm is doing the same for Windows on Arm. The hardware is arriving faster than the software ecosystem is stabilizing around it. That gap is where frustration lives today, and it is also where the opportunity lives.

What is clear from the MINISFORUM review framework is that the question is no longer whether consumer NPUs can run local LLMs. They can. The question is whether they can run them well enough to change buying decisions. The answer will determine whether the next wave of AI hardware purchases are driven by clock speed and tensor core counts or by efficiency, thermals, and what the NPU can sustain over a long inference session without throttling.

Japan is running that experiment right now. The world should be watching.