HBF vs HBM: Which AI Inference Data Belongs in Flash?

I see High Bandwidth Flash (HBF) as a new design opportunity, not a blanket replacement for High Bandwidth Memory (HBM). If it proves useful in deployed inference systems, it could make NAND design, memory controllers and AI software a more closely connected competitive field. SK hynix’s participation gives that possibility a Korean connection—but adoption and system performance must establish the benefit.

HBF uses NAND flash to expand memory capacity near compute. It can complement DRAM-based HBM; it does not make their access behavior interchangeable. The useful question is which inference data can tolerate flash’s restrictions, and what the system gains by putting it there.

HBF is intended to complement memory near compute. Its usefulness depends on data placement, not capacity alone. Conceptual comparison based on SK hynix, Sandisk, and Ma and Patterson.

What the announced HBF specification tells us

In August 2026, SK hynix and Sandisk announced an HBF technical specification through the Open Compute Project. SK hynix’s Korean announcement describes capacities up to 512GB across 8-high and 16-high NAND die-stack configurations, with three bandwidth grades spanning approximately 0.4TB/s to 3.0TB/s. It also identifies the UCIe chiplet interface as the processor interconnect.

Those are specification ranges reported by the vendor, not measurements from a deployed inference service. A stack’s maximum capacity and a bandwidth grade do not tell us how quickly a particular model answers a request.

The intended system role matters just as much. Sandisk describes coexistence with HBM, with guidance covering interfaces, packaging, reliability and software read/write operations. HBF is not simply a new name for a fast SSD, nor does this announcement establish that an existing GPU can accept it as an HBM replacement.

More room for weights is not the same as faster answers

Consider a frozen model whose weights exceed one accelerator’s available HBM. Adding capacity close to compute could make a smaller system configuration possible. That is part of the motivation described in Sandisk’s earlier architecture discussion.

Now change the example: the same model already fits, with enough room for its active cache. Extra flash capacity alone has not solved a problem. There would need to be another demonstrated benefit to justify the extra tier.

Three questions keep this comparison honest:

Question	What it measures	What it cannot establish alone
Does the data fit?	Capacity available for the intended data.	Whether it arrives before computation needs it.
How much data arrives per second?	Delivered bandwidth under the workload.	The wait for a particular dependent access.
How long does an access take?	Latency along the relevant access path.	Total service throughput under concurrent requests.

This is a comparison framework, not a benchmark. A design can move a large stream efficiently while still being a poor choice for small accesses that cannot be requested in advance. Bytes of capacity, bytes per second and seconds of waiting answer different questions.

The NAND restrictions do not disappear inside the package

In their research perspective on inference hardware, Xiaoyu Ma and David Patterson identify two important HBF constraints: finite write endurance and page-based reads with higher latency than DRAM. Flash does not offer an unlimited budget for repeated write/erase cycles. Reading a page can also move more data than a small request needs, reducing the useful share of the transfer.

Their argument favors infrequently updated data, including inference weights and slowly changing context. It does not turn every cache object into a good flash candidate. How often data changes, whether it will be reused and when it must arrive all matter.

That distinction is separate from how stacked dies are bonded. Better physical integration and a suitable workload are different requirements. A package diagram cannot establish the latter.

Weights and transient KV are different experiments

The August 2026 FLINT preprint investigates HBF as a capacity tier for model weights alongside HBM. Its proposed controller groups and pipelines reads, while specialized flash management handles the access pattern. It also addresses maintenance operations rather than assuming flash needs none.

The authors evaluate the proposal with trace-driven simulation. That makes it evidence for an architectural approach, not a benchmark of a shipping FLINT accelerator. The interesting point here is the work required to make weight traffic useful—not a performance multiplier that can be applied to any model.

A separate KV-centric serving preprint examines an SSD-style offloading system with HBF substituted underneath. Its modeled deployments can perform worse: the traffic reaching flash is write-heavy, and the system must account for thermal limits and changes to the nearer memory tier.

That study uses an extended serving simulator with projected HBF parameters, not measured HBF silicon. It challenges a drop-in deployment strategy, not every possible use of KV data in flash. The two studies ask different questions; they are not opposing product reviews.

Weight-focused and transient-KV studies investigate different access patterns. These questions summarize the design tradeoff, not measured product performance. Based on Ma and Patterson, FLINT, and the KV-centric HBF study.

A data-placement test before a capacity claim

The following matrix translates those constraints into questions worth testing. These are candidate uses and failure checks, not a universal placement policy.

Data or situation	Why investigate HBF?	What must be checked?
Frozen model weights	A large, read-heavy data set.	Useful read delivery and scheduling stalls.
Stable, reusable context	Repeated use can justify storing a larger working set.	Actual reuse after writing, plus retrieval delay.
Short-lived KV offloading	Pressure on cache capacity.	Writes versus useful reads, wear and thermal limits.
Model already fits in adequate HBM	Capacity alone supplies no justification.	A measured benefit that outweighs integration overhead.

The matrix combines the research above with a simple editorial rule: identify the constraint before choosing the medium. “KV cache is large” describes occupancy. It says nothing about how many useful reads a flash tier receives for each write.

Likewise, weight and KV quantization change the stored representation. Tiering changes where data resides. Neither choice can be evaluated by adding up nominal gigabytes alone.

The Korean opportunity is broader than another memory stack

SK hynix’s role is concrete: it helped develop the specification with Sandisk. That gives the Korean memory story a connection to interface and software requirements, not just NAND capacity. Google and Tenstorrent are named consortium participants in the release; participation is not a disclosed purchase order.

The OCP community directory also lists a High Bandwidth Flash workstream. A shared specification gives designers a common reference, but it does not settle implementation quality or establish customer qualification.

My view is that the opportunity lies in making the pieces work together. NAND and controller capability must translate into data that an inference engine can use efficiently. SK hynix’s involvement is a reason to follow that work, not evidence that revenue, profit or share price will necessarily rise.

Ask what was actually measured

A useful warning sits in Sandisk’s July 2025 fact-sheet footnote. Its HBM performance comparison models reads of 8-bit pretrained weights for Llama 3.1 405B, executes one kernel at a time and assumes unlimited HBM capacity. Those conditions are not an end-to-end serving test with arriving requests and an active KV cache.

For the next HBF demonstration, ask for the data placement, workload, retained HBM resources and sustained operating conditions. Then look at response latency and useful output under the same service requirement. A larger memory budget is valuable when the system can turn it into useful work—not merely when it produces a larger number on a slide.