SK hynix HBM3E delivers 36GB and around 1.2TB/s for AI accelerators
SK hynix HBM3E is high-bandwidth DRAM designed for AI accelerators, GPUs and other high-performance computing processors that need to move large amounts of data far faster than conventional memory architectures allow.
Rather than placing memory chips alongside a processor like ordinary graphics memory, HBM stacks multiple DRAM dies vertically. Through-Silicon Via, or TSV, connections link the layers, while the completed HBM package sits close to the processor inside an advanced semiconductor package.
SK hynix initially entered mass production with an eight-layer HBM3E product in March 2024. It followed that with a 12-layer version containing 36GB of memory later the same year. The 12-layer product operates at up to 9.6Gbps and remains relevant to current AI systems in 2026.
At Computex 2026, SK hynix displayed its 36GB 12-layer HBM3E alongside NVIDIA’s GB300 Grace Blackwell hardware. The company has also described HBM3E as widely deployed during the industry’s transition to the newer HBM4 generation.
TechnologyBlog.co.za has not independently benchmarked SK hynix HBM3E. This article therefore explains its verified specifications, architecture and use cases rather than making independent performance claims.
What is HBM3E?
HBM stands for High Bandwidth Memory.
Unlike a conventional DIMM or graphics card memory chip, HBM stacks several DRAM dies vertically and connects them through thousands of microscopic electrical pathways called Through-Silicon Vias.
This allows the memory interface to use a very wide connection to the processor.
The result is extremely high aggregate bandwidth without relying only on ever-higher clock speeds.
SK hynix describes HBM3E as an extended version of HBM3 and the fifth generation in its HBM family, following HBM, HBM2, HBM2E and HBM3. HBM4 has since become the sixth generation.
HBM3E is particularly important for artificial intelligence because modern GPUs repeatedly move huge quantities of model weights, intermediate results and other data between memory and their processing cores.
A fast accelerator can spend valuable time waiting for data if the memory subsystem cannot keep up.
HBM3E is memory, not an SSD
Despite sometimes appearing in broader storage or semiconductor categories, HBM3E should not be confused with an SSD.
It is volatile DRAM.
Data disappears when power is removed, just as it does with conventional system RAM.
An enterprise SSD serves a different role. NAND flash provides persistent storage for datasets, models, applications and operating systems. HBM3E provides extremely fast working memory directly beside an AI accelerator.
An AI server can therefore contain both.
Large datasets may reside on enterprise SSDs before moving through system memory and eventually into the HBM attached to GPUs or AI accelerators for processing.
Eight-layer HBM3E reached 1.18TB/s
SK hynix began volume production of its first HBM3E in March 2024.
The eight-layer version uses vertically stacked DRAM dies to provide 24GB of memory.
SK hynix rated that generation for data processing of up to 1.18TB per second per stack. The company said its Advanced MR-MUF packaging technology also improved heat dissipation by 10% compared with the previous generation.
For comparison, SK hynix has stated that its eight-layer HBM3 could process up to 819GB/s.
This illustrates why HBM3E became so important for generative AI hardware. The improvement is not simply additional memory capacity; considerably more data can also move between the memory and processor every second.
12-layer HBM3E increases capacity to 36GB
SK hynix began volume production of its 12-layer HBM3E in September 2024.
It stacks twelve 3GB DRAM dies to provide 36GB of capacity per HBM package, compared with 24GB from the eight-layer configuration.
Adding another four DRAM layers created a packaging challenge.
Simply making the stack taller could have caused problems for advanced GPU packages that have strict physical dimensions.
SK hynix instead reduced the thickness of the individual DRAM dies by around 40%. This allowed it to increase stack capacity by 50% while maintaining approximately the same total package thickness as the previous eight-layer product.
The dies connect vertically using TSV technology.
9.6Gbps operation provides roughly 1.2TB/s per stack
SK hynix specifies its 12-layer HBM3E for memory operation of up to 9.6Gbps.
In its 2026 technical material, the company describes widely deployed HBM3E as providing approximately 1.2TB/s of bandwidth per stack.
That is an important distinction.
The figure represents bandwidth for an HBM stack, not the total bandwidth of an entire GPU.
AI accelerators normally combine several HBM stacks around the processor. Their aggregate memory capacity and bandwidth can therefore be much higher.
For example, NVIDIA lists 372GB of HBM3E and 16TB/s of aggregate GPU memory bandwidth for its GB200 Grace Blackwell Superchip configuration.
The exact memory configuration ultimately depends on the accelerator and semiconductor package rather than the HBM chip alone.
Advanced MR-MUF helps manage the stacked package
HBM manufacturing involves more than producing fast DRAM.
Stacking many extremely thin dies creates challenges involving heat, mechanical pressure and package warping.
SK hynix uses its Advanced MR-MUF, or Mass Reflow Molded Underfill, process for HBM3E.
The process injects protective material between stacked semiconductor dies before hardening it.
SK hynix says the technology improves heat dissipation while reducing pressure on stacked chips and helping control package warpage.
Thermal characteristics matter because AI accelerators themselves can consume substantial power.
Memory placed immediately beside these processors must operate reliably while handling very high data rates.
SK hynix HBM3E specifications
| Specification | 8-layer HBM3E | 12-layer HBM3E |
|---|---|---|
| Memory type | High Bandwidth DRAM | High Bandwidth DRAM |
| Generation | HBM3E | HBM3E |
| DRAM layers | 8 | 12 |
| DRAM capacity per layer | 3GB | 3GB |
| Capacity per stack | 24GB | 36GB |
| Data processing bandwidth | Up to 1.18TB/s | Around 1.2TB/s |
| 12-layer operating speed | — | Up to 9.6Gbps |
| Vertical interconnect | TSV | TSV |
| Packaging technology | Advanced MR-MUF | Advanced MR-MUF |
| Main target | AI accelerators and HPC | AI accelerators and HPC |
| Commercial production | Began March 2024 | Began September 2024 |
The bandwidth figures are based on SK hynix documentation and should not be confused with aggregate GPU memory bandwidth. System-level performance depends on the processor, number of HBM stacks, package design and workload.
Why AI processors need so much memory bandwidth
Modern AI processing involves large amounts of matrix arithmetic.
A GPU may contain thousands of processing units capable of performing operations in parallel. Keeping those processing units busy requires a continuous flow of data.
If the processor must repeatedly wait for information from memory, adding more compute resources produces diminishing returns.
HBM addresses this bottleneck by placing high-capacity memory close to the processor and providing a very wide memory interface.
This is particularly valuable during large-language-model training, where enormous numbers of model parameters and intermediate values must move through the system repeatedly.
Inference creates its own memory demands.
Larger models require substantial memory capacity merely to hold model weights, while workloads involving long contexts, multiple simultaneous users and AI reasoning can increase memory requirements further.
SK hynix identifies AI training and inference as key drivers behind growing demand for HBM and other AI memory technologies.
AI GPUs are the main HBM3E use case
HBM3E’s most visible use case is high-end data-centre GPUs.
NVIDIA’s Blackwell generation provides a useful example.
NVIDIA specifies HBM3E for its GB200 Grace Blackwell systems, while SK hynix has publicly displayed its own HBM3E integrated with NVIDIA’s GB300 Grace Blackwell platform.
These are not conventional graphics cards aimed primarily at gaming PCs.
They form part of large AI servers and rack-scale systems designed for model training, inference and high-performance computing.
A single accelerator may use several HBM stacks, while a data-centre installation can contain hundreds or thousands of accelerators.
Large-language-model training
Training large AI models is among the most demanding computing workloads currently deployed at scale.
A training system must repeatedly move model parameters, training data and intermediate results while synchronising computation across multiple accelerators.
High memory bandwidth helps reduce the time the GPU spends waiting for data.
Additional capacity is also useful because larger models and training workloads require more information to remain accessible to the accelerator.
The move from 24GB to 36GB per HBM3E stack therefore improves more than a specification table. Multiple stacks can substantially increase the memory available to a processor without requiring a correspondingly taller package.
AI inference and reasoning
Inference is another major HBM3E workload.
Once an AI model has been trained, servers use the model to respond to users, generate images, analyse data or perform other tasks.
The rapid expansion of generative and agentic AI has increased the importance of inference infrastructure.
SK hynix said in July 2026 that demand for AI memory continues to broaden as AI moves towards agentic systems capable of performing more complicated tasks.
Memory capacity and bandwidth can become particularly important when servers handle many users simultaneously or run larger reasoning models.
High-performance computing
HBM predates the current generative-AI boom and remains relevant to high-performance computing.
Scientific simulations, weather modelling, engineering, computational research and other HPC applications frequently work with large datasets that must move rapidly between memory and processors.
Applications that are limited by memory bandwidth rather than raw arithmetic throughput can benefit substantially from an HBM-based accelerator architecture.
HBM3E therefore sits at the intersection of AI and more traditional HPC rather than serving only one application.
HBM3E versus conventional DDR5 memory
HBM3E and server DDR5 are both DRAM, but they solve different problems.
DDR5 DIMMs offer comparatively straightforward capacity expansion through server memory slots. They serve as system memory for CPUs and can provide very large total capacities.
HBM sits much closer to the accelerator.
Its wide interface provides dramatically higher bandwidth, but the memory must form part of the processor’s advanced package.
A data-centre server can use both DDR5 and HBM3E simultaneously.
The CPU accesses its DDR5 system memory while the GPU or AI accelerator uses its local HBM.
HBM3E versus GDDR memory
GDDR memory is widely associated with graphics cards.
Modern GDDR implementations provide substantial bandwidth while allowing memory chips to sit around a GPU on a printed circuit board.
HBM takes a more tightly integrated approach.
The stacked memory sits close to the processor and communicates over an extremely wide interface.
This allows enormous bandwidth, but advanced HBM packaging is substantially more complex.
As a result, HBM3E primarily appears in premium accelerators and high-performance systems rather than mainstream desktop PCs.
HBM4 has arrived, but HBM3E remains relevant
HBM3E is no longer SK hynix’s newest HBM technology.
The company has moved into HBM4, which doubles the interface from 1,024 I/O channels to 2,048 and substantially increases memory bandwidth.
SK hynix began shipping HBM4 products during 2026.
That does not mean HBM3E disappeared when HBM4 arrived.
SK hynix’s 2026 market outlook identified HBM3E as the primary memory used in AI servers and data centres while the industry transitions towards HBM4. The company was also still displaying 36GB HBM3E in current NVIDIA GPU modules during 2026.
The transition resembles other semiconductor generations: new architectures arrive while existing hardware continues shipping and operating in large numbers.
What HBM3E means for South African organisations
South African businesses are unlikely to purchase individual HBM3E packages.
The technology normally arrives as part of a complete AI accelerator, GPU module, server or cloud computing platform.
Its relevance is therefore indirect but significant.
A South African organisation training or running large AI models may encounter HBM3E when selecting GPU servers, renting accelerated cloud instances or evaluating AI infrastructure from international vendors.
Important specifications at that level include total GPU memory, aggregate memory bandwidth, GPU count, interconnect technology, cooling, power requirements and software support.
Knowing that a system uses HBM3E is useful, but the memory generation alone does not determine the performance of an AI server.
HBM3E remains one of the foundations of current AI hardware
SK hynix HBM3E helped move high-bandwidth memory beyond the capabilities of HBM3 by combining higher transfer rates with increased stack capacity.
The eight-layer product provides 24GB and up to 1.18TB/s, while the 12-layer version raises capacity to 36GB and operates at up to 9.6Gbps.
Advanced MR-MUF packaging, TSV connections and thinner DRAM dies allow SK hynix to fit that additional capacity into the tightly constrained packages required by modern AI processors.
HBM4 now extends the technology further, but HBM3E remains embedded in an important generation of AI infrastructure.
For organisations evaluating AI servers, the practical question is therefore not whether HBM3E is the newest memory technology. It is whether the complete accelerator provides enough memory capacity, bandwidth and system performance for the intended training, inference or HPC workload.
