Hacker News Reader

It's SRAM. It's the equivalent of 44 GB of CPU cache.

Exactly, which is why it’s so incredibly fast. And expensive.

traditional SRAM memory has 6 transistors per bit. Dense SRAM can have two transistors.

Wafers can have trillions of transistors, 10-16 times more than the 4 trillion of the WSE-3.

1 Terabyte of SRAM per wafer will be the upper limit for a few years [1].

[1] https://youtu.be/vbqKClBwFwI?si=gbBeZ1B66_eGI81Z

SRAM doesn't shrink as well as other types of circuitry do when process nodes advance.

SRAM is not very dense.

IMO the next big shift in AI hardware is when some of the upcoming logic-compatible memory types mature. Inference inherently likes many small local pools of memory, with their own interface each. This is impractical with currently available memory other than sram.

You mean it is unimaginably massive.