TSMC boss says one-trillion transistor GPU is possible by early 2030s | Hacker News Reader

TSMC boss says one-trillion transistor GPU is possible by early 2030s

theregister.com

39 pointsPaulHoule10 comments

We are about to witness ( although this has somewhat already happening ) where GPU or CPU are no longer measured by a single die but by package of multiple chiplets. You could have 4 x 250B transistor GPU Die packed together and the GPU will be 1 trillion transistor. And can continue to "claim" Moore's law is not broken.

The chiplet design is another innovation like FinFET and even EUV that is allowing for more transistors closer together. The interconnect for instance on the blackwell means no timing issues.

The transistor size is already closing in on 1nm, there will be innovation there eventually, but any step can be a stepping stone. I don’t see the issue here.

Transistors are nowhere close to 1nm right now. Current 3nm stuff is something like 20-40nm wide depending on how you measure it.

You are talking for Servers right? Else how do you plan to cool such a beast?

Won’t it have large surface area too?

Chiplets will be underclocked to run close to their most efficient power / performance frequency.

Exactly, this is what Apple are doing with their “Ultra” architecture.

It really is the way forward.

Doesn't Cerebras already have a trillion ?

Monolithic die?

Yes, 4 trillion on a single wafer. Apparently they're designed to be full wafer-sized. It's not a single multi-trillion monolithic core.

        4 trillion transistors

        900,000 AI cores

        125 petaflops of peak AI performance

        44GB on-chip SRAM

        5nm TSMC process

        External memory: 1.5TB, 12TB, or 1.2PB

        Trains AI models up to 24 trillion parameters

        Cluster size of up to 2048 CS-3 systems

https://www.cerebras.net/press-release/cerebras-announces-th...

Reply on news.ycombinator.com