Hacker News Reader

To clarify and contextualize a bit what you're saying:

The one big obstacle in creating larger chips is defects. There's just a statistical chance of there being a defect on any given surface area of the wafer, defect which generally breaks the chip that occupies that area of the wafer.

So historically, the approach was to make more smaller chips and trash those chips on the wafer affected by defects. Then came the "chiplet" approach where they can assemble those functional chips into a larger meta-chip (like the Apple M1 Ultra).

But as you're saying, changes in the way chips are designed can make them resilient to defects, so you no longer need to trash that chip on your wafer that's affected by such a defect, and can thus design a larger chip without fear of defects.

(Of course such an approach requires a level of redundancy in the design, so there's a tradeoff)

I'll add that many DRAM chips already do something like this, but ironically enough the re-routing mechanism adds complexity which is itself a source of problems, (be it manufacturing or design, such as broken timing promises)

Also, NAND Flash storage (SSD) is designed around the very concept of re-routing around bad blocks, because the very technology means they have a wear-life.

> I'll add that many DRAM chips already do something like this, but ironically enough the re-routing mechanism adds complexity which is itself a source of problems, (be it manufacturing or design, such as broken timing promises)

The best-performing solution there is probably software. Tell the OS about bad blocks and keep the hardware simple.

I think this is already implemented both in Linux and in Windows; you can tell the OS which RAM ranges are defective.

Doing this from the chip side is not there yet, apparently. I wonder when will this be included in the DRAM feature list, if ever. I suspect that detecting defects from the RAM side is not trivial.

> I suspect that detecting defects from the RAM side is not trivial.

Factory testing or a basic self-test mode could easily find any parts that are flat-out broken. And as internal ECC rolls out as a standard feature, that could help find weaker rows over time.

Yep, my last PC developed a defect in one of the RAM modules. Finding it using memtest86 was trivial; easier than figuring out exactly how to tell Windows what to do about it...

Of course it did take a little bit of a hunch to go from "the game I'm playing crashes at this point" to "maybe my RAM is defective". I suppose ECC would help spot this.

Spot and correct it too :)

Wikipedia link on microlithography if you want a rabbit hole about wafer making:

https://wikipedia.org/wiki/Microlithography

Being able to print something in nanometers is an overlooked technical achievement for human manufacturing.

If that rabbit hole appeals, the ITRS reports (now called IRDS[2]) are very good mid-level, year-by-year summary of the state of the art in chipmaking, including upcoming challenges and future directions.

> Being able to print something in nanometers is an overlooked technical achievement for human manufacturing.

IMO, a semiconductor fab probably is the highest human achievement in terms of process engineering. Not only do you "print" nanometric devices, you do it continuously, in a multi-month pipelined system and sell the results for as little as under a penny (micros, and even the biggest baddest CPUs are "only" a thousand pounds, far less than any other item with literally a billion functional designed features on it).

[1]: https://en.wikipedia.org/wiki/International_Technology_Roadm...

[2]: https://en.wikipedia.org/wiki/International_Roadmap_for_Devi...

is it the same reports accessed by the process in https://irds.ieee.org/home/how-to-download-irds ?

For IRDS, yes.

This is briefly touched upon in Neal Stephenson's "Seveneves", after The Cataclysm happens humanity regrows most technology, but for the most part ICs are limited to 8086-level devices as even thousands of years later they are unable to reach the level of semiconductor technology before it. The amount of technology and power in something as small as a cell phone or lowly raspberry pi is nothing short of a marvel, particularly at their price points.

The Cerebras WFE is design has on each wafer to disable / efuse a portion of itself to account for defects. This is what you can do if your control the wafer.

> change in the way chips are designed can make them resilient to defects

This is already happening for almost all modern chips manufactured in the last 10+ years. DRAM chips have extra rows/cols. Even Intel CPUs have redundant cache lines, internal bus lines and other redundant critical parts, which are burned-in during initial chip testing.

The other one big obstacle is chips are square while wafers are round.

it depends on the exact shape of your mask of course, but typically losses around the edges are in the 2-3% range.

It's not really possible to fix this either since wafers need to be round for various manufacturing processes (spinning the wafer for coating or washing stages) and round obviously isn't a dense packing of the mask itself. It just kinda is how it is, square mask and round wafer means you lose a bit off the edges, fact of life.

The interesting part is loss scales with the die size.

> changes in the way chips are designed can make them resilient to defects, so you no longer need to trash that chip on your wafer that's affected by such a defect,

no, it's basically "chiplets but you don't cut the chiplets apart". You design the chiplets to be nodes in a mesh interconnect, and failed chiplets can simply be disabled entirely and then routed around. But they're still "chiplets" that have their own functionality and provide a coarser conceptual block than a core itself and thus simplify some of the rest of the chip design (communications/interconnect, etc).

note that technically (if you don't mind the complexity) there's nothing wrong with harvesting at multiple levels like this! You could have "this chiplet has 8 cores, that one has 6, that one failed entirely and is disabled" and as long as it doesn't adversely affect program characteristics too much (data load piling up or whatever) that can be fine too.

however, there's nothing about "changes in the way the chips are designed that makes them more resilient to defects", you still get the same failure rates per chiplet, and will still get the same amount of failed (or partially failed) chiplets per wafer, but instead of cutting out the good ones and then repackaging, you just leave them all together around "route around the bad ones".

The advantage is that MCM-style chiplet/interposer packaging actually makes data movement much more expensive, because you have to run a more powerful interconnect, where this isn't moving anything "off-chip", so you avoid a lot of that power cost. There are other technologies like EMIB and copper-copper bonding that potentially can lessen those costs for chiplets of course.

What Intel is looking at doing with "tiles" in their future architectures with chiplets connected by EMIB at the edges (especially if they use copper-copper bonding) is sort of a half-step in engineering terms here but I think there are still engineering benefits (and downsides of course) to doing it as a single wafer rather than hopping through the bridge even with a really good copper-copper bond. Actual full-on MCM/interposer packaging is a step worse than cu-cu bonding and requires more energy but even cu-cu bonding is not perfect and thus not as good as just "on-chip" routing. So WSI is designed to get everything "on-chip" but without the yield problems of just a single giant chip.

"there's nothing about "changes in the way the chips are designed that makes them more resilient to defects"

Of course there is. Scarequotes invalid.

Ever since the first first pal/gal/cpld/fpga-like device, the essence of malleable silicon has existed, and has been incorporated into things more and more over time.

It's 40 year old stuff by now.

... that's the exact opposite of every economic and yield advantage that chiplet design addresses, isn't it?

Want to fine tune your chip offering to some multiple of 8 cores (arbitrary example of the # cores on the chiplet)? Just a packaging issue.

Want to upbin very large corecounts that generally overclock quite well? For a massive unichip described, maybe there are sections of the chip that are clocking well and sections that aren't: you're stuck. With chiplets, you have better binning granularity and packaging.

Want to fine-tune various cache levels? I believe from what I've read that AMD is doing L3 on a separate chiplet (and vertically stacking it!). So you can custom-tune the cache size for chip designs,

You can custom-process different parts of the "CPU" with different processes and fabrication, possibly even different fab vendors.

You can upgrade various things like memory support and other evolving things in an isolated package, which should help design and testing.

The interconnects are the main problem. But then again, I can't imaging what a foot-wide CPU introduces for intra-chip communication, it probably would have it's own pseudo-interconnect highway anyway.

Maybe you don't even need to reengineer some chiplets between processor generations. If the BFD of your new release is some improvement to the higher or lower cpus in the High-Low designs, but the other is the same, then that should be more organizational efficiency.

Intel and others have effectively moved from gigantic integrated circuits decades ago: motherboard chipsets were always done with a separate cheaper fab that was a gen or two behind the CPU.

Maybe when process tech has finally stabilized for a generation now that process technology seems to be stagnating more, then massive wafer designs will start to edge out chiplet designs, but right now it appears that the opposite has happened and will continue for the foreseeable future.

I said it upthread but maybe it's better to use the conceptual model of "chiplets but you don't cut the chiplets apart". You have some network of "chiplets" connected with some kind of interconnect (a mesh is the model everyone's used so far).

So a "chiplet" (a module) could still have some of its cores turned off, or be disabled entirely, and then you just route around them. The yields are thus the exact same you would expect from taking a wafer, printing a bunch of chiplets, and then not cutting it apart. Most of them (Zen3 is >90% with 8 functional cores now iirc) will be fine. A few will fail clock bins. A few will be dead. Etc.

https://cerebras.net/blog/wafer-scale-processors-the-time-ha...

A lot of the points you raise are fair, some of them can be engineered around and others not really.

* There is no reason that different "chiplets" can't have different clocks. big.LITTLE already deals with different clock domains and rates, your interconnect should be asynchronous and then it's just an aspect of the design. Maybe it's not desirable from a programming perspective but it's not a problem in hardware, and if you don't like an uneven canvas, then just disable those modules.

* However you can't do different voltages as easily. Having a FIVR (Fully-Integrated Voltage Regulator) in each "chiplet" is an option, just feed some high voltage like 2v or 3v and have the chip step it down. This is highly desirable for efficiency anyway. Or just feed a reasonably high voltage and if some of the chips don't clock well, then oh well, you use a little more voltage than necessary (Intel does this for Alder Lake - the chip uses the greater of "V_big" and "V_little" for whatever values are needed for frequencies at the moment - if the little cores get more voltage than they need, oh well). I don't think clock variations/chip quality generally varies as much as people imagine it does, especially once a process gets relatively mature, you can come up with some relatively good bars that almost all cores will pass.

* Wafer-scale chips have to be fabbed with everything on one process, that's true. With "tiles" and EMIB, or MCM, or regular stacking, you can mix and match them, including between vendors.

A lot of these "mix and match" bits don't really make sense/don't really fit the release model/don't really fit the physical engineering:

* A core block is already a self-contained (relatively) module that can be designed/tested/updated in isolation. Once you've got the bugs hammered out, scaling it arbitrarily large isn't a problem, and again, you probably are organizing it into "modules" of 8 cores or so anyway for organization's sake, even with wafer-scale.

* "version friction" is already a thing even in other chips. If RDNA2 isn't ready to go when AMD designs their Zen3 APUs... oh well, it doesn't go until the next generation. You could certainly iterate versions more quickly, but you can't change these behind your customer's back - if RDNA2 becomes available at some later date, you can't just swap it in, that's a new product and has to be validated and designed for separately.

* "steppings" already handle this in traditional chips. If there is some minor improvement in the memory controller, it becomes a new stepping of the chip. If it's a big change... it's a new release.

The big advantage of wafer-scale over current MCM/interposer tech at an engineering level is the much lower cost of data movement. MCM/interposer is expensive to push a signal all the way through a big interposer somewhere else, and it takes quite a lot of cache to even attempt to hide this. "Active interposer", EMIB, and advanced bonding technologies can help reduce this a lot, but it's "cheaper" to just push it to the module next to you and then have it pass the message along in turn.

MCM also really doesn't go to the scale wafer-scale does. What if I want a package with 64 CCDs on it? That's really what wafer-scale is about. But you can't do that with MCM since that's bigger than the reticle limit used to print an interposer. What you'd have to do is print "bridges" that go under the edges of the chiplets and carry the signal, and then mount the whole thing on an inert substrate (not printed like an interposer). And... that's exactly what EMIB does. But you'd have to do a ton of packaging work to get the size of chip that wafer-scale can gives you, each of those packaging stages (attaching each bridge, etc) has failure rates too. And from the sounds of it, it's actually quite high (AMD is supposedly taking a big hit on Milan-X yields due to the cache packaging step).

> Wafer scale integration [1] has come up periodically over the years.

Not merely years, but decades. I worked with a wafer-scale group in MIT Lincoln Laboratory back in the late 80s. I'd say there is a reason the technology hasn't taken off in the past 35 years, but who knows, maybe now is the time for wafer scale integration to shine.

Incidentally, that group was originally Ken Olsen's group, after he left to go found Digital Equipment Corporation (DEC). They had a lot of talent and produced some interesting wafers. The tech back then (4in wafers iirc) used laser reconfigurability to route around bad cells.

I think a part of this is that the clock speed increases made most other more expensive solutions not yet commercially viable, but now that 'the end' is in sight a lot of those more expensive solutions are making a come back. You could see the same happening in software to mirror the far more exotic hardware that we use now compared to single core CPUs which for a very long time were the norm.

Nothing new under the sun. Ivor Catt was proposing wafer scale computing in the '70s. Large numbers of processors with the ability to route around defective units.

https://www.ivorcatt.org/icrns86jun_0004.htm

Calling wafer-scale "no interconnect" is kind of misleading since it's still very difficult to stitch reticles and it has yield challenges.

Indeed, that Zdnet article hints at how they solved that but doesn't provide detail enough to make sense of it.

The typical problem is the ~30mm retical dimension limit. Unless you want to have the same pattern copied ~100x over the wafer you'll need to have more unique masks... A LOT of masks, and many more mask steps (because they can't step the same mask). For reasonable (finfet) geometries that is much too expensive, even if you could buy the required aligners.

I suspect stacked wafer is much more cost competitive, and you get to use appropriate (RAM, CMOS, Flash) processes for each layerso long as their sizes match. Via size and density are such that speed is less of an issue than going to package and the yield hit isn't more than 2-3x.

Sounds like a great development if it works out.

But consider also that you can stick chiplets on top of each other vertically. That means you can put chiplets much closer together than if they were constrained to exist on the same single plane of the wafer.

Now how about stacking wafers on top of wafers? That could be super, but there might be technical difficulties, which maybe sooner or later can be overcome.

> But consider also that you can stick chiplets on top of each other vertically.

The problem there is heat dissipation. Already the performance constraint on consumer chips like the Apple M1 is how well it can dissipate heat in the product it's placed in (see Macbook Air vs Mac Mini). Stacking the chips just makes it worse.

the problem is signal propagation

for light to cross 1 feet should take ca 1ns

3D circuits would be denser (shorter propagation distances) than a planar circuit. In fact "computronium" is sort of an idea about how dense you can conceptually make computation.

You just can't really cool it that well with current technologies. Microfluidics are the current magic wand that everyone wishes existed but it's a ways away yet.

Signal propagation latency is only a problem if both end points need to be fully synchronous, which they don't have to be if they're independent units. Even the cores within single-chip multi-core CPUs aren't synchronous to each other, as far as I know.

The fact that the M1 Macbook Air operates without needing a fan is very unusual for that level of performance.

AMD's 5800X3D and the upcoming generation of AMD/NVIDIA GPUs (both of which are rumored to feature stacked cache dies) are going to be real interesting. So far we haven't ever seen a stacked enthusiast die (MCM doesn't feature any active transistors on the interposer) and it will be interesting to see how the thermals work out.

This isn't even stacking compute dies either, stacking memory/cache is the low-hanging fruit but in the long term what everyone really wants is stacking multiple compute dies on top of each other, and that's going to get spicy real quick.

M1 is the other example but again, Apple's architecture is sort of unique in that they've designed it to run from the ground up at 3 GHz exactly, there's no overclocking/etc like enthusiasts generally expect. AMD is having to disable voltage control/overclocking on the 5800X3D as well (although that may be more related to voltage control rather than thermals - sounds like the cache die may run off one of the voltage rails from the CPU, potentially a FIVR could be used to drive that rail independently, or add an additional V_mem rail...)

And maybe that's the long-term future of things, that overclocking goes away and you just design for a tighter design envelope, one where you know the thermals work for the dies in the middle of the sandwich. Plus the Apple design of "crazy high IPC and moderately low ~3 GHz clocks" appears well-adapted for that reality.

Wide and slow is expensive. Very expensive. That's why Apple can do it and nobody else is doing it (in mobile Qualcomm is _cutting_ cache from Arm reference designs and in servers Graviton and Ampere are also cutting cache). It is cheaper for a given performance level to clock your cores as far as they'll go and cheap out on the width of your core if you know your customers either won't or can't care about power efficiency (because they have no better alternatives).

The problem is cost. Instead of getting N CPU’s per wafer you are now getting just 1 mega-CPU. So you need to sell the mega-CPU at N times the price to get the same ROI. So a solution for super computer applications. Not for everybody else.

Instead of microcircuits, megacircuits. I like it

speed of light ~ 1*10^9 feet/sec

To cross one foot - no less than 10^-9 sec = 1ns

There are some new chip manufacturing technique coming down the pipeline, which will lead to prices dropping and likely "wafer-scale" will get to the mainstream.

Could you elaborate? Would love to know more.

Unfortunately, I cannot.

Can you clarify what you mean? How can anything that would use a major share of a wafer or even a whole one be mainstream? Producing a wafer is expensive, the chips are only available to ordinary people in high income markets because you get hundreds or even thousands from a single wafer.