End of Moore's Law: It's not just about physics

For a nice dose of doom and gloom, I quite like the dark silicon paper[1], which explores the limited use we'll get out of Moore's Law, even if it manages to continue (and as far as I understand it, transistors/$ has already flat lined so it's at least temporarily over).

Since we're no longer getting good power scaling out of shrinking, if Moore's Law keeps up, we're just essentially getting price discounts.

A nice thought exercise for me is what would computing look like if you could fab wafers for free, today. Sort of the ad absurdum take on Moore's Law continuing.

We'd have more memory, and a ridiculous amount of flash storage, and high end graphics cards would become cheaper (but not faster). Desktop CPUs might look more like server CPUs, but single core performance wouldn't change one bit.

We'd probably see heterogeneous computing a whole lot more. Xeon Phi like processors on package next to Haswell cores at the very least.

Probably computers would start to have FPGAs in them, as well as large amounts of niche circuitry to compute hashes, encode videos.

We might see computation embedded inside memory (I think Hynix did this recently but I couldn't find a link). Maybe memory that accelerates garbage collection in order to accelerate modern workloads.

So there are some interesting places to go, especially from a Hacker News perspective, but even if transistors became free, we still wouldn't see the rate of progress we did in the 80s or 90s (in terms of speed up to the tasks we're doing).

Of course, this is all near to medium term stuff. I personally believe we'll see that rate of progress again when we move off of silicon, to a different computing paradigm, or (most likely) both - I can't believe we'll inch our way to science fiction computing, or that science fiction computing won't be possible at all.

[1] Dark Silicon and the End of Multicore Scaling (ftp://ftp.cs.utexas.edu/pub/dburger/papers/ISCA11.pdf)

I have, perhaps, a different perspective. I have been spending a lot of my income on computer hardware and power for the last decade or so; and being involved in spending other people's money on same for another half-decade back.

My take? When Intel thinks they are ahead, compute doesn't get cheaper.

In the DDR2 days, If you were on Intel, you had the choice between the stunningly inefficient and expensive rambus ram, or a stunningly shitty memory buss with (not very many) low-power ddr2 modules.

At the time, the AMD HyperTransport system was absolutely beautiful. Even on cheap boards, you could get more than 2x the low-power ddr2 ram modules per CPU that intel could. (at the time, lower-density modules were dramatically cheaper, per gigabyte, than higher-density modules) It worked way better when you had multiple CPUs, too.

Then ddr3 came, and Intel came up with their QPI systems, which were awesome. AMD came back with a competently built ddr3 platform, too; the G34 systems were a huge upgrade from the mcp55 chipset socket F platform.

But the benchmarks came out in Intel's favor, even when AMD had twice the cores. I mean, you could argue that the AMD systems had advantages in some limited situations, but they had lost the dramatic advantage.

As far as I can tell, intel has been largely resting on their laurels, price-wise. The E5-2620 is better than, but really not radically better than the E5520. Now, some of the higher-end E5s are pretty nice, but they are priced accordingly.

Until Intel gets some real competition again, we have to pay for our performance gains.

So yeah, really, until AMD gets their legs back under them, and I hope the A1100[1] will do it, I don't expect dramatic performance per dollar gains from Intel.

[1]http://www.amd.com/en-us/press-releases/Pages/amd-to-acceler...

It's always been economics. EUV already works. E-beam lithography already works. Carbon nanotube transistors already work. III-V transistors already work. It's just that none of these technologies work as cheaply as double patterned silicon.

We still have a magnitude order or two of computing power that we can squeeze out iff the semiconductor density/price stops.

We don't implement many optimization possibilities in each hw/sw layer because the "layer below" keeps changing all the time and we need to keep compatibility. Once we could say "this is it, this layer is as good as it will ever get", then (and not a day before) you can start to re-architecture everything above it to maximize performance by throwing away flexibility that won't be needed anymore.

E.g., instead of transistors being spent on translating x86 to the underlying microcode and cycles being spent on translating JVM/CLR bytecode (or javascript) to x86, we'd be able to define a single standard and adapt both processors and compilers to that. You can't break compatibility at every technological change - but if you have a reason to believe that it will finally be stable (which hasn't ever happened yet) then it does make sense to make a single final switch that disregards all compatibility and legacy issues - even if the benefits are small, they accumulate for each such layer and you only have to do it once.

I've been hearing about the end of Moore's Law ever since I was a kid. I remember smart people citing wave lengths and economics convincing me that there as no way we'd go beyond 300nm. Oh yes, and that would mean the end of x86 as well (since intel wouldn't have the process advantage to compensate for the less efficient CISC).

Well, maybe they are right this time, who knows, but I'm way more skeptical.

One thing I can predict with complete confidence is that we have no clue what the path forward for improving computer processing power will be from 2022. It's fun to speculate but that's a long time for new technologies, approaches, and architectures to take hold. Intel isn't the only company with an interest in improving the state of the art here.

I think maybe we might have been mis-reading Moore's Law for years, or rather, Moore himself might have mis-stated it.

The radical changing reality that it describes is the number of transistors a person could economically employ at any one time to perform work on their behalf.

So yes, physics and economics might provide us with a limit (or increasing slope of difficulty) for the construction of single chips, the practical effects of the Law continue unabated, at least as far as I can see. The average person continues to be able to employ more and more electronics to perform work for them. This is increasing geometrically.

I'm not trying to dismiss either this DARPA guy or Moore, just point out that the specific details of Moore's Law may not be as important as we make them out to be.

In my mind, the big obstacle we have now to continued growth is small-system, imperative thinking. Systems of the future will be massively parallel. I have no idea how long it will take the IT industry to truly transition, but that's the next big hurdle, not counting atoms inside a switch.

For the talk by the DARPA guy:

https://www.youtube.com/watch?v=JpgV6rCn5-g#t=15

Worst formatting I've ever seen. Anyway:

Colwell said that for the Defense Department, he uses the year 2020 and 7 nanometers as the "last process technology node." But he adds, "In reality, I expect the industry to do whatever heavy lifting is needed to push to 5nm, even if 5nm doesn't offer much advantage over 7, and that moves the earliest end to 2022. I think the end comes right around those nodes."

Let's assume that in 10 years time everyone agrees we have hit the wall on how small we can go. What then? Is there any reason to believe that popular architectures of today are necessarily optimal? I'm curious to hear people's ideas about what we do instead of shrinking dies.

My personal guess is that we move towards massively parallel systems with large numbers of low-power cores, but bare-metal programming completely and work on developing smarter and smarter compilers to take advantage of parallelism. My personal hope is that we find some sort of cold optical switching technology that lets us build ridiculously fast computers that look like glowing crystal cubes. Of course, I have no idea how that would work, or I'd be out pitching it XD

Yeah, I'm sure someone had similar doom and gloom to say about vacuum tubes.

The thing is, you just don't know where the next technological revolution will come from. Yes, we've about hit the limit for silicon, and we may very well stagnate for another decade because of it, but there's always going to be someone scrappy enough to try what the big slow incumbents won't.

There are gains yet to be realized besides reducing the size of the process. For example, reversible (a.k.a. isentropic or adiabatic) computing offers a way to reduce heat generation, which might combine with 3D construction in interesting ways. New ways of designing chips might allow progress to continue, but they're hard and risky. They're not terribly attractive as long as shrinking the process offers predictable advances and remains economically feasible.

Still, it's worth taking a moment to appreciate how crazy just getting under 10 nm is. The wavelength of light that is visible to the human eye starts at around 380 nm. Looking at a 10 nm chip with violet light would be like trying to navigate your house by sonar using a sub-woofer as your emitter!

One thing that seems to be glossed over in his argument is the impending move to 450mm wafer technology. That should allow Intel and others to continue shrinking at reasonable cost.

> it's time to start planning for the end of Moore's Law

Do we? Because all current software runs on current hardware. Even if in the next ten years we only get another 1% increase in speed, then the current software will run at the current speed (something which we are all perfectly fine with right now). I think it's indeed a doom and gloom article.

smaller devices will provide the economic drive, just as they did when 14" HDDs had enough performance.

Either this article or the guy's speech perhaps both are pretty lame. No supporting facts are presented as to why he thinks the 7nm generation will be the last frontier. He just basically states the obvious that if there is no profit motive or the technology isn't there then it won't happen. No kidding.

The whole article reads like an appeal to authority.

Are there any factual reasons to believe this time is actually different than the past 35 years?

For decades I have heard that 11nm would be the end of Moore's law as we know it. This is proving to be true. CPUs have been frozen at 1 to 4GHz for nearly a decade, with the latest advances going to power savings. Currently at 22nm, the next step is 14, and then 11. Perhaps they can eek out one more jump to 7nm or 5nm, but I expect that will barely be worth the effort and thus will drag out for at least decade itself.

But Moore fans need not worry. There is a clear next step, and I am wondering why no one is talking about it: Optical Interconnects. Connecting chips and circuits within chips with optical channels should allow plenty of room for speeding up processors and reducing power requirements well into the mid-21st century.