bippingchip

Same here. What’s this about? Anyone who can see willing to share some more info?

These gate and metal pitch numbers don’t tell the whole story. In the end it’s logic gate density that counts.

And while decreasing the gate and metal pitch, also the logic gates have shrunk to be smaller (typically expressed by measuring the height of a gate in amount of metal tracks) from 9tracks down to 6tracks.

Changing the transistor from planar to fins, and now hopefully to ribbons with eventually stacked pmos and nmos are a big enabler.

That said, we’re still not hitting the ideal scaling numbers. We’re just doing somewhat better than what’s suggested by poly and metal pitch only.

This looks like a treasure trove on what it takes in terms of algorithms to enable tools like Cadence Innovus or Synopsys ICC. It’s not a user guide on how to use these tools, but rather a perk behind the curtain.

I’ve worked with Andrew, one of the authors on occasion in the past, and he and his team of students are among the best academic teams in the world on this topic.

I do think a lot of the secret sauce lives as trade secret with Cadence, Synopsys, Mentor… They see all the real problems in designs from all their customers in bleeding edge nodes like 3nm and beyond.

Thank you for the write up! Very enlightening to see how things went for you. I was on the fence recently whether to go for contractor work but ended up just switching companies in the end. In part because I didn’t know what to expect. Articles like these hwould have helped!

Thank you! For now I am just binding c++ to Python but I expect/fear the lines might start blurring, so cython might come in handy then.

Thank you! - I’ll give pybind11 a go.

Thank you! Swig indeed can be a pain but having used it before I have become somewhat blind to it. But eg smart pointers are not easy to deal with well, I’ve found out recently… I’ll have a look at pybind11. I’ve worked on Cython codebases too, which indeed allows to really nicely compile Python code and interact with c code. It does get weird when using eg pyqt and native qt…

This is a somewhat tangential question to the new release, but there might be folks here that can answer this question.

Having used swig to create Python bindings for C++ code over 10 years ago, what’s the recommended way to do this in 2023? There’s still swig, there’s Cython, pybind11 and a few others. I do know swig and how complicated it can get with footguns abound when things grow more complex.

Is Cython the way to go? How does it hold up to the alternatives? Google search gives many articles on the topic, but many typical SEO optimized low-value, and those that do show a bit of depth, focus on the basic mechanics, not really on things hold up for larger projects…

If you just need a nice print: fmtlib is a really nice c++23 style implementation without needing c++23 compiler support. Highly recommend it. It’s simple. It’s fast.

Not really knowing Vercel, I thought, based on the title, that it might have been a new GPU competitor (accelerator card) but it’s a startup accelerator.

While I have no need for its online functionality and the SAAS part of plotly, I really do like plotly python + cufflinks [1]. It lets you make interactive plots in html/js format. Which means you can save the notebook as html, and while people won't be able to rerun the code, they can still zoom in on graphs, hover to see annotations etc, which is a really nice way to share the outcome of your work in a more accessible way.

[1] https://github.com/santosjorge/cufflinks

I was wondering what the goal of the project is. The README is not very clear on it, but the implementation document [1] does state design goals:

- Small memory usage within a fixed-sized memory region — no mallocs

- Practical for small scripts (extension scripts, config files)

- Concise source — less than 1000 loc

- Portable ANSI C (Windows, Linux, DOS — 32 and 64bit)

- Simple and easy to understand source

- Simple and easy to use C API

[1] https://github.com/rxi/fe/blob/master/doc/impl.md

A lot of companies are indeed trying to build AI accelerator cards, but I would not necessarily call them ASICs in the narrow sense of the word, they are by necessity always quite programmable and flexible: NN workloads characteristics change much much faster than you can design and manufacture chips.

I would say they are more like GPUs or DSPs: programmable but optimised for a specific application domain, ML/AI workloads in this case. Sometimes people call this ASIPs: application specific instruction set processors. While maybe not a very commonly used term, it is technically more correct.

+1 for UTM. I use it to run an arm Linux distro that uses Rosetta for x86 binary only programs. Works like a charm and blazingly fast!

The article overlooks a fundamental side of the social contract that is at least equally, if not more, important: how much time and effort (not money) do personally invest to spend time with your friends.

In my view, it's only awkward when the money side of things is not aligned with the personal investment.

That dinner example from the article actually shows this: I buy you all Olive garden dinner, or: I take the time to invite you to my home, spend some time clean the house think on what to by and prepare, what music to play, maybe a movie to watch after etc. in order to have a good time together. This is a much more thoughtful and mutually beneficial form of investment in friendship than just throwing money at it.

Another example could be: Hey, I bought a new board game (or PS5 or something else), wanna come over and play? You might have spent quite some money, but the goal is to be able to invest in spending time with your friends.

The moment that is (or is perceived to be) your main intent, most folks would have a hard time looking at this as bribery.

Oh these are nice. They are the modern equivalent, in a way, of the industrial photos of Bernd and Hill Becher [1], who with and archivist's diligence sought out, cataloged and captured industrial sites (often now long decommissioned and disappeared). It captures a picture of an era gone or disappearing.

If you have the chance: check out on of their exhibitions. I saw it back in December in SFMOMA. It's a special and humbling experience to see wall after wall full of all kinds of variations on the same industrial theme, like water towers. (almost like they were generated by a NN, and the Bechers just played with temperature, prompts to create variations...)

You could say the same of these oil rigs: Massive feats of human engineering and ingenuity, worth capturing for eternity, as over time many of them might disappear or will be replaced.

[1] https://www.metmuseum.org/exhibitions/becher#

Also a gothamchess viewer I see :) Levy's indeed always making a sport of it to have the most clickbaity titles and thumbnails, but the chess content is really really top notch. (especially for low ELO players like myself)

I 100% agree on verilator. It’s by far the most efficient tool for many simulation and verification needs. Hands down. Even more so if you are building CPUs

UVM definitely has its place still in verification flows. It for something as complex and flexible as a CPU for sure, it’s hard to beat the amount of cycles and coverage you get from verilator. (EDA license fees are a big part of that, but also the fact that verilator generates just C++ code you can integrate with the rest of your s/w flow is invaluable).

But: What is also important for CPU is good validation and compliance test, (random or replayed or directed) instruction stream generators, ISS to validate agains etc. There are so much more scenarios to check because a CPU needs to handle any sequence of any instructions flawlessly.

Those tools are also crucial. And they are being built or sold for RISC-V now too. And also: I much prefer combining those with verilator than with commercial eda tools. And don’t forget: before you start building a lot of RTL, you’ll typically will also build C based higher level performance models of the micro architecture, eg https://github.com/riscv-software-src/riscv-perf-model

The package is the new pcb.

But it’s going to be a chaotic gold rush with many competing solutions for a while, before a de facto standard will emerge (the way it usually goes).

Will it be UCIe, or something else? What will change when people introduce co packaged optics as well? Etc.

On the underlying packaging techniques there’s not many companies in the race.

I feel as a foundry, TSMC is technically most strongly positioned here: best offering across multiple technologies and definitely far ahead in offering complex chiplet based solutions.

Samsung and intel bring design into the picture as well, so they play multiple sides. Intel could have made a nice play with their foundry solutions to open up to fabless companies and their chiplet technology. But they failed to leverage that in house so far (their GPUs and accelerators, like pointe vecchio and others seem not commercially viable), which makes it much less likely to succeed in the broader market.

Yes I ran into the same thing - It's disappointing, though maybe understandable as this is an iteration on the Primephonic app they acquired a while ago. And though I cannot find any reference to it anymore, if memory serves right, that was mobile only too?

I hope that over time Classical will make it onto the Mac as well. I listen to a lot of classical music while I work and it's more than a bit annoying that the play/volume controls on my Mac don't work...

But don't underestimate how much 'middleware' there is in Cuda/CuDNN, NCCL etc. A lot of it is not in the compiler, but much more in hand crafted, carefully optimised libraries.

As an example: there are so many low level ways to run a conv2D kernel on a SIMT machine, but CuDNN will pick the best option for you based on the card you have, and the tensor sizes you are using.

Very nice explanation! It gets very complex very quickly... And then there's of course also the fact you never know if your last bytes in the Ifetch you get from the cache is a full instruction or not. You'll need to keep that around until the next line arrives which will have the rest of that instruction.

And then add to that the complexity of speculative and out of order execution: you will want to very early decide, i.e. (pre)decode, where branch instructions are, in order to make sure your branch prediction unit has the right inputs in time, to decide where to fetch the next instructions from.

And after all that, on a branch mispredict earlier on (or pipeline flush for another reason), you might need to throw almost all of that expensively decoded instructions away and start fetching from elsewhere...

CPUs are fun :)

Apart from the fact that people might not want it, even technically you might still have latency issues, our eyes are extremely sensitive… also keep in mind: the sensors and displays you need on head might still weigh a lot and consume a lot of power on your head, even if all processing power is relegated to an external unit.

As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a sudden. Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...

This is really interesting, thank you for the reference!

Having worked more with images based NN than language models before, I wonder: are LLM inherently more suited to aggressive quantisation, due to their very large size? I see people suggesting here 4b is pretty good, and 3b should be the target.

I remember ResNets etc can of course also be quantized, and up to 8-6b you get pretty good results with very little effort, with low-ish degradation in performance. Trying to go down to 4b is more challenging, though this paper claims with quantisation aware training 4b is possible indeed, but that means a lot of dedicate training compute needed to get to 4b (not just finetuning post-training): https://arxiv.org/abs/2105.03536

There’s a (short) list of countries where loot boxes are illegal [1].

Japan is on that list as well, though I think the Mario Kart micro transactions and gambling like mechanisms are still legal.

Belgium probably has one of the strictest laws: “Looking at various games, such as FIFA 18 and Overwatch, Belgium determined that the randomized risk/reward system innate to loot boxes is tantamount to gambling.” Mario Kart Tour, for example, is simply not available there.

[1] https://screenrant.com/lootbox-gambling-microtransactions-il...

I had the exact same question. I do like this - it's nice and ambitious to build something like this from the ground up. Looking at the todo list though, they still have quite some work to get to parity with qemu...

The RISC-V isa is specifically designed to be nice and regular and easy to decode, and work with, which definitely shows here (also in the RTL code if you look at some of the well designed RISC-V cores)

Of course, bot x86 and Arm started like that as well - but after 20+ years of evolution, they have to drag along a lot of history. (and one never really takes things away from an ISA, you only add new features, and at best deprecate old ones).

That said, with the extensibility of the ISA in mind as a design principle, I do have good hopes, RISC-V will stand the test of time reasonable well in that sense...

There are 2 main difference between many university (and even the much larger dedicated research lab/fabs like for example in Albany/NY Creates): - Research labs are typically equipped to focus on critical parts of the overal production process they want to do research on. But to get actual designs manufactured you need a fully working and sufficiently yielding process from bare wafers to transistors, interconnect, passivation etc. This is easily multiple hundreds of steps to 1000+ different process steps (just think about making a single metal layer: deposit resist, expose, etch, deposit barrier, liner and and then metal, CMP)

This works well for the uses cases you describe: dedicated inspection equipment or very niche small scale nano fabrication needed. But making a a fully working VLSI CMOS process needs much higher complexity and many more different process steps and equipment.

Which brings me to the second point: - Research labs typically don't care much about yield: they need sufficient amount of working devices to measure and show things are possible and publish about it, but they do not need to bring it up to reliably reproducible results. However when you make an ASIC with millions to billons of transistors and wires, you typically depend on ALL vias and ALL transistors working, not just 80-90% of them, to get a functional chip. This is a lot of hard and dedicated work, in a way it's an art, black magic almost to most: very few people and companies have the expertise to pull this off. It's not a big exaggeration to say only TSMC can do this really well. Just look at how much even Intel is struggling to get their next node out (it's been how many years of Tick Tock Tock Tock Tock.. now?)

e-beam litho is anything but fast though: the machines are cheaper but they are much much slower in wafer throughput than the insanely expensive ASML EUV machines.

Now the website claims a fast fab, but leaves it open what that means: fast production of wafers? Or slow production of wafers that run fast?