Hacker News Reader

Comment by tithe | Hacker News Reader

Peter Hofstee (one of the chief architects of Cell) gave a talk at UT Austin around 2008.

At the time, UT was building TRIPS[0], a processor capable of > 1 teraflop / sec.

In the middle of his talk (with several TRIPS members present), he made an off-hand remark "I think I could build a petaflop processer...without too much trouble,"* which caused a small stir in the audience!

[0] https://www.cs.utexas.edu/users/cart/trips/

* I might be misremembering the exact quote, but this was the gist of it.

View on HN

Perhaps relevant to the story, he was also on the team that delivered the first petaflop supercomputer (the most powerful supercomputer at the time) around that same time frame (it was around 24K processors):

Roadrunner is a 1.38 Pflop/s-peak (double precision) hybrid-architecture supercomputer developed by LANL and IBM. It contains 12,240 IBM PowerXCell 8i processors and 12,240 AMD Opteron cores in 3,060 compute nodes. Roadrunner is the first supercomputer to run Linpack at a sustained speed in excess of 1 Pflop/s.

https://dl.acm.org/doi/10.5555/1413370.1413372

https://en.wikipedia.org/wiki/Roadrunner_(supercomputer)

What a pain in the ass machine that was to write code for. You had 3 different processors to manage with 3 different architectures and all of the challenges the game dev community had with keeping the SPEs hot. But the SPEs were not enough to solely rely on for number crunching performance unlike today's GPU compute machines, so you also needed to do number crunching on the opterons and the main cell core for optimal performance unlike with a GPU where the CPU is mostly just keeping the GPU memory full. Then to make matters worse the cell had a different endianess than x86 making shared memory very annoying to work with.

I'm curious if you've ever heard of or used CellFS[0] , which was supposed to simplify programming for Cell?

[0] https://www.osti.gov/servlets/purl/1000498

Haven't heard of it. Internal communication was not a strong suit at LANL... You also had a ton of different teams who all got access to the hardware at roughly the same time and were shotgunning different approaches and coalescing on the better libraries wasn't really a priority at the beginning of roadrunner when I was there. LANL didn't really have project management at the code base level to direct people to upgrade stuff like that, its mostly independent researchers pulling the code bases in different directions. Most of those researchers were physicists in my world and didn't really care about the "engineering" of software, just that their code ran fast and correct enough to get to the next deadline. Then there is a small core of more computer science researchers who did projects like this. If they were successful enough they'd attempt to integrate it into the mainline codes. But their incentives generally were to publish papers about super computer code like this, not necessarily to integrate it into the mainline codes. So often getting something like this in the codes was giving a talk and hoping a physicist would see it and need the extra performance for their needs.

I was not running codes that would utilize even a small fraction of roadrunner so my projects quickly moved to the GPU based test beds for the next gen computer so I didn't get to see these research coalesce. My understanding from people who stayed on roadrunner is that for the most part people didn't adopt too much fanciness for roadrunner as the rewrites were too deep and GPUs were on the horizon. There was a lot of vision about making generic heterogeneous compute frameworks but too my knowledge they didn't pan out and just writing against CUDA, OpenMP, and MPI won the day.

FLOPS are easy; programming is hard. Cell was the wrong direction and TRIPS/TFlex hasn't happened either.

Reply on news.ycombinator.com