Hacker News Reader

#1) Mill cache is relatively conventional (except with 9-bit bytes). Everybody uses the EDA tools to create caches, so everybody will get similar power numbers. However, there's a lot more to the hierarchy power budget than the caches: read buffers, write buffers, pin drivers, etc. The Mill has neither read nor write buffering, and backless lines don't drive pins. We won't have good numbers until we have gate-level sims of the hierarchy, so for now all there is to go on is skill and expertise. We're not worried; YMMV.

#2) The bypass is similar for the FU-to-FU paths, but an OOO has to also feed from the renames that the Mill doesn't have.

#3) The OS port is a largely solved problem: we expect to use the L4 microkernel as a base and the existing L4-based Linux etc. implementations on that. https://en.wikipedia.org/wiki/L4_microkernel_family. Porting L4 to the Mill is pretty easy; we designed it that way :-)

As I understand it, Mill has no privilege mode so I don't see why you need even L4! Providing a Unix compatible API (all except fork()) is basically just building a modular emulator. Mostly grunge work once you work out the access rights matrix :-)

How far along are you on the OS work? Have you booted anything on the simulators?

One tricky issue in porting an existing OS to the Mill is their AS/400-style portability strategy (as outlined by Godard in the comments): binaries are distributed as "Mill IR", and compiled to local binary, which gets cached for the next execution. The problem is where to put this cache, and how the OS deals with it.

Say, for example, you put the cache in the file system. OK, who has write permission, and can the "cached local binary" be written on first execution by a user who doesn't have write permission on the binary being "localized"? (Or could the translator be run on "apt-get install ..." --- and if so, who trains apt to do that?) And so forth.

Putting the cache someplace that is hidden from the "normal" OS is possible, but that has problems, too. At the very least, you'd need to figure out what to do if the hidden whatever-it-is runs out of space. (And how doing I/O to it would interfere with other OS-level performance optimizations, like scheduling of disk seeks.)

IBM could finesse these problems on the AS/400 because they controlled everything about it, hardware to OS to UI. And there are niches with high performance requirements that could live with a nonstandard OS. But for general-purpose computing, it could get awkward. (Perhaps awkward enough to consider TransMeta's strategy of doing JIT translation to actual machine instructions, which let them keep the "real instruction" cache entirely in dedicated RAM --- though that has problems too.)

I sort of assumed that they would just use fat binaries, and store the cached, translated version in the same file as the intermediate code.

Exactly. Most will specialize at install-time. Code without an install step will specialize at first execution; the IR is designed for specialize speed.

The specializer can also be run free-standing as well, to create ROMS and where installations want to ensure distribution uniformity.

Specialization at install time is, in a lot of ways, the most reasonable answer. (It completely sidesteps the permissioning issues involved in write permission on a global cache at first execution, for example.)

However, hacking an existing package manager to do this job may not be entirely trivial --- and, in typical server environments these days, there will often be more than one package manager to hack. Language-based package managers like Rubygems and npm, for example, build binary extensions for Ruby and Node/Javascript as part of their job. Updating any of them to deal with an additional "specialization" step may not be much of a big deal --- but dealing with all of them might be. (Particularly if you have sysadmins that like to, say, run something like debsums to verify that the installed files are exactly the same as the ones in the package --- which might fail on a "fattened" binary.)

If you've already had coffee with the maintainers of, say, apt (or rpm), npm (or Rubygems, or PIP), and a few others, hashed out the issues, and have worked out that it's no big deal, that's great! But if not --- management of servers these days is complicated in ways that you'd never guess from just looking at the hardware, and a few of those coffee chats might be enlightening.

I don't think it's that difficult for package managers to deal with. All of them have a way to hook post-install actions to run an arbitrary script. You might wind up with another step added to the scripts used to produce the package which automatically generates this hook, and the checksums might be slightly difficult (but then storing specialised binaries separately isn't hard).

I guess the biggest infrastructure change might be running the specialisation on a dedicated machine and hosting your own packages. This way you could also checksum and sign the specialised binaries as well.

I'm not sure what you're suggesting here. Adding the "specialization" hooks to the post-install script in every single package is genuinely hard --- Debian, for example, has literally thousands of packages, maintained by hundreds of packagers, some of which squirrel .so files and the like in all sorts of places you might not expect them without detailed knowledge of the package in question. And that's before you even consider software installed on Debian boxes by other systems entirely, like npm or Rubygems.

Running the specialization on build machines is easier, if you have compilers there that produce binaries directly, and not IR. But that's exactly what the Mill crew is trying to avoid by producing the IR! (Though it may actually be a better fit to an infrastructure which is already set up to produce distinct binaries for different processor architectures (x86, x86-64, ARM, and several others), and which is not set up to expect the "specialization" hook as a necessary post-install step which each package must separately provide for...)

In one of their talks they also mention saving the branch prediction table _after_ execution. Clearly that can't happen at install time.

The compiler makes the first branch prediction table. (It's actually smaller than a branch prediction table, because it only needs one exit point for each entry point.) The table can be updated (in memory) during execution if needed. The on-disk version can be modified for optimization purposes for later runs, but the mechanism for that would obviously be software dependent and not reside on chip.

It seems like the specializer contains a lot of a compiler back end. (I am not a compiler guy, so my understanding is probably wrong.)

Different family members have different functional unit timings and different numbers and orders of functional units. ("orders" -- the order they drop results onto the belt. What's the right term?) Therefore the specializer has to schedule instructions.

Different family members have different belt lengths. Therefore the specializer has to insert belt spills, which seems to be analogous to register allocation/spills.

Different family members have different encodings, so the specializer has to determine the size of each basic block and link-edit them together. (Probably fast, just a lot of bookkeeping.)

It looks like there's a lot of work between the Mill IR and the actual machine code.

It transforms one language into another, so by definition it's a compiler. But what is so scary about that? I would be willing to bet that most people already run JIT'd code every day, in the form of Java, C#, Javascript, etc.

My question is, how slow is the specializer? It is portrayed as very fast, but it looks like it's doing a significant fraction of a real compiler's work.

Then again, if it's mostly used at program installation time, who cares?

Assuming their translation step is as cheap as they claim (single pass rewrite/substitute missing hardware for software macros), it's conceivable that there isn't much value in persistently caching the result.

In that case, everything could be contained to a single ld.so patch, or (doubtfully) a modification to the kernel ELF loader

Finally, and although it is less common now, in prior days Linux already had a post-install processing step for binaries on certain distributions - prelink(8) ( https://en.wikipedia.org/wiki/Prelink )

Also Mac OSX "prebinding" and (in later releases) dyld cacheing. Though just skipping the cache entirely is certainly the quickest way to get something running --- and quite a few plausible server workloads will pay the penalty mainly at startup, when it might not matter so much.

(But yeah, keep it in user space, if only to make it easier to debug!)

Sounds like Java and .Net Bytecode. I don't deal with Java much, but in .Net world IL is jitted on demand into memory and thrown away when the process is recycled unless you do ngen, which is difficult to do in many situations (web applications).

I think many underestimate the overhead involved in compiling IL to native including MS themselves. There are user perceivable delays in application startup time in larger code bases that lead to a worse user experience. Why they don't cache jitted code to disk after 5-6 version of .Net is beyond me, even Mono has a AOT compiler.

I am starting to appreciate the AOT approach of "native" code (C,C++,GO), do as much as possible one time, at compile time. Don't make the user wait because you want to distribute a single portable binary.

It's often recommended to run ngen as a part of the user install process for larger applications.

Sometimes ngen produces worse code than the JIT does since it has no knowledge of execution state. Compile your assemblies which can be easily optimized, but you may actually be better off losing a couple of cycles waiting for the JIT to compile a method on first call in certain cases.

I would like it very much if alternative to '70s era operating systems were to flourish on newer hardware platforms. If Mill hardware backs up the performance claims, it would open an interesting door that's been shut to non Multics-likes for too long.

>>All of which is to say, I think the 10x they're talking about is unrealistic.

I think that is putting it mildly and is a little strange to somehow claim in the first place. That can't possibly be true, their isn't room for a 10X increase in optimization on an Intel core chip and would be impossible to reach based on memory bandwidth and the amount of execution resources available on a chip alone. The ideas they have concretely put forth simply don't work or don't really provide a performance increase.

Take their virtual memoryless implementation. Getting rid of virtual memory doesn't buy you a whole lot especially when you need to add in a protection mechanism that looks a lot like a TLB in the first place(and must have the same general properties to provide protection, you just gain very marginal lookup costs).

If you do the math, this can't add more than 1-2% in performance in common application software at the cost of making every modern operating system unusable and increasing memory consumption(embedded systems anyone?). If getting rid of virtual memory was so great, why didn't someone do it in every other preceding clean room architecture? My answer: It isn't.

Or consider how they want to do branch prediction: add a separate ISA to do static branch prediction that is added by the compiler and loaded asynchronously by another cpu component and then supplied to the main cpu.

First of all, this doesn't work. The CPU can't have performance critical data pushed to it by another component. There is a reason the Branch prediction table and branch target buffers are small and focused and able to be accessed quickly. Secondly, static branch prediction is awful. You simply must be able to modify branch prediction data as the CPU executes to provide optimum performance. So it seems they want to be more power hungry, more complex, and have less performance than a mainstream CPU when it comes to branch prediction.

It is possible I have misinterpreted some elements of this scheme but basic design decisions like putting branch prediction into a separate component of the CPU simple don't make sense at all from a chip layout perspective.

Finally, I'm not really sure where the performance is supposed to come from with the 'belt' in the first place. Data dependency is incredibly complex in a modern pipelined cpu and while it is possible to reduce the cost by precompiling software for an optimized CPU the benefits are all very low level and really don't extend beyond reduced power consumption(assuming compilation cost can be amortized). At some point, to get more instruction level parallelism you simply have to bite the bullet and do dynamic out of order scheduling in the CPU to extract more performance. This has a well defined cost and an upper level limitation on how much total parallelism a CPU can extract from an instruction stream. Think of it another way: a static compiler has less information than a running CPU so one can't expect it to be able to extract more parallelism than the CPU itself.

The Instruction Encoding talk http://millcomputing.com/topic/instruction-encoding/ was the first talk, so explained these numbers in the first few slides.

DSPs are massively faster than your Out-of-order Superscalar Monster, just ... not on general purpose code.

The Mill is a DSP-like architecture with secret sauce so it can overcome the gotchas and go DSP-fast on general purpose code.

Right, and I'm saying that the data dependency limitations alone in general purpose code pose an upper limit on how much instruction level parallelism can be extracted from the instruction stream. You simply can't make use of execution resources if they need to wait on the results of another one.

EDIT: I'm not even considering pipelining, latency or transferring data among execution units, just assume every instruction completes in one cycle and makes its result available instantly.

True, if you go hopping all over main memory, we'll go at main memory speed just like everybody else.

Luckily there is a noticeable performance improvement between an i3 and an i7 precisely because normal app code doesn't go hopping long chains across main memory. This is the code we speed up.

An order of magnitude improvement is saying that - hand waving - you can have a hot monster at 10x OoO SS performance or a cool little chip equiv to the OoO SS but low power.

However, the faster you crunch the bits between main memory stalls, the more dominating those stalls become. Its diminishing returns. And hot is not good.

So we talk more about sweet spots like 3x performance and 3x less power, and temper them appropriately.

The numbers are based on sim and experienced estimates. Mill is faster because we can time x86 code and we can sim Mill code and we can compare them.

Am typing on a phone, apologises if brief.

I agree, you are always as slow as your critical path of executions. General purpose code, I imagine, often has very long critical path that no amount of parallelism will improve.

What I want to know is just how silent the pipelines of this design would be under multi-threaded general purpose code.

I think that the advantage the mill gives here is more room to compute things speculatively on the critical path. You can happily load from an invalid address and then realise that was wrong later without causing an exception in the CPU; same with FP arith etc. This allows you to parallise more sequential code than say x86. I think the other related advantage here is that they've done as much as they can to remove any idea of core global state (comparison flags and so on) so that more operartions can be run in parallel; do several comparisona at once, and then process all the results together.

You then assume your own conclusion when you ignore pipelining. If instructions are executed in sequence indian-file then necessarily none will be faster than any other.

The traditional rule-of-thumb is that programs have an ILP of two. The Execution talk (millcomputing.com/docs/execution) explains how the Mill turns that into an ILP of six. Then for the 80% or so of code that is in loops, pipelining has unbounded ILP - there will be a talk on pipelines upcoming.

>>All of which is to say, I think the 10x they're talking about is unrealistic.

Bear in mind they are claiming 10x improvement in MIPS/Watt, not MIPS. So I guess what they are aiming at is a 13W chip with i7 performance.

Even if they managed a 65W i7 they would be on a winner.

> The differences between how they and LLVM handle pointer is causing them trouble

Could you please elaborate? What is the difference in the way the Mill and LLVM handle pointers?

Ivan has said the problem is that the LLVM treats all pointers as integers.

Here is a longer list:

LLVM assumes a register-based target

LLVM assumes that pointers are integers

and it can only vectorize counting loops whereas the Mill does while-loops too.