ezyang

IMO, the big problem with Aider is that it's not agentic. This is good because it means costs are down, but most of the edit-test-fix loop magic in coding agents comes from the agent loop.

There's a few coding MCPs out there. I have also written one (codemcp) and the pitch for mine is that it DOESN'T provide a bash tool by default and checkpoints your filesystem edits every change in Git, so that it's all about feeling comfortable with letting the agent run to completion and then only inspect the final result. The oldest one in the space, I think, is wcgw.

The llama source code in the original repo has been updated for llama 2: https://github.com/facebookresearch/llama

If you want to try this in Python, you can use https://github.com/ezyang/expecttest which I wrote to do expect tests in PyTorch.

It's especially bad for rr, since it doesn't otherwise have any reason to talk to the Internet (I see people mentioning Firefox telemetry, but you know, Firefox is a browser, you expect it to talk to the net).

The best I can think of is to incentivize it other ways; e.g., telemetry only for bug reporting, or a "you ping us, we give you a nice hat" or something.

Doesn't help when it can take several days to land! (e.g., waiting for CI.) But I will admit this is a problem that is somewhat specific to the nature of our project :)

I definitely miss hg’s affordances like hg next/prev and restack. It doesn’t look like, though, that git-ps lets you submit multiple patches for review, this was the raison d’etre for ghstack

When you amend a commit and resubmit it to the PR, ghstack translates into pushing a new commit (not force pushing) to the PR. So yesish

While working on PyTorch, I also wrote an equivalent tool (funnily named nearly the same thing) for doing stack diffs (https://github.com/ezyang/ghstack/), which most of our team uses for more complicated PRs. The UX for working on commits is a bit different than this tool though; instead of pushing branches individually, you just run "ghstack" on a stack of commits and it will create a PR per commit in the chain (amending each commit so that its commit message records what PR it corresponds to). To update the PRs, just amend or interactive rebase the original commits. Personally, I find this a lot easier to handle than finagling tons of branches.

In an old draft of my post I called this out more explicitly. There are two things worth thinking about.

First, if you think about actually doing this in a traditional programming language, you can quickly see that this is going to result in a lot of boilerplate. REST APIs make this a lot easier, because it's easier to say something like, "change the meaning of this command, but have everything else stay the same." Functions and classes are not setup for this kind of metaprogramming.

Second, sometimes you are breaking BC because the old behavior is extremely inconvenient to provide under a new internal implementation. This is pernicious, because "I didn't like the old function name" is not a good reason to break BC, but "I can't rewrite the internals and support the old functionality" is a good reason to break BC. So you're in a situation where your mechanism for maintaining old versions breaks down /precisely/ when you would have gotten the most utility from breaking BC.

CuPy itself is just a framework, and you could slot TC in as a thing that generates operators for it. CuPy also famously has support for inline CUDA kernels; the equivalent TC kernels are shorter and autotunable.

With cuDNN, some expert had to sit down and write an optimized kernel for your ops. With TC, the idea is you write the simple, tensor comprehension for your op, and then throw the autotuner at it, and get something that approaches or even beats the hand-tuned example. The diagrams in the paper give a sense for some of these situations.

Apparently they are out of stock for the i7 motherboards, so you'll have to keep an eye out if you want one of those.

You don't get exhaustiveness checking, which is the killer feature of variants.

PyTorch/ONNX dev here. ONNX is a proto2 format https://github.com/onnx/onnx/blob/master/onnx/onnx.proto -- we definitely wanted to make it easy for people to parse and load ONNX exported graphs :)

Scala's type inference is not complete; it is marketed as a way to "omit certain type annotations". One of the reasons why Hindley-Milner is so influential is because you really don't need any type annotations: the inference algorithm always works.

As for DOT https://www.cs.purdue.edu/homes/rompf/papers/amin-wf16.pdf DOT is an explicitly typed calculus (similar to System F), for which no attempts at type inference are made. So... they're not really comparable at all.

Unfortunately, it is never as easy as "just" putting in a new type system to a preexisting compiler. In the case of ML and Haskell in particular, we are kind of crazy about GADTs, and GADTs and subtyping are known to interact in complex ways. See for example http://gallium.inria.fr/~remy/gadts/Scherer-Remy:gadts-subty... (PDF)

It makes some things easier, but you give up principality: without things like intersection types, you end up with programs for which there isn't a "best" type to ascribe them. You can see this in the parent's example: without subtyping constraints or intersection types, there's no way to usefully express the subtyping that is possible in this example. It's generally considered a bad thing if your language infers a type which is incompatible with another type you could have manually written, e.g., (float -> int) -> float -> int, that is also a valid type for the expression. We like there to be a best choice for type inference to pick, but if the type language isn't expressive enough, there won't be one!

It's a little difficult to explain why subtyping is so difficult. One answer is that Hindley-Milner inference is actually really simple; simple enough that I can ask my undergraduates to implement in a week long programming assignment. But how to add subtyping? Now there's a difficult question: one you could write papers about. Along the way, there are a number of desirable properties you want to achieve, like decidability, existence of principal types (for any expression you write, there is a "best" type you can give it), compactness of inferred types, support for records and functions... and it's been very difficult to get all of them. This thesis does it.

The thesis is about designing a type system that supports both type inference and subtyping. If you program Java but hate writing explicit type annotations all day, the thesis offers the tantalizing possibility of having a type system that supports all the subtyping you want, but at the same time has as convenient and predictable type inference as ML or Haskell. Conventional wisdom is that this is hard; in this respect, the thesis proves a very surprising result.

The lesson of "algebra before syntax" is one that I have seen before, and it is very gratifying to see it pay off such dividends here.

There is also discussion on LtU here: http://lambda-the-ultimate.org/node/5393

There are two responses here. The first is that in any language which has any sort of threading model at all, sequential code in multiple threads obviates the need for callbacks. If you block, fine; it's just a single thread, it's not obstructing the handling of other threads. Why don't Python and JavaScript have multithreading? Well, because programming with unrestricted concurrency and mutable state is really difficult. But there are ways to solve this problem, Haskell's purity being one among several.

The second is that do-notation is distinct from the IO monad. Even if Haskell didn't have green threads in the runtime, I could still write an async/callback library that looked just as natural as sequential code. Why? It has nothing to do with the IO monad: it has to do with the fact that "do x <- e; m" desugars to, in JavaScript notation, bind(e, function(x) { m }); it's been "callbackified automatically".

I like to tell people that the killer app for Haskell is writing IO bound, asynchronous code. The secret weapon is do-notation, which lets you write code as if it were sequential, but have it desugar into what is (essentially) a series of chained callbacks.

I like to point at Facebook's use of Haskell as a good example of being successful in this space http://community.haskell.org/~simonmar/papers/haxl-icfp14.pd... It would be disingenuous to suggest that Haskell is good in all situations, but if there was one place where it should be used, this it.

Replace every occurrence of "undefined" with "non-terminating program" and the point still stands.

Here is a transcript of the talk: https://github.com/ezyang/cusec2012-victor/blob/master/trans...

In my observation, the problem is recompilation avoidance. If you miss it the first time around (maybe you're just trying to get the thing to run) they won't come back unless you do a full rebuild. With a little care you can build a build system that will print them out every time you run, but it takes some care.

Note that R7RS does not really have any sort of sophisticated phase system for macros.

I didn't make it past the fizzbuzz example. But I wouldn't call the solution with cycle all that clever. Here's a better one: https://themonadreader.files.wordpress.com/2014/04/fizzbuzz.... the punchline reproduced below:

  fizzbuzz' :: Int -> String
  fizzbuzz' n = (test 3 "fizz" . test 5 "buzz") id (show n)
    where
     test d s x | n `mod` d == 0 = const (s ++ x "")
                | otherwise = x

  fizzbuzz :: [String]
  fizzbuzz = map fizzbuzz' [1..]

The article is very interesting and worth a read. In particular, this implementation doesn't come out of "thin air": it's been golfed from an implementation that defines a simple domain specific language to solve FizzBuzz. And if you understand your problem well enough, DSLs are an extreme force multiplier.

Credit where credit is due: nix-local-build is mostly Duncan Coutts work (I just wrote the announcement, and contributed some patches!)

Well, if you are a user using Stack, packages which are not in the LTS have to be added manually. So in a Stack universe, treating LTS as the universe of packages is not unreasonable.

I don't think either Stack users or Cabal users think Hackage should go away; in particular if you're a library author, the way into the LTS is to publish something to Hackage and then ask Stackage to take it.

As for Backpack, it doesn't really have much to do with the philosophy difference between Cabal and Stack. But I have, as necessary refactoring for implementing Backpack, been working on improving cabal-install, including the Nix-local builds feature which is supposed to make Cabal's usability story much better.