sakras

I figured this was pretty obvious given that MLPs are universal function approximators. A giant MLP could achieve the same results as a transformer. The problem is the scale - we can’t train a big enough MLP. Transformers are a performance optimization, and that’s why they’re useful.

What really made HNSW click for me was when someone described it as a "skiplist graph". It really does seem to share a lot of the same properties and intuition as a simple skiplist, where descending levels corresponds to finer-grained navigation through the space.

A couple of other thoughts with this perspective: - if a drawing of a skiplist is 2D, HNSW seems to be a 3D generalization. It makes me wonder if you could apply an HNSW-like scheme for a traditional relational database index (maybe a multicolumn index?). - if a skiplist is a probabilistic alternative to a B+-tree, what's HNSW a probabilistic alternative to?

One thing that still mystifies me is how in the context of vector indexing, the HNSW seems to "bootstrap" itself by using itself to find the neighbors of a newly-inserted point. It's not clear to me why that doesn't diverge to some garbage results as the index grows.

I think OP wanted to construct “sand by induction”, but I also think you need more than one grain. If you have 9 grains of sand and add one snowflake, you now have something that’s 90% sand and is therefore 10 grains of sand. Take your 10 grains of sand and add another snowflake to create 11 grains of sand. Continue with each snowflake one by one, and you’ve inductively created a sand pile.

I really enjoyed playing that! Just did the frog and the eye :) Seems like a good thing to mindlessly do for fun

Basically, quantum gates are mathematically represented as multiplying your state by unitary matrices. Unitary matrices are ones where their inverse is their own conjugate transpose (i.e. XX*=I). Since each gate has an inverse, for any computation you can reverse it (in other words find a circuit that recovers the inputs from the outputs).

In practice this means that you sometimes have to keep around extra bits called Ancilla bits that just contain garbage, but are necessary for reversibility.

Would highly recommend Zero engine - it was used by DigiPen for their summer camps (which I attended)! The scripting language is a custom one called Zilch, but it should be straightforward to pick up if you know JavaScript.

https://github.com/zeroengineteam/ZeroDocs/blob/master/getti...

There are a lot of steps involved in making the chips - lithography is only one of them. You have to have the supply chains set up for massive amounts of silicon, you have to have a process for doping the silicon properly, you need quality control, you need to actually build a fab to house the lithography machines, I could go on.

Am I understanding correctly that they are proposing to add Rust to only userspace packages, not the kernel like in Linux? I guess the real limitation then is that the base system is built in tandem with the kernel, unlike in Linux.

They operate in somewhat different domains - numpy is to numerical data as pyarrow is to dataframe data. This means Arrow has strings, maps, nested types, etc.

Note that I compared to pyarrow and not Arrow - Arrow refers to the arrangement of bytes in memory and isn’t tied to Python or anything.

Well, the array of Umbra strings will almost certainly be aligned to 16-bytes, since each string view is a fixed 16 bytes. For the secondary buffer containing long strings, they would not have any particular alignment because they contain bytes, which are 1-byte aligned. Enforcing an alignment for each string would waste a lot of space, and also it's not clear what that alignment would be. For example, if you had a 37-byte side string, what would you align it to?

Wow I didn’t know the old bridge was such a nightmare. This also makes me wonder why this line only opened now, if it was voted on all the way back in 2008.

I do usually take the 542 now, and it works well-enough, but it only comes every 30 minutes which is annoying.

And then them making to along I-90 instead of 520 also doesn't make a whole lot of sense. I'd LOVE a fast line from the east side to U-district. But I don't understand the logic behind the current scheme.

Unfortunately not :)

On a given implementation of C, a pointer has some specific size (let's call it S). This means that we can address 2^S bytes of memory. Each byte has CHAR_BIT bits, so can be in 2^CHAR_BIT states, meaning we have 2^(S + CHAR_BIT) possible states memory can be in. Since there's a finite number of states, it's not a Turing machine.

TimescaleDB's $110M series C would like to have a word :)

Ah also the multiplication operation isn't commutative, since struct { int A; float B; } is different from struct { float A; int B; }. So maybe it's a "noncommutative semiring".

> Let’s pause for a moment to remember that we’re dealing with types. And the expression 1 / (1 - a) contains both a negative and a fractional type, neither of which have a meaning yet.

This makes me wonder, is there a ring of types? There's addition and multiplication. Division and subtraction aren't necessary to define a ring, so their absence isn't particularly surprising.

One question I always have about these sorts of translation layers is how they deal with the different warp sizes. I'd imagine a lot of CUDA code relies on 32-wide warps, while as far as I know AMD tends to have 64-wide warps. Is there some sort of emulation that needs to happen?

The demo fell a little flat because the "before parallel DOM" and "after parallel DOM" boxes were almost exactly in sync for me (Firefox, MacOS).

Essentially, unless you `fsync`, there's no guarantee that your data will be durably written to disk. This is because the operating system keeps data buffered in in-memory caches, so if the machine crashes you may lose some data. The `fsync` system call forces the data to be flushed from the in-memory OS cache to the disk. As far as I could tell, the Flush you use does not `fsync`.

Yep - it's an exact copy at the page level, so _everything_ is copied.

Regarding data anonymization, you can use the pg_anon extension on your branch. There's a blog post about it: https://neon.tech/blog/easily-anonymize-production-data-in-p...

Disclaimer: I work at Neon

Neon makes this very ergonomic in a cloud setting (actually it works somewhat similarly to how the article does it). You can create a copy-on-write version of your prod database (a branch) and use it for testing. There's also some automation we've built up for doing it from GitHub CI: https://neon.tech/docs/guides/neon-github-app

This was fun, I went into this thinking "Ha I already know this theorem" but came out learning something about zero knowledge proofs!

I started reading it around a year ago with the personal goal of doing every exercise. I'm not done yet (almost done with chapter 4). It has definitely taken a lot of my time, but it really has changed my thinking about a lot of language constructs I find in other languages.

For instance, the part about tagging objects with their type reshaped my thinking about static type systems in general. Static typing, for example in C, is essentially just moving the location of the type tag from existing at runtime within the struct, to the "analysis" phase of the compiler. It's a pretty simple idea, but it made the "degree of dynamism" tradeoff click in my head. All the information _has_ to exist somewhere, and the only question is where. And Scheme is just at an extreme where the _everything_ is stored dynamically.

Is it an innate property of humans that the curly-brace style is more natural? I wonder if in an alternate universe where Lisp took off as the browser language people would find it more natural instead. It seems like somewhat of a chicken-egg problem.

I wrote a blog post about how we can use ideas from the HPC community to gauge how fast a query should run, and how we should compare databases to each other. Happy to answer any questions

This is very cool! My first thought is: can this be applied to converting raster graphics to vector graphics (eg PNG to SVG)? Seems like a very similar problem, though probably much more computationally expensive.

It's honestly my most enjoyed game of the last few years. The feeling of boarding the Pelican right at the end of the mission never gets old.

My real showstopper with Hare is the lack of multithreading. In the modern world, we need to be making parallelism easier not harder!

The trouble is that AI assistants have seen all of the contrived algorithmic problems before. I once interviewed a candidate for whom Copilot spat the answer out as soon as he typed the function signature. Whether these problems are a good idea in the first place is a separate discussion, but as it stands the AI seems to just sidestep the whole thing.

This actually addresses a huge hole right now in the ecosystem. At the moment, Arrow treats things that are logically the same as different types, for example a dictionary-encoded utf8 is different from a utf8. Really there needs to be a distinction between logical and physical types, and it’s a big source of rough edges in things like Acero. Hacking on my own query engine on the side, I’ve put some thought into how to group physical into logical types in Arrow.

I am however worried that the place to define the logical type system might be inside Arrow instead of Arrow consumers. If everyone has their own logical type system, we’re just going to end up with the same incompatibilities that Arrow was trying to solve.