Hacker News Reader

Comment by imranq | Hacker News Reader

I've found einsum to be amazing at consolidating my code into something more readable, particularly for implementing architectures from scratch.

Here's a good video that explains why its so good: https://www.youtube.com/watch?v=pkVwUVEHmfI

Also check out Lucid Rains Github, who uses it extensively to build transformer architectures from scratch: https://github.com/lucidrains \

* Example: https://github.com/lucidrains/alphafold2/blob/d59cb1ea536bc5...

View on HN

amazing at consolidating my code into something more readable

Readable if you're already familiar with einsum notation. Otherwise there's learning curve. An alternative to einsum is using multiple dot product and reshape ops, hopefully with each one having a comment - this would be a lot more readable imo.

I really don't find that einsum is less intuitive than reshape. Maybe they both take about the same time to learn the first time. And that's less time than it would take me to struggle through even a single function using reshape ops.

How many DL people are familiar with dot products and reshapes, and how many are familiar with einsum?

I really couldn't say.

It does occur to me, though, that you are probably talking about the elementwise product, not the dot product. The dot product includes a reduce_sum step and outputs a scalar.

But then to find some mistake you would have to corroborate n * 2 pieces of information, which is quite a bit more, before getting into the issue of whether or not the sequence of operations actually makes sense on top of all that.

So let’s say there’s a bug, and an einsum statement is a suspect. Assume I’m not familiar with einsum notation, but I know dot products and reshapes. How do I debug it?

Guess you can’t, then. Personally I’m a bigger fan of einops since the variable/index names can be a lot more self-documenting.

Reply on news.ycombinator.com