txus

At Manifest AI we have just released our open-source CUDA kernels to implement Symmetric Power Transformers, as described in our paper from back in August:

https://manifestai.com/articles/symmetric-power-transformers...

Since this is a variant of a linear attention, you get linear cost when training (as opposed to quadratic in regular attention), and constant when doing inference. This is especially attractive for longer contexts!

Have a look and play with it -- and of course contributions are very welcome! It's an early alpha!

Experimenting is good. I do it all the time. Just take a look at my repos on Github. Just a few of them are conceptually (and arguably) "new", while the others are just exercises to learn about new things. Experimenting about things, even about the basic ones, helps me a lot in my learning process. At a personal level.

But when you watch the bigger picture, with companies actually building software for clients that trust them, then choosing technology is a delicate, non-trivial matter. It's important, and I mean in actual dollars (or euros). There is where the expertise of older developers, people who have founded, sold or shut down profitable and unprofitable companies, who have worked in a gazilion of projects and made a lot of expensive mistakes, there is where that expertise comes to play. They can enlighten the sometimes childish, faction-like debates about technology that keep repeating themselves over and over. That can free us to think about new problems and new things, based on past experience. That's how science advances, and that's how software development advances too, in my opinion.

I chose that word on purpose because of both its semantic and emotional power. Besides, I'm pretty happy that all the critics have been focused on either using the term "cancer" or the swearing at the end, so I guess my points are valid to almost everyone, and you just didn't like the way I expressed them. That sound ok.

The 80% I was talking about.

My article has nothing to do with computer science, but with a human community built around a technology.

This is not (only) a rational argument, but a call to action, and that requires an appeal to emotion.

I'm really proud that so far no one has argued about any point I make in the article, but rather about the "cancer" reference and the swearing at the end. I guess that means I happened to write a pretty damn good article.

I am currently organizing the Barcelona Ruby Conference, and have attended many other conferences in Europe. I know a lot of people put much effort in doing things right. But that's not the majority of the community, unfortunately. I also agree that the Pareto principle (and many other points raised in my post) are a universal problem, not just Ruby's.

I did provide constructive advice. If you read the post thoroughly, you can see a call to improve our learning manners, stay away from hot trends, and even a simple pattern you can use in your day to day to compare technologies, or to understand better some debates (like threads-processes-reactors).

Heh, I really liked the analogy with Postel's law, and I guess you're right. The main criticising I've received about the blog post has been about the swearing, so I guess I could skip that next time.

I don't think that's Rails, or Ruby or whatever paritcular tech we're discussing. I think that's a common trait in human communities, and I'm worried about the one I feel mine, but I'm sure everyone can see a part of their community reflected on my post.

If you can't see the point because of the "harsh language" then I apologize, I might publish a politically correct version for you then :)

I didn't need to grow old to watch relatives and friends fighting cancer. I know how serious it is, and I really admire that you're working in that particular field. That said, I used the metaphor on purpose, because of its power, because I think it's a perfectly valid analogy.

Just like a tattoo, it shouldn't distract critical thinkers when judging the contents and the point of my argument.

I corrected a couple of those upon indication from readers. I hope there aren't any more :)

I didn't mean to give a bad impression of Ruby. I think (and many people have pointed out) that the problems I outline are valid for many developer communities, if not all of them. In the end, those communities are human, and share a good amount of our defects.

But swearing is a personal signature. (Anyway sorry if I offended anyone.)

I insist, this is fixed in version v0.0.5, thanks to your very warning.

On the CSRF concern, which is totally valid, I've pushed a patch. From version 0.0.5 it uses a token to prevent this kind of attacks.

https://github.com/codegram/rack-webconsole/commit/d5060d0e8...

You're right, that's a risk. I've opened an issue to implement a simple pseudorandom token to protect AJAX requests. What do you think? https://github.com/codegram/rack-webconsole/issues/4

It is meant for development environments only. Nobody would risk putting a Ruby console in production :)

In fact, when using it with Rails, it is loaded only in development environment. With other frameworks you should take care of what middlewares you use in which environment.