E-Reverance

Opus 5.5 came out around a month ago, I'd give it some time before coming to that conclusion

terrible quality video my friend sent but the idea seems genuinely good and would like to hear if hner's have done something similar before

Schmidhuber : https://arxiv.org/abs/0812.4360

Or we can finally decouple intelligence/ability and from (social) worth

Why do all the "frontier" models still occasionally output images with the horrible ai texture tells? I'm surprised whoever made the front page let those ones in lol. My hypothesis is that it is either an unpatching or feature loss shortcut/artifact. Until then I have no hopes of ai generated images being viable for anything actually beautiful or creative

Ex. https://optim-images.krea.ai/https---s-krea-ai-k3-presets-20...

I loved to hear! Just ping me on twitter (same account I sent the link with)

I never said its unpleasant, what I'm meant math in its current (social) form / interface is "solved"

1) yes, but with more than channels than 3 (and no correspondence to color, its "pixel space" in the sense of position, not value)

1D grid example for clarity (obviously meant to be done in 2D though)

So embed -> [N dim, N dim, N dim ... , N dim] instead of embed -> [RGB, RGB, RGB ... , RGB]

2) / [followup 1)] Stride of 1, so we place a window at each pixel (so lots of overlapping)

Also I wouldn't phrase it as "come up with a better image", the point is to give less spatial decoding pressure to the patch tokens so that they can almost completely focus on feature learning instead. There is no reason to have the model learn spatial decoding when the structure prior of images is comically strong (especially compared to text), its a waste of training time and parameters

[followup 2)] I haven't measured but it was passable is all I can say (my experiments setup are horrendous right now lol)

(Also don't mind the phrasing, I just wanted to be 100% clear)

But who is going to be the one solving them? Just query an LLM no?

I know it goes a tiny a bit against the spirit of what y'all are doing, but applying a few layers of pixel-wise local attention (so 1x1 "patch", with 3x3 or 5x5 attention window, basically treating it as a dynamic conv) has worked way better than both linear and conv unpatching in my recent experiments.

Diagram for reference https://x.com/1rreverant/status/2107546198093730287 (In my most recent recent experiment I actually removed the MLP and just used a linear project on the pixel's hidden states)

> You don't have ADHD or autism, you just suffered from trauma.

Yeah that's exactly what I'm saying, but more specifically I wanted to relieve myself of blame through a label my parents could more easily empathize with

> A diagnosis of ADHD or autism in adulthood often leads to incredible relief and grief in processing all the moments of being blamed and shamed for what was assumed to be a lack of effort, laziness and/or defiance.

This is definitely a factor for me when I tried to get diagnosed (which failed lol), because my parents whom caused my trauma would obviously refuse to blame themselves for my emotionally stunted development

What AI if is the only means for having those in your lifetime

Scale with compute*

not explicit to reference scaling laws of training

She actively lies on twitter

https://x.com/ByrneHobart/status/2105368471198380131

Apart from their own data, a lot of people's anecdotal experience is that its pretty accurate, why would they need to banned for that?

I'm talking about bringing up a specific outdated details, not about simplifying it

I get that this is for explaining GPT-2, but I really hope laymen don't use it as an example of how modern models work (ex. absolute positional encoding is no longer used)

edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!

Jacob Tsimerman claims [1] we might have superhuman expositors by April, so then what?

[1] https://youtu.be/H7_d_sgui6o?t=4436 (timestamped url)

"a tiger is just made of atoms"

Given the size of your team, have y'all considered going into niche like making a DLSS 5 competitor

In my opinion its a great idea because you get reuse hidden states which allows you to do something akin to thinking/[online learning]. If you only evolve use input space outputs you're dealing with more decoding pressure.

Also related to what I was saying see these:

https://arxiv.org/pdf/2609.16372v1

https://arxiv.org/pdf/2609.01449v1 (this one is quite mindblowing cause they noise they the actual input space every time but it still works)

https://arxiv.org/pdf/2609.11801

Slightly off-topic but thoughts on this https://arxiv.org/pdf/2410.08159

There are better ways to apply SVD to images than the way its usually taught

https://www.youtube.com/watch?v=ZGwVlnuuzt4

> there will be actual consequences for this.

Not implausible but what exactly because sometimes when people throw that phrase its just aggrieved cope

I actually strongly disagree, I think this is the natural tendency of many losers and there is a disproportionate amount of them on the internet

The HN poster of this owns an extremely anti-social page btw https://boomerdeathwatch.com/ . Before he claims that "they deserved it", note how the page itself does nothing but make them look like a weirdo lol

Yeah a lot of people have failed to internalize a lot of the meta-lessons of current culture wars

I don't share my friend's music taste and record stores for sure don't have what I like (its usually small soundcloud accounts). Also the things I like about a song are not genre bound, its usually very subtle things that are unsearchable, hence even of the songs I like I usually only like ~30% of the song itself. With a personally tuned reward model I can strictly focus on amplifying the subtleties instead of searching by means of exhausting indirection

Because there is a shortage of songs I like. I loop the same songs for weeks until I can find a new song