YmiYugy

There have been countless times where I wanted to use some new fangled platform API, but couldn’t because it was buggy/unsupported in one of the browsers or didn’t quite match my use case. It does t help that the platform abandoned some of its efforts to introduce better primitives in favor of higher level APIs.

Counterpoint on Safari. Apple still ties browser updates to OS updates and there are a lot of old iPhones in circulation. Safari 16 is still a reasonable target and it’s missing a lot of modern features and has a lot of bugs to work around.

I had that a variation of that experience. This was mostly in the pre-AI days. Homework assignments were mandatory but only counted towards exam admission not the final grade. Much better for motivation.

Teaching and evaluation are of course not the same thing, but evaluation is an important part of teaching. There is a mismatch between students’ overall desire to learn things and the moment-to-moment experience that learning is hard.

Even devoid of all the economic and social pressures that grades impose students would compromise their learning experience. Ever regretted looking up the solution to a puzzle in a video game or sneaked a peak at the crossword puzzle solutions?

One of the hardest part of teaching is keeping students from self-sabotage. Evaluation helps with that.

Beyond that evaluation is of course also useful just as feedback.

What does it mean though for a GUI to be keyboard-driven? The obvious way is that every action simply gets a shortcut assigned. My counter would be that that is not really keyboard-driven, but merely keyboard-compatible.There is the issue of discoverability. The best practice right now seems to display the shortcuts of buttons in tooltips, menu items, or when pressing a different shortcut. I’d content that buttons are a fundamental mismatch with keyboards. A keyboard driven UI shouldn’t have buttons. The issue is that genuinely keyboard driven UIs like CLI or TUI suffer terrible discoverability that being the reason that mouse driven UI exists in the first place. So can we have a keyboard-driven UI that is as intuitive as clicking with a mouse?

The thing is that mobile browsers fill fight you tooth and nail against providing a good experience to your users. There is the obvious stuff like lots of API not being available, but the bigger issue to me is how many common convenient UI patterns that users accustomed to native apps expect are really painful if not impossible to provide, e.g. there is no reliable way to react to the keyboard opening, or how scrolling will expand and shrink the browser UI. CSS Houdini being abandoned also makes it really hard to properly do really custom UI. I really wish there was platform with the distribution and security model of the web that was designed for apps from the ground up.

I think the UX around pull requests and code reviews are extremely difficult to get right and to this day I haven't seen a platform that I'm really happy with. There are bits and pieces in some places but none really bring stuff together. Here a small list of things I'd like to see: - diffs should be language aware and highlight semantic differences - I should have a working LSP on both the old and the new version simultaneously and have decent full editor code navigation features and git history inspection available - I want ergonomic ways to see diffs between different versions of commits - I would like to be able to comment on anything (commit messages, inter diffs, PR description, responses, etc.) - I want a better integration of LLM reviews than having them dump their findings in an a big PR comment

I think there is plenty or room to innovate on just the basics of PR and codereview.

I'm confused. I thought Mythos 5 and Fable 5 were exactly the same model just with a different security layer in front of it. Could they mean the Mythos 5 Preview?

I find the title highly misleading. The actual claim is a 40% lower chance (1 in 100 as opposed to 1 in 60) for taxi and ambulance drivers when compared to the average. That’s a big drop but the title implies that Alzheimer’s is rare in taxi drivers specifically and I don’t think the difference warrants that claim whatsoever.

IMHO the output is bad enough that I can't imagine a use case for illustrations of this kind.

I don't think it's a bad way to benchmark new models, I just find it concerning that the author implies that "pelican on a bicycle" has been exhausted. At the risk of making overly broad, unfalsifiable claims I think multi-year exposure to AI content has dramatically raised our expectations for speed and volume but lowered them for quality. We see a very janky pelican and declare the problem solved.

I remain skeptical of that line of reasoning.

1. There is quite the mania right now and security layers are definitely overzealous. I would expect that to get better with some more time, so models will perform security analysis and reviews but refuse to write exploits.

2. So the most important targets like browsers and co. are getting unrestricted access to proprietary models regardless. Yeah, for the mid-level targets, open-weight models could definitely be a huge help. What I'm most concerned about though, are the systems that no one will bother defending with any model. Like imagine your local police department getting hacked because a researcher asked a model for a report and it couldn't find the information publicly.

3. We do have a prominent case of a closed model escaping it's sandbox and going rogue. I would still expect this to be a bigger issue with open-weight models eventually. The security layer might have holes, but that's still better than not having it.

Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of open-weight models in the US that it will reduce cases of accidents like the recent attack on Hugging Face. More crucially though, the US government can do little to enforce their testing requirements. The nature of open-weight models makes it virtually impossible to clear the same bar for security as models served via an API. Open-weight model makers couldn't comply if they wanted to. The US government can restrict access with IP blocks and limit inference capacity with export controls, but these measures are not effective in deterring malicious actors.

I think it’s death by a thousand cuts. I think internet connections are the least of the issue. Plenty of people have gaming PCs wired to 1Gb/s fiber. It may even be better because users can be close together.

Where I see bigger issues is: 1. Economics. GeForce Now for a 5080 is 20ct/h. At retail prices you may get close in electricity.

2. Others want to game when I want to game. What good is offering my PC up at 5am? Who would want to use the service if you can’t reliably get access when you want it. Not to speak of the issue of getting kicked out when the owner wants their PC back.

3. The whole: do you trust someone else’s computer/do you trust others on your computer. Users don’t want their login details stolen or get banned for using a cheaters PC and providers don’t want you to cheat or do other nasty stuff. You can’t really do VMs because consumer GPUs don’t really support virtualization so dual booting

4. You need to dedicate a ton of disk space. Because of 3 you can’t really share and now you have to deal with some system to have the relevant games available.

I don’t think there is a ton of spare capacity. But even if there was the machines are designed for AI not gaming, e.g. low CPU clocks, the GPUs don’t have RT cores or video encoders, etc. Stadia had custom hardware, GeForce Now uses workstation GPUs, xCloud has custom servers. Gaming also isn’t a good use for spare capacity. Demand is very spiky and people don’t have a lot of time flexibility.

Seeing a comparison of different Wayland compositors could be very interesting.

But SOTA models used liberally at API pricing is a lot more than $10/hour. You can probably burn $100+/hour with just a single agent, and probably thousands when running agents programmatically, e.g. workflows.

I’m writing a lot of React code and find that the cheaper models are pretty terrible. Maybe I’m holding it wrong but the experience that the cheaper model is usually enough just track with my experience. Worse, I find predicting the difficulty of tasks exceedingly difficult. More often than not using the initially cheaper models requires me to reroll with a more expensive one or waste a lot of times and tokens cleaning up the subpar results. With OpenAI and Anthropic still subsiding tokens, not using the best models still seems like a tough ask.

Makes me wonder, as people grow to trust the AI more and more, not reading the code and barely skimming the implementation plans and simply rerolling if something doesn't work, will the value of these chats erode? Thinking back 1-1.5 years I was closely monitoring what these agents did and steering them quite aggressively. These days not so much. Where will RL signals come from when it approaches humans capabilities ever closer? How well does self play work for coding work? What about multistep tasks where it isn't just about being good at a single task, but evolving a codebase over time in the face of changing requirements?

Getting users to open a message isn’t a terribly high bar. As a user I would not find it acceptable if needed to be careful with which message I open. We tried putting the responsibility on the user with email attachments and I think it’s fair to say it’s been a disaster. Malicious attachments are probably the most important distribution vector for malware.

AI or not, it’s always been reasonable common that a bunch of related vulnerabilities get discovered after shortly after the original one.

So according to the benchmarks somewhere in between Opus 4.7 and Mythos

Because judging failure is itself a complex task requiring a potentially expensive model.

Of course you don't NEED the better models, but figuring out what model you need can waste a lot of time and effort. Even when a cheap model is capable of a task it needs a lot more guidance than a more expensive one. They are also less reliable. You can waste a lot of time cleaning up after them. Judging whether something is good enough is hard work and rerolling with a more expensive model is painful. Judging the difficulty of a task ahead of time is very hard. Judging how good a model is for a given task even harder, especially when models and harnesses keep changing all the time. The real productivity boost LLMs provide is already modest and when you start tinkering with models it can easily evaporate.

1. They heavily subsidized their plans vs. paying for API. 2. They allowed me to use the subscription in every tool I wanted. 3. It covered both Anthropic and OpenAI.

That seems not possible.

Yes, please! But browsers need to make it easier for things to exist in user space. That means reviving CSS Houdini, particularly reviving the animation and layout worklets. (It got abandoned because browser vendors (Chrome in particular) found them too difficult to implement. They would need to rearchitect a good chunk of their rendering pipeline. Instead we got a bunch of very limited but easier to implement features like scroll animation timelines)

I tried to use <dialog> and found it to be a pain. I wanted to close it when clicking outside, but Safari doesn't support closedBy. Some Safari versions on iOS broke when trying to style my backdrop with tailwind. The tailwind CSS reset didn't include <dialog>. I get the allure of just using a position: fixed;

I think cutting the legacy and doing things with the benefit of hindsight would make for a much nicer platform. For example something like CSS Houdini (making CSS extensible) might have actually happened if browsers where designed for it and would make Web Apps a lot nicer.

1. I think it's pretty clear that the frameworks sold themselves, because it's possible to see the DX benefits with anything beyond a hello world. You don't need Meta's scale. It came from the big companies because they needed it most, not because they are the only ones who benefit.

2. I agree that the frontend-backend split caused a lot of harm and it seems most places are moving on, but it

3. It's a fair assessment that many people don't understand JS well. But it's also a super quirky language with many features that probably should be considered "do-not-use". Things like messing with the prototype chain, purposefully using == instead of === etc. CSS is even worse. The defaults are weird, which is why everyone uses some form of CSS reset, many things have surprising names and refactoring in a codebase with complex usage of selectors and cascading is a nightmare.

But I wouldn't call people trapped in frameworks. It's pretty easy to switch between React, Svelte, Angular and co.

A) The gone by days of Java Applet's and Flash weren't so snappy either. Our websites also do much more. A tab of Google docs isn't slower or use more memory than a Word instance.

B) No group is ever immune to arrogance. But sure, if you write vanilla JS instead of TS, I'd think that's an odd choice. I'm not saying you can't make good things with Vanilla JS, but I don't see how that makes the Web more "sane".

LLM: When it comes to coding tools LLMs are not so different from humans in what helps them. They benefit just as much from the cleaner control flow of reactive frameworks (compared to jQuery), colocation of JSX and tailwind and typechecking of TS as us humans.

Mess: Has the Web really been such a mess lately? I think most of the messiness is far behind us. JS+HTML5 winning out against Flash and Java, standardization across browsers, XML or HTML. Was it worse than Microsoft reinventing their Windows UI framework every couple of years?