Wow! This is exciting. I have speculated that this would be possible. Since I've been learning about databases and dynamic db query optimization, it struck me as surprising that we didn't have similar things for more of our computational universe. Now that this is here, it makes me wonder what else we can optimize in using the same high level techniques.
The more you know about what you want to do ahead of time, the more optimally you can reorder your sequence of steps to give a better-than-naive solution. It makes me think about all software in terms of these abstract computation graphs and makes me wonder what else we can optimize automatically.
Of course, we do absolutely do need a formal model here, so we know what edits are possible, the same way db engines have relational algebra as their backing model. But this whole thing makes me feel like manual software optimization is soon to fall to AI. And I'm thinking that includes good-old-fashion AI first, not second, to LLM's. But I'm sure LLMs would be useful here too, especially for the formalization.
Woah this is so interesting. Confirmed it myself, you actually implemented elision!!! I really like your idea of capturing provenance in-document as well. This is humans speech, right? If you can do forced alignment on Latin, it gets me wondering if one could use a wav2vec especially turned to convert IPA into speech somewhat deterministically to compensate for how there is not heaps of training data in Latin.
I'm working on a new independent LaTeX engine written in Rust and no C dependencies at all. You can see it working at https://telox.dev/app/scratchpad/
There's currently not a way to invert LaTeX engine output for bidirectional editing. So you can't easily drag objects around or interact with the page on LaTeX. That's what I'm setting out to change in the name of mathematical exposition.
I've experimented with a few ways of making the editing realtime and I'm getting in the 10 millisecond ballpark on some edits, but I want to work backwards from the light speed path to see what it takes (besides a lot of time).
This is an excellent idea. I was curious about using DeepSeek OCR for exactly this purpose. But a tricky question is if we could do some sort of looping or something "energy based" and use classical search to find optimal parameters (LaTeX settings) to minimize the error (pixel difference). Me knowing I would get obsessed with the second half is what's keeping me from the first half. Maybe a vision JEPA would be good. If I had API credits to burn, I'd copy paste our two comments and see how far Fable gets.
I think I see how you're thinking about it, to see how I see it, consider the following: Memes don't have to be inherited from genetic biological parents. Memes can have their own memetic parents. Like the joke "I feel like I'm the average of my 5 best friends" - our habits, quirks, idiosyncracies, all fit into the framework. It just needs some reproductive medium (human brains). That's why evolutionary biologists would be interested in studying this and things like it. There are epigenetic traits that exist and aren't genes and might be able to self reproduce. To keep the analogy going, consider that things that are catchy are easier to have this "memetic reproduction" so a catchy song might get sung a lot, serving as its fitness condition. So it's a generalization of biological genes to not just dna and rna and physical molecules but any substrate that can carry information. It's familiar to how we call it a 'viral' video.
My experiment is examining the autonomy of Fable specifically in an auto research context. I don't believe I said in my message that no other model would have been able to complete my highly autonomous work. So it feels like my view has been misrepresented or misunderstood. This message makes it harder for me to share the things that excite me online and makes it more daunting to share my findings when this project completes, especially any comparative work. For an analogy, I feel like I said that I like Southern Butter Pecan Ice Cream and am being met with a response of the form "Sounds a bit sad that you have to wait for a weekly restock to enjoy any ice cream." I made a goal for myself to be more open with my feelings in life and share more of what I'm working on and not be so rejection-sensitive. I understand that even if I'm just sharing the positivity I feel, it can come across differently. I guess this is just the cost of communication in a lossy language.
For a course I'm taking, I made an "Exposure Triangle" demo inspired by Duolingo. This feels like I just struck gold. Whether I use this implementation or another, I'd like to develop a library of these Skeumorphic components. I want to make a later version of my application -- where I make explanatory mathematics and this kind of minimal diegetic interface is exactly what I need to pull that off
I am absolutely thrilled that they reset weekly limits. I have been experimenting with highly autonomous work (5+ hours continuous) and fable seems excellent at this, especially when using subagents. I ran out of Fable capacity and was bummed out that my experiment would take longer to complete. Now I'm super happy I get to continue it
I have been thinking we should do this for code too. I still haven't had time to execute on this plan, but I think there's probably part of the answer to how to use AI as a tool is how it doesn't automatically give us the same kind of window into what it does. In verbose mode, I've caught things earlier that I wouldn't have caught until later. But even beyond that, it speaks to how we draw lines between code and data and what happens if we erase those lines. What if we applied data vis methods to code? Maybe hierarchical semantic clustering colored by class hierarchy. Not just one floating architecture diagram. But different slices of the codebase.
Like some others, I also built this myself. Overly simply, with tmux, a memory tree, and handoff files and an orchestrator. And yet for how simple it was, it was so effective at minimizing the amount of duplicate context. It's like having shared specialist subagents who source and derive important shared knowledge from separate threads. It's useful because some skills just take too much of a token penalty to invoke and a single shared persistent session just lets that issue melt away. One agent pays the cost of that large skill once, and you don't have to keep paying for it in input tokens for the rest of that conversation.
We should discuss cache performance if we haven't already. That 33k tokens may be a cache hit (I am not certain it's automatically a cache hit) but after the first call, it should certainly be a cache hit. Cache hit tokens are billed at 1/10th the price of cache misses. This is quite opaque, but it's necessary when you're asking "is the system prompt worth its stay" if you can save 33k tokens worth of dynamic discovery across the next few turns, the break-even point is quick and if the system prompt makes task performance increase and/or makes the system more autonomous so that it can string together more cache hits in a row, it becomes way way better. On a personal note, I think of things as aa function of 'supervised time to desired result' and 'cost'. because I find it harder to reason about tokens. I do think they could introduce a "minimal" mode (something like this is probably doable with the Claude agent SDK today)
Other solutions get a lot right, but I don't know if I will ever use something other than LaTeX. Nothing gets math typesetting feeling so right. The justification and bin packing feels right. I might be esoteric for it. Maybe I care too much about things looking excellent. Maybe Claude is just good enough at debugging anything I'm facing. But I'm not interested in anything realtime/live if it means giving up on output quality. Overleaf has a live preview mode right there. I think a lot of the "problems with latex" are actually just problems with people developing packages in isolation in a decentralized manner. I don't care for LaTeX the language, or its implementation, but I do care for the output a whole lot, in a way that not the font alone or any approximation can reach. Maybe it's just inside baseball and people don't care about advanced typography. If I'm working on a piece of research that will carry my name on it, waiting for that compile is worth it. I still remember Scholarly Markdown. I might genuinely one day implement some other transpiler so I can always do my last pass in LaTeX. As long as your thing emits some decent IR, I'm happy.
Really reminds me that there's nothing in principle stopping us from storing parse trees and exposing them via something git like so we can avoid even needing to format, let alone also needing to resolve a whole category of merge conflicts based on that formatting. I mean a format is just a theme over your data -- I mean code.
Alternative Title: Claude Code puts words in my mouth (Self Prompt Injection)
I originally thought that this was just misunderstanding attribution in discussions. This does seem to be a harness bug. Or at least an ontology bug. In my work with LLM frameworks, it was always odd that tool call results are sometimes marked in the convo as coming from the "User", I think that could be fundamentally what's enabling this bug to happen. Neither the LLM nor the harness should be able to claim something came from the user.
This is command injection. I don't know enough to see if cryptography is part of the right answer but it might be. A hash of the user message, signed, public key private key, harness is coded to only allow signed messages issue instructions. Yes, that might be overkill, but thinking about the types of things agent harnesses are used for... I think the safety argument starts to speak for itself... This has never happened to me using CC though, for what it's worth.
Biggest limitation I see in this paper: the framing. Any time you have a lot of proprietary knowledge or you've just sorted out the right solution when it's not readily available from the model's parametric knowledge, that's when you should add a skill. Wrap it in a CLI that's easy to inspect. You don't need to store the whole help text of the skill either. The model can inspect it and its subcommands.
Reality doesn't force us to choose between skill or no skill, reality often doesn't give us a choice. You can either make a skill for your company's proprietary system or your model has to figure it out from scratch every time by searching wikis or reading code. If you use it right, skills are a compression mechanism. Instead of the process meaning your model needs to get all of theses files dynamically, it can simply statically run.
To steel-man the paper. It is worth looking at whether you should try to code something up first or try a skill first. And it may well be valid to say try first and if you can't work it out in 5 mins, install a skill. But there's a meta point of skills as software (where you deduplicate the effort of solving regressions).
For a reductio ad absurdum, If self-generated skills with no additional context _didn't_ eventually level off in performance, then we could reach AGI by making one big skill that keeps growing and solving harder and harder tasks, including improving the capability of its own skill-builder skill, all without embedding any signals from the environment or needing to interface with the real world at all.
I have been considering what it would be like to give each function name a specific color and a color for each variable's type followed by a color derived from the hash of the symbol name and keywords would each be their specific type. And essentially printing a matrix of this, essentially transforming your code into a printable matrix "low-lod" or "mipmap" form. This could be implemented like the VSCode minimap but I the right move here is to implement it as a hook that can modify the output of your agent. That way you can look at the structure of the code without reading the names in particular.
How did they leak it, jailbreak? Was this confirmed? I am checking for the situation where the true instructions are not what is being reported here. The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal.
This appears to be an observational result, so I'm genuinely perplexed by the reception here. I genuinely thought this comment shows a healthy amount of curiosity and asking important questions. Asking "what control group did this study use?" is usually well-received here.
Doctors aren't machines, they're humans. I have not yet read the full paper, only the article, but I already see something really big and important to look out for. When I read the full thing, the question I'll be asking is "what's the likelihood that the self-esteem of doctors was directly intervened on by the exam taking process itself." How do you control for the loss in confidence that learning of your test performance gives you? How are we certain that learning your score on the board exam doesn't make you more conservative (or riskier) with how you treat patients as a psychological effect?
We could probably fine-tune a tiny convolutional neutral net image classifier and just hold on the last good frames for longer to cover the frames with clear trolling and nsfw images.