upghost

Yeah that's fair, I concede the point.

I got a good snicker out of the YAYAMLMLOLOL :D

Seems like it's tools calling tools all the way down heh

I'm sure you realize it gets reassembled into "huge opaque strings" when it is fed into the LLM as context. The arbitrary transport of the context as JSON is just a bit of protocol theater.

You don't really have to parse the output, Python already has a parser in the form of the AST library[1].

But I get your drift. Depending on your workflow this could seem like more work.

[1]: https://docs.python.org/3/library/ast.html#module-ast

I think it's interesting and odd that tool calling took the form of this gnarly json blob. I much prefer the NexusRaven[1] style where you provide python function stubs with docstrings and get back python function invocations with the arguments populated. Of course I don't really understand why MCP is popular over REST or CLI, either.

[1]: https://github.com/nexusflowai/NexusRaven-V2

Another weird thing about Carmack, now that you mention it -- and Romero, coincidentally -- is their remarkable ability to remember technical challenges they've solved over time.

For whatever reason the second I've solved a problem or fixed a bug, it basically autopurges from my memory when I start on the next thing.

I couldn't tell you the bugs I fixed this morning, let along the "groodilizer" I optimized 20 years ago.

Oh btw Jank is awesome and Jaeye is great guy, and also a game industry dev!

Can we use this micro moment to try to bridge the gap? I was sold on cocaine but all I've gotten so far was corn starch. Is there like a definitive tutorial on this? I mean look I am proud of my work but if I can drop 200-1000/month for the "blue stuff" I'm not gonna turn my nose up at it.

I've been pretty deeply into LLMs myself since 2023 and built several small models myself from scratch and (SFT) trained many more so it's not like I'm ignorant of how it works, I'm just not getting the workflow results.

I've attended 4 of them (3 in person in Chicago, 1 remote during Covid), and I took the raft algo implementation course twice (second time remote). Half of the reason I take them is bc the content is good and the other half is because Dave is just genuinely an amazing instructor and human being. It takes someone of tremendous talent to present such complicated ideas with such simplicity and humility and lack of jargon, he talks about implementing consensus algorithms the same way someone might patiently teach you to tie your shoe. In the era of Big Ego in Big Tech, it's a truly refreshing difference in perspective.

Each course I took was accompanied shortly after by a significant increase in career, compensation, and understanding. Like anything else, it is what you make of it. All I can say is it was a very good experience for me.

If you want to speed run things and you have a little money to spend, I'd check out Dave Beazley's workshops, particularly his Advanced Python Mastery course[1].

One advantage you have over senior developers is that senior developers need to be generalists, you have the opportunity to go deep. It is entirely conceivable you could learn Postgres, JavaScript, or Python better than someone who has been programming in those languages daily for over a decade simply because they have a lot of other things to worry about. They can't read the manual cover to cover, no time. You can. Being a deep specialist in a technology has a lot of value.

If you find a programming community you love, start going to conferences as soon as possible. There are often financial opportunities available for new developers.

This is not going to happen overnight. In most cases to get proficient you can expect that working 3 hours a day, 3 days per week, it will take you about 3 years to reach a point where you might be employable -- although you will probably find you need to put in more time than that. But you can reach a point where it is enjoyable before then.

Good luck.

[1]: https://dabeaz.com/advprog.html

Monads are a set of annotated functions or methods that participate in shared encapsulating middleware. It's kind of like writing an interpreter for existing code by changing shape of the inputs, outputs, and possibly even flow control of the execution of those functions -- but without writing an interpreter.

The easiest example would be something like wrapping a bunch of arithmetic operations with a "cumulative" monad. Effectively this changes your add, sub, mul, div functions such that instead of taking 2 floats and returning a float, they take a hashmap and return a hashmap. The hashmap consists of the original args as well as the cumulative total, for whatever reason. The details of the hashmap are hidden from you, you use the functions as per normal.

You could also make the wrapper monad have some state, and then batch the operations while making them appear to execute sequentially, or make it appear you are doing pure logic when I/O is happening under the hood.

While you can do monads in dynamic languages, it can be hard to reason about changes to the code without strong compiler support, so typically you see it more often implemented in statically typed languages.

In dynamic languages such as lisp you might be better off writing a small interpreter, and in OO languages there are other patterns that might serve the purpose better.

I still don't know what a monoid is though. Or an applicative.

> When Captain Picard says in Star Trek: The Next Generation, "Tea, Earl Grey, hot," that's agentic AI, translating the voice command and passing the input for the food replicator. When astronaut Dave Bowman orders the HAL 9000 computer to, "Open the pod bay doors, HAL," that's agentic AI too.

More like GIR from Invader Zim.

Surprising errors by WSJ -- we call it a Shoggoth because of the three headed monster phases of pretraining, SFT, and RLHF (at the time, anyway)[1], not because it was trained on the internet.

Still, cool jailbreak.

[1]: https://i.kym-cdn.com/entries/icons/original/000/044/025/sho... (shoggoth image)

This looks absolutely fantastic, please accept my meagre professional jealousy. I have long bemoaned manual hyperparam fiddling . I have on occasion dabbled with nonparametric ("genetic") methods of hyperparam tuning inspired by AutoML... but then you still have to manually tune the evolutionary hyperparams.

Finding a way to derive this from the gradients is amazing.

Is there anyone else who feels kinda like declarative templating is actually kind of worse than jQuery? Don't get me wrong, I've been using React for nearly a decade. But the more complex my SPAs become, the more I wish I had imperative control of the DOM.

I think the reason is because the DOM is a leaky abstraction and at some level I would just prefer last write wins.

I realize declarative templating is supposed to handle that, but this starts to break down really quickly when you share mutable state between components.

For a second I got excited that openai was actually doing cool stuff again.

Literally was typing out "Unsloth, do your thing!!" but you are way ahead of me. You rock <3 <3 <3

Thank you!

> We now have machines that can mindlessly generate words, but we haven't learned how to stop imagining a mind behind them[1]

Good video essay. Learned the origins of the term "cargo cult", and to my surprise, has nothing to do with rust...

[1]: https://youtu.be/zKCynxiV_8I?t=26m04s

It's almost as bad as "manual" thinking!

Thanks, ok. I think that was this missing piece for me -- we are not just tracking points/pixels but related points/pixels that taken together constitute an entity or identity. Yes I can see how that would be quite useful.

Apologies, jargon meanings updated.

> The utility of optical flow (i.e., the instantaneous velocity of pixels [16]) toward this goal has long been obvious, yet it has remained challenging to upgrade flows into long-range tracks.

This sentence from the paper makes me feel a little bad that I don't understand why this goal is obvious. I am not tracking why we are tracking pixels.

Is this basically a competing technology with YOLO[1] or SAM[2]?

[1]: https://en.m.wikipedia.org/wiki/You_Only_Look_Once

[2]: https://ai.meta.com/sam2/

Edit: added annotations, should've done that initially

I know. The cover art is great and makes no sense. Some dude with an assault rifle (I guess supposed to be Milo/Asbel) riding a sandworm along with his buddy Dr. Doom I guess, a squid guy with a lightsaber (firewarrior probably?) and there's a guy flying pegasus for some reason? [1] Truly iconic.

Some of that is explainable but the pegasus, dude on the pegasus, and Dr Doom ... makes absolutely zero sense. I guess this is the rock opera version we never got. Maybe Dr Doom somehow sorta kinda was supposed to be Princess Kushana..?? I think the squid guy is a firewarrior, but so is the "sandworm" -- some kind of fever dream Ohm/Gorgon/Firewarrior mix. But why is a tiny adult fire warrior riding a baby giant fire warrior? lololol

I can only imagine the requirements gathering session that lead to this debacle.

Legendary.

[1]: https://i.pinimg.com/736x/00/fe/12/00fe121ffa3600cfb14bbbe1c...

Edit: Dear haters, I <3 <3 <3 Warriors of the Wind and all things Nausicaa, incl. the quirky cover. I even love your downvotes, it means you read this and cared enough to click the button, which hopefully means you are defending the cover. So I love you too!

Was the plot completely butchered...? Yes.

Were there gaping narrative discontinuities...? Certainly.

But that 80s voice acting in the English dub... chef's kiss. The guy who voiced Milo("Asbel" in the movie) was the voice of Leonardo from Ninja Turtles and Kaneda from original dub of Akira!! (Cam Clark)

If I watch any version, I watch Warriors of the Wind. It really is the "Hollywood version" of Nausicaa.

Yeah, the newer one is better, but it doesn't hold a candle to the manga so give me that 80s synth and big hair voice acting any day.

Also I was like 5 when I saw it so nostalgia goggles.

Do yourself a flava and read the Manga. The movie ends before where the first volume ends, but the story goes in a completely different and much more interesting direction.

I guess the 5th edition will need to be called "Artificial Intelligence: An Outdated Approach".

I absolutely love that book. Maybe that's why I get a little cringey at the "AI" monicker for LLMs.

Interesting that this used to be called "AI". I'm still trying to figure out what to call the umbrella field of Artificial Intelligence now that "AI" has come to mean the genAI subset of DL which is a subset of ML which is a subset of what used to be called "AI".

So what do we call "AI" now that "AI" has come to mean GenAI? You know how there used to be this field called "Artificial Intelligence", of which a subset was Machine Learning, of which a subset was Deep Learning, of which a subset was LLMs..? Does that have a name anymore...?

Does anyone have any idea what the DoD could possibly want from OpenAI? Less accurate/more sycophantic missiles?

You made my day. I am a long time lisp developer who held a similar view of Prolog for a very long time until for various reasons I stumbled on these videos, and then realized there was a whole aspect to the language I was unaware of that went beyond the logic. Really great community developing around this stuff particularly with Scryer and Trealla that are very active if you are interested in discussing any of this further.

The fact that we were able to have this exchange was great. I appreciate your points and look forward to incorporating your work into mine!

Datalog is a syntactic subset of Prolog[1], which this is... not.

I think the most misunderstood thing about Prolog (and Datalog, the functor-free subset of pure Prolog) is that the syntax is really, really important.

It's like, the whole gimmick of the language. It is designed to efficiently and elegantly query and transform itself. If you lose the syntax you lose all of intermediate and advanced Prolog (and Datalog).

[1]: https://en.m.wikipedia.org/wiki/Datalog

That's a great question. The answer to "why wouldn't you use Mercury instead?" should hopefully be more clear when you watch the meta-interpeter video. The syntax of Prolog is not accidental, and this is a case where it really matters. You sacrifice a lot of power by introducing things like dot access notation.

Additionally, there is nothing stopping you from writing pure monotonic Prolog, which does not involve cuts and uses pure IO.

Circling back to DCGs, the combination of DCGs plus interpreters, which can be expressed in a handful of lines of code, allows for powerful language crafting.

While constraint solvers are not unique to Prolog, CS+DCG+MI is a killer combination I've only ever seen in Prolog that when combined allow you to solve some wicked problems rather elegantly.

That being said, thanks for the reminder about Mercury, that looks like it could come in handy on some Java-only projects.

This is DCG+CS only, still quite powerful and I have adapted it for GOAP and Hierarchical Task Networks quite successfully: https://www.metalevel.at/zurg/

Finally, I will agree with you that there is a LOT of jank Prolog, but not ALL Prolog is jank, and some Prolog is truly exquisite-- I hope you will keep an eye out for it.

> Prolog is honestly kind of jank, and I’m not talking about good vintage jank like C.

Respectfully I would encourage you to consider this comment is confusing bad Prolog with all Prolog, and you are really missing out on some elegant and powerful concepts if you believe Prolog stops here.

I would love this to be an invitation for you to reevaluate modern Prolog and see what the language is really capable of.

I'll throw you right in the deep end with Prolog Definite Clause Grammars[1] and meta-interpreters[2].

There are a few things which are very special and magical to Prolog which are quite mind expanding. I hope you enjoy the resources.

[1]: https://youtu.be/CvLsVfq6cks

[2]: https://youtu.be/nmBkU-l1zyc