jaggederest

Yes I'm using the entire consultancy stack - define values, etc, and work your way down the "where do these not match reality on the ground and need change", but for little robot people instead of (arguably less messy) humans.

Yes, you need some kind of other source of truth. I think the best way to get that is to do clean room development with a different agent, but ultimately if you give them the wrong idea they'll do the wrong thing.

The other thing I do, not as much as I should, but it's very powerful, is to generate spikes and deliberately throw them away to understand how to prompt better. Like I generated a swift version of the react native app I'm working on, and Alloy provers for the state transitions. None of it is production quality but getting great results that way is useful to scope future work.

This is the fault of the distribution of the benefits, not of the technologies themselves, at least in so far as they don't actively accelerate the shift of value from labor to capital. Washing machines reduce drudgery no matter who owns them.

my stack in a sentence: refine the docs/prompts/skills often, that's your biggest job, use both frontier labs models reviewing each other, don't solve individual problems only the systemic ones (set standards strategically, don't define tactics)

If I had that many tokens/dollars I would be running canaries and adversarial verification in prod based on e.g. traffic replay, live fuzzing, all kinds of things to build confidence without direct human line-by-line review. If I had $100k to spend next month I could probably get through it, I'm running $2500+-api-equivalent a week at this point and I feel very token limited. Will be time for a 2nd or 3rd subscription soon for both labs I think.

Fable was a revolution, still learning how best to use it, 5.1 felt like a notable upgrade. At this point I launch a workflow with 10-20 minutes of interactive setup (and even that I feel might be too much), it runs for hours, and the PR is trivially mergeable (I still review every line, but 95% are just merge, maybe 4% are feedback needed, 1% are thrown away and regenerated, which implies I'm being insufficiently ambitious)

Other than that, Ms. Lincoln, how was the play?

I felt the same way at the time. Now, I believe we have better things to spend our collective time and energy on.

Far from the most frustrating thing they did, trust me. I could tell you stories for days and never run out of more frustrating anecdotes.

Instead of registering foo.com, and having bob.foo.com, they would register bobfoo.com - they had hundreds or thousands of certs, as a result, and had to renew them all annually or biannually. We didn't even manage all of the domains, we just had to renew the ones our group was responsible for, and it was in the dozens, and each form was ~30-40 actions to fill out correctly. I forget whether they were quarterly or we just ended up doing it all the time, but it was a maddening task.

Part of the explosion was that domains were not just in one TLD, they were bobfoo.com bobfoo.bar bobfoo.xyz and in many cases regionalized to bobfoo.co.uk or whatever. It wasn't "domain hoarding", because if you registered bobfoo.info you'd just get UDRP'd by corporate anyway, and nobody really wanted domains of the form somelongnamebobfoo.com

This was also before something like LetsEncrypt where you could automatically generate certs for programmatic usage, so they were all done manually.

Exactly what they did in many instances.

Okay, nowhere did I say it would be before they learned language. I said we could use models that can trivially build notetaking systems to auto-improve themselves by taking, refining, and updating notes as a memory system to exceed the x token context threshold.

> but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.

Why do you think that? You do a lot of assuming a thing to be true without any clear reasoning or evidence. I'm talking about a system, including a LLM, that acts in a way that is equivalent to or better than an average human at a large range of tasks, such that is generally capable of learning and problem solving similar to the degree of success humans have. All these specific criteria about what is "actually human" is not the point. We know how to make more humans, I'm talking about a form of intelligence that we are creating that is genuinely alien, so holding to "well it doesn't work the same as a human" is an impossible bar. If you reverse the situation, humans are by no means LLM-equivalent intelligences - perfect memory of long form content, rapid calculation, instantaneous tool calling - a hypothetical silicon based lifeform would dismiss us as "really not intelligent at all, they can't even do multiple floating point calculations per second in their head", and I think that would be similarly silly.

I was working for a company that had hundreds and thousands of domains. It was a fortune 50 company, and they had a policy generally against wildcard domains.

> If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors

I am not sure why any of that is necessarily true. I think you are crediting the human brain with a lot more than it actually does - memory is something in the direction of a fine tune on existing neural hardware, it's really not anything special, and I absolutely think that "just taking notes", when taken to a sufficient fidelity, is enough to pass that bar.

> They're insane and should not be in charge.

I remember back when we had to write mechanize scripts to drive a browser through the renewal process, because if you had dozens, hundreds, or thousands of domains there was no nonmanual way, especially if you wanted extended verification or something silly like that.

So what I'm saying is, agreed and that has always been true.

They're poking at SOX and Dodd-Frank I think, but it's a bit of a specious argument, in my opinion the rise of massive private equity (plus the absurd market orientation around short term results) is more of the story, as you say.

Or, to put it as Groucho Marx did:

"Please accept my resignation, I don't want to belong to any club that would have me as a member".

https://www.youtube.com/watch?v=kJHUres_2xU&t=228s

I'm pretty sure firefox itself is a direct contradiction to this, but ok...

I guess I fundamentally disagree, because on general knowledge they beat any human, and on processing speed they beat any human, and on problem solving likewise. I just don't see a domain besides "retain change over time" that they fail on, and that is something you can do in the system rather than the model - coding agents can and do build their own memory system and use it, like an amnestic human writing notes.

I think what I'm objecting to is the idea that if it can't do every task that every human can do better than any human, it's not AGI - I think we have to allow that this is a fundamentally different form of cognition, so requiring a 100% match feels uncharitable. Every human has cognitive gaps that AI doesn't have.

I also think it's moving the goalpost significantly. If you asked me what AGI looks like in 2024, I would have said "smarter on tasks that can be done via text than the average random human you'd meet in a NY bodega", and by that standard, it's long in the rear view mirror.

Now I think people are asking a different question, which is "better than any human at any task that can be done via text", which imo is superhuman, not artificial general intelligence.

I feel like making a human-proof benchmark is pretty clear evidence that they've exceeded even the highest human capacity in most respects, for things that you can do via text generation (and to a lesser extent image generation)

Yeah my experience in the past was that datacenter hardware (air cooled, at least, I've never dealt with water cooled) uses tiny fans at or about 15000RPM, so it matches basically what you're saying or worse.

I always loved this video: https://www.youtube.com/watch?v=tDacjrSCeq4

Don't think of it as being unable to work, just like github outages, it's two weeks a year of mandatory time off

It's been a while since I messed with serious hardware, what's the cooling noise level? I guess they have to be water cooled or the rack format 8u ones would be about 100db

Directionally I think social media is going to be regarded like tetraethyl lead and asbestos. (So is a lot of the ways we use plastic, though maybe not actual e.g. polyethlyene itself)

It's a bit like backtesting, in finance. Quite useful to establish whether your algorithm is sane and coherent, no use at all in determining if it is actually useful, to my understanding.

Seconded. No job is worth damaging yourself permanently like that, even if it's "just" a temporary burnout, pull the ejection handle. It's a life safety issue.

Start looking before you leave, preferably, but at the very least scale back your commitment so you don't burn out in the mean time. If it's bad, quit. This is why you need to have savings and spend less than you earn.

This is exactly and precisely why open source exists, for both the "never upgrade frozen version" case, and the extension into "fork it and make your own" case

I had to come back and comment because your "misunderstood genius" really resonated with me.

It's even beyond that, in my opinion. What if everyone had perfect health, physically and mentally, and had been educated to the absolutely top tier of their ability to accept? Perfect, individual tutoring? Excellent nutrition?

How much better off would the world be if we framed everything in terms of helping everyone be their best, every day, in every way? How much of the trouble in the world is the result of just... accidental awfulness, that we collectively could just choose to stop?

It makes me very sad that we live in a world where potential Nobel winners are dying of easily preventable causes, or working dead-end jobs.

Like Matt Uelmen's work in Diablo and Diablo 2... I haven't played the original in probably 25 years and I can still hear the Tristram soundtrack

Honestly it's really gratifying to hear that people around the world are getting the help they deserve, thanks for sharing.

I think I would actually characterize it as a mutual change - obviously the model doesn't change as far as I can tell, but changing the harness and context changes its behavior so wildly that it might as well. Bit of a pas de deux - I learn, I change the context to teach it, it teaches me in turn, rinse and repeat.