somewhereoutth

Following S.I. notation, we might have Tk for token, so kTk, MTk, GTk etc.

> Money doesn't appear out of thin air.

In fact [fiat] money does appear out of thin air (well, created by banks when they originate loans) - and has to to support a growing economy. Unfortunately, for various reasons, rather too much has been appearing, and has been funneled to the already wealthy.

So apparently grid forming or synthetic inertia can be provided electronically nowadays, but presumably it would not be too terrible to put a rotating mass in between the (e.g.) solar power source and the rest of the grid? So the electricity from the solar panels runs a motor that turns a generator. I suppose power is lost through friction and other inefficiencies, and you'd need to co-locate the solar panels into a big enough farm.

1. Both costs (going down) and performance (going up) likely are, or will shortly, approach asymptotic limits.

2. CFOs are seeing the token spend on the bottom line, and are not happy. CFOs don't care about 'the next big thing', they just count beans, and they are coming up short. CFOs tend to be the grown-ups in the C-suite, they will shut things down if they need to.

3. See 1. and 2.

Yes. As makers we tend to assume that the more that is made the better, and that simply by having lots of (shiny!) stuff we will get paid/honored/favored etc, whereas in fact often this stuff becomes someone's problem somewhere.

Banning online advertising would solve very nearly everything. Specifically, social media would die (and traditional media would get a new lease of life).

An intermediate step would be to ban personally targeted advertising, at least then the playing field can be leveled somewhat.

Push up asset prices mainly - so locking poorer people out of (e.g.) home ownership.

Money is not a tangible thing, you can't eat or drink it. Instead it is a signalling protocol for resource allocation. If the very wealthy have many empty homes, when many people are homeless or inadequately housed, then that signalling protocol has failed (from a social justice point of view), and 'trickle down' is not working.

lack of new content from the internet will make them go out of date. Not just facts and figures, but (for example) new programming languages/techniques.

Yes the huge discrete stepwise training spend is critical.

Maybe investors will realise that "the only winning move is not to play".

And so we are left with (as was) frontier models getting more and more out of date as whoever their post bankruptcy custodians are tries to eek pennies on the dollar for inference on their decaying property. Perhaps along with local and/or highly specialized models still feeding on the after-glow of the huge amount of training that was (and is no longer) done.

The next AI winter is going to be deep, savage, and long.

(sorry, nit pick, but I don't your usage of 'abrogate' is quite correct here, you can't abrogate to something)

The real test would be if an LLM makes an important conjecture.

But it is not a real economy is it? Vast sums of money are being spent subsidizing token processing with little to no tangible business benefit for the end user.

For those of us lucky enough to have the choice, the best bet is to sit it out for a year or so until it all comes crashing down, then re-engage with what's left of the software industry.

I'm also glad they have the option - and it does seem that many will take up that option. They themselves have judged that factory work is less bad than 'house' work, might be worth listening to them.

Right! I am currently living in a (relatively poor) Western European country that has recently experienced a tourism boom, and although the money pouring in has been nice for at least some, it has wrecked the existing social fabric in many ways - starting with housing.

On the flip side, at least the beaches are kept clean. In the UK (where I'm from) there is a big problem with sewage outflows. Meanwhile here an entire beach got washed away by the winter storms - so they are putting it all back! Maybe 100 000 m3 of sand!

Right, however over-tourism is a real problem.

Tourism provides low quality, transitory jobs, with income flowing more to wealth holders (property owners etc) than to wealth creators. It distorts property markets and sucks the oxygen out of other kinds of business. About time the Med weaned itself off of it.

Prior to the advent of LLMs, I had this concept of the 'complexity horizon' - essentially a [hand built] software system will naturally tend to get more and more complex until no-one can understand it - until it meets the complexity horizon. And there it stays, being essentially unmaintainable.

With LLMs, you can race right for that horizon, go right through, and continue far beyond! But then of course you find yourself in a place without reason (the real hell), with all the horror and madness that that entails.

The entire business proposition for LLMs is that they will replace whole armies of [expensive] humans, hence justifying the biblical amount of CapEx. So of course there is strong incentive from the LLM creators to anthropomorphize them as much as possible. Indeed, they would never provide a model that was less human-like than what they have currently, even if it was more often correct and useful.

Gambling is right. And not just the dopamine, habitual gamblers remember the wins and forget the losses, so tend to believe they are 'ahead'. In time the bean counters will come and sober everyone up, the bottom line can't be argued with.

Right, it was the perfect match: Humans for fuzzy touchy feely stuff, computers for hard edged correct calculations. How have we managed to screw this up so badly?

> When developer productivity tools become so valuable that engineers blow the entire budget in four months, the issue isn't the tool but that the budget was invented too early to forecast this adoption curve.

Where oh where can I find clients like these??

Tulips? NFTs? South Sea Bubble? Mortgage Backed Securities and derivative products (CDOs etc)? Beanie Babies?

Anti-Genism? Antija for short.

Conversely, just because an invention causes a bubble doesn't mean it is useful and world changing.

I believe wealth taxes (really, wealth restitution) should go into sovereign wealth funds - not least as then the public can see how that money is working for them, and so support the continuance and expansion of such taxes.

LLM proponents believe that these higher level encodings in latent space do in fact match the real world concepts described by our language(s).

However, a much simpler explanation for what we see with LLMs is that instead the higher level encodings in latent space match only the patterns of our language(s), and no deeper encoding/understanding is present.

It's Plato's Cave - the shadows on the wall are all an LLM ever sees, and somehow it is expected to derive the real reality behind them.

depends if post-correction it is worth anyone's money to keep training new frontier models. It could be that it isn't, so we are left with models that were trained in the bubble, but are now increasingly out of date, or (open?) models that are trained much more cheaply somehow with consequent lack of utility.

I like open note exams (and perhaps open book exams, as you need to know the book well to know which page to look at) - it forces you to condense the material to the salient points and operationalise it to solve what would be more challenging problems than a simple recall exam.

When I see 'cheat sheets' - designed to be hidden on the back of calculators or whatever - then I see true application of human ingenuity and intellect.

My understanding is that the major part of the cost of a given model is the training - so open models depend on the training that was done for frontier models? I'm finding hard to imagine (e.g.) RLHF being fundable through a free software type arrangement.

The first 90% of this is standard set theory.

I'm unclear what the last 10% of 'category theory' gives us.

Really not the same. Assembly / machine code is entirely deterministic - they are a notation for your thoughts. LLM produced content is more a smorgasbord of other people's thoughts, and cannot help you with clarity, conviction, etc etc.