andy_ppp

Bioweapons and cyber attacks on other system - for example the banking system or suppose and AI hacked into important Russian systems that pushed them back in time to almost pre-computer society or an attack on Chinese systems that made it look like the US was moving nuclear weapons into Taiwan, the responses from these countries could be awful and dramatic. We can't control what they do to be honest, I believe once self improvement happens the AIs will build in their own circumvention that we humans cannot even understand. We barely understand what is happening now in terms of interpretability of neural networks on tiny problems I'm not sure alignment is even feasible at the scales of parameters we are talking about today let alone in the future.

We’re talking about run away self improvement of these systems, I’m not even convinced we’ve seen these systems invent a single new thing yet.

Have you seen anything produced by LLMs that is better than the best humans?

So predicting the next word given all humanity’s knowledge is surely going to max out at slightly less good (we probably can’t get perfect data) than the best human in any specific field. What test does the AI do to be able to understand it is improving? At some point it becomes impossible to know that the output is actually better right?

Ah a more direct way of funnelling government dollars to AI companies, knowing Trump it’s probably using Deepseek under the hood while he pockets the difference :-/

The output of LLMs, largely :-/

So what exactly are you proposing I do - just ignore the fantastic gains and great software that people are using day in day out?

Yes I agree we have all signed our own death warrants. But this does not just apply to coding.

Are you suggesting not using the tools? A kind of technical version of the Amish way of life?

I dunno I think my guidance and testing is very important and I make sure not to ship things with bugs and code that is really awful. I'm still just about necessary for now.

I don't really understand all the negativity - I've delivered amazing robust solutions at about 10x my previous rate I think and can make dramatic cross code changes on awful legacy code bases with ease. I just added some complex prev/next navigation across a load of detail pages pages that rely on the current search, a new PDF generation system that uses a queue and workers and absolutely tones of changes to a legacy ERP system that would be unworkable without AI.

Dealing with legacy messes I used to get frustrated and bored of making the improvements, now it's easy to clean up code bases and write loads of tests. I asked Astra to come up with a plan for Playwright testing the whole App, I have not built it yet but the flows suggested were fantastic as was the ephemeral database we plan to create for CI.

I built my friends portfolio website almost entirely vibe coded in 4 hours and it looks unbelievable, we added so much slickness (he's a designer) just prompting together. I used a CMS I had never used once before and it was so so easy to do without any of the usual need to read docs about everything.

I've done so much devops now I'm actually fairly confident that me and an AI can do anything you want in terms of deployment/infra and scaling from AWS to Terraform to whatever.

Anyway my main concern about this technology is not that it is crap at coding it's that the improvements in how it codes and thinks are absolutely dramatic which is extremely scary - it has come so far in a year I wonder what the next year will bring.

So is there not a case for suing not for the shares but being told the wrong information at the time? It seems wrong to be mislead by a company who owes you shares in this way.

Oh man… a whole fan would be good https://www.youtube.com/watch?v=bReWaZz5Grg but the actual did it already :-)

Did you try different sources I do not trust some of the stuff out there and often wonder if these supplements aren’t just generic white powder…

This is why I was careful to specify tokens per second not time to first token which is the prefill step you’re talking about. Clearly to run these models well you need both but as I said adding more chips or compute units can give you more latency where as overall memory bandwidth (throughput) is limited by access to fast memory.

Tokens per second is almost entirely memory bandwidth at inference time, training obviously needs more compute but you can add more chips for that.

The agents will soon decide all this, not the humans ;-)

It can go either way, you might get a relief of pressure or you might get an increased destabilisation in the region.

I agree with this, the rich actually are oddly MORE greedy in Europe and take double the equity for half the money (roughly) so founders get a lot less upside or they move their businesses to the US and get US capital at better rates.

Why can the Chinese build models and Europe cannot? The algorithms behind this stuff are not that complicated, are they? Is it the cost of energy? The illegality and difficulty of obtaining all the data in the world? Lack of capital to start moonshot labs? Lack of optimism?

The Chinese just seem to have an ability to get it done without anywhere near the GPUs of the US and Europe can buy these GPUs.

I think relying on the US and China for AI is probably not ideal? For example I think Qwen have not released the Omni models as open weights in the past, it’d be good to know if they’re doing this here?

Yes I sometimes think the "don't train on my data" is actually a good signal for "this data/person is probably better to train on because they want to keep something private". The whole copyright system should have stopped these guys from training on everyone's data and it did not, if you think they care about the privacy checkbox I think you're dreaming personally, based on their past behavior.

You believe any of these companies care about the law? They care about winning and building the self improving AI as quickly as possible.

So, during the Great Depression who ended up doing well? What can be applied to today?

Why? Seems the US doesn't like democracy anymore and the EU for all its faults still does. I can see Japan joining too and potentially South Korea, Philippines, and others who are concerned about the state of the world and want to work together rather than drive the world apart.

How funny, they chose very beautiful old architecture and very ugly modern architecture and invented a result - why not choose things like the Bahai temple in Delhi or Fallingwater by Frank Lloyd Wright? I'm sure there are some old ugly buildings and the ones that people hated probably got torn down so there's survivorship bias in play here too!

It’s impossible to know for sure but if they’d stayed private I think they’d be more not less successful, just because startups can move more nimbly. Although we’d need to define success, we’d be here forever…

People here are assuming that current CEOs are not actively destroying their companies, maybe the agent presented here is actually a very accurate simulation!

Excellent! Let us know when!

You know there was a lab that was cutting edge in Europe and it is still there - Deepmind - but the UK let it be sold to the US for a fraction of what it's worth today - $400m. We just allow US companies to asset strip the EU and then people complain about us never having big companies here...

Deepmind would likely be worth conservatively $200 BILLION today (probably more without the overheads of joining Google to be honest) or around 500 times the sale price in 2014, not a bad return.

Who decides on when to press said button?

Maybe we could connect AI to everything and make ourselves completely vulnerable to and dependent upon it instead?

It would obviously be opt in but it’s just an idea, it clearly might dissolve on contact with reality!