hermitShell

That would be like regulating who gets access to the best food on a sinking ship. The USA could allocate capital and attention toward much better things, for the country and for their people. I just hope the next couple administrations are wise enough to not experiment with the idea that ASI can get the country out of this 'fall of Rome' slow motion decline.

Ironically what I think people really want here is a better recommendation system, one that almost certainly would need ML algorithms (or 'AI' as we call it nowadays) to implement.

So Valve will need to train on descriptions of games and perhaps demand more out of developers who want to sell games on their platform.

All notable app stores face this already or will face it soon. Maybe the apt GUI for your favourite debian derivative won't get too flooded...

So do you think there will be a big crash and memory will get really cheap in 2030, or do you believe that AI demand is strong enough to sustain all this buildout?

It seems like this list also doesn't account for China and state sponsored memory industry (which will be more immune to a downturn)

To be completely honest, I don't know. Here's the kind of thing that I was referencing though:

https://youtu.be/tj1dFIUAs7A?si=a4KYg6QEaxALHnWX&t=207

I've always thought Bill Gates was pretty smart, smarter than me, and has dedicated a lot of his time to researching things like pandemics. More time than I have dedicated to such research.

Seems like Meta's release of Muse Spark Open does not move the needle on things like this, since GLM-5.3 was already released back in August.

1. Not entirely, it has two degrees of freedom. Think of it like a bridge.

2. The opposing springs are normally in equilibrium, so in the absence of forces the center mass moves back to position zero.

3. Think of it like a bridge. The points of contact and support are to the sides and stationary, while the spring mass system is suspended in air or in a vacuum.

4. Is simple enough if you start with two disconnected layers of materials, and then etch the top layer. You could also start with a layer of something like photoresist spincoated on the bottom layer, except for the adhesion points...

An interesting real-life scenario was the possibility of helium atoms being able to get inside the MEMS clock of some smartphones and cause the phone to stop working until enough helium can escape.

It basically enabled the plot and mostly happened 'off camera'. The parts about the management of radiation was pretty cool though. Because if it I earmarked "Ichi-F: A Worker's Graphic Memoir of the Fukushima Nuclear Power Plant" to read later on.

It seems like you're talking about chemical weapons.

Which is fair, they are very easy to make without the assistance of an LLM, and I don't hear about that kind of thing in the news.

I was talking about biological weapons like gain of function and that kind of thing.

The implied question was 'Should companies like Meta publish open weight models when they could theoretically be subverted by malicious users?'. So far as I'm aware there are few people worrying about this kind of thing. Bill Gates for one. But apparently it's off topic and/or not interesting, or not welcomed as a discussion point.

Another instance of 'Super Ice' or as named in the article 'Pykrete' in science fiction is in Neal Stephenson's Novel from 2015: Seveneves.

The substance is used in space for the construction of temporary habitats. I suppose this is plausible if you manage to find a large source of ice and bring it into a stable orbit. It won't melt into liquid, and perhaps the reinforcing fibers could help prevent sublimation.

A lot of the novel strained my ability to suspend disbelief, but the overall scope and ambition is so awesome I was able to forgive the poor editing. It's attempting to be hard sci-fi, much of the novel is taken up by technical descriptions that could only appeal to an engineer-minded person.

Not at an appreciable token rate. I wonder if these large open weight models are considered safer to release for that reason? I'm thinking no because any small to medium size criminal organization or state funded group will be able to cobble together the resources to achieve some kind of runtime. Even at 10 tok/sec, making bioweapons at 10 tok/sec is concerning.

I tend to agree, it's not for me. Objective 'goodness' I will not assess. But it's barely sci-fi. More of a Eldridge horror book awkwardly set in space, with... vampires? Scientifically defensible vampires

It sounds like your regulatory environment has properly incentivized infrastructure.

Where I live in Canada, it's extremely unclear whether anyone is making their ROI on solar installations.

Does anyone know of some good scripts or workflows to install Ubuntu (because it 'just works' e.g. comes with a lot of drivers) and then scale back to a Debian-like experience?

I've had issues starting with Debian because it doesn't ship with broad support for wifi hardware, and internet over USB to iphone hotspot also didn't work. The only option was to get wired ethernet to bring up wifi.

Didn't Spotify also start by streaming a whole bunch of pirated music?

I am also very glad for Meta's investment into the future, so second your voice here. In a sense though it's Palmer Luckey who punched the tech into a defensible product. John Carmack surely tried to do the right thing, but I think part of why he left is that it's too difficult for any individual to make significant (risky) design decisions.

Do I have faith in Meta to 'win' in VR or AR commercially? Well... not really, but the technological advancements are real.

The AI generated character is a weighted average of many characters written by humans.

Which are not created from nothing. People write characters based on a combination of other fictional characters, real characters, and perhaps some 'RNG'.

Seems the distinction is that the AI generated character can not have any direct bearing on reality, because the LLM never got to know anyone directly. Not derived directly from experiences rooted in reality.

It's ARM.

I fail to see how they will take a significant market share or even break even on this venture.

It's an overcrowded market. Far better would be to focus on semiconductor supply chain, which Japan already supplies some elements, to sell to fabs.

Because the bottleneck is the fabs. If a new 2 nm fab came online today, it would immediately sell all its capacity to 2030 no problem, without Fujitsu trying this gambit.

It's not 'sovereign', the architecture is not designed in Japan, the silicon is not fabbed in Japan. This whole thing is sideways.

You have to load the model weights into VRAM over PCI-E (from RAM). So the (PCI-E) bandwidth strongly affects time to first token.

You have to run inference on the GPU by reading and writing to VRAM. So TFLOPS of the compute matters, and bandwidth to the VRAM (Always integrated with the GPU, rarely a bottleneck), and this strongly affects tokens/s

If you're doing training workloads or offloading to system RAM, it gets more complicated. (And mostly bound up trying to feed compute on time)

(Edits for clarity.)

People sometimes call EU the sleeping giant. Big potential compared to the actual results.

I'm more aware of what Canadian government does with its tax dollars. And I'm just assuming that the EU is similarly quite expensive for what it achieves, and the budgets only change in one direction.

There are deeper structural problems when you look at the incentives inherent to government driven society.

While I do appreciate and even support the idea that the immense wealth available on planet earth should be more evenly distributed, and people should mostly be focused on raising families, deepening their philosophy, and perfecting their craft... (some societies in Europe are pretty laid back in terms of hyper-productivity culture)

Regulation is possible without paralyzing industry. Laws about the internet are possible without stifling freedom. Central planning is possible without crashing the economy so bad you end up with a regime change. It all comes down to execution and for the most part, it's been a series of failed experiments.

As much as I want a happy future for Canadians and Europeans, I just don't see the odds being super favourable.

Manny comments I agree with - it's a mistake, it's a reason to leave the distro...

I'd like to point out that Ubuntu/Canonical is now seemingly philosophically separate from the origins of GNU and Linux. If you're curious, read "Hackers" by Stephen Levy. They've lost the plot, so to speak, about the core spirit of open software and hardware, which is what gave the projects life in the first place. A philosophical understanding and unity between many, many top notch independent developers.

Another note is how AI contributions to such libraries and programs is going to have an unknown effect on quality. It's almost like there's a business case to rip all the good FOSS written by humans out of the hands of github and apt, and curate all the best source code to ensure it remains in circulation, and extant copies are available that are not washed out by loose standards WRT AI contribs. Or if not a business case, perhaps a reasonable reaction and a good personal vendetta.

> "until the context rot and sampling problem is fixed forever"

I agree, prompt adherence seems to get worse when operating on large inputs. Does anyone have some notion of the SOTA with this? Can we expect big improvements by this time next year? (hopefully in open weights)

Your hardware can do way more than 64k tokens context window, can't it? And with Ollama it's very easy, superficially you just drag the slider.

I'm now reading "Friends Don't Let Friends Use Ollama" linked in another comment so a lot of problems with that approach are surfacing for me right now.

So yeah. Along with others, I think you should come up with some empirical means of understanding if your preprompt is doing anything good since I doubt that it's all necessary and helpful. Second maybe you and I need to fix our runtimes.

This is good guidance, but what do you have to say about convincing your team of developers to live it out?

I've found that developers usually like writing code and avoid contributing to documentation. For some, it's actually scary because (edit: for them,) high quality writing is harder than high quality coding, and it can be avoided quite a bit.

On the project side, it's rare for the implementation and verification stages to not consume all the budget and more, and delivery creeping past the original optimistic date. So there's no time or money to spend on documentation.

The combination is that even with your great advice in hand, it's hard to navigate to really solid and comprehensive design documentation underpinning the products.

It will happen eventually, especially with AI generated books published electronically. I wonder if there is a cutoff date or something to try and avoid this problem of low effort books.

If you've not read Fred Brooks MMM, give it a shot.

It seems like you're implying that teams of 60-80 developers should be expected to outperform teams of 12. This is simply not true. The most important feature of source code as a language is that it allows precise mindshare among close knit teams. It doesn't guarantee it, but it makes it possible for people to talk about the product at a level that is otherwise very difficult.

A convenient side-effect of the source code is that it instructs the machine what to do. But instructing the machine was never the bottleneck, the essential difficulty of software development is in understanding what are the correct instructions to achieve some objective, not typing them out.

The problem is that communication doesn't scale at all. Having just 3 developers with good alignment about mental models, best practices, and design direction is hard enough, and if you found the right three people at the right time with the right ideas, you could generate billions of dollars of value.

Large monolithic teams on the order of 80 are a product of people in control not understanding how software development works, and how to make it work well.

As a user of local models, does this mean that there are 'magic incantations' that can increase the performance of some local models?

I see some details about recovering information via whatever technique. It's interesting, but appears not generalized.

So for a specific question, yes, but this is not about techniques like adding a good embedding that just generally tends to improve open model performance on certain tasks.

When I look at the technology and capability of frontier models for ideation, brainstorming, and research, or when I look at the open source community and open weight models, and running local LLM's... I'm convinced that this is a technology revolution the size of the internet.

When I look at businesses trying to turn this research and technology into profit, I think that this is a bubble that will burst. You are not alone, I am very doubtful of the current concoction of AI features being offered. The true winners and legitimate applications will emerge over the next 30 years. Google, Nvidia, and Microsoft could all become IBM, Xerox, and Kodak. Or maybe they find a way to pivot. Because truly, there is an amazing amount of nonsense BS in the AI hype train.

Acknowledging the outside chance that recursive self-improvement works and we just scale directly into a Kardashev 1 civilization within my children's lifetime, and K-2 in theirs'. Can't rule it out no matter how hard I try.

honestly that would still probably mean more than story points. maybe all this AI insanity has an upside that the new cargo cult washes out the old one.

In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity. AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans. The problem is further compounded by the fact that searching and accessing the internet will be noticed by AI agents that will generate still more swamp content in response.

Unfortunately, it seems that this fiction ended up being prophetic. The open internet will fall to entropy, not legislation or one-sided international trade agreements. I think we need more projects like Anna's Archive, where the public uses torrents and distributed infrastructure to save and organize the world's information. Google has abjectly failed in its original mission to organize the world's information and make it universally accessible and useful.