Tuna-Fish

Learning the basics of how LLMs work is legitimately doable in a weekend if you have ever passed a linear algebra course.

If you are actually good at linear algebra, it's doable in an afternoon.

They are fantastically simple in structure, all the intelligence is in the massive piles of data in them, not their structure. Then there is of course a lot of room in optimization once the basics are done.

If interested, maybe start here: https://www.deeplearning.ai/courses/how-transformer-llms-wor...

Too bad for all the horrible misfeatures that got designed in and that we still have to live with, like the entire I/O Manager/Object Manager/filters/driver stack mess, with no caching anywhere above the drivers.

Distilling just works too well. It costs much more to aggregate the data in the first place and train the first model, but once that exists and you are selling access, the second guy can train their model by using the first model to generate the training data.

The only ways to fight this is trying to either: 1. Block it by cutting off access, which is pretty tough because it basically means you cannot offer unmonitored access which is a no-no to a lot of customers, and even then the workload looks very similar to actual real workloads and if the company doing the distilling is a bit sneaky and using intermediaries etc it's almost impossible to stop, or:

2. Try to use IP law to block the practice, which is pretty gross given how they got the data in the first place, and so far no court has supported them on this. And even if they can make it illegal, it won't actually help because there are countries that see having top-tier AI as a national priority and will actually support their companies breaking laws to get there.

People misunderstand why Anthropic is being so stingy about granting access to their top models. It's not some kind of marketing ploy, they know that if they release them fully openly then there will be a Chinese open-weights model in 3 months that is just as good. Also the reason why they try to hide chain-of-thought and the probability distributions.

SRAM is not very dense.

IMO the next big shift in AI hardware is when some of the upcoming logic-compatible memory types mature. Inference inherently likes many small local pools of memory, with their own interface each. This is impractical with currently available memory other than sram.

it's the widening (for example: fp16 * fp16 = fp32) instructions, which are useful on some loads.

What's actually used is banks of old, used phones with broken screens, controlled via usb. They can be had for rather cheap.

They care tremendously about virtualization security, when the threat model is one of their tenants attacking another one of their tenants.

They care not one bit about virtualization security when the threat model is themselves attacking one of their tenants.

One of the 384-bit keys was literally 4. All of them fit in a byte.

Yes, it more efficiently utilizes the land. But, if you are not the person who owns the plot that gets developed, more housing being built next to you harms you.

Vulnerabilities are typically fixed, and then reported.

The "hypothetical people" are in actuality all the other voters in your country. Lots of people are "pro-development, just anti-development near themselves".

Do you think that the Japanese system where permitting is centralized away from local government antidemocratic?

Yet less than 20% of the 80+ major vulnerabilities in confidential computing have been found by cloud companies, with the majority being found by universities and independent researchers.

It's not actually in their interest to look too closely. They know they are selling snake oil, and they do not care if it works. The only function of the technology, as far as they are concerned, is to be something they can market to customers who are scared at running on public cloud, and either they never intend to exploit it, in which case it doesn't matter if it doesn't work, or they do, in which case they don't want the vulnerabilities to be found and fixed. Either way, there is no upside to them for going spelunking.

> I think homeowners are too close to the problem to make rational choices.

The homeowners are making rational choices. That's the problem. As the most expensive asset a typical homeowner will ever own is their home, it makes rational sense for them to support any policy that supports it's value and oppose any policy that undermines it's value. Building more housing undermines it's value.

And this is why the local community absolutely should not have any more say about a new development than a hypothetical someone who would like to live there, but can't, because there is no available housing.

It doesn't need to run indefinitely. I don't think anyone sane thinks it's going to run indefinitely. It just needs to run long enough. Moore's law was what, 50 years? If AI keeps improving at the rate it currently is for 20 years, the world will be very different when that curve flattens out, in a way that makes most of our current concerns are entirely irrelevant.

AFAIK, there has never been a memory encryption scheme that successfully protects the VM client from the hypervisor, they've all been found to be defective within a few years on the market.

It is probably an useful layer because some of the attacks are not feasible for a remote attacker who has compromised the host but require actual physical access. But the marketing spiel of protecting your data from Amazon itself is just lies, physical access is more powerful than root access, no exceptions.

Either frontier AI will continue to get better steadily, until it's better than people at basically everything, and very weird things start to happen, or, it plateaus and the open weights models will eventually catch up.

Full mask sets for leading edge nodes cost tens of millions these days, and it takes months from committing to the design to actually getting the silicon in hand.

Companies that design CPUs today commonly simulate the designs on massive server farms to find bugs before they commit to building masks. Running purely on CPU, such simulations of leading edge chips struggle to reach 1kHz, which is a problem given how long it takes to just boot an operating system these days. (And this is at full scale-out -- dedicating sufficient compute to the problem that it no longer gets faster if you use more, due to communication latencies.)

Because of this, it is now common to use FPGAs to emulate better understood parts of the circuits, to get better speeds and do more validation work in software before committing to the masks. This can get them up to a MHz.

This is the kind of thing where everything will be fine until it rm -rf:s everything you care about, and then you will be shocked that it never did that before. This has already happened publicly to hundreds of people, in private probably thousands. Be better than them and learn from the mistakes of others.

The actually sane default is to treat it like a junior trainee you would never trust with rights to anything critical. Use existing operating system facilities, namely user accounts and file permissions, to make everything it doesn't need protected from writes, and everything it shouldn't blithely broadcast to the world like private data and keys, also protected against reads.

The fingerprint is a visual hash, which is actually a low-resolution grayscale "thumbnail".

This is actually a terrible idea, because to keep the air breathable you'd have to raise the oxygen fraction, making fire risk much worse. (Inert gases in air suppress fires by carrying away heat, making it harder to sustain a flame.)

If your AI agent has the privileges to delete prod, the problem is not that it is stupid.

Yes, current gen hardware was not built to run such models, and are obviously inefficient and bad at it.

Significant investment is being spent on producing hardware that is more suitable for running mid-sized (100B-ish) models. Starting with flash chips with wide enough interfaces to provide the weights (NAND flash can be ~5x more energy-efficient for reads than DRAM), and going into more exotic solutions like compute-in-memory.

Just die.

That's not actually a problem, because dangerous GRBs are simply not that common. Earth gets hit by one every day, but, that includes GRBs from very distant galaxies that are so weak when they hit us that they would not be any danger even to a completely unshielded person in orbit. GRBs of sufficient power to be an actual threat to people in a tin can is one/millions of years, at which point it's a propulsion problem again.

Yes, there are lots of things that could kill the ship on route. No, none of those are actually a significant problem compared to the basic "how to get there before everything breaks down from internal issues".

Fusion is ridiculously efficient and effective power source. The problem with it is that it is very hard to created conditions where fusion can happen. The one reliable way we have that doesn't take more energy in than it outputs, is to detonate a fission bomb and use the x-rays to compress a fusion secondary.

The history of fusion research for the past 70 years is trying to figure out a cost-effective way to trigger enough fusion to make net energy without a nuke. So far, we've got bupkis. (The magnetic containment approaches using high-temperature superconductors seem promising, but we won't know they actually work until we've built them.)

The people running the company are people who lived through the tech crash, and saw how quickly the entire economy around them can evaporate. At Sun, only slightly exaggerating, they went from flying high and controlling a market segment, to seeing all the demand evaporate and customers vanish, leaving the company holding the bag on just massive piles of inventory, basically in a day.

When you sell shovels, even slight downturns are massively exaggerated because the shovels can be resold, and if some of their customers go under, it's not just that demand that goes away, it's all the demand that can be satisfied by their customers purchasing second hand servers from the bankruptcies instead.

No-one can really say how long this bubble will last. Might pop tomorrow, might be a decade. But if Oxide funds inventory on debt, and the bubble pops, they go with it. Whereas if they fund on equity, and it pops, they will come out the other side just fine. The cost of this is that the old stockholders have a slightly lower upside.

Note that the LG takes regular screenshots, uploads them to their servers, and sells them. If you don't allow it on the internet, it saves the screenshots to internal storage and uploads once it is connected. If you don't attach it to the internet, or give it permission to upload them, and later sell the screen to someone who does do that, all your data is uploaded then.

I would not resell a LG tv that has been used for anything but content consumption.

There is enough humidity around the Nile, that is, the places where people actually lived, to rot papyrus. We have some remaining papyrus archives, but those are exceptions, not the rule. There is absolutely nothing like the library of Ashurbanibal.

Papyrus was amazing material for the time, but it was also a curse on future archeologists. The Achaemenid Persians went from being illiterate, to writing on clay, to capturing Egypt and writing on Papyrus, in one human lifetime. We have more surviving Persian writing from the short time they wrote on clay than we have of the two hundred years of their history after that.

The fibers don't need to be long. About 1 inch is enough for any purpose. The trick is to twist each fiber individually in one direction (fe. clockwise) while twisting them around each other in the opposite direction (fe. counter-clockwise). This creates internal tension that makes the fibers push against each other with substantial force, creating sufficient internal friction for them to bind against each other. And now you have string, and you again switch twist direction to bind multiple strings together to make thicker string, a switch again to bind multiple of those together, etc, until you get to proper rope thickness, which is basically always substantially thicker than the length of the fibers that make it.

It's a bit magical when you figure out how to do it, it doesn't seem to make any sense yet it works.

Actual databases pay up to two orders of magnitude of speed for durability. If you can regenerate your dataset in case the DB drops it (or there is a power outage, or whatever), being able to complete a complex, write-heavy query in 10s instead of 15 minutes is actually very useful when doing analysis.

There are many mosquito species in competition for the same resources, only a few (iirc 10% in the US) of which sting humans and carry diseases. Wiping out the ones that attack people would allow other mosquitoes to take over the ecological niches.