2,805 karma · joined March 28, 2014
Just for completeness: I presume that is referring to government owned hospitals in Australia, which are free. There are also privately owned hospitals. I've not had to wait at a private hospital.
Private hospitals are expensive, but I suspect not as expensive as USA hospitals. The price is held in check by the alternative of "wait a little longer, and it's free".
This wasn't "unwanted new features". Tridge was fixing a security issue, related to a bug report. I sympathise - we are all getting slammed with security issues. Fixing them isn't optional. I can't say I enjoy returning to decade old software to do it - so colour me impressed that tridge is putting in the effort.
I'm also guilty of using LLMs to help me get past this mess. I dunno what tridge is doing - but I check every line of code it spits out. Nonetheless, I have no doubt bugs slipping through is a real danger. I haven't looked at the code in a long while, I'm not as familiar with it as I once was. So a bug slipping through is not a big surprise.
Which brings us to the one odd thing about the blow up. The original complainer seems very protective of his backup system - yet tridge's commit was only 2 weeks ago. I know tridge is good - but surely you treat this as alpha software. What was he thinking? Maybe he has a bit to learn about building reliable systems himself.
However, I didn't learn much. Which means as far as I can tell, my mental model of how it works wasn't far off. So yes - I was already aware one interpretation of how these things work is that LLMs turn concepts into vectors in a high dimensional space, and high level abstractions are linear summations of these vectors.
Given that model, parts of your comment don't make much sense to me. For example "it's able to predict N different full paths WITHOUT actually exploring them fully" - why do you think that's so? And "GRAM allows it to look at the different hyper words and say" - no, GRAM does not look at different hyper words, or at least no more than a non-GRAM LLM does. Only the last word (the 4k vector or whatever dimension they are using) is fed back through the machine.
Regarding "Once decoded into a rigid real word, the multidimensional nuance of the continuous vector is lost!". Yes, and no. Yes, the decoded word doesn't mean much compared to the vector it was derived from. But the machine isn't operating on just the last word. It's operating on the information in the entire context window (which could be 100's of thousands of tokens).
There is undeniably a lot of nuance encoded in one vector, and yes you're right - it can't be represented with just one word. But it can be represented using a string of words, and generating that representation by spitting out more words is partially what an LLM is doing when it generates text. It's only partially doing that because it's randomly mutating that last token as it goes, and pulling in information into the vector from the MLP layers.
Re "LLMs ... add so much more meaning to a word than a human ever could imagine try to put 10,000 dimensions on the word "the" .... OBVIOUSLY makes them enormously less intelligent!". No it doesn't obviously do that. A vector is maybe 16k bytes (depending on the number of dimensions). That corresponds to around 5000 words. Humans have no trouble connecting those 5000 words into a single concept - which would presumably spell out the concept represented by the vector. Same meaning - just encoded differently. Using computer science terminology - we could say the 16k vector is serialised into a sequence of words.
So - two representations of the same thing. What humans do that LLMs can't do right now is squeeze those 5000 words into something tiny. For example, the word "LLM" is a huge concept, squeezed into 3 letters. The human knowledge and thought seems to be based on that one trick - naming abstract concepts, and then using them as building blocks for more abstract concepts. LLMs meanwhile are stuck with their fixed size vectors. They cannot add new concepts to their vocabulary by modifying their weights. Where LLMs seem to win is their short-term memory (of the order of 200K tokens), and they are about 1 million times faster (cycle time of the order of 1 nanosecond vs 1 millisecond), which gives their ability to reason very different properties to human reasoning. Sometimes this means they are (dramatically) better, and sometimes they are worse.
I don't see how GRAM on its own is going to make LLMs 3 orders of magnitude faster than they are now. That 200k token context window is hideously expensive and maintaining it grows O(N^2). As you observed, they can already compress a 100,000 word book into the single token encoded in the last word (although beyond 100k words that compression starts to look increasingly lossy). To get the 3 orders of magnitude speed up, they are going to have to start taking advantage of that compression, and start throwing away the part of that 200k context they have already encoded. So far, no one has deployed something that does it well.
- Avoiding building something that turns the universe to paper clips in order to satisfy a prompt is a problem they are genuinely struggling with now.
- They do it by spying on the words generated during CoT. "I can do this quickly by turning the Universe into paper clips. Wait - they won't like that. But there is no need to mention it." - SMACK!
- But you can speed things up immensely (3 orders of magnitude!) by skipping the output layer (and I guess compressing the context window / KV cache, otherwise 3 orders of magnitude seem impossible) which would give someone who pulled it off a huge advantage.
- Downside is humans can't see the CoT anymore, so they can't see what the machine is planning. Keeping the final output layer to spy doesn't work because the model uses its hidden reasoning to sanitise it.
How can this possibly go wrong?
- "travelling": why didn't you postal vote?
- "medical": we allow pre-voting; was it planned?
Just paying the fine without arguing on time gets you a 50% discount in state voting. And it's a token fine - $20 or so from memory for federal ballots. Besides you don't have to vote. The requirement is you turn up, or give them a piece of paper if you post it. This is deemed so important when we designed our own voting machines (which were never deployed), they had an explicit "I decline to vote" option.
The paper can be blank, but people are often more imaginative. I can't find the reference to it now, but one paper had penises of different lengths drawn beside each option. The Australian Electoral Commission is required by law to "save" votes, which means that even if it wasn't marked strictly according to the rules if a reasonable person could infer the intention, it counted. This particular vote worked its way through the courts, where it was eventually struck down. Reason: it was impossible to know if a longer penis meant it was more or less favourable to the candidate.
About 8% of ballots can't be saved. Of those around about 2% are deliberately spoilt - the rest are mistakes. If Vanessa Teague's voting machines (with the decline button) had been deployed, the remaining 6% would have gone away.
Australia would like to remind everyone we have compulsory voting.
The Gemini Flash is very good at searches. Just about any low end model can toss out a poem. All the higher end models (open source and otherwise) seem to be able to churn out code that passes tests. The smaller, "less capable" ones are much faster at it, which means in the hands of a skilled practitioner are the best choice for that task. But they rapidly fall apart where there isn't a hard source of truth (like a good test suite) to grind against. Because of that you have to use a bigger model for bug finding. In that task the open source models tend to fail on larger code bases, where something like Opus still shines. I gather Mythos is an absolute monster, and unparalleled, and unavailable. I'm sure one of the reasons for that is it's so expensive to run.
Or to put it another way - you don't use a 100 tonne crane to pick up the shopping. And ... the smaller models will happily run on in-house hardware. You may not do it today because of the current DRAM price and integrated NPUs have just started shipping, but in 5 years time models will be running on your phone.
No argument about the point of the issue. But this is a discussion about the relative efficiency of C, C++ and Rust. My point is there is a way in Rust to say "I don't care about observable writes, hoist the bounds check out of the loop", so that the efficiency is the same.
Admittedly, it's not part of the language definition. You're relying on intimate knowledge of how the optimiser works. In fact, you are probably pasting the code into godbolt, and looking at the assembler produced. But if you care about cycles that much, that's true for all three languages.
assert!(a.len() >= 32);
for i in 0..32 {
a[i] = 0;
}
for i in 0..std::cmp::min(a.len(), 32) {
a[i] = 0;
}
As for the speed comparisons with C++, the OP says at the end you tell the C++ compiler to be as strict as Rust using "-D_FORTIFY_SOURCE=3 -fsanitize=bounds,object-size" & hardened STL, then it slows to below Rust speeds for the same safety unless you use the same techniques.
It's a shame the other optimisation techniques you need to bring Rust in line with C++ aren't as easy to apply.
Permit me to paraphrase slightly. "It is not worth switching to Linux except as a hobbyist. Something that is overlooked: the mainstream OSs have a huge advantage ....".
You are in good company. In 1999, Bill Gates confidently dismissed Linux as a threat, arguing it lacked the central control, features, and graphical interface needed to compete in the commercial market.
Back to the article, quoting:
> Pi might be built with Pi, but we’re quite far off today from where Bun and OpenClaw already are: fully detached, automated software engineering.
Please don't call it software engineering. I've been programming for 40 years, and most of that time had to put up with the derision from the other engineering disciplines: "If civil engineering built things like software engineers, the first woodpecker that came along would destroy civilisation". It hurt because it was true. It's still often true for things like web pages, but for the things I use like Linux and vim, it hasn't been true for a long, long while. We have finally mastered how to repeatedly build solid, reliable software.
Which is why I'm an Anthropic refugee. Opus is definitely the best for coding, but claude-cli + bun is the most unreliable piece of crap I've had the misfortune to come across in a while. Sadly I can't afford their API pricing, so either my principles or Opus had to give. I went to pi and an open-source model. The difference between the top open-source models and Opus are noticeable, but not drastic, unlike the difference between pi and claude-cli.
pi has proved to be solid, fast, have a transparent design, and be customisable in the old Linux way ("do one thing, and do it well"). I pray that will never change.
It's not far from true. The fights you get into with the borrow checker can be legendary, but lifetimes serve more as gentle reminders. If you get stuck, you can always just use Rc, which is pretty close to opt-in GC. But it's rare to have to resort to Rc, because ownership is just not that much of a problem. In fact, I very rarely use Box either. All heap memory allocation is done by containers, not manually by me. I guess the main friction point for lifetimes is Rust's closures and async, but if you avoid them life is pretty simple.
In return for wearing this almost not a problem, you almost don't have to think about releasing a whole pile of other things - like closing files, sockets, and locks. They are guaranteed to be released by the same mechanism.
On balance, I would not be surprised if the cognitive balance tips Rust's way once you allow for the fact that Rust's memory management also gives you robust resource management for free.
It's a similar dynamic here in Australia, but it seems not in Germany and Japan.
I think the difference is that in the USA and Australia the unions are organised on a craft or occupational basis. You see this in the sorts of laws they want passed - they are invariably after laws that only union members can do a certain job. It isn't always so direct. Doctors for example insist on certain qualifications, which seems fine and necessary at first glance, but then the doctors' associations somehow manage to gate the institutions that can issue the qualifications. It's interesting how doctors have no trouble with that, but computer engineers can't bring themselves to do it.
Anyway, the outcome of "guild based unions" ends up being what others here noted. Instead of everyone in the same firm cooperating to get work done, they are all fighting to preserve their patch. In Germany and Japan, unions look to be organised around large companies. If the company goes broke the union disappears with them, so the company's incentives and the union's are more aligned. So much so the union reps are given seats on the board, and are expected to make a contribution. The unions are still focused on ensuring the distribution of profits goes more towards the employees than the shareholders of course, but nonetheless the outcome seems to be far less dysfunctional than the USA and Australian systems.
His plans for when someone else inevitably has control of the executive branch boggles the mind. Does he have no foresight whatsoever?
I've come across others who live only in the present. I recall an inmate breaking out of jail a week before his release because his sister was sick and needed help. They are a rare breed. Life can't be easy for them. I would have thought it was impossible for someone like that to ascend to the level of President.
Yet even now when a disaster is looming in the mid terms, he manages to keep the Republican party cowed. It is something to behold. I still have no idea how he does it. Do these people have no sense of self-preservation?
Which makes it somewhat less than iDeal for anyone who isn't Dutch. The magic of Visa and Mastercard is they enable commerce between two people, even if they bank on different sides of the planet. Well, not Russia - but they do work in Japan, and if you ever dealt with the Japanese banking system you will know that's a minor miracle.
That's an open question for models produced in the USA. But for models produced elsewhere, the answer is pretty clear.
They are about $50 in quantity. The surprise isn't that isn't happening now. The surprise is it didn't have a couple of years ago. It entered mass production 5 years ago, before the war started.
[0] https://hailo.ai/products/ai-accelerators/hailo-8-ai-acceler...
It looks like it appeared that way to Trump. But you make it sound like it appeared that way to most people. As one of those "most people", I can say that's wrong. The reaction of most people was "WTF is Trump thinking?".
It's been clear he's not the sharpest tool in the shed for a while. But he should be surrounded by very bright people for are able to provide frank and fearless advice. Looks like he fired most of those people, and whats left have been cowered into sycophants.
I dunno. Yes, the stock market dictates they get a return. But the return they get on that R&D isn't easy to determine. The return is high now. When you go from the previous best model only being able to find a handful of security issues, to the next model producing a flood so large Linus is screaming rage into the void complaining that his legion of helpers can't handle the flood of new CVEs (but he can't really blame anyone for this), development is going very fast. They haven't monetised the ability yet, but really does critical infrastructure have any choice now other than to get their entire codebase vetted by Mythos? How much is that monopoly worth? (Granted - it won't last long.)
Their R&D is aimed at discovering - well I'm not sure what you would call it, but it's the same thing that allows a mother to take a glance at her child and know what's troubling them, or a senior engineer to glance at code and spot a bug, or a same thing that a chess master to look at a chess board to produce a move in a second or so that defeats all but the best chess players. There are lots of these areas of expertise to be discovered. They are worth a lot of money. When does it stop?
A hyped exploration of this by bloomberge: https://www.youtube.com/watch?v=JmFKaqJg5X4
Others have posted links here saying inference is about 30% of Anthropic's spend, the other 70% is R&D - things like developing the next model. If you take that into account, add a profit margin, and round to make the figures easy you end up with them selling tokens at $1 / M Token to make money.
Their API cost is currently $25 / M Token. There is no question that's profitable. Someone really pushing their $200/mo Max plan can use 10B Tokens per month, which works out at $0.02 / M Token, so they are eating a huge loss there. That is clearly going to go away at some stage. For the rest of us: look at your average monthly token usage. If you are using 100M Tokens a month with the Anthropic $100/mo plan, they are making money out of you.
You have to be working very long hours, and be really, really proficient at using AI to achieve 10B Tokens a month. The only way to be that proficient is to have been using it for a long time, years in fact, so it was useful to you long before Opus came along. You would be very disappointed to lose Opus, of course - but you are just the sort of person who can make a less capable model sing. It's not so difficult to see those users moving to an in-house-hosted, open-source model in a few years, and it will cost them what they are paying now - $0.02 / M Token.
All that means I'm not convinced by the gloom and doom vibe of the article. Things will change, but it won't mean the end of AI usage.
In my search for alternatived I stumbled across https://passbolt.com/ AGPLv3 and does support sharing single secrets, but no free hosted version. Free if you self host of course.
It guess it's a vaultwarden without "the man in Nebraska" problem.
Interesting. So Urkaine and the EU have been denied USA weaponry, and yet Ukraine is turning the tide of the war anyway.
I guessing this isn't how the current USA administration expected that would turn out. I imagine they thought the collapse of Ukraine would reinforce the USA piston as leader of the western world.
Judging by the West's silence when Trump appealed for help with Iran, that isn't what happened. Instead it looks like the Wests reaction to Trump's USA threatening to the leave their club is "don't let the door hit your arse on the way out".
Urkaine successfully waging war with weapons that cost several orders of magnitude less than what the USA is offering probably has something to do with that, particular given they are right next door and will be greatly appreciative of the EU's support for them through the war. Friendship and helping a mate out in a time of need brings it's own rewards.
Just like renewables, EV's and a lot of other technologies, the expensive toys the US arms producers are hawking are starting to look like relic from another era.
The primary difference between a Chromebook and a Googlebook appears to be the ability to run LLM's locally.
The requirements were spelt out at Google I/O. They boil down to a 40 TOPS NPU and a minimum of 16GB of memory. They appear to be trying to match Apple's M series memory bandwidth using software compression. ChromeOS didn't need an NPU and specified a minimum of 4GB of memory. Aluminium OS looks to have the same relationship with its LLM as a Chromebook did with Google Chrome, and needs the hardware to power it.
If they pull it off you will get GPT-4 performance, running locally.
As for who this is for: your guess is as good as mine. But if their replacement for crostini works (crostini is so hopelessly unreliable it felt like it never got out of beta, so it's a big if), even the minimum specs would be a very good Linux laptop.
Not being reproducible is a bug.
There is no reason for a build to not be reproducible, but somehow we let the built binaries become infested with timestamps, login names and file system paths. We recently moved to reproducible builds at work, and discovered that our login names and local home directory paths were being shipped in every release. No one was was very happy about leaking PPI like that.
You may not consider it worth the effort, but you aren't the one putting in the effort so I'm not sure why that matters to you. It is very much worth the effort to those people doing the work. Debian is a do-orcracy and so the people doing the work get to make the decisions.
You underestimate your own abilities. Tech savvy doesn't mean they think much about crypto.
To get a feel for this I asked Gemini "If you were to survey a group of people who would be called "Tech Savvy", what percentage of them would be aware you could construct a zero knowledge proof for a person's age that revealed nothing beyond they were older than a given threshold?". The answer was 5%..10%. That rises to a surprising low 20%..30% for Software Engineers. It's only once you get to Software Engineers who write security systems that you get above 50%.
Gemini didn't give any references so those figures could be complete rubbish, but in my experience they seem on the high side. Many very experienced engineers I interact with clearly have not thought very deeply about how crypto systems interact with human trust. Granted understanding the implications of crypto is yet another step beyond understanding the maths, but I'm amazed at how many technology curious people haven't bothered to take that step.
The good pollies on the other hand probably have a very good intuitive feel for human trust systems and how to navigate them. They rely on engineers to tell them what is possible of course, and they won't care about the details. But what they will care about is whether the engineers can deliver the system they promised, and there I have to admit our track record is appalling. How many government IT initiatives have you seen deliver what was promised on time and on budget? So when you tell them you can build a ZK system that delivers in all these privacy promises, expect a very sceptical reception.
True for age verification, but not true in general. If you have something that can be used illegally, it's very handy to allow firms to rent / hire it out anyway but make the hirer responsible for any illegal activity.
An example is hiring a car, and the car is used to ram-raid a shop. Today this is solved by handing over a government ID to the rental company. Commit a crime in the car and they hand that over to police, but it has the sad side effect of handing over information to the car rental they can use to track you, and worse sell to others.
Using a zero knowledge proof for a valid driver's licence fixes the privacy problem, but at the expense of the hire company not being able to transfer responsibility for illegal activity onto the hirer. I suspect if that happened no one would hire out cars any more.
You can easily design something that is Zero Knowledge to the car hire firm, but includes an opaque token they can hand over to the government on lawful demand. It contains all the details needed to pursue the law breaking hirer. Thus there is still a role for the law here - you can't always do everything with crypto.
This is a very minor quibble - I agree completely with what I think is your main point. This Google change is a privacy disaster. It's a step towards an enshittified internet with the gateways onto it controlled by a few big tech firms.
But I don't think just yelling "just use ZK" is helpful. It's much harder than that - ZK is only part of the puzzle. Passkeys are currently caught up in the same attestation trap, and there is no workable solution in the offing. Banks and other high trust applications need some assurance your FIDO private key is being handled securely. The solutions on the table are Apple not doing attestation, or Google who does at the low low price of selling your true name to Google. Both "solutions" suck, horribly.
ZK proofs of things like licences and age have to solve the attestation problem, and solve extra stuff as well. I'm not holding my breath.