Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console
github.com
github.com
Those projects should default to hosts less geared toward corporate governancy; custom internet domain being the first requirement.
That's one of the best things with git (among with other DVCSs), it is incredibly resilient. I remember during an interview of Linus Torvalds about backups, he says he doesn't backup his work in the traditional sense, he has no real reason to when everything he does is copied millions of time.
The real point of GitHub for such a project is because GitHub is really popular, and it has a social network aspect to it. So yes, you are going to lose it if you get kicked out of GitHub, but by choosing a lesser known platform you won't get it in the first place. Also, their website is not on GitHub, so kicking them out of GitHub won't make them disappear completely.
To your point, it was briefly possible to buy them for under $100 until word spread that they make decent steam machines, so I'm guessing that Sony doesn't need to sell any games to turn a profit. Even at their current prices it would be far cheaper to use the BC-250 for local inference.
I don't follow your logic. If the BC-250 was made from PS5 quality control failures, then why would their pricing have any bearing on the PS5's profitability, except as an effort to increase margins? Does cost not remain independent of pricing?
As far as I cam infer from what you said, it could be that the PS5 was sold at a loss and still is even with the BC-250 helping, or that it was sold at a profit and is still is with the BC-250 helping. What am I missing?
The only way they can lose money is if their existing users stop buying games and that's not happening yet. Maybe when vibe coded community servers for online games get better and everyone jailbreaks their console because they don't have to worry about getting banned.
But even with that every unit sold is a net positive for them because they make money off the sale, and if they run out they can use it to justify higher prices.
And nothing is stopping you from renting a VPS from a provider based in that country and running gitlab or something.
The worst part is sharing around a sketchy URL but, that just comes with the territory when you're in the DMCA dodging parts of the internet.
Renting a VPS from a country that genuinely doesn't is pretty risky for numerous reasons, least of all international sanctions.
Looking for similar UX to https://archive.softwareheritage.org/save/ (which archives an source control repo on demand from a url, but is still a single point of failure)
Private keys and private endpoints is all we need
>Radicle is an open source, peer-to-peer code collaboration stack built on Git. Unlike centralized code hosting platforms, there is no single entity controlling the network. Repositories are replicated across peers in a decentralized manner, and users are in full control of their data and workflow.
GitHub isn't just used to host your actual git work tree and everyone knows that, it's an issue tracker, an easy way for projects to get contributions as everyone already has a GitHub account, an easy way to receive bug reports, CI, a website for your project, a popularity metric like GitHub stars (For better or worse).
I struggle to think of anywhere on the entire open internet you could put something like this where it could not be taken down.
I hate the centralization of centralized git watering holes, but it is what it is. And frankly getting older and not as able to keep up, having all the cool projects in one place would massively help my street cred.
The problem, of course, still arises when you want to guide users to your hidden service, when sharing the link anywhere will get it taken down. This already happens with magnet links.
It's been proven time and time again that the only way to pull something like this off is to be completely untraceable. If the feds can ever find you, that's it. Which creates difficulty because in order for somebody to be able to visit your site with a normal web browser, they need to be able to trace something, and that can always be attacked. Even if that doesn't reach all the way to the source, it means you have to both produce infinite new traceable points and somehow distribute the knowledge to your clearweb users too.
> in order for somebody to be able to visit your site with a normal web browser, they need to be able to trace something
I've been thinking about these problems for a long time. A hard truth is that this statement applies to every protocol, not merely normal web browsers. For a TOR hidden service, there are six intermediaries that exist in the real world and can be traced one after another. The system relies on this being prohibitively hard for attackers (based on the rule of law with supporting laws, among other things).
The same truth even applies to content centric networking! You can go after whomever may be selflessly hosting a specific piece of content, and keep turning up the heat to crack down on it. Maybe you won't be able to find the publisher if they decide to lay low, but you can certainly make it so that a given piece of content is not easy to access - by accessing it the easy ways and applying outside-the-system pressure to whomever provides it.
The implication of this is that in addition to technical means, communications freedom also intrinsically relies on social means. Those social means are bolstered by most people having access to free communications, so they support the types of protections that allow the technical means to operate. Think of a hidden service working to make themselves accessible from the open web as public outreach.
I'm just pointing out the huge middleground of essentially legitimate activity that ends up bothering some entrenched US interests and inevitably gets squashed on Github. That dynamic itself ends up being a bait and switch that kills a lot of project momentum. It would be great if these communities developed a best practice of using some code host that simply shrugged off nastygrams from the western legal system.
- monitoring server error rates
- reviewing security footage
- watching plant growth / controlling watering
etc. Any long running task you can now just slap a PS5 on it. 70 cents a day in power cost means you now get ~2M output tokens per day (~$3.50 worth of usage). It's not the worst thing in the world if you have one lying around. 2bit isn't even that terrible - it's ~80% the accuracy of 27b at full quant, which should put it at around GPT-5 levels of intelligence (which honestly is crazy in its own right).
I'm sure someone will point out some spec that's better on the PS5, but the extra $200 will make up the difference if you spend more time looking than the 15 seconds I just did.
(or if you just build a PC from parts yourself)
They're not even in the same ballpark - the PS5 is more than an order of magnitude better in GPU performance.
Despite how much it does matter, RAM is not the only thing you need for running LLMs - that N150 is not going to run anything but really tiny models at any sort of reasonable speed - certainly nothing that approaches using all of its ram. The GPU in the PS5 is ~25x faster. And if you're running games the RAM doesn't really affect performance at all once you have "enough" for a given game.
You're right that you probably could build something quite a bit faster than a PS5 for the same price though - if you can build/find everything but the GPU for ~250 USD and slap a 16gb 9060xt (~25tflops @32bit compared to the PS5 gpus ~10tflops @32bit) in it for another ~350 USD it should theoretically do a better job - the GPU in the PS5 was on par with mid-high end PC GPUs on launch, but that was like 5 years ago.
[1] https://www.amazon.com/dp/B0GGM6BKN9 [2] https://www.amazon.com/dp/B0FX2TV3HX
Shameless plug - I methodically rank the best possible Mini PC deals on https://MiniPCs.zip for this exact purpose.
Ever since RAM prices grew 6x, these old appliances suddenly became a lot better value for money.
Tools like these will probably just spike the prices of second hand PS5s though.
9060XT: 128bit GDDR6, 322.3GB/s, 102.6TF FP16, 160W TDP
PS5 GPU: 256bit GDDR6, 448.0GB/s, 20.58TF FP16, 180W TDP
2: https://www.techpowerup.com/gpu-specs/playstation-5-gpu.c348...
And at that price point or a little bit more you might as well just get an RTX 3060 and save yourself a lot of hassle (you also get DLSS, DLAA etc.)
https://www.ncsa.illinois.edu/2003/05/27/playing-the-superco...
As sibling said, you won’t hit the same quality using a 2 bit quant like this project, which is unfortunate but that’s the reality today.
The ~issue is though that they're using a pretty low quant, 2 bit. 4 bit are _pretty_ close to full ability, maybe degraded a bit. 2 bit is _noticably_ worse. It's about as small of a quant as you can get while still pretending it works, for this model.
This won't replace claude for you, unless you baby it a lot and are okay with it being kind or derpy.
However you can run comparable or better models than gpt4. Try to stay under 30b
The readme oversells it by pretending it would be something actually usable, when instead it is a cool achievement to make it happen.