I'm incredibly lucky (even in the US) that I have symmetric 5gbit connection and can go up to 8gbit if I wanted (but I don't have everything wired for 10gbit so it doesn't make sense right now). I'd love to see more of that in the US, but know it's not happening anytime soon.
In many states voter lists are publically available online. And those same databases will tell you what party the person voted in for primary elections. The amount of information publicly available about everyone in the US is astounding.
I really haven't seen any 16gb Mac Mini cluster setups running large models at any appreciable speed for the reasons previously provided. Do you happen to have examples that folks are actually using?
That's not quite the case. Effort and focus by amazing groups, governments and individuals are the reasons for improvements, not just that producing more or costs being driven downwards.
Innovation with AI is happening as well, but it obviously will need more time. It's already being used to diagnose tumors and disease earlier in the process, which will improve outcomes for patients, but obviously it's going to take time to see the results of those efforts.
I think of AI (especially self hosted) as the most helpful computer assistant that can do things I otherwise wouldn't dedicate time to do. I have a massive media library that is a mess of naming conventions and want to clean it up. Writing specific scripts and processes for doing that is possible, but I don't want to invest the time. Or I can ask the local Qwen model to go through and do it for me with a single prompt, "Please rename these files in each folder following this specific format, and when needed access this local service to look up the additional information needed" and then extend that to when I dump new media in, it's automatically organized and renamed. Works pretty darn well and I spent a whopping 5 minutes getting it going, rather than the hours of writing scripts and doing it by hand.
I think a lot of it is because the majority of improvements are not seen by those who already comparatively have so much. AIDs used to be a death sentence, and now it's a manageable disease, and there is a lot of work going on to vaccinate against it. These are massive things that I think a lot of people ignore. It's always easier to see when things are on fire rather than to celebrate the victories along the way.
Do you think monthly AI 'subscriptions' are going to be $100 a month in 5 years? These people using these would probably be on $200/month subscriptions and with that OPs assertion of 'shoving away and paying with interest' makes no sense.
A Pi isn't a NAS with an old CPU and minimal RAM which is running a bunch of other things as well. I have run Plex on my Synology, and it's fine, but I also have 8gb ram and honestly always have a better experience running Jellyfin on a small PC. Easier that way, and you can run other services. Before the rampocalypse you could find mini-PCs with 32gb ram and 1tb SSD for like $350, and have great transcoding ability when you are on the go or have multiple streams going.
I have had an impossible time getting 120B or better models running on Strix Halo (especially under Windows) with any large context windows. And 30-40 tokens/second is fine, but not the fastest.
For the most part lately I have been sticking with Qwen 3.8 27b and that thing will easily suck up 64gb of ram. Add in docker with some additional programs running and it's really easy to eat up 128gb of ram.
Aren't you kind of stuck with smaller models on the mini though? Even with the pro you'd be stuck with a max of four daisy chained over thunderbolt and with the 160 gig memory bandwidth you'd probably be far better off with other configurations.
As long as everything is using 10gb, faster transfer speeds when moving data between computers. For downloads, it wont help unless you have 10gb fiber, but for most folks, 2.5gb is quite fast. Hell my spinning media NAS has a hard time saturating when moving files between internal servers.
I have had a difficult time with running 120b models on my 128gb setup, especially with any larger context size. The 6bit of Qwen 3.5 is already just over 100gb, and when you go down to 4bit it seems a bit lobotomized.
For me it's be Strix Halo, 128gb machine, especially running Qwen models. Except when I bought it, it was $1,900, now it's $4,600 for the same box. (Wow that's insane)
For tinkering and learning, it's been great. Tie it into something like Hermes and you have a pretty powerful AI assistant in a box. And when you need to step up your model, you just do something like OpenRouter and it makes it pretty easy.
How are you liking Omarchy? I saw a video on it recently, and it looks 'pretty' but still looks like it's a lot of memorization of shortcuts and feels like the 40% keyboard of OS's. Like some people it's absolutely amazing, but lets be honest, it's going to be really difficult to be as productive as a full fat keyboard.
Not 'more simply', there are basically zero savings accounts that are going to net you a 5%+ interest rate to give you that $100 a month. And that $25k becomes less valuable over time. $25k is now only worth $19k because inflation.
Sure, when you look a little wider, since 2000 we have seen the following major improvements:
- Extreme poverty has dropped from 30% to under 10% globally.
- Child mortality rates have dropped in half
- Internet access has exploded from 10% to 70%
- Solar energy costs have dropped 90%
- Cancer death rates have declined by 30%
All of these massive improvements in less than 30 years.
While there certainly are issues to solve, and if you simply follow journalism you may think the world is worse off, but for many, their lives have been significantly improved.
Big ole pool of very fast ram that can be accessed by the CPU and GPU. Lets you run larger models. AMD does the same thing with Strix Halo. I have a 128gb machine at home, and have had difficulties running 120b models, but 70b and below run pretty well.
At least in the US it's not even a good method either. Own a house? Boom, your information is publically available and you won't be able to remove it without a court order, which in most states is impossible to get. In most states, you don't even need to own a house, decide to vote? You information again is publically available. Tie your real name to a single account to purchase something, and you are trackable. Want privacy? You'd need to remove yourself from the internet, dump your phone, destroy your ID and work underground for cash. We can push against iot, and people have been forever, but at the end of the day the government will always they have a vested interest in being able to identify it's citizens.
It's not sad, it's always been the case, and just expanding areas that are implementing it. It seems a lot of people here haven't travelled, or especially haven't travelled to 'restrictive' places in the past.
Yep, travelling outside your country of origin, expect that your phone/laptop/ect is subject to search. Anything on you is, it's literally stated. Want to push against that? Sure, makes sense, but just carry a burner phone/device and not worry about it. Literally what the US government recommends when traveling to places like China.
The US being the 'free-st' place on earth at anytime is quite the misnomer. Sure, you can be 'free' if you are white, wealthy, oh and land owning. Tale as old as time. Sure, you can protest, but many many other countries have far surpassed the US on that one, and have been doing it a whole lot longer.
I want to use this as internal AI guidelines, I am so tired of people in my workplace simply copying and pasting whatever garbage AI gives them and expecting everyone to just go along with it and try to decipher what crap they were provided.
You could say that same exact thing for the entire Industrial Revolution, but that doesn't mean we are going to destroy the looms even though some tried unsuccessfully.
Sure. Realistically you simply regulate them. Espresso martinis are already long a popular 'after dinner' cocktail with the assumption that you'll be buzzed from the caffeine and able to drive better.
At least for the most part, if you are downloading from huggingface, you should be able to saturate your connection. I know I usually can pretty easily even with a 5gig connection at home.