Yea my m4 max with 128gb has ended up making a lot of sense for me. I do video editing, I train ml models, I run large open AI models, I do 3d modeling, rendering and cad work. I never do all of this 100% of the time, I’ll setup a ml training to run over night and check results in the morning, during work I’ll set it up as a server and run local models, on my own time I’ll edit video and work on 3d modeling. It’s an incredibly versatile machine - and all of this is done while keeping your data on your device and giving you full control over your workflows.
The really interesting thing that I think is going on inside of the DS4 repo is exploring all of the interesting knobs that frontier labs have hidden from users, and then thinking about how they can fit into real dev/interaction workflows. It's really cool to see different interaction modalities being explored and thinking about for example how steering can be worked into a user interface in a helpful way. I think that once the cat is out of the bag as they say, and users understand the level of control and utility they can get from models that are sort of turned inside out in this way, it will start to be an integral part of their tool belt, and it'll just make sense for this level of control to be expected from your models or model providers.
Just want to mention that I've been pulling down and using DwarfStar locally and it's incredible. I actually have it running on my personal macbook m4 max with 128gb of ram and I am running the server to share it through tailscale with my work laptop and just have pi running there.
The long context reasoning is something I haven't even seen in frontier models - I was running at 124k tokens earlier and it was still just buzzing along with no issues or fatigue.
I am amazed at how well it works, I'm using it right now for some pretty complex frontend work, and it is much much faster than, for example running a dense 27b or 31b model (like qwen or gemma) for me (The benefits of MoE) - but the long context capabilities have been what have been absolutely flooring me.
Super excited about this project and hope Antirez can keep himself from burning out - i've been following the repo pretty closely and there are a ton of PR's flooding in and it seems like he's had to do a lot of filtering out of slop code.
If you can’t be bothered to provide sources for any of your claims, when responding to an article that actually provided sources for its claims, we can just assume you’re making it up. When you become emotional when someone asks you to provide even a sliver of evidence, I have to assume you’re just spreading misinformation that aligns with your views. If you want to discuss in good faith, I’m here.
Just wanted to maybe make a light suggestions that, for marketing purposes, this really doesn't need any suggestion of TikTok and also might benefit from less heavy handed mentions of AI. I think it provides a real value proposition on its own without needing to rely on those two things to sell itself. They are pretty polarized terms at this point and I can sort of understand the initial revulsion from hearing TikTok next to scientific papers.
"In some instances, AI data centres are powered completely by renewable energy. Unfortunately, unless the data centre builds new renewable energy sources to supply 100% of its power, this still results in an increase in fossil fuel usage. Why? Because if the data centre is using pre-existing sources for renewable energy, it is taking that energy away from other consumers that need it. To pick up the slack, we must generate more energy, and most of that generation is done using coal or natural gas."
-- strangely I had never actually thought about this
Does anyone know if there is another printer manufacturer that has an equivalent to the Bambu A1S with it's custom AMS system. I don't think people realize how good that printer and AMS system is (the AMS system for the X1C pales in comparison), and I'd love to support another company, but haven't really seen another bed slinger with the simple center-rotating AMS style system seen on the A1S AMS. For context I run a business where I sell 3d printed parts for old film cameras - and the A1S is a workhorse.
I'm particularly interested in it being REALLY fast - do you have any rough tok/s numbers for the flash model? I'm excited for unsloth to drop some quants that I can try and run locally, but really curious how it's been performing speed wise. In general I actually over-index on speed over intelligence. I'd rather a model make mistakes quickly and correct in a follow-up than take forever to get a slightly better initial result.
This black box approach that large frontier labs have adopted is going to drive people away. To change fundamental behavior like this without notifying them, and only retroactively explaining what happened, is the reason they will move to self-hosting their own models. You can't build pipelines, workflows and products on a base that is just randomly shifting beneath you.
In what ways has fetch never caught up to axios? I have not encountered a situation where I could not use fetch in the last 5 years so I'm just curious what killer features axios has that are still missing in fetch (I certainly remember using axios many moons ago).
Sure, but Google also leaves little to no overlap between models and often will leave models in preview mode (which many companies cannot use in production for legal reasons) - right up until the point that the previous model is deprecated.
The point is that if you want to build a platform that customers can rely on based on their own schedules of feature development, you need to support models for longer periods of time. For example, OpenAI is still offering older models like gpt4 which was released in 2023 - this gives customers plenty of time to test, experiment and eventually migrate to a newer model if it makes sense.
Consistency, new models don't behave the same on every task as their predecessors. So you end up building pipelines that rely on specific behavior, but now you find that the new model performs worse with regards to a specific task you were performing, or just behaves differently and needs prompt adjustments. They also can fundamentally change the default model settings during new releases, for example Gemini 2.5 models had completely different behavior with regards to temperature settings than previous models. It just creates a moving target that you constantly have to adjust and rework instead of providing a platform that you and by extension your users can rely on. Other providers have much longer deprecation windows, so they must at least understand this frustration.
It's interesting that, of the large inference providers, Google has one of the most inconvenient policies around model deprecation. They deprecate models exactly 1 year after releasing them and force you to move onto their next generation of models. I had assumed, because they are using their own silicon, that they would actually be able to offer better stability, but the opposite seems to be true. Their rate limiting is also much stricter than OpenAI for example. I wonder how much of this is related to these TPU's, vs just strange policy decisions.
I use LMStudio to host and run GLM 4.7 Flash as a coding agent. I use it with the Pi coding agent, but also use it with the Zed editor agent integrations. I've used the Qwen models in the past, but have consistently come back to GLM 4.7 because of its capabilities. I often use Qwen or Gemma models for their vision capabilities. For example, I often will finish ML training runs, take a photo of the graphs and visualizations of the run metrics and ask the model to tell me things I might look at tweaking to improve subsequent training runs. Qwen 3.5 0.8b is pretty awesome for really small and quick vision tasks like "Give me a JSON representation of the cards on this page".
I'm crossing my fingers they release a flash version of this. GLM 4.7 Flash is the main model I use locally for agentic coding work, it's pretty incredible. Didn't find anything in the release about it - but hoping it's on the horizon.
I've had really good success with LMStudio and GLM 4.7 Flash and the Zed editor which has a baked in integration with LMStudio. I am able to one-shot whole projects this way, and it seems to be constantly improving. Some update recently even allowed the agent to ask me if it can do a "research" phase - so it'll actually reach out to website and read docs and code from github if you allow it. GLM 4.7 flash has been the most adept at tool calling I've found, but the Qwen 3 and 3.5 models are also fairly good, though run into more snags than I've seen with GLM 4.7 flash.
It's really easy to setup with any OpenAI compatible API and I self host Qwen Coder 3 Next on my personal MBP using LM Studio and just dial in from my work laptop with Zed and tailscale so i can connect from wherever i might be. It's able to do all sorts of things like run linting checks and tests and look for issues and refactor code and create files and things like this. I'm definitely still learning, but it's a pretty exciting jump from just talking to a chat bot and copying and pasting things manually.
Absolutely. So my codebase is huge, it's a monolith. But my work is in very specific parts of the codebase, I don't pull the entire code base into context (and I don't think that is common practice even with claude) - I start at a specific point with a specific task and work with the agent to achieve something clearly defined, for example writing tests, extracting things into separate files, refactoring or even scaffolding a new feature. You have to periodically start new threads, because you'll start hitting the limits of the context, but I max it out at over 200k because I have the memory overhead on my 128gb mbp to do that, so I can get quite a lot done.
I really recommend trying the Qwen models - 3 coder next is really incredible. GLM 4.7 flash is also incredibly performant on modest hardware. Important things to consider is setting the temperature and top_p and top_k values etc based on what is recommended by the provider of the model - a thing as simple as that could result in a huge difference in performance.
The other big leap for me was switching to Zed editor and getting its agent stuff just seamlessly integrated. If you run LM Studio on your local machine it's super easy and even setting it up on a remote machine and calling out to LM Studio is dead simple.
In my experience qwen 3 coder next is better. I ran quite a few tests yesterday and it was much better at utilizing tool calls properly and understanding complex code. For its size though 3.5 35B was very impressive. coder next is an 80b model so i think its just a size thing - also for whatever reason coder next is faster on my machine. Only model that is competitive in speed is GLM 4.7 flash
I use Qwen 3 Coder Next daily on my mac as my main coding agent. It is incredibly capable and its strange how you are painting this picture as if its a fringe use case, there are whole communities that have popped up around running local models.