oofbaroomf

A little bit unrelated, but the rise of Transformers in 2022 was because of the compute available - in 2018, it would have been almost impossible to make something like GPT-4.

GP didn't say Cline/Roo charged anything on top.

How big do you think your index is compared to Google?

Things like Hunyuan 3D are nice for game assets and the like, but they aren't able to really do CAD well. That would be like using Stable Diffusion to code.

Yes, there has been. Unfortunately, there are a few core issues blocking this from becoming a big thing:

1. The majority of 3D modeling is not done parametrically, meaning there is not a lot of data. The little data there is is generally in OpenSCAD, which isn't very powerful or extensible for useful CAD. 2. Generally, when you want to do CAD, you need to come up with a way to define everything precisely. Like I want this hole 2 millimeters from the bottom, and this exact edge next to the hole to be beveled, etc. Saying all that to an LLM is slower than just making the whole.

That said, these still can be useful for beginners, and there are things like Adam AI that are starting to catch on for simple stuff.

Many models that use test time compute are MoEs, but test-time compute is generally meant to refer to reasoning about the prompt/problem the model is given, not about reasoning about which model to pick, and I don't think anyone has released an LLM router under that name.

First off, they are basically completely different technologies, so it would be disingenuous to act like it's an apples-to-apples comparison.

But a simple way to see it is that when you pick between multiple large models that have different strengths, you have a larger amount of parameters just to work with (e.g. Deepseek R1 + V3 + Qwen + LLaMA ends up being 2 trillion total parameters to pick from), whereas "picking" the experts in an MoE like has a smaller amount of total different parameters you are working with (e.g. R1 is 671 billion, Qwen is 235).

No. Each expert is not separately trained, and while they may store different concepts, they are not meant to be different experts in specific domains. However, there are certain technologies to route requests to different domain expert LLMs or even fine-tuning adapters, such as RouteLLM.

Yeah, it was for the Llama team because they love playing in ball pits instead of releasing good models.

Probably one of the best parts of this is MCP support baked in. Open source models have generally struggled with being agentic, and it looks like Qwen might break this pattern. The Aider bench score is also pretty good, although not nearly as good as Gemini 2.5 Pro.

If you would use LLM-assisted CAD for real industrial design, you would have to end up by specifying exactly where everything has to go and what size it has to be. But if you are doing that then you may as well make an automated program to convert those specific requirements into a 3D model.

Oh wait, that's CAD.

Cynical take aside, I think this could be quite useful for normal people making simple stuff, and could really help consumer 3D printing have a much larger impact.

This is even more incomprehensible to users who don't understand what this naming scheme is supposed to mean. Right now, most power users are keeping track of all the models and know what they are like, so this naming wouldn't help them. Normal consumers don't really know the difference between the models, but this wouldn't help them either - all those letters and numbers aren't super inviting and friendly. They could try just having a linear slider for amount of intelligence and another one for speed.

It's there now in the web app for me.

Same...

Claude got 63.2% according to the swebench.com leaderboard (listed as "Tools + Claude 3.7 Sonnet (2025-02-24)).[0] OpenAI said they got 69.1% in their blog post.

[0] swebench.com/#verified

They didn't provide a comparison either in the GPT-4.1 release and quite a few past releases, which is telling of their attitude as an org.

Finally, a new SOTA model on SWE-bench. Love to see this progress, and nice to see OpenAI finally catching up in the coding domain.

Leader is debatable, especially given the actual comparisons...

Oh, ok. But it's still quite telling of their attitude as an organization.

Reasoning models have the o first, non-reasoners have the digit first.

I'm not really bullish on OpenAI. Why would they only compare with their own models? The only explanation could be that they aren't as competitive with other labs as they were before.

I (not GP) would like to be able to choose between the options. Inference provider isn't super necessary though (can do that through huggingface).

Yeah, but Openrouter has a 5% surcharge anyway.

The Seattle/Bellevue area.

Questions.

Do you think Claude Code is "better", in terms of capabilities and token efficiency, than other tools such as Cline, Cursor, or Aider?

Unified memory is great because it's fast, but you can also get a lot of system memory on a "conventional" machine like OP's, and offload MOE layers like what Ktransformers did, so you can run huge models with acceptable speeds. While the Mac mini may have better value for anything that fits in the unified memory, if you want to run Deepseek R1 or other large models, then it's best to max out system RAM and get a GPU to offload.

The M4 Max has up to 546 GB/s. The M4 Pro, what GP was talking about, has only 273 GB/s. An M4 Max with that much RAM would most likely exceed OP's budget.

The bottleneck for single batch inference is memory bandwidth. The M4 Pro has less memory bandwidth than the P40, so it would be slower. Also, the setup presented in the OP has system RAM, allowing you to run models than what fits in 48GB of VRAM (and with good speeds too if you offload with something like ktransformers).

That was the R&D cost. According to OP, the cost of building one of these is around $1500.