It will be interesting to see how these cars perform over years and decades. One of the major cost saving mechanisms BYD uses is lumping components together. An issue with one part of the drive train means the whole drive train may need to be replaced. The battery pack being part of the chassis means a small accident could become a write-off. Initial cost savings at manufacture will be passed on to the owner in maintenance.
Is this only an issue if you are using agents? Or are there any theoretical risks of serving a model over say llama server (llama.cpp) and accessing it for chat in browser on machines on your local network?
I like the analogy. Interesting to note that a lot of the really high end furniture is still made using hand tools. But it is not affordable for most, who will buy mass produced furniture made by machines.
Thanks to reading your post on HN, I was able to put mitigations in place before the patches came out. So for me, your original intention of: "alert others in a similar position to be aware of the importance/urgency of the updates" held up.
Can you explain how this would help protect a company's secrets? I would think that if you want to keep your IP secret surely using something like Deepseek's API directly is not ideal and hosting something locally is best practice.
I have a friend who drives a Tesla (hates it but signed a lease). On several occasions he has been unable to get into his car when his phone had no battery and he uses it as a key. I'm sure he could use a real car key but that would probably be incredibly inconvenient to carry around.
In the mid 90s I used Fortran to model a distillation column in a chemical engineering plant. Here I read that it is still the go to language for nuclear engineering. I am amazed but for some reason happy about this.
I thought it would be interesting to see if the local AI I use for quick question and answer stuff could crack the code.
I use Deepseek V4 flash 0731 Q8 quantisation from unsolth with reasoning turned off on a local server for quick easy tasks. Note I use another model locally for harder tasks that need reasoning.
I was a bit dissappointed that it knew it was Poe's cypher so didn't need to even try to crack the code.
It seems very expensive. I built a 5m x 3m 'log cabin' garden office (in Scotland in 2018), with electricity and cat6 cabled internet for £5k (approx $6800). The shed itself was a kit-build so easy and for foundations I used stakes (concreted in).
Ollama uses the llama.cpp backend for inference. I find Ollama noticably slower. Llama.cpp has had a built-in webui (used as llama-server) for a long time now so have owned the user experience too.
I'm sure this is a wonderful breaktheough but I'd rather have the current, or a recent, generation of semiconductor technology available at a reasonable price.
For a fraction of the price of 96GB vram, I built a desktop based on a supermicro server mobo and EPYC 9 series CPU, with just under 400GB rdimm ram (approx $4500 all in but this was before the ram price hike). Works really well for serving larger local modals at a decent enough speed (I consider anything more than 10 tokens/second usable and value accuracy over speed).
Coding, my own proprietary code hence my desire for local hosting, a decent amount of legacy code. General troubleshooting of anything and everything from running Linux servers to fixing my car. Summarizing and translation of large documents occasionally. Also, image generation and other automations but obviously not LLMs for this.
Basically it comes down to memory bandwidth of server CPUs being decent. A bit of oversimplification here but... The model and context have to be pulled through RAM (or VRAM) every time a new token is generated. CPUs that are designed for servers with lots of cores have decent bandwidth - up to 480GB/s with the EPYC 9 series and they can use 16 channels simultaneously to process memory. So, in theory they can pull 480GB through the system every second. GPUs are faster but you also have to fit the entire model and context into RAM (or VRAM) so for larger models they are extremely expensive because a decent consumer GPU only has 24GB of VRAM and costs silly money, if you need 20 of them. Whereas you get a lot of RDIMM RAM for a couple thousand bucks so you can run bigger models and 480GB/s gives output faster than most people can read.