I hope they somewhat fixed the hallucination and forgetting plagued V4 previews and that it wasn't just benchmaxxed but the numbers hold in reality. Then it would be my choice for 2x DGX Spark or 2x RTX Pro 6000.
What really helped me was to run any .md/prompts through an LLM to find contradictions, duplicates or ambiguities, repeatedly. That led to agents much better following instructions.
There was a presentation at CS 25 Transformers United about some phenomena like chain-of-thought emerging only after training LLMs with at least 1T tokens and only in LLMs of certain size.
He didn't mention outright banning open source LLMs, just that their safe release would be a much harder problem, which to me implied "the easiest way is to ban the open source models".
If your brain has a local metabolic failure leading to the inability to replete neurotransmitters, how taking a walk is a solution to your problem? You need to improve energetics first. Or what if some virus/auto-antibody is occupying some receptors in the brain, messing it up? Pushing the brain would likely just exacerbate the underlying failure.
This feels like Nvidia's strategy doesn't really care about the few companies they invested to pushing frontier AI, but about scaling up the amount of GPUs/datacenters needed; open models forcing more GPU purchases would drive that faster than 2 frontier labs ever could. It's likely also attractive to anyone running a GPU rental business like AWS, Azure etc. who can offer more models.
Aren't depressions coming with side-effects like inability to do anything at all? The block is somewhere else to overcome before even thinking about exercise and the brain might have a good reason to block it (e.g. some local energetic/metabolic failure and the brain forcing a "safe mode" to prevent further damage). It's like telling to people with endothelial damage that exercise would do them good when in fact it would cripple them further (e.g. from long covid).
Dunno, now the workflow is like agent makes code changes, ruff complains, agent fixes complaints at the cost of code bloat, agent makes a PR, another agent reviews the PR, agent makes changes, PR is approved and merged. Nobody reads PRs anymore with the agentic development velocity.
One could argue that the interesting parts of simulation require higher precision. When you are in stable conditions, that's where your intuition might be sufficient already; once you hit those ugly parts then you start requiring as high precision as you can get to have any correlation with reality. Even a simple solar system simulation through gravity interactions has significant drifts in FP64.
Borland was way ahead of its time, nowadays there is no non-AI way to build a whole UI app in on afternoon like with Delphi/C++ Builder. Pity their greed prevented them from dominating via simplicity.
I doubt they did any distillation as Hinton defined it (requiring logit access). They most likely ran a bunch of prompts/conversations and captured the results. Those conversations already missed thinking tokens, replaced by some confusing quasi-summaries. Then they took those and ran basic SFT or maybe DPO if they had competing responses. As there is no copyright on the output of AI, I am not sure where is the "covert industrial distillation" part of the problem.
DeepSeek V4 hallucinates like crazy and often forgets explicitly mentioned parts of the context. I guess compressing tokens and cherry-picking attention comes at a cost.
Windows Phone aesthetics was repulsive to most people at that time; we finally got TrueColor 4k screens and all MS could do was to use 10 colors everywhere and start the flat fad that destroyed UX on most systems. What a waste.
Capacitive screens were out of possibility for them as Apple bought 2 year production in advance, a trick Tim deployed repeatedly in many areas. MeeGo had a chance but US funds didn't want to allow a state where an EU company would rule the fastest growing market of that time and their darling MS slips into irrelevance and its trojan horse killed it off quickly.
They are orthogonal; preference optimization like RLHF can be done on the base model which can later be quantized, or it could be done on a new LoRA that is then converted to QLoRA.
First they tried to approve software patents during an agriculture and fisheries council session, now they are bending procedural rules to hack it in before summer vacations. Some weird form of democracy™.
Math is static, CS is dynamic. In math you describe static idealized "worlds", in CS you look at any discrete dynamic process in detail via algorithms. Many folks doing math can't understand algorithms, and many coders can't understand math. Just ask a mathematician what does A = A + 1 mean. There is some inherent impedance mismatch.