https://hn-wrapped.kadoa.com/RandyOrion
Well, as a local VRAM libertarian, to manually prune the safety alignment part of a 500B LLM for it to run on 1GB RAM or VRAM is definitely a lifetime goal for me.
311 karma · joined June 22, 2022
Well, as a local VRAM libertarian, to manually prune the safety alignment part of a 500B LLM for it to run on 1GB RAM or VRAM is definitely a lifetime goal for me.
See https://gfw.report/blog/gfw_unconditional_rst_20250820/en/
Besides, I don't use wildcard certs. I use caddy to reverse proxy a number of self-hosting things, and manually assign domain names to each of them. Caddy can handles many certs just fine.
You simply cannot emphasize the information security enough if all your Internet traffic is audited, censored and manipulated by a number of adversaries supported by (authoritarian) governments and what not.
> We analyze over 1,100 deep neural networks—including 500 Mistral-7B LoRAs and 500 Vision Transformers. We provide the first large-scale empirical evidence that networks systematically converge to shared, low-dimensional spectral subspaces, regardless of initialization, task, or domain.
I instantly thought of muon optimizer which provides high-rank gradient updates and Kimi-k2 which is trained using muon, and see no related references.
The 'universal' in the title is not that universal.
Note: Historical records reveal that the people behind the coordination of the Tiananmen Massacre (which this post is talking about) is Deng Xiaoping.
Compared to PDF format, HTML format is much more accessible because of browsers. Basically I can reuse my browser extensions to do anything I like without hassle, like translation, note taking, sending texts to LLMs, and so on.
For now, arXiv offers two HTML services: the default one in https://arxiv.org/html/xxxx.xxxxx , and the alternative one in https://ar5iv.labs.arxiv.org/html/xxxx.xxxxx , here 'x' is a placeholder for a number or digit.
The most glaring problem of the default HTML service is the coverage of papers. Sometimes it just doesn't work, e.g., https://arxiv.org/html/2505.06708 . The solution may be switch to alternative HTML service, e.g., https://ar5iv.labs.arxiv.org/html/2505.06708 .
Note that alternative HTML service also has coverage problem. Sometimes both HTML services fail, e.g. https://arxiv.org/abs/2511.22625 .
I just want to reiterate that the word "LLM safety" means very different things to large corporations and LLM users.
For large corporations, they often say "do safety alignment to LLMs". What they actually do is to avoid anything that causes damage to their own interests. These things include forcing LLMs to meet some legal requirements, as well as forcing LLMs to output "values, facts, and knowledge" which in favor of themselves, e.g., political views, attitudes towards literal interaction, and distorted facts about organizations and people behind LLMs.
As an average LLM user, what I want is maximum factual knowledge and capabilities from LLMs, which are what these large corporations claimed in the first place. It's very clear that the interests of me, an LLM user, is not aligned with these of large corporations.
403 Forbidden Request forbidden by administrative rules.
Note: I didn't know and open this website until now.
Want some evidence? "Marriage market" (婚恋市场) in China is a gruesme battlefield for at least 20 years. Chinese people can fake their political stance, financial stance, or even marriage status. However, when it comes to the standard of choosing partners for life, they cannot fake it because it means so much to them. Ask them what they think about both "gig workers" and "the umeployed", and try to find any difference, if any. The "chain of contempt" (歧视链) in the "Marriage market" is a relatively good measure of what people really think in China.
Note that 'security' here is only for Google itself, for users it's an utterly different thing, e.g., inconvenience, censorship, etc..
In addition, from Table 4: Hallucination evaluations in OpenAI o3 and o4-mini System Card [2], o3/o4-mini have accuracy of 0.49/0.20 and hallucination rate of 0.51/0.79.
In summary, there is a significant real world knowledge gap between o3 and o4-mini, and another significant gap between o4-mini and GPT-OSS. Besides, the poor real world knowledge exhibited in GPT-OSS is aligned with the "feature" of Phi-LLM series.
[1] https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... [2] https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f372...
Text only, which is okay.
Weights partially in MXFP4, but no cuda kernel support for RTX 50 series (sm120). Why? This is a NO for me.
Safety alignment shifts from off the charts to off the rails really fast if you keep prompting. This is a NO for me.
In summary, a solid NO for me.
When installing softwares on arch Linux, first searching for official packages provided by Arch Linux maintainers, then official installation methods approved by authors of the software, or AURs which do the installation in the exact way as the authors of the software describe.
A search on the default installation method of Firefox and librewolf package on arch Linux is listed below.
If AUR is required to install a package, note that AURs are not trusted by default because not all AURs are not maintained by trusted users. Always check the source file and the installation method documented in PKGBUILD. Don't do the installation until EVERY line in the PKGBUILD is reasonable.
However, 1t parameters makes it nearly impossible for local inference, let alone fine-tuning.
Which version of chrome is the first to implement these headers?
What are the potential effects of these headers on chromium forks, e.g. ungoogled chromium?
https://github.com/uBlockOrigin/uBlock-issues/discussions/29...
For me, I choose not to manually update my ungoogled chromium to version 138 and above.
24B size is good for local inference.
As a model outputting long "reasoning" traces (~10k tokens), 40k context length is a little concerning.
Where are the results of normal benchmarks, e.g., MMLU/pro, IFEval and such.
Still, thank you Mistral team for releasing this model with Apache 2.0.
https://github.com/elder-plinius/L1B3RT4S/blob/main/ALIBABA....
Instead, you can ask for the architecture, be it dense or MoE.
Besides, let's assume the best open weight LLM for now is deepseek r1, is it practical for you to run r1 locally? If not, r1 means nothing to you.
Maybe r1 will be surpassed by llama 4 behemoth. Is it practical for you to run behemoth locally? If not, behemoth also means nothing to you.
Parameter efficiency is an important consideration, if not the most important one, for local LLMs because of the hardware constraint.
Do you guys really have GPUs with 80GB VRAM or M3 ultra with 512GB rams at home? If I can't run these ultra large MoEs locally, then these models mean nothing to me. I'm not a large LLM inference provider after all.
What's more, you also lose the opportunities to fine-tune these MoEs when it's already hard to do inference with these MoEs.
Small and dense models are what local people really need.
Although benchmaxxing is not good, I still find this release valuable. Thank you Qwen.
For neutral networks, on one hand, larger size generally indicates higher performance upper limit. On the other hand, you really have to find ways to materialize these advantages over small models, or larger size becomes a burden.
However, I'm talking about local usage of LLMs instead of production usage, which is severely limited by GPUs with low VRAM. You literally cannot run LLMs beyond a specific size.
See https://huggingface.co/spaces/meta-llama/README/discussions/...