RandyOrion

I guess I have to say thank you Meta?

A somewhat sad rant below.

Deepseek starts a toxic trend of providing super, super large MoE. And MoE is famous for being parameter-inefficient, which is unfriendly to normal consumer hardware with limited vram.

The super large size of LLM also disables nearly every people from doing meaningful development on these models. R1-1776 is the only fine-tune variation of R1 that makes some noise, and it's by a corp not some random individual.

In this release, the smallest Llama 4 model is over 100B, which is not small by any means, and will prevent people from fine-tuning as well.

On top of that, to access llama models on hugging face has become notoriously hard because of 'permission' issues. See details in https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/dis...

Yeah, I personally don't really see the point of releasing large MoEs. I'll stick to small and dense LLMs from Qwen, Mistral, Microsoft, Google and others.

Edit: This comment got downvoted, too. Please explain your reason before doing that.

Don't really know why this comment got downvoted. Are you serious?

Nice brain storming.

I think the name of the Chinese company should be DeepBaba. Tencent is not competitive at LLM scene for now.

Interesting observation.

I prefer things with low color contrast in general, just to leave some color space for important things. Maybe this preference stems from the time I tweak color themes in IDEs.

In contrast, I also found more and more photography pieces which show vibrant colors and high color contrasts.

I didn't see many papers on solving this problem.

I see non-stop response as a generalization problem because normally every training sample is not of infinite length.

Targeted supervised fine-tuning should work, as long as you have enough samples. However, supervised fine-tuning is not good for generalization.

First, this is not an open source / weight release.

Second, it has the problem of non-stoping response.

Just a random note.

As someone in this thread already stated, if you want smooth remote desktop experience with security requirements, you may try sunshine + moonlight for streaming with tailscale / zerotier for connection.

Thanks for these cool models!

One suggestion (or just rant): Less censorship for local models, PLEASE.

One question: 100+ elo gains from gemma 2 to gemma 3 on Chatbot arena is really something, any estimates on how this is achieved?

The combined population of China and Russia is less than a fifth of the world (15-16 bn vs 80+ bn). Edit: should be 1.5-1.6 bn vs 8 bn.

I'm only discussing Trump's behaviour and its effect on totalitarian governments, I don't have enough knowledge to discuss the rest of what you wrote.

I think the recent series of Trump's actions against Ukraine have failed to send a message to totalitarian governments that matches his own words. This has nothing to do with how much of the population Trump rules.

When totalitarian governments all start applauding what you're doing, using what you're doing as a distraction from a bad domestic situation as well as a justification for their dictatorships, you should know that something is totally screwed up.

Yes, I'm talking about the totalitarian governments of China and Russia.

Ungoogled chromium is a good alternative in terms of privacy because all google-related services are gutted and there are no other built-in telemetry things.

There are some downsides, too.

First, you have to do some research on learning to make this browser work conveniently, e.g. finding alternative services to sync and backup your settings, bookmarks, accounts and passwords, etc.

Second, changes pushed by Google like Manifest V3 is still hard to deal with.

Bitwarden app itself already integrates two-factor authentication code support.

I use the app on both PC (chromium extension) and phone, and I'm happy about it.

Well, although R1-671b is way too expensive for me to self-host, given their past open source (or weight) contributions, I DO have high expectation of them.

Each and every contribution to open source community will be helpful. Thanks DeepSeek!

A very random note.

I try to check the forum post and found out that I was blocked by https://forum.palemoon.org , e.g., https://offline.palemoon.org/blocked/index.html . Don't know and haven't visited this site before.

https://www.palemoon.org works though.

It's not nice to assume that people don't read then proceed to comment.

I read the link and I upvoted the "just support all GPUs you recently produced" comment.

I don't think the solution to bad software support is the prioritization. The prioritization is causing even more discrimination among different GPUs and different customers.

You can say whatever you want, and downvote whatever you want. However, that doesn't solve the real problem.

Why are people in AMD assuming other people don't want more software support for their GPUs by default? This is not nice.

Thanks for the reply. Would like to see whether the chat template is cause of this strange behavior.

Hi. It's nice to see these fixes.

I got a question after checking results on the open LLM leaderboard[1].

Comparing the result of NyxKrage/Microsoft_Phi-4 and microsoft/phi-4 or unsloth/phi-4, I can see fixing both the tokenizer and chat template causes the performance of both IFEval and BBH to increase. However, the performance on MATH, GPQA and MUSR degrades A LOT.

Is there any explanation on why this is happening?

[1] https://huggingface.co/spaces/open-llm-leaderboard/open_llm_...

Haha. Nice little idle game. Buy and sell Bitcoin for what win.

Interestingly, gitlab.com is pulling out from China, too.

Using ungoogled chromium for now. Ublock works without hassle.

Note that it's MORE complicated to setup ungoogled chromium because it doesn't contain any basic syncing service and update service. You need to do the first time setup manually as said in its FAQ. You also need to setup your own or use existing account / bookmark syncing service as well as software update service for convenience, e.g., vaultwarden server and bitwarden extension for account syncing.

An interesting read on alternative tokenization methods.

Questions:

1. What's the goal of entropy based byte token grouping as tokenization? Is this tokenization method best suited for the goal?

2. What about simply using byte level sequence to sequence autoencoder with down sampling for tokenization?

From a customer's perspective: NO, I don't like the BIG-little core architecture of CPUs on desktop platforms, and I don't enjoy the quality issues of the 13-14th gen CPUs; I also don't like the lack of software support for middling performing GPUs. I used to like intel's SSD, but the division was sold.

Chromium handles two versions of PDF just fine except that

use of LLMs to be fully justified.

EDIT: seems like spaces between words is nullified.