Hacker News Reader

Comment by leetharris | Hacker News Reader

CEO of Scale said Deepseek is lying and actually has a 50k GPU cluster. He said they lied in the paper because technically they aren't supposed to have them due to export laws.

I feel like this is very likely. They obvious did some great breakthroughs, but I doubt they were able to train on so much less hardware.

View on HN

CEO of a human based data labelling services company feels threatened by a rival company that claims to have trained a frontier class model with an almost entirely RL based approach, with a small cold start dataset (a few thousand samples). It's in the paper. If their approach is replicated by other labs, Scale AI's business will drastically shrink or even disappear.

Under such dire circumstances, lying isn't entirely out of character for a corporate CEO.

Could be true.

Deepseek obviously trained on OpenAI outputs, which were originally RLHF'd. It may seem that we've got all the human feedback necessary to move forward and now we can infinitely distil + generate new synthetic data from higher parameter models.

> Deepseek obviously trained on OpenAI outputs

I’ve seen this claim but I don’t know how it could work. Is it really possible to train a new foundational model using just the outputs (not even weights) of another model? Is there any research describing that process? Maybe that explains the low (claimed) costs.

Probably not the whole model, but the first step was "fine tuning" the base model on ~800 chain of thought examples.

Those were probably from OpenAI models. Then they used reinforcement learning to expand the reasoning capabilities.

800k. They say they came from earlier versions of their own models, with a lot of bad examples rejected. They don't seem to say which models they got the "thousands of cold-start" examples from earlier in the process though.

every single model does/did this. Initially fine tuning required the expensive hand labeled outputs for RLHF. Generating your training data from that inherently encodes the learned distributions and improves performance, hence why some models would call themselves chatgpt despite not being openai models.

Check the screenshot below re: training on OpenAI Outputs. They've fixed this since btw, but it's pretty obvious they used OpenAI outputs to train. I mean all the Open AI "mini" models are trained the same way. Hot take but feels like the AI labs are gonna gatekeep more models and outputs going forward.

https://x.com/ansonhw/status/1883510262608859181

I would think the CEO of an American AI company has every reason to neg and downplay foreign competition...

And since it's a businessperson they're going to make it sound as cute and innocuous as possible

Or, more likely, there wasn't a magic innovation that nobody else thought of, that reduced costs by orders of magnitude.

When deciding between mostly like scenarios, it is more likely that the company lied than they found some industry changing magic innovation.

It's hard to tell if they're telling the truth about the number of GPUs they have. They open sourced the model and the inference is much more efficient than the best American models so it's not implausible that the training was also much more efficient.

If we're going to play that card, couldn't we also use the "Chinese CEO has every reason to lie and say they did something 100x more efficient than the Americans" card?

I'm not even saying they did it maliciously, but maybe just to avoid scrutiny on GPUs they aren't technically supposed to have? I'm thinking out loud, not accusing anyone of anything.

Then the question becomes, who sold the GPUs to them? They are supposedly scarse and every player in the field is trying to get ahold as many as they can, before anyone else in fact.

Something makes little sense in the accusations here.

I think there's likely lots of potential culprits. If the race is to make a machine god, states will pay countless billions for an advantage. Money won't mean anything once you enslave the machine god.

https://wccftech.com/nvidia-asks-super-micro-computer-smci-t...

We will have to wait to get some info on that probe. I know SMCI is not the nicest player and there is no doubt GPUs are being smuggled, but that quantity (50k GPUs) would be not that easy to smuggle and sell to a single actor without raising suspicion.

Facts, them Chinese VCs will throw money to win.

Man, they say China is the most populated country in the world, I’m sure they got loopholes to grab a few thousands H100s.

They probably also trained the “copied” models by outsourcing it.

But who cares, it’s free and it works great.

Bro, did you use Deepseek? That shyt is better than ChatGPT. No cards being thrown here.

Thanks to SMCI that let them out...

https://wccftech.com/nvidia-asks-super-micro-computer-smci-t...

Chinese guy in a warehouse full of SMCI servers bragging about how he has them...

https://www.youtube.com/watch?v=27zlUSqpVn8

Deepseek is indeed better than Mistral and ChatGPT. It has tad more common sense. There is no way they did this on the “cheap”. I’m sure they use loads of Nvidia GPUs, unless they are using custom made hardware acceleration (that would be cool and easy to do).

As OP said, they are lying because of export laws, they aren’t allowed to play with Nvidia GPUs.

However, I support DeepSeek projects, I’m here in the US able to benefit from it. So hopefully they should headquarter in the States if they want US chip sanctions lift off since the company is Chinese based.

But as of now, deepseek takes the lead in LLMs, my goto LLM.

Sam Altman should be worried, seriously, Deepseek is legit better than ChatGPT latest models.

I haven't had time to follow this thread, but it looks like some people are starting to experimentally replicate DeepSeek on extremely limited H100 training:

> You can RL post-train your small LLM (on simple tasks) with only 10 hours of H100s.

https://www.reddit.com/r/singularity/comments/1i99ebp/well_s...

Forgive me if this is inaccurate. I'm rushing around too much this afternoon to dive in.

The CEO of Scale is one of the very last people I would trust to provide this information.

Just to check my math: They claim something like 2.7 million H800 hours which would be less than 4000 GPU units for one month. In money something around 100 million USD give or take a few tens of millions.

If you rented the hardware at $2/GPU/hour, you need $5.76M for 4k GPU for a month. Owning is typically cheaper than renting, assuming you use the hardware yearlong for other projects as well.

Only the DeepSeek V3 paper mentions compute infrastructure, the R1 paper omits this information, so no one actually knows. Have people not actually read the R1 paper?

I've also read that Deepseek has released the research paper and that anyone can replicate what they did.

I feel like if that were true, it would mean they're not lying.

You can't replicate it exactly because you don't know their dataset or what exactly several of their proprietary optimizations were

Alexandr Wang did not even say they lied in the paper.

Here's the interview: https://www.youtube.com/watch?v=x9Ekl9Izd38. "My understanding is that is that Deepseek has about 50000 a100s, which they can't talk about obviously, because it is against the export controls that the United States has put in place. And I think it is true that, you know, I think they have more chips than other people expect..."

Plus, how exactly did Deepseek lie. The model size, data size are all known. Calculating the number of FLOPS is an exercise in arithmetics, which is perhaps the secret Deepseek has because it seemingly eludes people.

> Plus, how exactly did Deepseek lie. The model size, data size are all known. Calculating the number of FLOPS is an exercise in arithmetics, which is perhaps the secret Deepseek has because it seemingly eludes people.

Model parameter count and training set token count are fixed. But other things such as epochs are not.

In the same amount of time, you could have 1 epoch or 100 epochs depending on how many GPUs you have.

Also, what if their claim on GPU count is accurate, but they are using better GPUs they aren't supposed to have? For example, they claim 1,000 GPUs for 1 month total. They claim to have H800s, but what if they are using illegal H100s/H200s, B100s, etc? The GPU count could be correct, but their total compute is substantially higher.

It's clearly an incredible model, they absolutely cooked, and I love it. No complaints here. But the likelihood that there are some fudged numbers is not 0%. And I don't even blame them, they are likely forced into this by US exports laws and such.

It should be trivially easy to reproduce the results no? Just need to wait for one of the giant companies with many times the GPUs to reproduce the results.

I don't expect a #180 AUM hedgefund to have as many GPUs than meta, msft or Google.

AUM isn't a good proxy for quantitative hedge fund performance, many strategies are quite profitable and don't scale with AUM. For what it's worth, they seemed to have some excellent returns for many years for any market, let alone the difficult Chinese markets.

> In the same amount of time, you could have 1 epoch or 100 epochs depending on how many GPUs you have.

This is just not true for RL and related algorithms, having more GPU/agents encounters diminishing returns, and is just not the equivalent to letting a single agent go through more steps.

Alexandr only parroted what Dylan Patel said on Twitter. To this day, no one know how this number come up.

R1 is double the size of o1. By that logic, shouldn’t o1 have been even cheaper to train?

Why would Deepseek lie? They are in China, American export laws can't touch them.

Making it obvious that they managed to circumvent sanctions isn’t going to help them. It will turn public sentiment in the west even more against them and will motivate politicians to make the enforcement stricter and prevent GPU exports.

I don't think sentiment in the west is turning against the Chinese, beyond well, lets say white nationalists and other ignorant folk. Americans and Chinese people are very much alike and both are very curious about each others way of life. I think we should work together with them.

note: I'm not Chinese, but AGI should be and is a world wide space race.

Reply on news.ycombinator.com