Hacker News Reader

fspeechon China announces retaliatory tariffs on some Canada farm, food products

100% tax on EV imports and 25% metal tax imposed on China by Canada about half an year ago. Canada is in a tough spot. It needs to sync up its tade policy with the US in order to maintain its free trade with the US.

fspeechon DeepSeek's Multi-Head Latent Attention

Matrix absorption is unnecessary. What is needed is the order of multiplication associates towards the direction of the absorption. This and the modified Rope are needed to make the caching work.

fspeechon AI firms follow DeepSeek's lead, create cheaper models with distillation

The drive so far has been to scale up to push performance, not to reduce cost. Although everyone offers cheaper versions of models they don't base their marketing on these. Now that the momentum of scaling up on training is slowing down they are pushing inference time computes and efficiency comes to the forefront.

fspeechon AI, China and Copium

https://github.com/deepseek-ai/open-infra-index/blob/main/20...

Statistics of DeepSeek's Online Service All DeepSeek-V3/R1 inference services are served on H800 GPUs with precision consistent with training. Specifically, matrix multiplications and dispatch transmissions adopt the FP8 format aligned with training, while core MLA computations and combine transmissions use the BF16 format, ensuring optimal service performance.

Additionally, due to high service load during the day and low load at night, we implemented a mechanism to deploy inference services across all nodes during peak daytime hours. During low-load nighttime periods, we reduce inference nodes and allocate resources to research and training. Over the past 24 hours (UTC+8 02/27/2025 12:00 PM to 02/28/2025 12:00 PM), the combined peak node occupancy for V3 and R1 inference services reached 278, with an average occupancy of 226.75 nodes (each node contains 8 H800 GPUs). Assuming the leasing cost of one H800 GPU is $2 per hour, the total daily cost amounts to $87,072.

fspeechon AI, China and Copium

In addition Deepseek just showed that they are serving all that demand (admittedly inadequately) from a cluster of only 2000 H800 GPUs.

fspeechon DeepSeek-V3/R1 Inference System Overview

Gross margin in theory. They are using API pricing to project revenue but not all traffic is API. They don't charge for chat traffic so this is theoretical. It is in response to a raging debate on whether their gross margin is negative with their API pricing. Apparently not if one can saturate the traffic.

fspeechon DeepSeek-V3/R1 Inference System Overview

There are a lot of shared prefixes from user prompts. You can save by first looking into the cache to find the longest prefix for a prompt. Their MLA makes the KV cache particularly efficient.

fspeechon DeepSeek Open Source Optimized Parallelism Strategies, 3 repos

It's probably more about the size of the firm. Once a company is large enough with middle layers and silos it seems some form of KPI is inevitable. This could also explain why Deepseek doesn't mind open sourcing as they have not raised outside capital for expansion so they can't really scale rapidly on their own.

fspeechon DeepSeek Open Source Optimized Parallelism Strategies, 3 repos

It appears that they have really good toolings so they can focus on hiring really smart new grads and giving them a lot of leeway on choosing what to work on without worrying about how productive they will be. One particular memorable response when the CEO Liang was asked about not using KPI goes roughly like this: since we all reviewd and worked with the candidate, if he is not productive it must be our fault to not have given him the right support to make him productive.

fspeechon DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Efficiency could lead to cheaper hardware for everyone, themselves included.

fspeechon DeepSeek Open Infra: Open-Sourcing 5 AI Repos in 5 Days

You need that to optimize load balancing. Unfortunately that gain is not available to small or individual deployment.

fspeechon Grok3 Launch [video]

A better approach is to split the model with MOEs running on CPUs and MLAs running on GPU. See the ktransformers project: https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...

This takes advantage of the sparsity of MOE and the efficient KV-cache of MLA.

fspeechon It's not just AI. China's medicines are surprising the world, too

China proves that you don't need 100 times return on capital to get potential innovators to take risk. 100% may be enough. People say China couldn't innovate due to its weak IPR protection, but it is turning out that you don't need to be an absolutist. Sure outright theft is bad, but imitation can be good for competition. You want to give the innovators a head start so they can profit from their efforts, but you don't want to make the protection so tight that they can sit on their initial efforts and profit for years to come without further innovation.

fspeechon Ask HN: Is there a way to use Baidu cloud legally from US/EU?

You need a Chinese number to sign up and facial verification to make payments, as far as I can tell.

fspeechon SambaNova Cloud Launches the Fastest DeepSeek-R1 671B (198 tok/s)

The smaller cluster is not necessarily a good thing. It may not be able to take advantage of unbalanced load on experts.

fspeechon Alibaba's Tsai Says Apple iPhone Will Use His Firm's AI in China

Deepseek doesn't have the infrastructure to support Apple. I suspect that they are also not interested in tailoring to Apple's needs given their mission and small size.

fspeechon BYD to offer Tesla-like self-driving tech in all models for free

In a polytheistic context the invocation is much less severe, roughly equivalent to Apollo's Eyes or Zeus's Eyes.

fspeechon BYD to offer Tesla-like self-driving tech in all models for free

That's due to the translation. The original term 天神 comes from the polytheistic Chinese folk religion so it doesn't have the same connotation.

fspeechon Run Deepseek from fast NVMe drives

Deepseek's open source inference code, while correct, may not be fully efficient. For example the MLA needs the right associative matrix multiplication order to be efficient.

fspeechon Run Deepseek from fast NVMe drives

Do you have any benchmark run yet? I am interested in knowing how many tokens/sec you can get to. Though in the end it should be more efficient to run the model on distributed server clusters.

fspeechon The Race to God Mode

Thanks for the thought provoking take.

fspeechon SemiAnalysis says DeepSeek spent more than it claims

Right. And the number is based on rental rates of GPUs so how many GPUs they own is irrelevant to the claim.

fspeechon Understanding Reasoning LLMs

Recent developments like V3, R1 and S1 are actually clarifying and pointing towards more understandable, efficient and therefore more accessible models.

fspeechon Trump slaps tariffs on Mexico, Canada and China

Unless the world demand for these raw material goes down or production elsewhere goes up someone will still buy these raw materials. If tarriff is high enough to force US to source from and Canada to sell to more distant places the world will be less efficient.

fspeechon Nvidia’s $589B DeepSeek rout

The key question is: has demand elasticity increased for Nvidia cards? An increase in elasticity means people are more willing to wait for hardware price to drop because they can do more with existing hardware. Elasticity could increase even if demand is still growing. Not all growths are equally profitable. Current high prices are extremely profitable for Nvidia. If elasticity is increasing future growth may not be as profitable as the projection from when Deepseek was relatively unknown.

fspeechon Nvidia, ASML plunge as DeepSeek triggers tech stock selloff

That's not how commodities are priced. Commodities are priced at the marginal cost of production.

fspeechon Nvidia, ASML plunge as DeepSeek triggers tech stock selloff

Inference can be done on much more diverse hardware.

fspeechon Nvidia, ASML plunge as DeepSeek triggers tech stock selloff

You need to also look at valuation and how much growth justifies that valuation.

fspeechon Nvidia’s $589B DeepSeek rout

You are not taking into account why people are willing to pay exceedingly high prices for GPUs now and that the underlying reason may have been taken away.

fspeechon The impact of competition and DeepSeek on Nvidia

It doesn't make sense to compare individual models. A better way is to look at total compute consumed, normalized by the output. In the end what counts is the cost of providing tokens.