Hacker News Reader

Comment by callc | Hacker News Reader

If US regulation prohibits China from training LLMs? How?

If you mean trying to stop GPUs getting to China, US already has tried that with specific GPU models, but China still gets them.

Seems hard/impossible to do. Even if US and CCP were trying to stop Chinese citizens and companies doing LLM stuff

View on HN

Not prohibit training, but make it illegal for US companies to embed, use or distribute LLMs created by China. Basically the general consumer will need to go through the effort of use the LLM that they want, which we know means only a small fraction of people.

That assumes that only the US is interested in using LLMs commercially, which isn't really true. Even if you can get America to sanction Chinese LLM use, you aren't even going to American allies to go along with that, let alone everyone else.

China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).

Canada would be reluctant. So would Europe, South Asia and so forth. The biggest hurdle for China is the CCP. It is one thing to use physical products from China but relying on the CCP for knowledge may be a step too far for many nations.

Most people won't care if the products are useful. Chinese EVs, Chinese HSR, Chinese industrial robots, Chinese power tech, they are already selling. LLM isn't just a chatbot, it could be a medical device to help in areas without sufficiently trained doctors, for example.

Most people may not care but that doesn't matter if the government cares. I will not be surprised if countries restrict LLMs to trusted countries in the future. Unless there is a regime change in China, seeing it adopted in other countries may be an issue.

It is a very large world though, and nationalism is more of an American shtick at the moment. It is totally possible that countries have to decide to trade with China or the USA (if they put down an infective embargo), but then it really depends on what America offers vs. China, and I don't think that is a great proposition for us.

We are definitely heading into untreaded territory. It is one thing to use Chinese EVs, but using a knowledge system that will be censored (not that other countries won't be censored) and trained in a way that may not align with a nation's beliefs is a whole different matter.

> using a knowledge system that will be censored (not that other countries won't be censored) and trained in a way that may not align with a nation's beliefs is a whole different matter.

this is the exact same story told to the public when Google was kicked out of China. you are just 15 years late for the party.

Nationalism is not merely having having a moment as an "American shtick", though I don't know much about how widespread it is in non-Western developing countries. Certainly might not be so much, there.

The real possibility exists that it would be better to be an independent 'second place' technology center (or third place, etc) than a pure-consumer of someone else's integrate tech stack.

China decided that a long time ago for the consumer web. Europe is considering similar things more than ever before. The US is considering it with TikTok.

It's not hard to see that expanding. It's hard to claim that forcing local development of tech was a failure for China.

Short of a breakthrough that means the everyday person no longer has to work, why would I rather have a "better" but wholly-foreign-country-owned, not-contributing-anything-to-the-economy-I-participate-in-daily LLM or image generator or what-have-you vs a locally-sourced "good enough" one?

> They are still behind in semiconductors, maybe 10 or 20 years

i dont believe they're as behind as many analysis deems. In fact, making it illegal to export western chips to china only serves to cause the mother of all inventions, necessity, to pressure harder and make it work.

They will definitely throw more resources at it, but without even older equipment from the west, they have a bigger hill to climb as well. There are lots of material engineering secrets that they have to uncover before they get there, so that’s just my estimate of what they need to do it. I definitely could be wrong though, we’ll see.

> China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).

You don't need the most efficient chips to train LLMs. those much slower chips (e.g. those made by Huawei) will probably take longer for training and they waste more electricity and space. but so what?

And lest we forget, China is not bothered by building as many nuclear power plants as it needs.

because Chinese is investing heavily on all sorts of renewable energies. its annual investment is more than the total US and EU amount combined.

"China is set to account for the largest share of clean energy investment in 2024 with an estimated $675 billion, while Europe is set to account for $370 billion and the United States $315 billion."

https://www.reuters.com/sustainability/climate-energy/iea-ex...

Economy isn’t efficiency, China has a yield problem on the chips they need, and that reduces their progress.

Making something illegal isn't gonna work if there's clearly value in doing it, and isn't actually harmful. Regulatory unfairness is easily discerned.

Look at how competitive chinese EVs are, and no amount of tarriffs are gonna stop them from dominating the market - even if americans prevent their own market from being dominated, all of their allies will not be able to stop their own.

Like I've said before, this isn't a physical good that is being sold. LLM is knowledge and many governments and people are going to be concerned with how and who is packaging it. Using LLMs created by China will mean certain events in history will be omitted (which will not be exclusive to China). How the LLM responds will be dictated by the LLM training and so forth.

LLMs will become the ultimate propaganda tool (if they aren't already), and I don't see why governments wouldn't want to have full control over them.

Reply on news.ycombinator.com