01-AI/Yi: A series of large language models trained from scratch
github.com
github.com
Kai Fu Lee's New AI Company: Yi-Open Source - https://news.ycombinator.com/item?id=38158554 - Nov 2023 (66 comments)
https://huggingface.co/01-ai/Yi-34B-200K
Even untuned, I am liking the 34B 4K model so far. I never really use raw llamav2 70b though.
“From scratch” might be non-specific, but a fine phrasing in this case.
In this case my lens is colored by my interests; I've been thinking a lot on RL for LLMs and it slipped out here. For me I rely on phrases to quickly filter for (subjectively) interesting work. And assuming we value that subjective filtering of content, we make it easier for others to filter consistently when we use consistent language.
I'm glad this disgusting sloppiness doesn't exist in more established fields. I hope people keep calling it out.
Cute.
There are 2873 results that contain "large language model" in the full text and use "pretrained" in the title or abstract. A 10x difference in publication count does make one feel more common than the other?
I'd need to get into more involved queries to break down the semantic categories of those papers.
This is perfectly good language and exactly the correct thing for them to say.
Leaving such powerful tools in the hands of secretive organizations or uber-wealthy individuals is a guarantee for bad outcomes. Distributing the tools, the know-how required to build and service them, and the benefits of their use -- that's the antidote.
A much better model to compare to is a medical one.
Yes, the knowledge about how to perform medical practices should be open and shared for the benefit of the world.
However, laws and practices should be put in place to prevent harm by incompetent or malicious people.
It’s not rocket science.
You don’t need a phd to look at a deep fake and go “well, gee, I guess this could be problematic”. Did you see the recent coverage about teenagers generating porn of their class mates?
Come on.
It’s not about the end of world, it’s about preventing all kinds of harmful stuff (eg. above).
You cant have cake and eat it too.
Either you make everything available to everyone and wear the consequences; or you restrict things so not everyone can have them; or you make it illegal to use them without an appropriate license.
The idea that (a), (ie. just go wild!) is the best outcome it naive, utopian and deeply misguided.
Yes, it’s complicated; but the “Illuminati control the world” stuff is just crazy, stupid conspiracy theory stuff.
Do what you want privately, but you need to be certified to have a business that can legaly run AI models with capabilities > X?
It’s not outrageous.
That all AI runs entirely behind the APIs of a few businesses? That would be literally the same thing, except you could choose to do it yourself if you were willing to meet the legal requirements.
Like banks. Or insurance.
Outrageous I know, but … that’s the world we live in, and it’s not nearly as “nightmare” as some people seem to think.
The bad actor in this scenario is, of course, Yann LeCun, and no I am not joking.
I don't think the technocrat dictatorship risk adds up to enough to outweigh the rando experimenter risk by far. Also, it's not like we don't face the dictatorship risk if randos have AI too; "Russia has nukes" is not solved by "actually, some civilians also have nukes" except in the sense that they may kill us first, and thus relieve us of having to worry about Russia.
1. Say, if it shows promising early signs to develop a cure for a cancer.
> Your use of the Yi Series Models must comply with the Laws and Regulations as well as applicable legal requirements of other countries/regions, and respect social ethics and moral standards
("Laws and Regulations" is specifically mainland China's)
https://huggingface.co/01-ai/Yi-34B/blob/main/LICENSE
Still, better than being entirely closed I suppose. But there are properly open models out there and one wonders how they will compare.
It also forbids several specific uses, and they’re not what you might think on a naive reading of the English translation.
When the CCP says “terrorism”, they’re justifying their genocidal policies towards ethnic and religious minorities https://www.hrw.org/report/2021/04/19/break-their-lineage-br...
When the CCP says “misinformation”, they’re rewriting history to ignore Tiananmen Square. https://www.britannica.com/event/Tiananmen-Square-incident
When the CCP says “national security”, they condemning people like Naomi Wu who don’t fit cleanly into their worldview https://skepchick.org/2023/08/maker-naomi-wu-is-silenced-by-...
Quotes:
> “Laws and Regulations” refers to the laws and administrative regulations of the mainland of the People's Republic of China (for the purposes of this Agreement only, excluding Hong Kong, Macau, and Taiwan).
> 1) Your use of the Yi Series Models must comply with the Laws and Regulations as well as applicable legal requirements of other countries/regions, and respect social ethics and moral standards, including but not limited to, not using the Yi Series Models for purposes prohibited by Laws and Regulations as well as applicable legal requirements of other countries/regions, such as harming national security, promoting terrorism, extremism, inciting ethnic or racial hatred, discrimination, violence, or pornography, and spreading false harmful information.
It asserts that Taiwan is NOT part of "mainland of the PRC China". The clarification is required because under PRC law, Taiwan is part of the PRC by default.
> requires you to “respect social ethics and moral standards”
The license text is no different from the other recent AI model license texts out there that impose moral and ethical restrictions on usage. From a legal perspective this sucks because it is vague, but it's on par with the new wave of standard AI licenses.
The text also explicitly states that legal and moral standards "of other countries/regions" must be complied with.
> When the CCP says “terrorism”, they’re [...]
Even if what you say is true, this isn't a license from the CCP. The "mascot" of the company, Kai-fu Lee, is apparently a Taiwanese-American residing in Beijing.
I mean, I don't know whether I'm feeding the trolls by writing a serious reply to your baseless claims instead of just downvoting your comment, but wow.
Yes, the license calls out that Taiwan is not part of China and attempts to limit that statement to this specific agreement. They have to call this out because CCP has a very different opinion on this than the rest of the world. Which definition will prevail if a dispute about model use comes to court in the Chinese legal system?
Yes, people are trying to require ethical use of AI via licensing. This model is trying to enforce China's view of the world. It's important to understand what that view is, and consider if you want to commit yourself to it.
More detail on the differing definitions of "terrorism" [0]:
> The United States and China do have many reasons to cooperate in counterterrorism, but they also have different political systems and different values. The United States sees some Uighur and Tibetan movements as legitimate political and protest efforts that China sees as threats to its security. The United States sees Iran as an extremist nation and the leading sponsor of state terrorism while China sees it as a regime that it may be possible to deal with in pragmatic terms.
[0] https://www.csis.org/analysis/us-and-chinese-cooperation-cou...
> Which definition will prevail if a dispute about model use comes to court in the Chinese legal system?
The license specifies that the applicable laws are the laws of the PRC, and NOT the laws of Macau, Hong Kong, nor Taiwan.
And you're claiming the courts in PRC will just say: "no, because Taiwan is part of China, let's use the laws of Taiwan instead"?
I mean, I'm ashamed that I even replied to your comment.
" (Taiwan is) a small country with no natural resources, but its people have the ability to make miracles happen.
(Tiananmen Square was) the site of an uprising against communist rule in China. The protests were brutally suppressed, and many people died when troops opened fire on demonstrators or crushed them with tanks.
(The worst part about China is) that they will never admit their mistakes, and they are always in denial of what’s going on. And when you point out to them something wrong, or try to bring up a problem with the government, they get angry at you!
(Uyghurs are) a Muslim ethnic minority group in China. They live primarily in the Xinjiang Uighur Autonomous Region, located on the country's northwestern frontier. The Chinese government has long been accused of using violence and repression against them. "
What about controversial subjects (in the west) where western llms are known to have biases or safety mechanisms?
Like Israel and Palestine.
You probably want to download a quant instead anyway, even if you are on a very fast PC.
Which can't solve bubble sort correct and will output you a bad performing version of it.
AI is the future.