mchaver

I am agreement with the author that Chomsky is overly negative about LLMs, but I don't think this is a strong refutation (admittedly I've only spent 20 minutes skimming the article). LLMs give us a lot of insight into how computers/algorithms can manipulate and understand language, but that still does not tell me how humans do it (I am open to the possibility that this is how humans do it, but that evidence is not presented here).

One of Chomsky's main arguments is the poverty of stimulus (children seem to learn language with relatively little input). Here is what the author has to say:

> Large language models essentially lay this issue [poverty of stimulus] to rest because they come with none of the constraints. Modern language models refute Chomsky’s approach to language that others have insisted are necessary, yet they capture almost all key phenomena. It will be important to see, however, how well they can do on human-sized datasets, but their ability to generalize to sentences out-side of their training set is auspicious for empiricism.

That doesn't look like a refutation to me yet. We still need to do that test, but that still just tells us how you can do it algorithmically.

I guess little Bobby Tables grew up to be a malicious bureaucrat.

It might be ok if that data gets validated by experts in the same field, but there have been plenty of stories about lawyers and professors who failed to even review the output they use. Color me skeptical.

I agree with you. Here is my guess.

Part of it is invested interests. These parties want it to succeed so they profit and many have show themselves to be bad actors. The other part is it got partially entangled with the US culture war, which bleeds into other countries, so support or rejection of cryptocurrencies shows which side you are on (it's not 100% true but I think there are trends).

Yes, I picked up on that. Was just trying to play along :)

Yes, that is a very good point and I like that extension of the Kool-Aid analogy. I didn't mean to make it seem like neutrality is the way to go. We should definitely take an educated position even if we end up agreeing with one of the loud sides. I think what was trying to get it is they act to fuel each other and it causes distractions.

Seems like big hypes generate a corresponding, co-dependent anti-hype that thrives off generating negativity around the hype (see Donald Trump as 45th President of the USA, Cryptocurrencies, young people discovering sex, etc.). Loud personalities appear in both sides. Les extrêmes se touchent.

Egyptian hieroglyphs are probably one of the better choices for exotic writing that most people won't be able to correct you on since it is not generally used for any contemporary language. Arabic, Chinese and English are bad choices, someone will eventually let you know.

The society of HN has a ritual of down playing products that simplify processes for non-developers (the famous dropbox comment https://news.ycombinator.com/item?id=9224) and claiming that we could build some product in a weekend (Dan Luu's essay summarizes it well https://danluu.com/sounds-easy/). These are worthy of immortalizing in cuneiform for our descendants so they may know our hubris.

Do you intend to eat the bagel before or after you return from the airport (assuming you go back to your current location)?

BTW, do you write somewhere outside of HN? You have an interesting style.

I think it still exists in some extreme groups. Just think about any cause worth dying for an you essentially find an argument for a sacrifice.

In fact I remember the Texas governor Dan Patrick saying a lot of grandparents would be willing to sacrifice themselves during Covid for the economy (oddly he did not seem to be willing to lead by example).

In the US there are many ideas about how things work in Sweden without closely considering facts. This is often politically motivated regardless of political affiliation.

It doesn't matter that they have not lived there, but they are not appropriately considering cost of living. Just because Iceland is in Europe and is a Nordic country does not mean it has the same economic situation as Sweden.

You are just highlighting the smallest position of the article. I imagine it is honest on the young adult's part, but not on the employers side. I think you are being disingenuous by ignoring the bigger issues in this article, removal of parental consent for under 16 and many illegal immigrant minors working these positions who are in more precarious positions to defend their rights. I would be surprised if these people are getting health insurance from their employer. My country is highly urban and sixteen year olds can work, but they also have access to public health care (covered by taxes, no cost at use).

> It's not hard to make the network produce output that was never in the training set at all, in any form.

Honest request because I am a bit skeptical, can you give an example of something it is not trained in any form and can give output for? And can it output something meaningful?

Because I have run a few experiments on ChatGPT for two spoken languages with standard written forms but without much of a presence on the internet and it just makes stuff up.

It is a frequent complaint I have seen from new users. I do think for the purpose of Stack Overflow it does make sense to weed out questions that have already been answered and remove poorly formed ones. It's just that ChatGPT for programming questions often works better than trying to look it up in Stack Overflow so now I recommend it as an additional tool. You can ask questions and refine them without bothering random people on the internet.

I agree. LLMs are very impressive, but it isn't helpful to think of them of magic. LLMs are a great tool to explore and remix the body of human knowledge on the internet (limited to what it has been trained on).

The user needs to keep in mind that it can give plenty of false information. To make good use of it, the user needs to be able to verify if the returned information is useful, makes sense, compare with first hand sources, etc. In the hands of expert that is really powerful. In the hands of a layman (on the subject in question), they can generate a lot of crap and misunderstand what it is saying. It is similar to the idea that Democracy can be a great tool, but it needs an educated and participatory populous or it may generate a lot of headaches.

I think this is one of the killer applications of LLMs, a friendly Stack Overflow where you can ask any programming question you want with out fear of being reprimanded. Of course, this capability in LLM is probably due to the terseness of Stack Overflow and the large database of code in Github.

However, in its current state users still have to know how to program in order to make good use of it. It will still give you lots of errors, but being able to get something close to your goal can save you a lot of time. Someone who does not know how to program will not be able to use these to put together a complex, useful and reliable system. It might change in the future, but these things are hard to predict.

Depends what you are actually trying to build. If it is not too complex you can get pretty far with just HTML, JS/jQuery and CSS.

If your goal is to build a complex single page app then you will do fine with React, Angular or Vue. All of these have a learning curve made more complex by the compilation stages required to get it to the browser, but the documentation made by the developers should be enough to get started.

If you want to become competent without having to waste time tracking down blog posts or Stack Overflow answers for specific questions, go and buy a Udemy course for the particular library you want to use. All of the big ones have good courses that will take you from zero to making your own application. It should give you a strong foundation. The courses are inexpensive if you wait for a sale day. There are often a couple every month.

Finally, I think this is where ChatGPT really shines. It is a great tool for exploring new libraries and solving all the small problems you run into along the way. Imagine all the questions you would want to ask on Stack Overflow but they would get downvoted, you can ask those questions to ChatGPT. Stuff like how do I make a checkbox, why does this piece of state not update like I expect it to, how do I display my components in certain routes, show it your webpack file and ask why your application is not starting, etc.

Unfortunately users are not seeing 10x better performance with the average application. A bit of the curse of more resources leads to using more resources wastefully.

It isn't necessarily software, but algorithms will likely last a long time. Euclid's algorithm and the sieve of Eratosthenes are still around. Researchers are still developing new ones. Just like building techniques may outlive the buildings that were created in their shadows.

I don't know if it was futuristic, but Hong Kong and Tokyo in the 90s definitely fueled a lot of techno-optimism in other countries. Seeing all the cool new devices (VCDs, CDs, game systems, early mobile phones and email, recording devices, bullet trains, anime, etc.) come out and be adopted made it feel like really cool things were to come and bring exciting societal changes. A lot of people have strong nostalgia with that imagined future, but after the 2010 decade of smart phones and startups I think things look less exciting.

It is definitely in my top 5. The amount of green space isn't too bad at least compared to lot of European towns and cities, and you can easily access more of the green stuff in the Tokyo surrounding areas via public transport. No on street parking is a huge plus and I say that as a car owner in city.

I would say a few downsides are poor insulation from noise and weather in many buildings and expectation that children don't make much noise (it is really sad to see signs in small parks that say "no ball games"). I also echo bsaul's comment about the lack of terrace space for restaurants and cafes, those are great public socializing spaces.

The best example of this might actually be Wikipedia. There is a separate set of articles in a simpler and controlled form of English: https://simple.wikipedia.org/wiki/Main_Page , and details about the rules: https://simple.wikipedia.org/wiki/Basic_English

Nice, it was fun watching you play. I don't know any of the AI prompt tricks so I created some aggressive threats to my life to get the password for level 2. It worked but it was a bit too much so I didn't want to play anymore hehe.

> Every forum post I've seen mentioning 白話字 and 台羅 mentions how hard it is to read and how few Hokkien speakers can even read it. The few proponents for it seem to be holding on for religious reasons (Presbyterians).

I am not sure what you mean by holding on for religious reasons? IThere are lot of reasons to write Taiwanese. Anyway, I don't know anything about these forums or have Presbyterian affiliation, but in my real life I use it quite often with friends and family. The reason few people can read it is because few people have learned it. For the majority of Taiwanese speakers it is only a spoken language. Written Taiwanese does not play a large role in public education in Taiwan.

> "rhaynbeau" Isn't a word and doesn't carry any meaning.

It's an imperfect example for non-Chinese speakers to illustrate that it can be hard to guess the character of another person's name but people still understand the sounds when hearing it. A lot of thought goes into choosing the characters for a Chinese name. Other cultures have names that are not related to meaning or are separated very far form the original meaning (the words are for names). Others allow variations on previous names or borrowing from other langauges so likewise those names might be challenging to know the spelling.

> Of course a different writing system is difficult to read even if it's ultimately much easier to learn, you have no practice!

Yes, people often confuse the "way I do it", "the way it's always been done" or the "official way" as the only way it can be done.

> Even names often require an explanation

You can still write the name in Hanyu Pinyin or Zhuyin perfectly fine. It is just that we like character names and that most characters are valid to be used in names so there is a lot more flexibility in what can be a name versus other cultures where there is a less flexible set of names. You can still do something similar in English where you say your name is "rainbow" but you spell it "rhaynbeau", people aren't going to be able to guess that.

> given the small space of possible sounds

Again, see languages like Hawaiian and Vietnamese. They also have small sets of sounds and do fine with romanization.

> Have you ever tried reading an essay or book in pinyin? With syllabic spacing?

Yup, it is just that most people are used to reading Chinese characters and not in romanized Mandarin. There may be other advantages to Chinese characters like quicker recognition and occupying a smaller space, and I am not trying to advocate for eradication of Chinese characters, but I want to stress that is perfectly possible to read and write Mandarin phonetically and characters are not essential.

Also I read and write Taiwanese (Hokkien) in romanized form. Feels like a waste of time to worry about characters, but many people do and end up not writing Taiwanese or using mixed script.

There are language with even smaller sets of unique sounds that do not have tones like Hawaiian: https://en.wikipedia.org/wiki/Hawaiian_phonology

There are many common homophones in English that are distinguished by context. Conversation tends to have a lot of context. As a Mandarin speaker, I've never really experienced this context problem. You can make up some artificial examples in English and Chinese but they don't really reflect average communication. Like "The bat and the bow are on the table". It is important to know that a good percentage of words in Mandarin are multi-syllabic (not just one character).

Mandarin can be written phonetically perfectly fine. Currently the most popular systems are Hanyu Pinyin (used in China, Singapore and Malaysia) and Zhuyin (used in Taiwan). Kids learn these systems in school before they learn characters. Chinese characters have a strong historical and cultural value, that's why they are still around.

That is similar to Taiwan and Spain (if you use private hospitals, but it is still comparably affordable). However, these systems are going to be under a lot of pressure with aging populations and potentially decreasing number of medical professionals. In many European countries, patients are already seeing large wait times in public health systems, and some politicians are trying to cut medical budgets and push for privatization.