thorum

You might check out this list from espnet. They list the different corpuses they use to train their models sorted by language and task (ASR, TTS etc):

https://github.com/espnet/espnet/blob/master/egs2/README.md

It has a big problem with hallucination in Chinese, random extra syllables all over the place.

I’ve found telling it to pay attention to nuance helps as well.

> Furthermore, the study also reveals 21% of those with a 3G handset aren’t willing to upgrade, and will simply continue to use their phone without internet connectivity.

Seems reasonable. Is this supposed to be a bad thing?

My experience is that GPT-4 is quite good at most languages, including hard ones like Chinese - especially when paired with a “verification” step, where you copy something it wrote and start a new session and ask, “Is this sentence well-written <language>?”

It has a surprisingly deep understanding of nuance and context. You can copy some Chinese text that Google Translate completely fails to translate, paste it into GPT-4 and ask “Please explain this text including any nuanced meaning that a foreigner would miss“ and get a detailed, coherent explanation.

At least in the grassroots language learner community, there seems to be a growing sense that the most efficient way to achieve real fluency in a second language is: regular long-term effort to read or listen to text or media in the language that is just a little above your level of understanding. Notably different from how duolingo works.

https://en.wikipedia.org/wiki/Input_hypothesis

FYI, this is not a new project. Here’s an HN discussion from 6 years ago: https://news.ycombinator.com/item?id=15399132

> Despite targeted outreach, we've noticed that community members are not returning to use the tool after their initial interaction. There has been minimal engagement apart from a spike during a mid-November focus group. This indicates that Sawt is not helpful enough yet for people to want to come back.

Perhaps they have trouble thinking of additional questions to ask? Their first few interactions probably cover the things they care most about. They get an answer and that’s it. Maybe a “subscribe to updates whenever this topic comes up in a meeting again” feature would be useful?

Wonder how much longer Poe app can hold out against this kind of competition.

The fact that it’s technically true doesn’t really help her. Getting hung up on the technical details of Harvard’s code of conduct just comes across as uncaring about the big picture issue, that is, the thing people outside Harvard actually care about. Part of the job involves clear and sensitive public communication about politically charged topics, and that’s where she failed.

> let's talk more.

Unfortunately if that happens, we won’t be able to!

My experience with other tools like xtts is you really need to have a studio-quality voice sample to get the best results.

The analogy falls apart with AI, though.

- American AI “safety” practices have little or nothing in common with best practices in other fields of engineering, to the extent they can be said to exist at all (most AI safety work focuses on making sure the AI doesn’t say anything that might offend someone).

- When a rocket blows up, people die but we learn from the mistake. When an AI seriously “threatens humanity”, humanity dies and we very possibly don’t get a second chance.

Children are born with endless curiosity about the world and in too many cases our schools systematically squash it.

I’ve never understood why the default assumption when we see a kid struggling to pay attention in class is that there is something wrong with the kid, instead of the obvious: the class is boring.

Has anyone tried using an LLM for the retrieval stage? Instead of using vector embeddings, have a (small, fast) LLM literally scan the entire corpus in chunks extracting relevant sections?

The average person’s storage needs will likely increase a lot over the next decade: AI models, AR/VR, and six-degrees-of-freedom video are very storage intensive technologies that are close to going mainstream.

The OpenAI moat is private datasets, a sizable chunk of the best talent, proprietary discoveries (like whatever Q* is) that keep them ahead of everyone else, and lobbying advantages that will help them to survive future AI regulations. It doesn’t matter that much if you can build your own AI system with open source tech as long as OpenAI’s solutions are better and cheaper to use.

Apart from the advantages for privacy and Siri, imagine if the OS included a shared system AI that any app could query. That would unlock so many new features that are either impossible or impractical today.

Anecdotally, I finetuned Mistral 7B for a specific (and slightly unusual) natural language processing task just a few days ago. GPT-4 can do the task, but it needs a long complex prompt and only gets it right about 80-90% of the time - the finetuned model performs significantly better with fewer tokens. (In fact it does so well that I suspect I could get good results with an even smaller model.)

Important to note this is not a ruling about copyright infringement: Stability has not yet even presented a defense on that subject. Instead they argued that although they are a UK-based company, no people or computers residing in the UK were involved in the development of Stable Diffusion, so a UK court has no jurisdiction on the matter. The judge didn’t find the evidence for this (which includes the CEO saying he asked around and is “confident” it’s true) conclusive enough, so things will move forward. It may turn out to be true later, but Getty will get a chance to present counter-evidence, and so on.

OpenAI said in Spring that GPT5 was not being trained yet:

https://www.theverge.com/2023/4/14/23683084/openai-gpt-5-rum...

Then hinted a few weeks ago that work had started, but probably not actual training:

https://decrypt.co/206044/gpt-5-openai-development-roadmap-g...

Even if training has started since then, that doesn’t seem like nearly long enough to have it completed by the end of the year, especially if they intend to go through an extended period of testing & aligning work as they did with GPT4 (which they tested for six months before releasing).

Current ~70B models like LLAMA 2 70B are on par wih ChatGPT 3.5. The best smaller models can appear on par at first glance, but they hallucinate at a much higher rate and lack knowledge of the world. GPT 4 ‘gets’ things at a deeper level and no open source model is even close.

A year is a good timeframe to evaluate things: the rest of the world seems to lag behind OpenAI by around 12-18 months, at least with LLMs and image generation.

On the other hand open source tech usually has additional features for controlling output that OpenAI never bothers to implement, like llama.cpp’s grammars or ControlNet. So in that sense open source is usually ahead of OpenAI in terms of customizability.

I believe the primary outcome was users switching from the popular Apollo app, which closed down, to Narwhal which stayed active, added a paid plan, and rolled out a redesign to integrate many features from Apollo.

What are you basing that on?

> Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL.

For image quality, sure - language understanding is still an issue. SDXL can generate a beautiful image, but if it doesn’t show exactly what you asked for in the prompt, on the first try, there is still room for improvement. The gap between LLMs and image generators in this regard is huge.

When we reach the point when average people can easily run today’s models on their own devices, today’s models will no longer be SOTA and there will still be demand to run better models in the cloud.

It doesn’t seem to detect subdomains set up with Kubernetes ingresses, based on results for one of my domains, so that might be a place to start research.

Threads was a breath of fresh air at first and I followed a bunch of people I already followed on Instagram and Twitter. Opening the app today there is exactly 1 post from someone I follow and pages upon pages of ‘recommended’ content from people I don’t care about, much of which is outrage bait and political content. Talking to other people I know this experience seems to be common.

I understand the idea of wanting to make the app feel alive on day one by including content from popular influencers, but I genuinely think they would have had better retention if they went the other way and only showed you what you wanted to see - or at least separated the recommended posts onto another tab and gave better controls for filtering it.

Honestly seems like the market itself should have safeguards against this kind of thing.

5G surgery has been done, though: https://www.huawei.com/en/huaweitech/industry-insights/outlo...