Hacker News Reader

Comment by fintechjock | Hacker News Reader

Correct me if I’m wrong, but this is a big bet on a company that just had their lunch eaten on another product by open source, no? Didn’t see much of Dall-E after Stable Diffusion release

View on HN

It would be incredible if a chat gpt competitor came into the self host able, open source, niche like stable diffusion.

The options I've seen today are no where near as good as chat gpt

There are open source / OS model-based projects competing with OpenAI’s text generation. At least one, NovelAI, also has a Stable Diffusion fork incorporated into their paid product.

At least so far none of them are close to what’s been achieved with ChatGPT, and that could in itself be driving some of the hype. ChatGPT was probably more impressive to me on the first impression being familiar with the limitations of existing AI text gen projects.

I think the problem is time. To replicate the OpenAI RLHF architecture, they need their own high-quality dataset, which takes time to create. Without details about the hyperparameters, RL architecture, and omitted steps, they need to test a lot of things, which takes time and money. It requires more resources than it took SD to replicate DALL-E 2, which took months and was an easier task.

Check out Constitutional AI from Anthropic. They automated "RLHF" by simply writing a few rules.

Thanks, just skimmed the paper, I think that "they automated RLHF" statement is maybe too strong here, there is still manual process but it seems like they optimized away a lot of manual labeling work.

There is Open Assistant by LAION (WiP): https://github.com/LAION-AI/Open-Assistant

Reply on news.ycombinator.com