devetec

Distributed GPT training doesn't really work (sadly). Maybe someone will get a good solution to that, though.

There is a FOSS version of GPT-3 being worked on by EleutherAI, but they are only at 6 billion parameters so far, while the largest model of GPT-3 is 175 billion (more parameters is better (usually)). EleutherAI is getting more compute from CoreWeave to actually train the GPT-3-like FOSS model, so that's something to look forward to eventually :). Github Copilot uses Codex though, which seems to be a GPT model trained on just code by OpenAI. It wouldn't be too hard to train a FOSS version of Codex on the open-source code of Github and other sources.

Seems reasonable, however you can make the case that this isn't the course of action that should be taken. But this is Singapore, the country with only 2 infections per 100k people every week. Along with their high vaccination rates, at least compared to nearby nations, at 44.1%, this is a reasonable route that Singapore is choosing to take.

Now, you can get digital e-cards from many major libraries around the US without even having to step foot in a library, giving you access to e-books galore. I think that the library card won't die, but the fact that e-cards are now a common everyday occurrence doesn't bode well for them.

Doesn't something like that already exist: Lenovo's Yoga Book C930? It had 2 screens, with one being an e-Ink keyboard. Very useful tech.

Also filled with many nuts, though I wish more people would actually use it. It's a very good alternative to YouTube though.

...wow. Congratulations!

Technology tends to follow a sigmoid curve. First nothing happens, and then it seems like the world is changing and it is unstoppable, and then it falls into a new normal, not changing much. We never know when it slows down, reaching the third stage, until it's happened. For all we know, this is the furthest we get. Personally, I don't believe that, but food for thought.

Large enough dataset + some way to see context. There could be an ambiguous word that is only found through accompanying media. A multimodal AI would be needed for that task. Maybe something like https://blog.google/products/search/introducing-mum/ would be up for the task, maybe not. But you are correct, people say a machine can't replace them until they do.

It's not an API. It's just a demo site :)

They're not a company. They made some decision to not add more to the Pile, as they think it is complete enough. However, you could go to the Discord and propose a restarting of the Pile project, they allow those. On the Github, it looks like the next step after a restart would be the Pile v2, a multilingual dataset. Good luck!

Nope, probably not. It's using JAX, which works on both GPUs and TPUs.

Part of the Pile dataset is the Ubuntu IRC stuff