344 karma · joined May 15, 2021
At this point it’s a build or buy type deal for models using a model registry service.
Data versioning still feels unsolved to me.
Not sure what Spacy is doing under the hood these days but always thought of “neural net” as word2vec type model which won’t hit the above definition.
I hope it did not detract too much from the point of focusing on subtasks and modalities for FOSS as GPT 4 was built on a $163 million budget.
Finally, good point. We’ve got no idea of what OpenAI’s MoE approach is and how it works. I went back to Metas 2022 NLLB-200 system paper and they didn’t even publish the exact details of the router (gate).
I had to read this two times as it’s poorly worded.
An any rate, it should be 80% and not 35%. You don’t need to go into an office to make software.
There’s something just sinister how employers are handling this as it’s got nothing to do with actual output.
FANNG hires?
I am just going to leave my hit here.