HNHacker News
TopNewBestAskShowJobs

ak3ra

5 karma · joined August 18, 2019

submissionscomments
ak3ra··on Distributed AI Research Institute
Distributed AI Research Institute, founded by Timnit Gebru
ak3ra··on Geoff Hinton 2021 – How to represent part-whole hierarchies in a neural network
This paper does not describe a working system. Instead, it presents a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM. The advances include transformers, neural fields, contrastive representation learning, distillation and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy which has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language
ak3ra··on Distill Pub – Curve Circuits
We reverse engineer a non-trivial learned algorithm from the weights of a neural network and use its core ideas to craft an artificial artificial neural network from scratch that reimplements it.