The next iteration depends on NVIDIA & co, what we need is sparse libs. Most of the weights in llms are 0, once we deal with those more efficiently we will get to the next iteration.
that's interesting. Do you have a rough percentage of this?
Does this mean these connections have no influence at all on output?