Pure-PyTorch Implementation of Kolmogorov-Arnold Network (Kan)

github.com

41 pointsadif_sgaid3 comments

KANs are equivalent to MLPs

https://www.reddit.com/r/MachineLearning/comments/1clcu5i/d_...

Yes, that should not be surprising at all to anybody with a minimum of knowledge in statistics.

Neurons in MLPs are linear models, and KANs simply use basis expansions for splines. That's exactly the same trick that was used to make additive models, ie modeling arbitrary smooth non-linear effects within the linear regression framework. Known since 1989.

Not ranting towards you in particular but rather modern ML academic community.

Yes, i agree with that. Kolmogorov-Arnold theorems most work has stuck with the original depth-2 width-(2n + 1) representation, and did not have the chance to leverage more modern techniques (Like.. back propagation techinque) to train the networks.

The biggest bottleneck of KANs lies in its slow training.from the research paper they had observed that KANs are usually 10x slower than MLPs, given the same number of parameters. So, KANs’ slow training is more of a engineering problem to be improved in the future rather than a fundamental limitation.

It's more practical for ML Infrastructure industry leaders to take charge of solving this engineering problem.

umamaheswar edara

Reply on news.ycombinator.com