efficient-kan

Thanks for the link to Conv-KAN!. I had a quick look and I have a few points. Firstly, they use a different implementation compared to the original KAN paper, i.e. they use efficient-kan (https://github.com/Blealtan/efficient-kan) - maybe some of the issues are negated in this alternative implementation? Secondly, it could be that there is something that's a bit different when applying KAN through convolution layers - again, something that negates some of the holes pinpointed in that post. Thirdly, Conv-KAN authors actually claim "At the moment we aren't seeing a significant improvement in the performance of the KAN Convolutional Networks compared to the traditional Convolutional Networks".

Nice implementation I've been playing with, https://github.com/Blealtan/efficient-kan alongside @GistNoesis's.

Hi Noesis, I just noticed that your implementation, combined with the efficientKAN by Blealtan (https://github.com/Blealtan/efficient-kan), results in a structure very similar to Siren(MLP with Sin activations). efficientKAN first computes the common basis functions for all the edge activations and the output can be calculated with a linear combination of the basis. If the basis functions are fourier, then a KAN layer can be viewed as a linear layer with fixed weights + Sin activation + a linear layer with learnable weights, which is a special form of Siren. I think this may show some connection between KAN and MLP.