I am not aware of any DL startups that are interested in techniques which are incompatible with GPUs and back propagation. I think this is a solo journey through the dark forest for now.
Enjoy your adventures in the Dark Forest.
I don't know that classical architectures are infeasible to perturb as noted in other comments. I just think we have a really interesting knob to turn in this specific case that doesn't seem to exist in other architectures.
How would you go about constraining the resource consumption (by up to half!) of a classical ANN while preserving realistic hopes that it will recover between training epochs? And, we are talking real resource reduction here not some academic thing. You can see the difference in task manager. That's actually the primary tool I've been using to determine how to tune the parameters of this scheme.
See for example: https://arxiv.org/abs/2404.02258
They have a fixed compute budget which is lower than what the LLM need, and dynamically decide to allocate this compute budget to different part of the network.
So its not exactly what you propose since here the compute budget is fixed (that's the point of the paper: to make the network learn how to allocate the resources by itself) but its dynamic for each part of the network, so it shows that its possible.
In academia there's quite a bit of stuff on neuromorphic computing going on.
Still it, like alternate activation functions, data beyond matrices and vectors and non-linear networks, remain very unstudied.
Is your thought that it is some novel approach that nobody is doing and has obvious market implications? Even then, the implications matter only if you are in a position to take advantage.
The Internet used to be a place people shared cool shit because it was cool, not for some monetization scheme or personal gain beyond the joy of sharing. Content creators (as a job) and hustle culture are a blight.
I am an academic researcher (not in this field) and everything I do is 100% free and public, but I still either patent or publish it, because getting the credit is what allows me to keep doing what I love. I actually hate that part, and would rather just do the work, but I already push that line too far sometimes, and risk losing my funding aka "publish or perish."