undefined | Better HN

0 pointscolinator2y ago0 comments

Well, our brains are closer to spiking neural networks than 'regular' neural networks. And they work pretty well. For the most part.

I feel like SNNs are like Brazil - they are the future, and shall remain so. I think more basic research is needed for them to mature. AFAIK the current SOTA is to train them with 'surrogate gradients', which shoe-horn them into the current NN training paradigm, and that sort of discards some of their worth. Have biologically-inspired learning rules, like STDP, _really_ been exhausted?

0 comments

ilaksh2y ago

But this group claims to have demonstrated a way to use SNNs to run LLMs effectively and with vastly less energy usage.

p1esk2y ago

If OpenAI or DeepMind makes such claim I'd pay attention. Otherwise it's always some (usually hw) guys trying to get a grant, or even just publish a paper.

p.s. People interested in biologically inspired data processing algorithms should look at Numenta's papers (earlier ones, because recently they switched to regular deep learning), and especially learn their justification for not using spikes.

esafak2y ago

They switched? Why? There goes their whole raison d'etre!

j / k navigate · click thread line to collapse