What does it matter if your interest is common for people with your background or not? I would be just as happy if I knew I were the only one in the world that was interested in something, or if I were one of millions with the same interest. Trust yourself and your own mind.
Having said that, I know or have heard of several people that have gone from EE/CS into neuroscience. One of my friends is a programmer and EE graduate like you. He spent quite a bit of time studying neuroscience and taking biology/chemistry prereqs as a grad student recently. Jeff Hawkins (Palm, Numenta, wrote a book called On Intelligence) is yet another person that fits the pattern. Does this mean anything important? I don't think so, unless you're contemplating changing careers. I think it's just that this is a relatively new but promising field (I know it's over a century old, but bear with me), just like molecular biology was the hot thing to get into during the 50s if you were into physics.
