Counter anecdote. I've never seen a Gadsden flag flown by a confederate flag. It's mainly flown by libertarians and others who respect privacy and freedom.
Either way, confederate flags aren't normally flown in racist intent. Most people fly them to represent their heritage. The Dixie line, Outlaw country, etc.
I wouldn't be surprised to see a libertarian from deep Georgia fly both and not be racist in the slight.
I think you need to leave your bubble and learn about the people you are trying to demonize.