The addicts and lonely line up waiting for these places to open, they spend everything they've got and the gambling places encourage it by providing them with free coffee, snacks and in some cases dinner.
No, gambling isn't immoral, but praying on the addicts, the mentally challenge and the lonely is. If you can't stay in business without exploiting the weak, you have no right to exist. The only negative consequence I see from banning gambling is the potential dangers of a black market.