The fact that I need to connect to some peer-to-peer network or third party service for every transaction is a limiting factor. It reduces the efficiency of the system and increases latency, and it adds additional points of failure.
In fact it can be solved for electronic transactions and there is a mountain of published research on the topic that dates back to the late 80s. Here is a recent research paper:
https://eprint.iacr.org/2017/1220.pdf
The idea is to ensure that users who attempt to double spend can be identified and penalized later (e.g. by being added to a blacklist; some systems actually ensure that all transactions in which the cheating user participated can be identified, so merchants who try to evade the blacklist can also be penalized). The money has to be issued by a bank under the security definition. If anonymity is not a requirement -- and presumably a Bitcoin enthusiast does not care about anonymity since Bitcoin is not anonymous -- this can be easily achieved by using ordinary digital signatures.
