Google tried this in 2015. It was called Project Ara.[1] A similar click-together system, with the same problems. Plus the problem that there's no good reason to make a phone that way. Google acquired that from Phonebloks and killed it.[2]
It's not a bad idea if you solve the mechanical problems and have the components produced in sufficient quantity to be very cheap. A little bigger, a lot more rugged, with a good wire management solution, and you'd have something useful.
[1] https://www.cnet.com/tech/mobile/project-ara-everything-we-k...