If you allow self signed certificates, anyone who can MITM traffic can masquerade your site just like with http
Self signed does however stop passive fibre taps - to intercept you need to MITM.
There then the “remember this cert” option. If I visit www.selfsigned.com on a secure network, my browser remembers the certificate. If I then travel to another network with a MITM, my browser can flag up a warning. This is how SSH works.
However I’m not too concerned by SSL certificates as a centralised point - my browser trusts dozens, probably more than 100, root certificates. That’s not centralisation.