If you get to the point of having a performance or security team, you realize your org is big enough you need a couple people focusing on it full time (and their role is often helping other developers implement best practices.) By ditching them, you’re acknowledging literally that you’d like to focus less on that area in the future. As a result, product teams will do their best, but it’s impossible for every dev to be up to date on every best practice for every topic in web development. So you won’t do as well.
Maybe that’s ok as a small startup, but when your audience is incredibly large and you operate at scale, it’s not reasonable to have subpar security, performance, accessibility, etc. If you are subpar in these areas, you loose money because your audience is large enough that many are impacted.