Mainly the mental overhead and the risk of confusion or mistakes. Presumably you still need some co-ordination to make sure two people don't try to release at once. (What do you do if someone else merges a feature to develop while testing on staging is ongoing?)

> we merge it to main and then out to production.

Hmm, so what you deploy to production is a different artifact from what you tested on staging? I'd find that worrying.

> I think you can release small parts with testing around it. I often release half of a back end feature, then the other half, then the front end all in separate branches and releases. All I really need is to have the parts broken down into logical testable chunks.

It's sometimes unavoidable, but my thinking is: yes you can unit test, but how can you possibly know that you're testing the right thing if your change isn't user-visible? You can confirm that your code works the way you think it works, but you can't confirm that it actually delivers the functionality you want. IMO it's only worth putting something in the shared branches once you know you're not going to significantly rework it (otherwise you're causing as many conflicts as you're avoiding), and you can only know that when you've actually tested it end-to-end.