I concur.
I do think software like gotosocial is the future of ActivityPub, it certainly isn't there yet, indeed.
Mastodon, I'm afraid, has tainted ActivityPub¹. Maybe even beyond fixing. Mastodon is architectured and developed for large servers. It's performance is terrible (It's Rails) for a one-man-show, but optimized and pretty good, for a server (or fleet) for (tens)of thousands of people. There's no incentive to change this. If a choice has to be made between "ease or operation for a single person instance" over a "ease of operation for a thousands-people instance" the latter will always win. Same for performance, security and features.
Gotosocial, OTOH, has the incentives reversed. I'm convinced it will only become easier to host for your personal instance. Only gain more features for such an instance and prioritize fixing issues so self-hosters benefit, rather than community-admins.
¹Edit: AP isn't designed around large federated servers. The protocol, albeit convoluted, is perfectly fine for a situation in which each user is her own server. It's mastodon that has turned AP into a de-facto "one-server-thousands-users" model. Without Mastodon, AP wouldn't exist, I'm not trying to make Mastodon look bad, I like the project.