Once those basics are setup it's a matter of attracting users. First and foremost your software needs to solve a problem that people are encountering. Once there's a reason to use the project, then there need to be incentives to assist in the development. Tell users clearly how they can help and provide some issues in the style of first-timers-only issues. There's going to be a good number of drive by contributions, but if they're handled correctly some of them can turn into repeat contributions. In that vein, issues/PRs/etc need to have prompt responses to keep people interested, they need to point out resources, and otherwise convince the community members they're being heard. Overall growing and cultivating a community is a ton of work.
For context I've been maintaining a fairly popular open source synthesizer project for over a decade. Feel free to ask more specific questions and I should be able to point out some ideas.
I do not and will not use Discord, and do not like Discourse. For fast communication, I would use IRC (although do not currently have one set up); if someone else wants to provide a bridge of the IRC to Discord I would accept that though. For slow communication, I would use NNTP (which I already have set up), but I would accept contributions to provide a web interface and mail interface to the same messages (the messages are stored in a SQLite database, and I can provide a link to the database schema and documentation for those who are interested).
In the case of my projects, I do not allow others write access to my repositories. However, if someone writes a bug report or other message with a patch included, then I can review it and then possibly apply it.
I agree with your second paragraph ("Once those basics are setup it's a matter of attracting users ... Overall growing and cultivating a community is a ton of work") and unfortunately, I am not really sure what to do. Hopefully, if someone on here looks and is interested in these projects (whether mine or someone else), then they can go on there. You can also post here, if wanted.
I also agree that documentation is important (comment 26106008). (I often see projects without good documentation.)
It's 100% fine to develop things which are openly available, but not try to build a community. That seems to match your description with your current setup.
Documentation is always important, and what I am working on for my own project exclusively right now.
