Yup, you could. Although connecting your own PDS hosts to the Bluesky production network won't be possible for a little while longer. Creating custom algorithms (feeds) is already possible, as it participating in the developer sandbox network, including PDS hosting.
The architecture is explained briefly here: https://blueskyweb.xyz/blog/5-5-2023-federation-architecture
Code/docs are here:
1. AT Protocol (atproto) is documented here: https://atproto.com
2. App Web/Android/iOS app: https://github.com/bluesky-social/social-app
3. API (AppView) and PDS (Personal Data Server): https://github.com/bluesky-social/atproto
4. The Relay (formerly BGS) https://github.com/bluesky-social/indigo
5. Custom algorithm (Feed Generator): https://github.com/bluesky-social/feed-generator
6. Sandbox PDS distribution: https://github.com/bluesky-social/pds/blob/main/SANDBOX.md