Did not even know VPC peering was a thing. In the past I've often used Amazon RDS with Heroku Web servers for hosting Rails apps. For my current project I initially started out with Aurora Serverless which is VPC only, so I thought my web servers also had to be in the same VPC, so I decided to use Elastic Beanstalk instead of Heroku.
I do wish Amazon had an offering like Heroku. While Elastic Beanstalk works pretty good it's nowhere near as nice or easy to use as Heroku. It has gotten a bit better with Amazon Linux 2 [Procfile, platform hooks], but still leaves a lot to be desired. I wish I could scale up/down faster like you can with Heroku. There is probably a way to improve it, but right now the way it's set up any deploy, config change, etc bundles the gems and precompiles the assets which take a while. So deploy/scale up take around 8-10 minutes. It's been a while since I've used Heroku but from what I remember it's pretty much instant to add more dynos.
I also ran into a couple of other annoying issues with EB.
1. There is a bug if using Amazon Linux 2 Ruby 2.7 deploying a RoR app where assets are not precompiled on configuration changes. It was an easy fix, just needed to add a script to .platform/confighooks/predeploy but it should have just worked.
2. When migrating to Amazon Linux 2 I deployed a version of the app that needed some code changes to work on Amazon Linux 2. There is no way to disable rollback on failed deploy on EB. Setting the "Ignore Health Check" option to true does not help here. I ended up in a situation where I needed to both push a code update and update some environment variables. However any code update I would push would fail because of the missing environment variables and rollback and any configuration change to update the environment variables would rollback because the current code had an error. I ended up having to tear down the environment and recreate it to get out of this loop.
3. Another minor issue is the Monitoring/Health dashboard. It will show warning/degraded/severe during scaling operations. For example when you trigger a scale down event it will complain:
"Environment health has transitioned from Ok to Warning. ELB processes are not healthy on 1 out of 6 instances. ELB health is failing or not available for 1 out of 6 instances."
Like of course it's not responding, the autoscaling group just terminated it.
I feel like Elastic Beanstalk really could be a Heroku competitor if AWS cared enough to put the work into it. It just needs some polish and quality of life improvements, but I guess this way they still make their money and Heroku has a good business middle-man'ing it.
I'm wondering if perhaps I would be better off using ECS or EKS, or if that's what AWS wants me to use hence why they are not putting much effort in EB.