Yeah, this kind of thing is part of what I meant when I criticised AWS encouraging VPC use instead of end-to-end security.
But off the top of my head, you could always use the firewall API from the lambda to open network access between it and the RDS when the lambda starts. (In addition to using certs or IAM security on your TLS connection to the RDS db)