If you're not using AWS, the use of Parameter Store (or Credstash) becomes a turtles problem--because you need to provision AWS credentials. At that point, whatever you provisioned AWS credentials into might as well store your other secrets, too.

If you are using AWS, EC2 itself is a trusted third party that grants the appropriate permissions for your executing system (container, instance, whatever).