How to track costs in multi-tenant Amazon EKS clusters (opens in new tab)

(aws.amazon.com)

64 pointsAjayTripathy5y ago6 comments

6 comments

Another way to do that is to buy commercial SaaS vendor monitor AWS costs: CloudHealth or Cloudability.

webb5y ago

There would be a number of differences with these products. First, Kubecost does not egress any data from your infrastructure, it's based on open source technologies (e.g. Prometheus), and because it is build specifically for Kubernetes it takes a pretty different approach for determining the cost of a tenant, workload, etc.

Disclaimer: I'm a founder of Kubecost.

simonebrunozzi5y ago

Congrats on being the founder.

>The default installation of Kubecost includes an optimized Prometheus server that only contains metrics that are useful to Kubecost. This optimized version retains 70-90% fewer metrics than a standard Prometheus deployment. You can also use an existing Prometheus installation.

I really like this approach. What do customers say about it?

1 more reply

gbrayut5y ago

Is there any way to setup a sharding solution for Prometheus? Like instances per AZ or that only scrape a certain set of nodes? On our larger clusters we had issues even with the optimized Prometheus instance in kubecost, and it's starting to look like metrics collection/processing may be a big driving force on our cluster scaling designs.

AjayTripathyOP5y ago

There's definitely ways to do this by configuring your scrape configs to ignore sets of nodes. Curious though when you last tried kubecost out? We've built out some caching mechanisms in the product over the last month or so that should dramatically reduce load / memory consumption on prometheus. If you reach out on our slack kubecost.slack.com we can discuss more about expected Prometheus resource consumption.

j / k navigate · click thread line to collapse