undefined | Better HN

0 pointspattycake234y ago0 comments

Seems like it's a much higher rate limit, but it exists none the less, and Shopify's scale has also grown significantly since 2018 (when that article was written) - so it was probably a valid way for them to go.

0 comments

sciurus4y ago

I think two things happened that are covered in that blog post

1) The performance per partition increased

2) The way AWS created partitions changed

When I was at Mozilla, one thing I worked on was Firefox's crash reporting system. It's S3 storage backend wrote raw crash data with the key in the format `{prefix}/v2/{name_of_thing}/{entropy}/{date}/{id}`. If I remember correctly, we considered this a limitation since the entropy was so far down in the key. However, when we talked to AWS Support they told us their was no longer a need to have the entropy early on; effectively S3 would "figure it out" and partition as needed.

EDIT: https://news.ycombinator.com/item?id=30373375 is a good related comment.

j / k navigate · click thread line to collapse