undefined | Better HN

0 pointsTechIsCool2y ago0 comments

Yeah its been nice to leverage this while working on some of our larger multi TB non-partitioned clusters. We have seen snapshots restore in under 10 minutes across AWS Accounts (same region) as long as you already have one snapshot shipped with the same KMS keys. We have been upgrading DBs to lift out of RDS into Aurora Serverless.

If anyone here knows how to get LSN numbers after an upgrade/cluster replacement. I would love to hear about it since its always painful to get Debezium reconnected when a cluster dies.

0 comments

brentjanderson2y ago

I looked at getting LSN numbers after an upgrade/cluster replacement, and IIRC restoring from a snapshot emits LSN information into the logs, but it's a bit of of a mixed bag as to whether or not you get the __right__ LSN out the other side. Because the LSN is more a measure of how many bytes have been written within a cluster, it's not something that meaningfully translates to other clusters, unfortunately.

TechIsCoolOP2y ago

Agreed, the snapshot does output a message in the logs but based on our conversations with AWS it was suggested that we use the SQL Command to determine the LSN. Sometimes depending on revision you won't get the logs and other times the log line is emitted twice based on the internal RDS consistency checks. Makes me long for GTIDs from MySQL MariaDB Galera[1]. They worked super well and we never looked back at my last company.

[1] https://mariadb.com/kb/en/using-mariadb-gtids-with-mariadb-g...

gunnarmorling2y ago

Can you expand on what those problems were you had with reconnecting? Would love to better understand this and see whether there's anything which can be improved in Debezium.

j / k navigate · click thread line to collapse