undefined | Better HN

0 pointsbob10291y ago0 comments

Not all data lives in a SQL database. Much of the extraction code I write does things like loading flat files from unusual sources and querying APIs.

If the source data is already in a SQL store, then the solution should be obvious. You don't need any other tools to produce the desired view of the business at that point. Transforming for an upstream schema is a select statement per target table. This doesn't need to be complicated.

0 comments

mulmen1y ago

Yeah I extract a lot of data out of Dynamo. It’s still the easiest part. Change capture just isn’t complicated. You need some basic constructs and then you’re golden. The data mart design phase is orders of magnitude more effort.

j / k navigate · click thread line to collapse

0 pointsbob10291y ago0 comments

Not all data lives in a SQL database. Much of the extraction code I write does things like loading flat files from unusual sources and querying APIs.

0 comments

mulmen1y ago

j / k navigate · click thread line to collapse