one of the problems we've noticed with ETL pipelines is cleanly getting data from spreadsheets into data warehouses. many businesses still have a lot of data in spreadsheets that don't look like a database table with rows and columns.
on the other hand, there are data engineering teams that are building highly powerful data pipelines that require that data be in a relational format.
right now - to get spreadsheet data into a data warehouse like Snowflake, data engineers have to write their own ETL code for spreadsheets or use something like stitch data's google sheets integration to get the data into the data warehouse.
however, this is error prone because there is no immediate feedback to the business user when they make edits to the spreadsheet - it just silently fails later with an error in the logs when you are trying to extract the data.
it also requires the spreadsheet user to format their spreadsheet to look like a relational table (which is annoying for finance and operations users just wanting to use a spreadsheet) and they have to keep track of mentally where they can or cannot edit the spreadsheet.
so - to solve this problem, we built structure sync, an easy way to setup spreadsheet ETL inside of google sheets so that business users can edit spreadsheets with immediate feedback on whether they will sync correctly and data engineers can easily build a transform pipeline that turns any spreadsheet into data that looks like a relational table.
we currently are waiting for google to approve our app - so please let us know if this is a problem in the comments and sign up on our sign up page!
we are also looking at supporting microsoft excel and other data warehouses, let us know in the comments if this is something you'd be interested in.
Check us out @ http://sync.structure.rest
No comments yet.
