Show HN: Ascend.io – Smarter Data Pipelines
I’m Sean, the founder of Ascend.io (https://www.ascend.io). I’m really excited to post here and announce the launch of Ascend.io, a radical new way of designing, scaling, and automating data pipelines. Ascend is the result of nearly 4 years of development effort for a team that is now 30-strong, and I would love for you to give it a test drive and let me what you think.
I’ve felt this pain since I wrote my first MapReduce in 2004 (using Sawzall @ Google), and in the 15 years since, things have not improved at the pace of other parts of the technology ecosystem. When I went about trying to fix this, I took a page from other technologies, particularly the migration to declarative models such as those found in React and Kubernetes.
With Ascend, you just declare what you want the end-state of your data to be and the Dataflow Control Plane automates the orchestration, infrastructure, and persistence to make it happen. Data can arrive late, schemas can shift, logic can be updated, and the control plane generates new pipelines to propagate, backfill, and repair existing datasets. With this approach, pipelines take a lot less code to build (current record is reducing LOC from 100k to 2k), they break less, and you get to actually sleep through the night without being paged.
I certainly think this is a huge step forward for the world of pipeline development and making pipelines suck less. But, I really want to know what you all think! Anyone who is interested, I can get you setup with a free trial (https://www.ascend.io/get-started). Can’t wait to hear the feedback!
~Sean