Open-source infra for data scientists (opens in new tab)

(magniv.app)

57 pointssachinkesiraju4y ago21 comments

21 comments

It reads open source core which means it's not fully open source. So the product page isn't exactly clear to me what's being offered. Although it is possible to have a sass with a fully open source code base of course.

jzone34y ago

We will have an open-source core framework (the Python decorator part). Kind of following the DBT model. Still very early (in alpha) so will not open source until the beta development begins.

dang4y ago

"Request early access" is not a valid Show HN. Please see the rules:

https://news.ycombinator.com/showhn.html

jzone34y ago

Hey- Jared here! I am one of the founders of Magniv. Let me know if you have any questions. We are excited to get this in the hands of beta testers in the community :)

eatonphil4y ago

The site says open-source but there's no link to code, only a suggestion to sign up for early access?

jzone34y ago

Yes we are still very early! The core framework will be released to design partners first then to the community.

1 more reply

adolph4y ago

It looks like Magniv is targeting Python in general. This is similar to ClearML. What are the differentiating points to Magniv compared to similar products?

It seems like the product also integrates with SCM systems. Are you using gitea and then containers to push code and data to execution like CodeOcean?

https://github.com/allegroai/clearml

https://codeocean.com/

jonpon4y ago

Yeah, so one of our differentiating points is that we can integrate with whatever scheduler you are already using.

On top of that a lot of the libraries that exist are focused on ML, focusing on resources/gpu etc. We are more focused on the data science side of this problem.

Similar though.

alexvybe4y ago

We’ve been trying to plan out building our data team at my company— been so hard finding a data scientist to do infra too. Wonder if this would help...

jzone34y ago

Yes! Please sign up on our website, would love to chat further and see if it can help. Our goal is to decrease reliance on data engineers when building data teams

alexvybe4y ago

Sounds good. Just signed up

keshav554y ago

Very hyped to see where this goes. Much needed for data science teams.

maxmalkav4y ago

The “our experienced ex-Facebook engineers” sounds like a bit weird claim to me, TBH. I imagine them securing a continuous flow of ex-FB employees to guarantee this in the future.

jonpon4y ago

are you ex fb, if so you are hired.

jakrim4y ago

The two founders are two of the smartest people I know. I think Magniv will be a game-changer for data science teams

jzone34y ago

Thanks man!! Now back to building..

jonpon4y ago

j / k navigate · click thread line to collapse

21 comments

FloatArtifact4y ago

jzone34y ago

We will have an open-source core framework (the Python decorator part). Kind of following the DBT model. Still very early (in alpha) so will not open source until the beta development begins.

dang4y ago

"Request early access" is not a valid Show HN. Please see the rules:

https://news.ycombinator.com/showhn.html

jzone34y ago

Hey- Jared here! I am one of the founders of Magniv. Let me know if you have any questions. We are excited to get this in the hands of beta testers in the community :)

eatonphil4y ago

The site says open-source but there's no link to code, only a suggestion to sign up for early access?

jzone34y ago

Yes we are still very early! The core framework will be released to design partners first then to the community.

1 more reply

adolph4y ago

It looks like Magniv is targeting Python in general. This is similar to ClearML. What are the differentiating points to Magniv compared to similar products?

It seems like the product also integrates with SCM systems. Are you using gitea and then containers to push code and data to execution like CodeOcean?

https://github.com/allegroai/clearml

https://codeocean.com/

jonpon4y ago

Yeah, so one of our differentiating points is that we can integrate with whatever scheduler you are already using.

On top of that a lot of the libraries that exist are focused on ML, focusing on resources/gpu etc. We are more focused on the data science side of this problem.

Similar though.

alexvybe4y ago

We’ve been trying to plan out building our data team at my company— been so hard finding a data scientist to do infra too. Wonder if this would help...

jzone34y ago

Yes! Please sign up on our website, would love to chat further and see if it can help. Our goal is to decrease reliance on data engineers when building data teams

alexvybe4y ago

Sounds good. Just signed up

keshav554y ago

Very hyped to see where this goes. Much needed for data science teams.

maxmalkav4y ago

The “our experienced ex-Facebook engineers” sounds like a bit weird claim to me, TBH. I imagine them securing a continuous flow of ex-FB employees to guarantee this in the future.

jonpon4y ago

are you ex fb, if so you are hired.

jakrim4y ago

The two founders are two of the smartest people I know. I think Magniv will be a game-changer for data science teams

jzone34y ago

Thanks man!! Now back to building..

jonpon4y ago

j / k navigate · click thread line to collapse