Show HN: Open-source AI video editor (opens in new tab)

(github.com)

268 pointsdrochetti1y ago50 comments

Hey HN community! I'm one of the lead devs of this project at fal.ai and we created an open source lightweight video editor powered by the latest media AI models. The main goal was to tackle some challenges when dealing with complex media handling and encoding on the browser.

It all started as an internal experiment but as we tackled some of the issues it was clear there could be some value sharing it with the open source community.

Some of the key points and tech stack details:

- It uses IndexedDb, so all data is local (i.e. no auth, no cloud db)

- Multiple AI models for video, image, music and voice-over. APIs are provided by fal.ai

- Built with the typical React+Next.js, Shadcn front-end

- Used remotion.dev for the realtime video preview (this is such a great project, without it the codebase would be twice as large)

- File uploads so you can bring your own media by uploadthing.com

- ffmpeg for encoding the final video and also some ui tricks, like the audio waveform

We deployed a version of it and for now it's free to use. We do plan to add some rate limiting and a bring your own API Key next, but it's open source and I'm curious about what the community will build on top of it, or derive from it. Customize your own video app and if you do, please share.

If you have any questions, hit me up!

50 comments

nikolayasdf1231y ago

someone please add "AI Video Editing". I don't care about generating assets with AI.

scrolling frame by frame through hours of hundreds of tiny videos, fixing their exposure, fixing their dropped frames, fixing transitions, fixing and matching visual narration to textual and audio narration, aligning music to video, making "vlog" like templates. there is just so much to be fixed in this industry, but everyone is concerned with generating assets. that's just not the biggest issue video editors are facing. like not at all.

burningion1y ago

I'm working on this! Here's an example using Claude and Model Context Protocol: https://www.youtube.com/watch?v=bEDA9lkkGVM

You can sign up at https://www.video-jungle.com/ and I'll send an invite over.

farzd1y ago

very cool, been looking to make some applications using something like this. What kinda of pricing model is this going to have ?

nikolayasdf1231y ago

nice. is there any examples of final results what the best it can do?

EyMaddis1y ago

Because for generation you just have to make a prompt to OpenAI and get a VC to throw money at you. Why bother with the hard thing

nikolayasdf1231y ago

you don't even need VC. people would love it and more than happy to pay for it, high price! I bet China has already some of such apps. I see many very similar vlogs in Little Red Note, likely edited automatically by some kind of software. build it and. ship it

t0bia_s1y ago

Absolutely agree. I don't care about generating footages with non existing content, just let me help with editing of already taken real life footages to speed up post-production!

lastdong1y ago

Thank you for open sourcing the project, the UI looks fantastic. I noticed that several comments have mentioned the potential to improve functionality by allowing users to replace the backend with local models, such as Hunyuan. Would you be open to considering this feature in the future? Additionally, if other developers were to contribute, do you think this could be a possibility?

drochettiOP1y ago

Absolutely. The idea of being open source with a permissive license is that we're encouraging anyone to do whatever fits their use case.

You can replace anything, deploy on your own server, port it to other stacks... whatever brings value to you.

We're also open to PRs, cut an issue in the repo and we can get the conversation going.

voiper11y ago

Wow, that's so fun. Just upload and generate assets super easily... queue the next one while it's working.

popalchemist1y ago

Wow, very impressive. Is there any way to use the video editor UI component in a package, separately from the rest of this project?

drochettiOP1y ago

Thanks! Not as a component, but you can clone or copy it and modify as needed.

It's a quite complex UI, so it's not easy to export it as a single component.

newusertoday1y ago

you can use remotion which this project is using

dgreensp1y ago

Thanks for posting it!

Remotion does look really cool for in-browser video editing. People should note that you can’t make free-to-use software with Remotion. It is a paid product, and you can download and run the code for free in very limited circumstances that let you experiment with it. You can’t use the output of your own software built on Remotion for any for-profit purpose, nor can anyone else, without a license.

jerriclynsjohn1y ago

Hey, co-founder of oss.incredible.dev here. @drochetti this is an amazing project, I love the codebase and I honestly think this is the future of video production.

During the days of oss.incredible.dev we found that the biggest friction for our user in video production was themselves, It's extremely hard for anyone to come in front of a camera and record themselves, while we were able to solve for everything else like scripting, story boarding, layouts, animations, collaboration, block based recording etc it was the inability to record themselves that caused them to leave the workspace hanging. With AI, users won't have to record themselves, they can have a saved avatar and just give it a script to make it work! This was the missing puzzle.

I'm planning to revive oss.incredible.dev, but with the intention to make it AI first product. Let me know if anyone here would like to join hands.

grugagag1y ago

It is quite a terrible thing if AI technology completely replaces the human with their own flaws with something that mimics human but is completely devoid of it and obediently executes the deed, reads the scripts that is fed and never fails. Yes it is more efficient this way but is this consumption geared towards humans or just to the human as cattle? Seems perfect for propaganda though.

jerriclynsjohn1y ago

I think about this as democratization of content creation, now people who were shy but have immense knowledge can share what they know to the world. Think about the accessibility unlock that happens with a product like this. The biggest problem we saw when we created incredible.dev was that extremely talented developers were reluctant to come in front of a camera, but they wrote amazing scripts. They knew what they were talking about, but the only blocker was themselves. This product can unlock knowledge like never before, people get to stay relevant in the fast-paced video first world.

asadm1y ago

this looks great! I would like standard timeline scrup, drag-drop etc.

Some eta on how long a generation will likely take.

kfarr1y ago

Second on timeline scrub (move timeline position depending on x position of click on timeline) and drag and drop asset from media into timeline. Also spacebar is usually a basic universal shortcut to play/pause

drochettiOP1y ago

Drag and drop the media to the timeline, and drag the media along the timeline track is already supported.

We will keep improving the UI, including shortcuts. Thanks a lot for the feedback.

drochettiOP1y ago

Thanks for the feedback, eta would be great indeed. I'll look into it.

satvikpendem1y ago

I was just looking for something like this, is this similar to RunwayML and their editor?

Jayakumark1y ago

Great release , where is the source code for FFMPEG library used?

oldgradstudent1y ago

Cool project, I played with it a bit a few hours ago.

A small bug report: if you have both a still image and a video in the timeline, the video does not play in the preview.

aykutkardas1y ago

Glad you liked it. And thanks for reporting, I'll look into the issue.

atentaten1y ago

Nice work! It would be cool to be able to do a basic cross fade along with fade in fade out with the clips.

wanderingmind1y ago

Please add a dockerfile to generate a docker image that makes it easy to setup and test

steve_aldrin1y ago

can we add smooth zoom in effects to particular part of vids? like the ones used in product demos etc

yigitkonur351y ago

great stuff! nice to see that remotion is becoming more popular on such projects.

rebalh1y ago

hey man cool thing, one advice, add a video on the GitHub page that shows how the thing works.

peppertree1y ago

Open source part is just a thin layer of UI. Built to peddle their closed backend, which is built on others' open source work. Nice.

echelon1y ago

Fal wants to sell compute, and since they're straggling behind the compute-as-an-API leaders they've attached their cart to the art, image, and video subset of the market.

They think they've got a leg up on their model partners. That the foundation model companies will mostly become generic copies of one another, servants of the compute layer. There are so many foundation video models now, and they'll battle it out over dwindling margins. Pika, Runway, Kling - they're all the same. And there's also growing open source foundation models.

The thing that stands in Fal's way is that the future of AI video for artists is local. Hunyuan and Comfy can run on desktop machines, and real artists hate the SaaS model and non local stuff. It doesn't look like we'll even need A100s to reach Pixar levels of quality.

The ones to watch in the art space are Comfy and Invoke. And Adobe.

Fal probably has a future in powering the API-driven YouTube Shorts slop, though there's probably an upper limit to the number of talking head startups.

But there's no way they win artists over to cloud. Sophisticated tools are going to be local. Any SaaS art tools that do achieve escape velocity will buy and manage their own compute and won't go through Fal as a reseller.

varispeed1y ago

Call me thick, but it's like back in the day, rich bored person goes to an artist and says "paint me this and that". Except now the "rich" person calls themselves an artist and "actual" artist is replaced by AI contraption that basically regurgitates real artist works. It's kind of comical.

But also makes sense. If you wanted e.g. music, you could either look at catalogue of what's out there or try to make your own. But if no one is making what tickles you and you yourself have no talent to make what you would like to listen, the AI comes to rescue.

2 more replies

drochettiOP1y ago

That's fair. But I know a lot of smart folks out there that have trouble building that "thin layer of UI". So if that helps them, mission accomplished.

Anyone can replace the AI layer with their own local models, other services... whatever suits your use case and preferences is fair game.

dylan6041y ago

Am I missing a demo video or anything, or is the only way to see it to install and run it? The only thing I'm seeing is the poster image with what I'm assuming is a very cropped image of part of the UI???

Just seeing the interface gives a lot of clues as to potential usability as if it's targeted at novices or hardcore editors. Not being able to see if there's any kind of lag when trying to scrub/navigate the video is also something that tells me if I'm going to get upset with it from being web based or if it feels native.

Just food for thought on what people might be looking for instead of having to install something that might not be right for them.

thangngoc891y ago

It’s here: https://fal-video-studio.vercel.app/

dylan6041y ago

and you found that link how? am I just missing it in TFA?

Edit: What do the buttons next to the play/pause button do? Typically, I'd expect to frame advance, or jump to next the next clip. I click and see nothing happening. I dropped two images I uploaded into the timeline, yet there's nothing visible in the timeline. The monitor shows something, but nothing in the timeline indicating how the clip fits in the timeline. There's not indication of what the frame size of the timeline is. The images look like they are being stretched/squeezed to fit, but there's also no indication in the media bin about the media like duration/size/etc.

This really feels targeted for people with no previous knowledge of a video editor. It also just has that feel of "do something quick to make a post with it's AI to HN".

1 more reply

j / k navigate · click thread line to collapse

50 comments

nikolayasdf1231y ago

someone please add "AI Video Editing". I don't care about generating assets with AI.

burningion1y ago

I'm working on this! Here's an example using Claude and Model Context Protocol: https://www.youtube.com/watch?v=bEDA9lkkGVM

You can sign up at https://www.video-jungle.com/ and I'll send an invite over.

farzd1y ago

very cool, been looking to make some applications using something like this. What kinda of pricing model is this going to have ?

nikolayasdf1231y ago

nice. is there any examples of final results what the best it can do?

EyMaddis1y ago

Because for generation you just have to make a prompt to OpenAI and get a VC to throw money at you. Why bother with the hard thing

nikolayasdf1231y ago

t0bia_s1y ago

Absolutely agree. I don't care about generating footages with non existing content, just let me help with editing of already taken real life footages to speed up post-production!

lastdong1y ago

drochettiOP1y ago

Absolutely. The idea of being open source with a permissive license is that we're encouraging anyone to do whatever fits their use case.

You can replace anything, deploy on your own server, port it to other stacks... whatever brings value to you.

We're also open to PRs, cut an issue in the repo and we can get the conversation going.

voiper11y ago

Wow, that's so fun. Just upload and generate assets super easily... queue the next one while it's working.

popalchemist1y ago

Wow, very impressive. Is there any way to use the video editor UI component in a package, separately from the rest of this project?

drochettiOP1y ago

Thanks! Not as a component, but you can clone or copy it and modify as needed.

It's a quite complex UI, so it's not easy to export it as a single component.

newusertoday1y ago

you can use remotion which this project is using

dgreensp1y ago

Thanks for posting it!

jerriclynsjohn1y ago

Hey, co-founder of oss.incredible.dev here. @drochetti this is an amazing project, I love the codebase and I honestly think this is the future of video production.

I'm planning to revive oss.incredible.dev, but with the intention to make it AI first product. Let me know if anyone here would like to join hands.

grugagag1y ago

jerriclynsjohn1y ago

asadm1y ago

this looks great! I would like standard timeline scrup, drag-drop etc.

Some eta on how long a generation will likely take.

kfarr1y ago

drochettiOP1y ago

Drag and drop the media to the timeline, and drag the media along the timeline track is already supported.

We will keep improving the UI, including shortcuts. Thanks a lot for the feedback.

drochettiOP1y ago

Thanks for the feedback, eta would be great indeed. I'll look into it.

satvikpendem1y ago

I was just looking for something like this, is this similar to RunwayML and their editor?

Jayakumark1y ago

Great release , where is the source code for FFMPEG library used?

oldgradstudent1y ago

Cool project, I played with it a bit a few hours ago.

A small bug report: if you have both a still image and a video in the timeline, the video does not play in the preview.

aykutkardas1y ago

Glad you liked it. And thanks for reporting, I'll look into the issue.

atentaten1y ago

Nice work! It would be cool to be able to do a basic cross fade along with fade in fade out with the clips.

wanderingmind1y ago

Please add a dockerfile to generate a docker image that makes it easy to setup and test

steve_aldrin1y ago

can we add smooth zoom in effects to particular part of vids? like the ones used in product demos etc

yigitkonur351y ago

great stuff! nice to see that remotion is becoming more popular on such projects.

rebalh1y ago

hey man cool thing, one advice, add a video on the GitHub page that shows how the thing works.

peppertree1y ago

Open source part is just a thin layer of UI. Built to peddle their closed backend, which is built on others' open source work. Nice.

echelon1y ago

Fal wants to sell compute, and since they're straggling behind the compute-as-an-API leaders they've attached their cart to the art, image, and video subset of the market.

The ones to watch in the art space are Comfy and Invoke. And Adobe.

Fal probably has a future in powering the API-driven YouTube Shorts slop, though there's probably an upper limit to the number of talking head startups.

varispeed1y ago

2 more replies

drochettiOP1y ago

That's fair. But I know a lot of smart folks out there that have trouble building that "thin layer of UI". So if that helps them, mission accomplished.

Anyone can replace the AI layer with their own local models, other services... whatever suits your use case and preferences is fair game.

dylan6041y ago

Just food for thought on what people might be looking for instead of having to install something that might not be right for them.

thangngoc891y ago

It’s here: https://fal-video-studio.vercel.app/

dylan6041y ago

and you found that link how? am I just missing it in TFA?

This really feels targeted for people with no previous knowledge of a video editor. It also just has that feel of "do something quick to make a post with it's AI to HN".

1 more reply

j / k navigate · click thread line to collapse