I've been building my own video editor for about 8 months now. I'm slowly realizing that marketing this software might be the end of my solopreneur dreams. If I can get video creators on calls and show them the software in action, they like it, but so far nothing else works.
So I've decided to open-source the most polished part of my app which is video smart cutting.
It uses PyAV and libavcodec (ffmpeg internals) to encode a small part of the video near the cutpoints and then uses libavformat to stitch the recoded segments and parts of the original video back together into a whole video. The benefit of this is that reusing parts of the original video is way way faster than recoding, so doing it this way we can have very fast frame-accurate cutting.
While this is not a new idea, and there are a couple open-source implementations already, I believe mine is the first foss one to really try to solve the problem, and not just treat it as experimental feature that might work sometimes.
I've written a test suite that checks the implementation with various codecs (h264, h265, vp9, av1), container formats (.mp4, .mkv) and audio codecs (mp3, vorbis, opus, aac, flac, wav). The tests generate bunch of synthetic videos and sets of randomized cutting points and verify that input and output videos are the same.
I want to keep improving it further, but in my experience it works pretty much on everything that is produced in a reasonable manner. E.g. anything from obs, my phones, YouTube, Twitch and everything that my ~10 test users have tried throwing at it. I think there was one video at https://kodi.wiki/view/Samples that failed and I'll try to get to that eventually. I think sometimes it has problems with files where the seeking is difficult. Best would be if we didn't have to seek at all, but that would require some changes that I have to think through.
Please check it out and offer any feedback on either the command line tool or the full GUI app. https://github.com/skeskinen/smartcut
And if mifi or someone else would be interested in trying to incorporate this into lossless-cut, I'd be down to help.
