If you're willing, could you try using AdaptiveDetector? It should have better defaults for handling fast camera movements a bit better.

The threshold values themselves can be tuned if you generate a statsfile and plot the result, but that can sometimes be tedious if you have a lot of files (thus the huge interest in methods like TransNetV2). Glad to see the real world applications of those in action. You can always just increase/decrease the threshold by 5-10% depending on if you find it's too sensitive or not sensitive enough as well.

Thanks for the response!