>As a practical example, figuring out where a given pixel moves from one video frame to the next one, when working on real-world videos, the best known algorithms get about 50% of the pixels correct. With clever filtering, you can maybe bump that to 60 or 70%, but in any case you will be left with a 30%+ error rate.
what do you mean by this? optical flow isn't really a learning problem? it's a classical problem with very good classical algorithms
https://www.mia.uni-saarland.de/Publications/brox-eccv04-of....
https://people.csail.mit.edu/celiu/OpticalFlow/
https://github.com/pathak22/pyflow