For such things it seems you'd need to check every shot change.
Even today, directors rarely seem to want to film in actual color. They'll tint everything sepia, or that hideous blue-orange scheme that is so popular these days.
http://priceonomics.com/why-every-movie-looks-sort-of-orange...
In any case, any such colorizing system would be designed to accept a bit of guidance here and there from the artist. This is much like when an OCR'd document needs a bit of touch-up.
And even if it wasn't perfect, many BW movies would be made much more watchable, like the 1927 Wings, which is crying out to be colorized (and have a soundtrack added).
Aren't we talking about a B&W film? In that case the colour (or its lack of) wouldn't communicate any information.