I agree that reaction outcomes depend on many other factors besides the reagents. In the future, I'm sure we'll create reaction prediction frameworks that also take these other factors as inputs. The problem right now is that there aren't many datasets that include these extra factors.
We're not advocating turning retrosynthetic analysis over to machines yet. These are just baby steps.