Ah, but take a look at this chart:
https://xkcd.com/1633/
For anything more than 10^6 km away and less than 10 metres in size, we just wouldn't see it. We have only just noticed some enormous lumps on Ceres, and it seems a whole planet has evaded detection until now.
I don't think it's particularly likely we will find a vN probe. But to consider their non-detection to be of statistical significance is, I think, an overestimation of our technological capabilities.