If we were talking about the out-of-tree VirtualBox drivers I would agree with you. But we're not -- WireGuard has proven itself to be incredibly solid for the past 3 years and supports all kernels since 3.10 (with each commit getting tested against all of those kernels).
To be honest, that is far more stringent requirements than most subsystems in the Linux tree. Being in-tree is better for a variety of reasons, but just because something is in-tree doesn't make it significantly more stable or safe (I can think of several counter-examples where Linus hasn't motivated maintainers to fix mistakes and breaking changes).