The downside is that a junior teammate was probably less effective than they could have been because of a lack of in-person mentoring -- their learning was slower than I would have expected in-office, but, then, that could be this particular person rather than remote work.
I don't doubt your experience, to be clear -- I just mean to say that I think the situation is more complex than "people are less productive" or "people are more productive."