It depends on how much you're leaking, and how fast.
Best case scenario, it just slows down garage collection a little bit, as you're holding into a lot of references that aren't going anywhere.
On the other hand, I recall a bug in a particular version of AngularJS where component DOM nodes wouldn't get cleaned up when navigating with the router unless you manually set all of the scope values their templates used to null.
We had a data dense application with tables and what not, and you could clearly watch memory jump tens or more megabytes flipping between pages in the chrome dev tooling.
Eventually (this was a SPA intended to be used for upwards of hours at a time) the page would crash.