Strictly speaking, a pointer points to 1 address, not a set.
Abstractly speaking, a lifetime represents a set of memory that lives a certain length of time, and thus also represents the set of memory a reference with that lifetime can reference.
It's just a different way of thinking about lifetimes which helped me understand them.