SSNs are sequential; the numbers adjacent to your own are valid. Surely you cannot bypass data privacy requirements by applying a trivial and reversible transportation like +/- 1, so knowledge of your own social security number would be illegal under your logic.
It doesn’t make sense to think of SSNs as stolen identities unless the thief knows the names associated with them.