So no, it doesn't account for you to secretly discriminate against you. What it does do, for the penalization part, is that the schools that are in affluent neighborhoods have access to much better resources than ones from Compton or Watts.
To me, a kid from Compton who scores 1500 on the SAT far outweighs someone from Palo Alto High who also scores 1500 because of all the resources the latter received to be able to reach it.
Moving is orthogonal to the problem, except that the district could be brutally hard for some kids.
Also, imagine the other folks in your neighborhood who got literally everything. Driving to school in a Maserati and spending for 4 different tutors over the course of 12 years.
Who is more deserving? You, who scored 1500, or him, who scored 1500? I sure hope you don't say, "it should equal!"