A merit lottery also keeps everybody from having to waste shitloads of time studying for this one test.
Standardized testing is a good predictor. In my country people are admitted based on standardized testing only (country wide subject exams and sat-like test) or results of university-adjacent "preparation courses" and it works fairly well. The affirmative action is realized directly as a bonus points to your scores based on the background, which reduces the effect of ethical gaming.
The core issue is the pyramid shaped system where not being at one of these super places means that you are out of the competition for best work in general.
Do you want only rich kids going to college?