1. Like it or not, plenty of families in America have had it worse than you had, and would not find it at all impressive that you have never gone on a vacation or eaten out.
2. You should save your anger for the huge number of spots saved for legacies and athletes. Being the child of alumni means you have a 45% greater chance of getting into a college. (https://www.cnbc.com/2019/04/07/harvards-freshman-class-is-m...) If you must get angry at someone, don't get angry at the people who are struggling as hard, or harder than you are--get mad at the rich assholes who quietly sail into these so-called elite institutions.
3. If you have good grades and good test scores, there are tons of good colleges you can go to. They just might not be Harvard. Is not going to Harvard worth pushing yourself to the right? Worth aligning yourself with the guy pushing a trade war with China, which is going to do wonders for the image of Asians in America? Personally, I don't think so.
4. The adversity score is designed to ensure equality of opportunity, dude. If they were capable of making a good adversity score, they would certainly take into account your family's struggles. They probably won't be able to, because these things are unquantifiable, but frankly the SAT and the whole college admissions process are bullshit anyway, and you'll be happier once you let go of your belief that because your family sacrificed a lot, the system ought to reward you. It should, but it won't, because it's a capricious and unfair monster.