Roughly, the number of upvotes on their stories and comments minus the number of downvotes. These don't match up exactly; some votes are dropped to prevent abuse.
Awesome, back to 51 points. Is it because I'm talking about karma and get automatically downvoted? I don't usually mention karma and yet still my little 1 point doesn't get added to my "score."
Well, certainly don't post questions complaining about karma. Instead, post interesting non-duplicate content that people want to read, or make useful or insightful comments in discussions.
Looking through the number of points you've gotten from your submissions (not many, join the club) and your current amount of karma, most of your karma has come from your comments.
Thank you. So for example, and I realise this isn't a great news story, this Ask HN post of mine is showing me it has 6 points (upvotes?), but my karma is still 51.