2 of the axioms are:
1. For all x, x*0 = 0
2. For all x, y: x*S(y) = x*y + y
(Now having written that and looking back, I see that, in my previous post https://news.ycombinator.com/item?id=43442074, I wrote "Despite the name, in the usual mathematical meaning of the term, Peano arithmetic does not define arithmetic at all, only the successor operation, and everything else is built from there." Perhaps this infelicitious-to-the-point-of-wrong wording of mine is the source of our difference? I meant to say that Peano arithmetic does not axiomatize arithmetic at all, but that arithmetic can be defined from the axioms. Thus the specific definition x[pt] = [pt] is eminently sensible if we consider the distinguished point [pt] to be playing the usual role of 0; but the definition x[pt] = x is also sensible if we consider it to be playing the usual role of 1, and even things like x[pt] = x + x + x + x + x can be tolerated if we think of [pt] as standing for 5, say. The axioms cannot distinguish among these options, because the axioms say nothing about multiplication.)
Enderton, “A Mathematical Introduction to Logic, 2nd Ed.”, p,203,269-270
Kleene, “Mathematical Logic”, p.206
EDIT: It seems like you're talking about Peano's original historical formulation of arithmetic? That's all well and good but it is categorically not what is meant by "Peano Arithmetic" in any modern context. I've provided two citations from pretty far apart in time editions of common logic texts (well, "Mathematical Logic" is a bit of a weird book, but Kleene is certainly an authority) and I hope that demonstrates this.
There's a lot of reasons that the theory is pretty much always discussed as a first-order theory. The biggest, of course, is that when taken as a first-order theory it fits neatly into the proof and statement of Godel's Incompleteness Theorems, but iiuc it's just generally much less useful in a model theoretic context to take it as a second order theory (to the point where I only ever saw this discussed as a historical note, not as a mathematical one).
EDIT 2: This is all a digression anyway. Both first- and second-order PA label the start of the Z-chain as 0; so any model of PA contains 0 when interpreted as a model of PA.
But these are all referring to Peano arithmetic as a model of the theory of the natural numbers. And that seems a bit silly: the impact of Peano's work wasn't because he showed that there was a model of the theory of the natural numbers, which everybody believed if they bothered to think about it, but because he showed that all you needed to make such a model was a successor operation satisfying certain axioms. Yes, they may be less model-theoretically congenial because they're second order, but to change Peano's work from what he did historically and still call it Peano's seems strange to me. (I'm fine with dressing it up in modern language, and calling it an initial object in the category of pointed sets with endofunctor, which perhaps is biased but still seems to me to be capturing the essential idea.)
Certainly I was taught the second-order approach, though it was as an undergraduate; I've never taken a model-theory class. As I say, I'm away from my library and so can't consult any other sources to see if they still teach it this way, and anyway I am a representation theorist rather than a logician; but, if the common logical approach these days really is to discard Peano's historical theory and to call by Peano's name something that isn't his work, even if it is more convenient to use, then I think that's a shame from the point of view of appreciating the novelty and ingenuity of his ideas. But just because I think something is a shame doesn't mean it's not true, and so far you've produced evidence for your view and I can't for mine, so I can't argue any further.
Ah, good point that this was the actual source o# the discussion. This one at least can be argued, because the question is about how things should be axiomatized/defined, not how they are. And certainly the theory of the "natural numbers starting with 1" can be axiomatised just as well as the "natural numbers starting with 0." All these axioms are made by humans, and an appeal to existing axioms here can only say what's been done, not what should be. (And I say this as someone who does start my naturals at 0.)