Right, and calling someone English when they're Welsh or Scottish may rather annoy them.
There's an animosity towards the English from the Welsh. I believe the Scottish generally feel the same way towards them. England are like family that we love, but we don't really like very much.
Northern Ireland is a bit of a tricky one because people from Northern Ireland generally consider themselves "Northern Irish", "British" or "Irish".
I don't know if anyone would take offence at "Northern Irish", but you don't want to incorrectly guess either of the others!