The mobile (UE) IP anchor is not the S-GW but the P-GW in the core network. For an established PDU session the IP address is fixed, and does not change during a handover.
For information MIP is used between S-GW and P-GW (PMIP IIRC), so there can be IP addresses changes when the HO is such that the S-GW changes. Still, this is transparent to the UE: it's a change at the tunnel level, and the UE traffic flowing inside this tunnel is not affected.
BTW what "anchor" means here is that the IP address the UE gets is routed to the P-GW. From then on the traffic is tunneled to the UE across the cellular network core and radio access networks, but all this is transparent for the end device. There's a first leg from P-GW to S-GW, then from S-GW to the eNB (the base station), then the over-the-air and last leg to the mobile device. That's 3 tunnels joined together to transport data seamlessly from P-GW to UE. During a HO the radio one always changes, the S-GW to P-GW rarely changes. And the P-GW is fixed to provide IP session continuity.