This is really not a good argument.
If I swap the parts myself, I either get the new parts from apple themselves (in which case it IS their fault if something goes wrong or they can blame it on me for wrongly assembling it; e.g. "You were holding it wrong")
If I go to a shop or a friend to get my phone repaired and they replace parts, this would be different. If the device malfunctions I would obviously blame the guy who last fiddled with it first. Look at car repair shops, household appliances etc...
In the case of batteries its even more simple. If its an apple battery, its obviously apples fault if the device catches fire. Because either the battery was bad or the manuals were. If the swap is so terribly difficult that many users will damage the phone while swapping batteries, it's also apples fault, but they will (as usual) deny any claims so no harm done.
If it's a third-party battery, I can't see how the blame would fall to apple.
All in all, the reputation argument is on them. I don't see how e.g. Ford can be blamed for a car where the independently repaired brakes didn't work and caused an accident.