I am not at all concerned with what is or isn't possible in a computer for the purposes of this discussion. My only point with the link is that dealing with inverses in particular situations (i.e. where multiplication has or doesn't have certain properties) frequently requires particular considerations, and the properties of division defined as multiplication by the inverse will have different properties as a result.
To be clear, do you disagree that it is commonplace in complex analysis to extend the complex plane by {infinity} and define 1/0 = infinity, 1/infinity = 0? I find it hard to imagine that you can't have encountered that given how much you seem to know about abstract algebra. Or do you just think that it is a bad idea, despite being commonplace? In either case, to say that mathematicians would not call that operation division as a result is contradictory to my experience, even if those two special cases don't fit the category of multiplication by the inverse.
Also to be clear, I know of no counterexamples in abstract algebra and it would make sense to me that in that context division would mean something very particular, in order to be able to talk about it with any generality. But as it happens, abstract algebra isn't all of math.