(I guess you could argue that an empire that invaded another culture/nation/territory and then enslaved its people was effectively enslaving them based on something akin to "race," but that argument is slightly weakened by the fact that the Roman army didn't only conquer "barbarians" but also went to war against their contemporary civilizational peers, often driven by familial disputes between emperors.)
So while enslaving someone from birth is undoubtedly bad, I'm not sure it's fair to characterize it as enslavement based on racial attributes, if the slave's ancestor was originally enslaved for reasons unrelated to race. Like, I guess you could call it racism if you squint, but only in the sense that skin color (or whatever other physical attributes you might associate with race) is an inherited characteristic, which seems like self-referential reasoning.