It's even more complicated than that. The Byzantine Empire was conquered by a group of crusaders in the Fourth Crusade in 1204, with several rump states cropping up. The Empire of Nicaea reconquered Constantinople in 1261, so people usually call it the continuation of the Byzantine Empire. By the 1400s, this Byzantine Empire quite literally existed at the whim of the Ottomans, even if they only entered Constantinople itself in 1453. The third rump state of the Byzantine Empire was the Empire of Trebizond, whose last bastion in Theodoro held out against the Ottomans until 1479.
And, by the way, practically none of these countries were so named to their contemporaries. Basically everyone I've mentioned here considered themselves the Roman Empire.