Asphyxiation with inert gases causes hypoxia very quickly. It's not like holding your breath. Lungs don't actively pump oxygen; gases just diffuse along their partial pressure gradients. Air has higher partial pressure of oxygen than blood does, so the oxygen diffuses from the air to the blood. If you fill the lungs with inert gas the oxygen diffuses back out again. You're effectively breathing in reverse.

Filling the lungs with vacuum has the same effect. You have about 5 to 10 seconds before you're incapacitated:

https://en.wikipedia.org/wiki/Time_of_useful_consciousness