Yes, although you can have multiple sheets each with a million rows. Kind of like sharding.
But I’d be fascinated what kind of system spec is required to get good performance on those kind of numbers. I’ve been on a Mac for years, where Excel is crippled by limitations.
