Note that a "foreign business" is a technical term in this context, simply meaning that a business operating in one state that is incorporated in another state. (See for example https://www.californiaregisteredagents.net/incorporation/for...) For example, almost every YC company is incorporated in Delaware, but is registered as a "foreign business" in CA where they are physically headquartered.

At the international level you usually form subsidiaries, due to the vast differences between laws at the national level, but some companies will simply register a "branch office" in other countries in which they have small operations. In fact, many countries, like the U.S., have tax forms specifically for these companies: the Form 1120F...