that would be the foreign ministry office in the country.
in china i have my visa renewed once in a small countryside town. they were big enough to have an office there. every town has one. in germany too. the local government office has a branch of the foreign ministry.
i have never had to leave a country to renew my visa. that's just insane. in fact that's even true for the US, at least for non-immigrant visa: https://www.usa.gov/extend-visa
what the US makes different is that it separates the work permit from the visa, and they get different validity times. and while having a valid work permit is enough to stay in the country, a new visa is required to reenter. which other country does that? everywhere else the visa itself is required to stay, a work permit is not enough, and consequently, you also get to renew that visa in the country.