Only when people are being sloppy, either through ignorance, or, e.g, because its convenient for spin they'd look to put on something.

Less sloppily, referring to an entity rather than a building, it refers to either the White House Office within the Executive Office of the President, or the whole Executive Office of the President (both of which are headed by the White House Chief of Staff.)