Examples: I can call some business on the phone and people won't know I'm a foreigner. No accent. I've gotten in a cab late at night, talked to the driver for 10 minutes, and then had him freak out went he turned around to collect the fair because based on conversation he thought his ride was Japanese. I bring those examples up only to point out having zero accent and being able to speak in a way that I'm perceived as being fluent.
And yet I still run into people who's brains seems unable to accept that I'm speaking Japanese to them. It's a common complaint by friends who've been here 15-20-25 years that they still get this. Over time I've learned to accept they are trying to be helpful and I'm the 1 in 50 or 1 in 100 foreigners that can speak Japanese well so just thank them ... in ridiculously polite Japanese.
