The truth is, majority of worker stay very long hours in the office, because leaving on-time gives you the notion that you are not getting enough work; hence you would be treated as you are slacking off.
Also, there's often unspoken rules, for example, someone leaving earlier than their boss being rude.
When they were in growth (back prior to 1980) where lifetime employment and salary increase by age were common, there was at least some incentive to put up with that, but they are now gone as the thing in the past, they are just getting the short end of the stick, as losing your current job might mean that there aren't any other jobs. (Getting hired in Japan outside of new hire -- that's something other than right after graduating, can be very tough.)
Now, having said that, of course, there are people out there who have more liberty in deciding how much they should work each day, and doing it as their choice, but considering this being nationwide issue, I wouldn't think that's a majority.
Note: If you are someone from outside of Japan, you will get some perk as a foreigner, that you are not expected to go along with the above. Merely appearing like the Japanese would lock me into same expectation as Japanese. (Which I envy "foreigners" every time I visit Japan, as my mindset is as foreign from Japanese.) Japanese people usually don't really discriminate foreigners, but they do have different social expectations. (Just you never can be "one of them.")