This issue reflects the way that people have had very different ideas about what the point or purpose of free and open source software is (in some ways, reflecting the split between people who preferred to say "free software" over "open source" and vice versa).
It's also a very concrete issue today in whether people call, say, the Chrome browser "open source". Most of their source code is downloadable, derived from the fully open-source Chromium project, but in Google's current practice, users never get the complete source code to the Chrome binaries that they run. If you're focused on the development process, it might almost make sense to call Chrome "open source" because almost all of its source code is distributed, licensed, and developed in an open source manner -- but if you're focused on what users can do with the software, it's obviously just a proprietary application (with a proprietary EULA, to boot).