I really have to dispute that.
First, it's not really clear what you mean by "detrimental" - is that to software quality, or development schedule, or suitability? Or all of them, or others? Agile is very clear about the trade-offs - working products being more important than documentation for example, or responding to change over following plans.
Second, I don't think it's true. I've directly observed, in multiple organisations, that agile development is perfectly capable of delivering good systems on-budget and on-schedule. If anything, it's especially good for complex systems, which are those most likely to require flexibility and have hidden requirements.
I also dispute the idea that "ghetto Scrum" is invalid. Development processes should be tailored to the requirements of the product, organisation and team. Not every team needs epics or planning poker or scrum of scrums, and it's totally valid to take some of the ideas of scrum and apply them to your own development process without having to buy the whole thing.
I consistently see agile development hand waved away with this argument that it doesn't work because there are companies which do it badly, and it's baffling to me.
And you can't compare software with a house, or even a spaceship :) For one thing, we have far fewer constraints.
When we build software we are (generally, though not in every case) building a solution to a problem rather than a strictly defined product - and in many cases, we are unsure about what the final product will look like! Spotify did a good illustration of this process in this slide: http://www.chrisyin.com/images/spotify-mvp.png - the point is to continuously deliver features to customers and using the feedback generated to inform ongoing development, which like you point out, is absolutely essential in some cases.
, but that's what Agile indirectly makes you do. Although Waterfall is the other dangerous extreme, it makes you think, correctly estimate, and plan.
That's perhaps the idealistic view, but there is a cost there too - change becomes much harder. I also suspect that there is no such thing as a correct estimate!
You can't apply Scrum to every project - especially those with firm deadlines!
Maybe not, and Scrum is not the ideal methodology in all cases - but it's also not the only way to be agile. It's also a fallacy that planning in advance means a project will meet a deadline - let's remember that Scrum doesn't typically mean you are doing different work, just that you are slicing it and delivering it differently. I'd argue that one of the main differences is that if a Scrum-developed project misses it's deadline, you still have a working product - it just doesn't necessarily have all the features you'd like. But if you are correctly building the most important and complex features first, then that's potentially much better than the "waterfall" outcome of "the product isn't complete and can't be used".
can you build a spaceship using Scrum?
I probably wouldn't want to try it, but this is an example of a project with incredibly tight constraints on what can be built - far more than software typically is. But I'd imagine it's also possible to build components of any product in an agile manner.
I still use Gantt charts, and that's another thing that makes people make funny faces in Scrum.
That's great, and it's something that should be encouraged - development methodologies should be determined by what works well for you as a developer, team, or company! A Gannt chart, assuming you are changing it as product requirements change, is basically the same as a Scrum burndown chart with additional planning thrown in. And we do the same sort of thing - "these are the epics, this is how long we estimate they will each take, and here is the order we will do them in." It's not so different.
Ultimately these methodologies are designed to help projects cope with change, which has been a long-term problem in correctly sizing and executing software projects. Maybe they won't always work for every project with particular constraints, but I reckon they are far more applicable than some people believe.
