The people I know who regularly take this approach end up with a really fantastic code architecture in the end, and avoid a lot of pitfalls from their first naïve attempts.

When dissatisfied with an approach I'll often copy a class/function or three into new versions. I'll append the names of the old with 0. Then I start writing new ones with the new approach. In effect throwing the old ones away, even if sometimes it is a gradual process.

This happens frequently in a new project, less frequently in a maturing project, to never in a maintenance project.

However, it has a lot in common with (a severe) refactoring. Generally you're keeping the "overhead" docs, code, and even files, so it is not a complete rebirth, perhaps ~30% of the core code.