As I discussed in my talk "Testing is Overrated", programmer testing finds a different class of bugs than manual testing or code reviews.
One of the problems is that programmers are not very good at writing tests. They are much more likely to test "happy path" scenarios. Steve McConnell reports in Code Complete that mature testing organizations have 5 times more dirty tests than clean tests, while immature testing organizations write 5 clean tests for every dirty test.
Another big problem is that unit tests are never going to show you that your software blows. Only user testing can find that.
If you want to ship good software, I think you need to do a combination of code reviews, unit tests, system tests, user studies, and manual testing. Different organizations will find a balance that works for them.
That last is a key point. I like writing tests, but I hate the dogmatism that if you're not writing tests you're doing it wrong. Obviously, good software has been written without tests, and buggy crap has been fully TDD'ed. In the end, the team matters more than the methodology.
http://www.scribd.com/doc/8585284/Testing-is-Overrated-Hando... http://www.scribd.com/doc/8585444/Testing-is-Overrated
For example, TDD encourages code churn: you write tests for a lot of throwaway code.
Personally, it usually takes me two/three attempts before I'm satisfied with the version of the code that I have. Writing tests for these throwaway attempts is a waste of time.
As you become more experienced in writing tests and testable code, you realize that you no longer need TDD to write code that can be easily tested, so I see TDD more as "training wheels" for beginners.
Code should be developed with as large a view of the design as possible. TDD restricts development (by definition) to satisfying the tests. The effect is of 'moving the bugs around' since the existing tests rarely completely describe the whole design. You get into a loop of "oh, yeah, well we Really meant to write a test to cover that case too", repeated ad-nausuem.
Behavior Driven Development really shines in the aspect of "completely describing the design" as it focuses on driving development against a well defined behavior (using scenarios). BDD lets you know where the bug is: in the code or in the specifications (the behavior as specified by the customer). So there is a lot less "chasing around". If you are trying to cover behavior of a system only using TDD then I agree you can end up chasing around a bug.
However, both are necessary and highly effective when used together.
So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done.
Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.
I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.
"Dependent Types: A Look At The Other Side" by Larry Diehl
It talks about using Types for Driving your Development. So it is TDD, but without tests, without runtime and programmer's overhead of them.
When using TyDD, you isolate core that should go as bugless as possible and write DSeL for it in language with strong type system. The talk above tells about Agda, but we and some number of other companies in the world (Eaton http//eaton.com/, Echo http://aboutecho.com/ - from the top of my head) successfully employ Haskell for that task.
That way you shoot two birds with one stone: you get provably correct software and you get an ideal language to express your core problems.
Best of all, you get free "requirement tracking" - whenever your requirements change, you express them in types and then apply changes where they should, guided by friendly compiler. Still getting provably correct software (modulo types) at the end.
However, I actually came here to say something else about his material: if it is harder to decompose the business requirement into Haskell than it is to write monkey tests for one's Blub language, I am not certain anyone wins when going with a provably correct implementation.
My experience using advanced type system tells me that you can get away with functional tests. Types, also, takes much less code space-programmer's time than tests.
So I do not see how they could be harder to use. Certainly types will be unusual, but not much harder.
https://webcache.googleusercontent.com/search?q=cache:grokco...
