undefined | Better HN

0 pointspeterwwillis9y ago0 comments

Testing doesn't detect failure, it only detects the failure of a test. Real failures happen more often than test failures, for the same test on the same code with the same input and output. The best systematic solution would detect real failures, not see what happens when you fail a test.

0 comments

nostrademons9y ago

That's monitoring, then. As Steve Yegge's Platforms Rant [1] mentioned, testing and monitoring are two sides of the same coin. Google does both, but the original thread-starter here was asking about how to detect failures when the system itself is designed to mask & recover from failures. (FWIW, most such systems do log when they've encountered a failure condition and recovered from it, and this stat is available to the monitoring system.)

[1] https://plus.google.com/+RipRowan/posts/eVeouesvaVX

peterwwillisOP9y ago

Basically, yes. But we don't have to make a traditional monitor, or have it be an extra component. Monitoring all the facets of, say, a code deployment, or a software build, or performance testing, is a dynamic thing. It may fail, or it may succeed, or it might be suspicious.

Normally we design systems for humans to determine that 3rd part; in this case, there should have been a system where humans could see the one or two pieces of unusual activity and investigated. But there wasn't, or it didn't work right. So a "fix" would be to develop software that adapts to nondeterministic behavior the way a human does. I wouldn't exactly call that monitoring, though.

j / k navigate · click thread line to collapse

0 comments

nostrademons9y ago

[1] https://plus.google.com/+RipRowan/posts/eVeouesvaVX

peterwwillisOP9y ago

j / k navigate · click thread line to collapse