典型的测试错误有哪些？(20)

　　困难的问题是很难知道你是否圆满地完成了测试工作。可能只有当产品已交付后才能真正知道。可以理解的是，这使得经理们不舒服。有时候你会发现他们热心采用代码覆盖率，认为只有那些简单的数字可以鼓舞士气。候测试员也变得倾心于覆盖率，虽然他们的兴趣没有那么高，而且结束得也快。

　　What is code coverage? It is any of a number of measures of how thoroughly code is exercised. One common measure counts how many statements have been executed by any test. The appeal of such coverage is twofold:

　　什么是代码覆盖率?它是代码是否全面执行的数字衡量。一个常见的衡量是计算所有测试共执行了多少条语句。对这种覆盖率的呼吁有两方面：

　　1. If you've never exercised a line of code, you surely can't have found any of its bugs. So you should design tests to exercise every line of code.

　　如果你从未执行过某一行代码，你当然不能找出它的任何 bug 。所以应当设计一个可以执行每一行代码的测试。

　　2. Test suites are often too big, so you should throw out any test that doesn't add value. A test that adds no new coverage adds no value.

　　测试套件常常很大，所以应该抛弃任何不能增值的测试。一个不增加新覆盖率的测试不能增加任何价值。

　　Only the first sentences in (1) and (2) are true. I'll illustrate with this picture, where the irregular splotches indicate bugs:

　　句子(1)和(2)中，只有第一句是正确的。我将用下面的图说明，其中的不规则黑点指示的是 bug ：

　　If you write only the tests needed to satisfy coverage, you'll find bugs. You're guaranteed to find the code that always fails, no matter how it's executed. But most bugs depend on how a line of code is executed. For example, code with an off-by-one error fails only when you exercise a boundary. Code with a divide-by-zero error fails only if you divide by zero. Coverage-adequate tests will find some of these bugs, by sheer dumb luck, but not enough of them. To find enough bugs, you have to write additional tests that "redundantly" execute the code.

　　如果你仅编写需要满足覆盖率的测试，你会发现 bug 。那些总是失败的代码不论怎样执行，你都肯定能发现它们。但是大多数的 bug 取决于如何执行某一行代码。例如，对于“大小差一”(off-by-one)错误的代码，只有当你执行边界测试时才会失败。只有在被零除的时候，代码才会发生被零除的错误。覆盖率足够的测试会发现这些 bug 中的一部分，全靠运气，但发现得还不够多。要发现足够多的 bug ，你必须编写其他的测试“冗余地”执行代码。

　　For the same reason, removing tests from a regression test suite just because they don't add coverage is dangerous. The point is not to cover the code; it's to have tests that can discover enough of the bugs that are likely to be caused when the code is changed. Unless the tests are ineptly designed, removing tests will just remove power. If they are ineptly designed, using coverage converts a big and lousy test suite to a small and lousy test suite. That's progress, I suppose, but it's addressing the wrong problem.

　　同样的原因，因为有些测试不能增加覆盖率而将它们从回归测试套件中去掉也是危险的。关键不是覆盖代码;而是测试那些当代码更改时容易被发现的 bug 。除非测试用例是不熟练的设计，否则去掉测试用例就是去除作用力。如果它们是不熟练的设计，可以使用覆盖率将一个大而粗劣测试用例套件转化成一个小而粗劣的测试用例套件。我想这是进步，但是与这个问题无关。

　　A grave danger of code coverage is that it is concrete, objective, and easy to measure. Many managers today are using coverage as a performance goal for testers. Unfortunately, a cardinal rule of management applies here: "Tell me how a person is evaluated, and I'll tell you how he behaves." If a person is evaluated by how much coverage is achieved in a given time (or in how little time it takes to reach a particular coverage goal), that person will tend to write tests to achieve high coverage in the fastest way possible. Unfortunately, that means shortchanging careful test design that targets bugs, and it certainly means avoiding in-depth, repetitive testing of "already covered" code.

　　代码覆盖率的一个重大危险是它是具体、主观而易于衡量的。今天的许多经理都使用覆盖率作为测试员的绩效目标。不幸的是，一个重要的管理规则适用于这里： “告诉我如何评价一个人，然后我才能告诉你他的表现。”如果一个人是通过在给定的时间内覆盖了多少代码(或者是在多么少的时间内达到了特定覆盖目标)来评估的，那么那个人将倾向于以尽可能快的方式达到高覆盖率的测试。不幸的是，这将意味对以发现 bug 为目的的仔细测试设计的偷工减料，这当然也意味着避开了深层次、重复地测试“已经覆盖”的代码。

原文转自：http://www.uml.org.cn/Test/200709289.asp