一旦一个程序通过了测试,那么以后就不太可能不通过该测试了。这种说法导出了一些论述(没有一个被明确赞成),即自动化测试有危险,因为它给我们一种很舒服的错觉,认为程序没有问题。即使程序今天没有出现昨天出现的问题,但是依然还有很多情况可以使程序出问题。而如果你总是在没有bug的地方寻找,你就永远找不到它们。
6. Of the bugs found during an automated testing effort, 60%-80% are found during development of the tests. That is, unless you create and run new test cases under the automation tool right from the start, most bugs are found during manual testing. (Consensus)
在一次自动化测试工作中找到的bug里,60%到80%是在开发测试中被发现的。也就是说,除非你从一开始就用自动化工具创建并运行新的测试用例,不然大多数bug都会在手动测试中被发现。(一致同意)
Most of us do not usually use the automation tool to run test cases the first time. In the traditional paradigm, you run the test case manually first, then add it to the automation suite after the program passes the test. However, you can use the tool more efficiently if you have a way of determining whether the program passed or failed the test that doesn’t depend on previously captured output. For example:
我们大多数人都不是第一次使用自动化工具来运行测试用例。在传统规范中,你首先要手动运行测试用例,然后在程序通过测试后把它添加到自动化套件中。但是,如果你有一种办法不依靠先前的截取的输出结果就能确定程序是否通过了测试,那么你就能更加有效的使用工具。例如:
Run the same series of tests on the program across different operating system versions or configurations. You may have never tested the program under this particular environment, but you know how it should work.
要在不同的操作系统版本或配置上运行相同系列的测试。你可能从未在一种特殊配置环境下测试过程序,但是你知道它应该怎样工作。
Run a function equivalence test. [11] In this case, you run two programs in parallel and feed the same inputs to both. The program that you are testing passes the test if its results always match those of the comparison program.
要运行一个功能等价测试。这时,你可以并行地运行两个程序,并且给它们同样的输入。如果程序的结果总是与对照程序一样,就说明它通过了测试。
Instrument the code under test so that it will generate a log entry any time that the program reaches an unexpected state, makes an unexpected state transition, manages memory, stack space, or other resources in an unexpected way, or does anything else that is an indicator of one of the types of errors under investigation. Use the test tool to randomly drive the program through a huge number of state transitions, logging the commands that it executes as it goes. The next day, the tester and the programmer trace through the log looking for bugs and the circumstances that triggered them. This is a simple example of a simulation. If you are working in collaboration with the application programming team, you can create tests like this that might use your tool more extensively and more effectively (in terms of finding new bugs per week) than you can achieve on your own, scripting new test cases by hand.
要在测试中操作代码,这样就能在出现异常情况时生成日志,比如,程序到达了一个异常状态、做了一个异常状态转换、以异常方式管理内存、堆栈或其他资源、或者做了任何一种错误的事情。先通过大量的状态转换以及记录执行的命令使用测试工具随机地驱动程序。然后在第二天,测试人员和程序员通过跟踪日志来寻找bug和触发它们的环境。这是模拟出的一个简单例子。如果你正在与应用程序团队合作,那你就能像这样,相对自己手工编写新的测试用例脚本,更加扩展地、有效地(每周都可以找到新的bug) 用你的工具创建测试。
7. Automation can be much more successful when we collaborate with the programmers to develop hooks, interfaces, and debug output. (Consensus)
文章来源于领测软件测试网 https://www.ltesting.net/