Product Comparisons
Testing includes many activities. A variety tools are available to automate these activities, but because they are so diverse, no one tool provides a complete solution. Rather users must select a suite of tools that best support their testing goals.
The following table includes the types of testing activities.
| Testing Activity |
Overall Testing Effort |
Tools Providing Support |
Tool Classes |
| Test Planning |
10% |
10% |
Requirement Tools
Test Management Tools
|
| Test Case Design |
30%-40% |
5% |
Functional Test Generators
Data(base) Generators
|
| Test Driver/Script Creation |
20% |
25% |
(GUI) Capture Playback
Test Harness Generators
Template-based Drivers
|
| Test Execution & Results Analysis |
10%-20% |
40% |
(GUI) Capture Playback
Performance Testing
Load Testing
|
| Test Measurement |
10% |
10% |
Requirement Test Coverage
Structural Test Coverage
Code Complexity Analysis
Code Coverage Analysis
|
| Test Management |
10% |
10% |
Test Management Tools
Configuration Management
|
For each activity the table also includes a
- Estimate of the effort required by the testing activity as a percentage of total testing effort
- Estimate of the tools that support the activity as a percentage of all testing tools
- Classes of tools commercially available that support the activity
The following indicates the testing activities some representative testing tools support.
The following table includes the types of testing activities.
| Testing Activity |
T-VEC RAVE |
Starbase Caliber RBT |
Winrunner
Silk Test
Rational Robot
|
McCabe |
T-VEC Tester for Simulink |
ADI AUTT |
| Test Planning |
|
|
|
|
|
|
| Test Case Design |
X |
X
(bool and enum only) |
|
|
X |
X |
| Test Driver/Script Creation |
X |
|
X |
|
X |
|
| Test Execution & Results Analysis |
X |
|
X |
|
X |
|
| Test Measurement |
X
(req cov) |
X
(req cov) |
|
X
(complex measure) |
X
(model cov) |
X
(model cov) |
| Test Management |
X
(reqmnts) |
X
(reqmnts) |
|
|
|
|
As highlighted in the first table, notice that there are relatively fewer tools that support creating test cases (Test Case Design), although this is typically the most expensive testing activity. A large majority of testing tools support test execution. This partly because there are many unique environment in which tests are executed, and these tools are often specific to an environment.
|
