Product Comparisons
Testing includes many activities. A variety tools are available to automate these activities, but because they are so diverse, no one tool provides a complete solution. Rather users must select a suite of tools that best support their testing goals.
The following table includes the types of testing activities.
Testing Activity |
Overall Testing Effort |
Tools Providing Support |
Tool Classes |
Test Planning |
10% |
10% |
Requirement Tools
Test Management Tools
|
Test Case Design |
30%-40% |
5% |
Functional Test Generators
Data(base) Generators
|
Test Driver/Script Creation |
20% |
25% |
(GUI) Capture Playback
Test Harness Generators
Template-based Drivers
|
Test Execution & Results Analysis |
10%-20% |
40% |
(GUI) Capture Playback
Performance Testing
Load Testing
|
Test Measurement |
10% |
10% |
Requirement Test Coverage
Structural Test Coverage
Code Complexity Analysis
Code Coverage Analysis
|
Test Management |
10% |
10% |
Test Management Tools
Configuration Management
|
For each activity the table also includes a
- Estimate of the effort required by the testing activity as a percentage of total testing effort
- Estimate of the tools that support the activity as a percentage of all testing tools
- Classes of tools commercially available that support the activity
The following indicates the testing activities some representative testing tools support.
The following table includes the types of testing activities.
Testing Activity |
T-VEC RAVE |
Starbase Caliber RBT |
Winrunner
Silk Test
Rational Robot
|
McCabe |
T-VEC Tester for Simulink |
ADI AUTT |
Test Planning |
|
|
|
|
|
|
Test Case Design |
X |
X (bool and enum only) |
|
|
X |
X |
Test Driver/Script Creation |
X |
|
X |
|
X |
|
Test Execution & Results Analysis |
X |
|
X |
|
X |
|
Test Measurement |
X (req cov) |
X (req cov) |
|
X (complex measure) |
X (model cov) |
X (model cov) |
Test Management |
X (reqmnts) |
X (reqmnts) |
|
|
|
|
As highlighted in the first table, notice that there are relatively fewer tools that support creating test cases (Test Case Design), although this is typically the most expensive testing activity. A large majority of testing tools support test execution. This partly because there are many unique environment in which tests are executed, and these tools are often specific to an environment.
|