| Store | Cart

ActiveState Docs

TDK 5.2 Documentation


Tutorial: Coverage Analysis, or How to Write a Complete Testsuite


This tutorial shows how to use the Tcl Dev Kit Debugger with coverage to ensure that a tcltest-based testsuite for a Tcl script library covers all paths through the code. When the "Plain coverage" option is selected in the Project Settings, the Tcl Dev Kit Debugger will highlight covered or uncovered lines in an application and generate data about the number of calls to each line, along with the file names and line numbers.

The project for this tutorial contains a library of code written in Tcl, and a testsuite written in Tcl that uses the tcltest package. We'll use the debugger to check which parts of the library are exercised by the testsuite. If we find that there are areas of code that are not covered, we'll extend the test suite to cover those paths as well.

The testsuite and the code library that we'll test come from tcllib, a Tcl library distributed with ActiveTcl. The module under scrutiny is csv, which is used to read and write CSV ("comma separated values") delimited data and to CSV data to and from Tcl lists.

The first testsuite we'll run, csv.test, was extracted from the core Tcl file csv_new.test.

Step One: Open the Tutorial Program

  • UNIX: Change to the directory /demos/TclDevKit/TclDebugger/Coverage/ beneath the root Tcl Dev Kit installation directory, and run:
    tcldebugger csv.tpj
  • Windows: Click Start | Programs | ActiveState Tcl Dev Kit | Tcl Dev Kit Demos | Profiling and Coverage | Coverage Testsuite I

The Tcl Dev Kit Debugger will open and load the csv project. The Code display portion of the debugger's main window will show the file csv_frame.tcl.

Step Two: View Unexecuted Lines

Run the application in the debugger as usual. You can watch variables, use the eval console, and perform all other debugging tasks while coverage information is being gathered in the background.

As the debugger executes lines in the application, lines that have not been executed will be displayed with a yellow background.

When the script has finished running, select View | Code Coverage, or click the Coverage button on the toolbar. The Code Coverage and Profiling dialog will be displayed.

Step Three: View Covered Files

The Code Coverage and Profiling dialog has two tabs, "Files" and "Calls". The Files tab, relevant to this tutorial, is displayed by default. It displays a list of all files for which the system collected coverage information.

Three files are listed:

  • csv_frame.tcl
  • csv.tcl
  • csv.test

To view a covered file in the Code display area of the debugger's main window, click on the file in the list, then click the Show Code button.

Two of the files (csv_frame.tcl and csv.test) contain the framework to call the testsuite, and the testsuite itself. For the testsuite to be comprehensive, all code in the program (csv.tcl) should have been executed. Otherwise, the test is incomplete.

Step Four: View Uncovered Code

View the uncovered code for the file csv.tcl. In the Code Coverage and Profiling dialog, select the file, then click Show Code. Scroll the Code display down to line 235. Notice that there are code paths that have not been executed.

If this were a real testsuite, you would analyze the uncovered code and create a test that exercises it. In this example, the original testsuite was deliberately altered by removing the tests for checking the "empty" and "auto" expansion modes of the split2matrix routine.

Step Five: A Complete Testsuite

Open the second tutorial project csv_new.tpj (Coverage Testsuite II). Follow the procedure you used to open Coverage Testsuite I. This project contains the original, complete testsuite. As with the previous example, run the program and view the uncovered code in csv.tcl. All paths through the code have been covered by the full testsuite.