Coverage and Hotspot Analyzer - Graphical Interface

Overview of the Interface

The Coverage and Hotspot Analyzer has two main interface windows in GUI mode: the Project View window and the Experiment View window. The Experiment View window also functions, with slight differences, as the Comparison View window.

Project View

The Project window displays summary information on the projects and experiments in the curent Database.

The Project View pane has the following columns:

• Projects: A collapsable tree view of projects and its experiments.
• # Files: Number of files run during the experiment.
• # Runs: Number of times perl was run (i.e. the number of `perl` executions that the Analyzer has captured data for).
• # Subs: Number of Subroutines in the experiment.
• Coverage: A histogram illustrating how much of the code was run during the test (coverage). Green indicates covered code, red indicates uncovered code.
• # CLOC: Covered lines of code (i.e. lines that were run during the experiment).
• # LOC: Total lines in the tested code.
• Runtime: How long the experiment took to run (absolute/calendar time).
• Last Modified: When the experiment was run.

Main Toolbar

The toolbar in the Project Window contains the following buttons:

• Refresh from database: The Analyzer keeps all experiment results in a database. This button refreshes the current view from that database.
• Add Project: Adds a new project. This button brings up a dialog box with the following options:
  • Project Name: This can be any string up to 40 characters.
  • Default Command: The default Run Command to use (see Run Command under Add experiment below).
  • Default Working Directory: The default directory to run the script or command in.
  • Run Experiment on Project Creation: If selected, will seed the project with an initial experiment based on the defaults above.
• Delete Project: Deletes the currently selected project.
• Lock Item: Locks the currently selected project or experiment. Locking a project prevents the addition of new experiments to it or the deletion of existing ones from it. Locking an experiment prevents it from being deleted. This is useful in configurations where the Analyzer's database is shared.
• Add experiment: Create a new experiment in the currently selected project. This button brings up a dialog box with the following options:
  • Experiment Name: A default name is given based on the date and incremental number of the experiment. This can be replaced with any string up to 40 characters.
  • Run Command: The command to run for generating the experiment data. Note: Commands other than `make test` must use perlcov-perl rather than perl, either in the command itself (e.g. `perlcov-perl testscript.pl`) or in the '#!' line.
  • Working Directory: The directory to run the script or command in.
  • Collect Information for Core Modules: Include coverage and hotspot data for modules specified in use statements.
• Delete experiment: Delete the currently selected experiment.
• Mark for compare: Mark the currently selected experiment for comparison. This makes the marked experiment the baseline for comparisons. Mark an experiment, select another experiment to compare it to, then click Compare Coverage or Compare Hotspots.
• View Coverage: View code coverage for the selected experiment. Opens the Experiment View window.
• View Hotspots: View hotspots for the selected experiment. Opens the Experiment Veiw window.
• Compare Coverage: Compare the code coverage of the selected experiment with the baseline (i.e. marked) experiment. Opens the Coverage/Hotspot Window in compare mode with the baseline experiment on the right.
• Compare Hotspots: Compare the hotspots of the selected experiment with the baseline (marked) experiment. Opens the Coverage/Hotspot Window in compare mode with the baseline experiment on the right.
• View Active Runs: Opens the Active Runs dialog which lists the commands and output of the last experiments that ran in this session.

Experiment View

Clicking on the View Coverage or View Hotspots buttons or View menu items opens a new Experiment View window in either Coverage or Hotspot mode.

Coverage/Hotspot Toolbar

The Experiment View window has the following toolbar buttons:

• Coverage/Hotspot View: Toggles between Coverage and Hotspot mode. Coverage mode displays the coverage histogram and sorts the subroutine list sorted by Coverage (percentage of covered code). Hotspot mode displays the subroutine list sorted by Time.
• Flat/Tree View: Changes the grouping of the listed subroutines. Flat View shows the subroutines sorted individually; Tree View shows the subroutines as branches of the file they are found in.
• View code only: Removes all coverage and hotspot highlighting and statistics from the Code pane.
• View uncovered code: Highlights the code not covered in the experiment (light blue) and shows the call count statistics (how many times a line of code was called during the experiment).
• View hotspots (experiment normalized): Highlights the code that is run most often in the experiment (in shades of orange) along with the code coverage highlighting (light blue). The statistics column shows how much time was taken for each line of code as a percentage of the total time the experiment took. Darker shades indicate more time (i.e. "hotter" spots).
• View hotspots (function normalized): Highlights code as above, but with the percentage calculated against the time spent in the individual function rather than the entire experiment.
• Next Item: Steps through each code item in the code view pane, moving to the next subroutine in the sorted list when then end is reached.
• Previous Item: As above, but backwards.

Coverage Histogram Filter

Below the toolbar is a histogram similar to the one in the Project View pane, but with a black triangular slider. This slider can be used to filter the contents of the Experiment View pane, to show only subroutines with a certain level of coverage.

By default the slider is positioned on the extreme right of the histogram, showing subroutines in the entire (0% to 100%) range of coverage. Moving the slider left reduces this range (e.g. showing subroutines in the coverage range of 0% to 99%, 0% to 89%, and so on).

Subroutine List Pane

The Subroutine List pane displays the following columns:

• Sub or File - Sub: The name of the subroutine or file.
• Coverage: Percentage of code in each subroutine covered during the experiment.
• Covered: Number of functional lines in the subroutine run during the experiment vs. the number of functional lines in the subroutine.
• Time: Elapsed (real) time for the execution of the function in seconds.

The list can be sorted by any of these columns. By default, it is sorted from most to least covered. The subroutines can be viewed as a flat list or grouped by file (using the Flat/Tree View toolbar button or selecting Tree View from the View menu.

Code Pane

The Code Pane displays the code run in the test. Clicking on the various subroutines or files in the Subroutine List pane automatically scrolls to the appropriate place in the code.

With View uncovered code selected, uncovered lines of code are highlighted in blue. With View hotspots selected, hotspots are highlighted in orange.

The Code Pane has three columns to the left of the code itself:

• The first column is a condensed view of the coverage or hotspot highlighting. The black outlined box is a transparent scrollbar, used to quickly move to uncovered lines (shades of blue) or hotspots (shades of orange).

• The second column shows the line number of the source file. The line numbers corresponding to the currently selected subroutine are highlighted in pale yellow.

• With View uncovered code selected, the third column shows the call count statistics (i.e. how many times that line was called). With View hotspots (experiment normalized) selected, it shows the percentage of the total experiment runtime spent on that line. With View hotspots (function normalized) slected, it shows the percentage of the subroutine's runtime spent on that line.

A line of code in bold indicates that it has a "hover tip", which appears when the mouse pointer is positioned over it. The tips ('Branch is always true: "eval statement"' or 'Branch is always false: "eval statement"') often indicate why a related block of code was not covered by the test.

Experiment Details

Selecting Experiment Details from the View menu opens a pane at the bottom of the Experiment View window displaying the number of runs in an experiment, the timestamp of each run, the command which was run, the hostname, and the directory the command was run in.

Comparison View

The comparison view functions much like the Experiment View but shows two versions of the code: the current comparison experiment on the left, and the baseline (i.e. marked for comparison) experiment on the right.

The Subroutine List pane shows two sets of statistics: Coverage (percentage of code covered) and Covered (lines of code covered) apply to the current experiment, C2 % and C2 apply to the baseline.