Coverage Evaluation

To check your unit test coverage:

$ brownie test --coverage

When the tests complete, a report will display:

Coverage analysis:

  contract: Token - 82.3%
    SafeMath.add - 66.7%
    SafeMath.sub - 100.0%
    Token.<fallback> - 0.0%
    Token.allowance - 100.0%
    Token.approve - 100.0%
    Token.balanceOf - 100.0%
    Token.decimals - 0.0% - 100.0%
    Token.symbol - 0.0%
    Token.totalSupply - 100.0%
    Token.transfer - 85.7%
    Token.transferFrom - 100.0%

Coverage report saved at reports/coverage.json

Brownie outputs a % score for each contract method that you can use to quickly gauge your overall coverage level. A detailed coverage report is also saved in the project’s reports folder, that can be viewed via the Brownie GUI.

Viewing Coverage Data

For an in-depth examination of your test coverage, first open the Brownie GUI:

brownie gui

Click on the drop-down list in the upper right that says “Select Report” and choose “coverage”. A new drop-down list will appear where you can select which type of coverage data to view (branches or statements).

Relevant code will be highlighted in different colors:

  • Green code was executed during the tests
  • Yellow branch code executed, but only evaluated truthfully
  • Orange branch code executed, but only evaluated falsely
  • Red code did not execute during the tests
Viewing Coverage Data

How Coverage Evaluation Works

Test coverage is calculated by generating a map of opcodes associated with each statement and branch of the source code, and then analyzing the stack trace of each transaction to see which opcodes executed. See “Evaluating Solidity Code Coverage via Opcode Tracing” for a more detailed explanation of how coverage evaluation works.

Improving Performance

During coverage analysis, all contract calls are executed as transactions. This gives a more accurate coverage picture by allowing analysis of methods that are typically non-state changing. A snapshot is taken before each of these calls-as-transactions, and the state is reverted immediately after to ensure that the outcome of the test is not affected. For tests that involve many calls this can result in significantly slower execution time.

Some things to keep in mind that can help to reduce your test runtime when evaluating coverage:

  1. Coverage is analyzed on a per-transaction basis, and the results are cached. If you repeat an identical transaction, Brownie will not analyze it the 2nd time. Keep this in mind when designing and sequencing setup fixtures.
  2. For tests that involve many calls to the same getter method, use the no_call_coverage marker to significantly speed execution.
  3. Omit very complex tests altogether with the skip_coverage marker.
  4. If possible, always run your tests in parralel with xdist.

You can use the --durations flag to view a profile of your slowest tests. You may find good candidates for optimization, or the use of the no_call_coverage and skip_coverage fixtures.