Improving Code Coverage Reporting in Monorepos

Improving Code Coverage Reporting in Monorepos

Tracking code coverage is an essential aspect of maintaining the health and robustness of a software project. However, this task becomes more challenging when using a monorepo, where multiple applications and packages coexist within a single repository. Despite the multitude of available tools, I found it impossible to create a simple report for our weekly reviews. This article outlines a method to streamline code coverage reporting in a monorepo setup using Node.js, aiming to produce an easily interpretable summary report.

The ultimate goal is to have a straightforward, console-friendly table output in the following format:

Let’s break down the process of achieving this.

NOTE: Please remember that this code was developed with a specific objective and a precise Turborepo structure. Although it could be generalized, I currently have no immediate need for such modifications.

Prerequisites

The repository should already have unit testing with jest configured for individual packages and applications.

Process Overview

The process of creating this summary report involves the following steps:

1. Configuring TurboRepo to collect coverage in JSON format per package or application. This will result in a coverage-summary.json file in the coverage folder of each package.
2. Writing a script to collate these coverage summaries. The script will:
   1. Identify all application and package directories,
   2. Retrieve paths to all the summary files, and
   3. Generate a combined JSON file representing both individual and total coverage.
3. Rendering the collected data in a console-friendly table.

Configuration changes

First of all we should create coverage reports for every single package and application in the repository. To do so, we should have proper jest.config.ts files for them at the first place. The coverageReport property should be added:

  "coverageReporters": ["json"]

NOTE: You can have a huge variety of the report formats listed in the array, but for this article only json is needed

Moreover, the package.json file for each package should be configured with the following task:

  "coverage": "jest --coverage"

To verify we have the reporting working we should run the following command for individual package/application.

yarn coverage

Than take a look a the new directory coverage created in the directory from which you’ve run the command above. It should contain the file called coverage-summary.json.

NOTE: I’m using yarn, but it will just as good with npm run

The next step will be to configure the project to run command above for each package and application. I’m using turborepo for managing mono repository. It has configuration file turbo.json at which I should add just a single line

  "coverage": {
    "dependsOn": ["^build"]
  },

As the result, I can run turbo run coverage command which will trigger coverage calculation for each application and package.

The only component we’re missing is the script that collects all the coverage reports and creates a summary. Let’s name the script like this monoRepoTotalCoverage.js NOTE: Details are covered in the section below. It’s executable with node monoRepoTotalCoverage.js command.

For the sake of simplicity it would be nice to have yarn/npm tasks in package.json to make preparation of summary report a single line command.

"scripts": {
  ...
  "coverage:per-package": "turbo run coverage",
  "coverage:total": "yarn coverage:per-package && node coverage-total.js",
  ...
}

The only needed command now is yarn coverage:total.

Script review

The script involves several stages and performs the following functions:

1. Collect Paths to Coverage Summaries: The script navigates through the directories of all applications and packages within the monorepo, assembling paths to each package’s coverage-summary.json file.

function getAllPathsForPackagesSummaries() {
  const getDirectories = source =>
    fs
      .readdirSync(source, { withFileTypes: true })
      .filter(dirent => dirent.isDirectory())
      .map(dirent => dirent.name);

  const appsPath = path.join(_dirname, "apps");
  const appsNames = getDirectories(appsPath);

  const appsSummaries = appsNames.reduce((summary, appName) => {
    return {
      ...summary,
      [appName]: path.join(
        appsPath,
        appName,
        "coverage",
        "coverage-summary.json"
      ),
    };
  }, {});

  const packagesPath = path.join(_dirname, "packages");
  const packageNames = getDirectories(packagesPath);

  const packagesSummaries = packageNames.reduce((summary, packageName) => {
    return {
      ...summary,
      [packageName]: path.join(
        packagesPath,
        packageName,
        "coverage",
        "coverage-summary.json"
      ),
    };
  }, {});

  return { ...appsSummaries, ...packagesSummaries };
}

2. Generate Consolidated Coverage Report: Each package’s coverage-summary.json file is read, and the coverage statistics are aggregated into a unified summary report. This provides a consolidated view of the total code coverage across the monorepo.

function readSummaryPerPackageAndCreateJoinedSummaryReportWithTotal(
  packagesSummaryPaths
) {
  return Object.keys(packagesSummaryPaths).reduce(
    (summary, packageName) => {
      const reportPath = packagesSummaryPaths[packageName];
      if (fs.existsSync(reportPath)) {
        const report = JSON.parse(fs.readFileSync(reportPath, "utf8"));

        const { total } = summary;

        Object.keys(report.total).forEach(key => {
          if (total[key]) {
            total[key].total += report.total[key].total;
            total[key].covered += report.total[key].covered;
            total[key].skipped += report.total[key].skipped;
            total[key].pct = Number(
              ((total[key].covered / total[key].total) * 100).toFixed(2)
            );
          } else {
            total[key] = { ...report.total[key] };
          }
        });
        return { ...summary, [packageName]: report.total, total };
      }

      return summary;
    },
    { total: {} }
  );
}

3. Format Report for Visual Presentation: The script transforms the coverage report into a console-friendly format, appending the percentage difference next to the current coverage if a comparison with a previous report was made.

function createCoverageReportForVisualRepresentation(coverageReport) {
  return Object.keys(coverageReport).reduce((report, packageName) => {
    const { lines, statements, functions, branches } =
      coverageReport[packageName];
    return {
      ...report,
      [packageName]: {
        lines: lines.pct,
        statements: statements.pct,
        functions: functions.pct,
        branches: branches.pct,
      },
    };
  }, {});
}

4. Print the report: The final coverage report is printed to the console in an easy-to-read table format.

console.table(coverageReportForVisualRepresentation);

The final execution script, built from the functions mentioned above, is as follows:

// Execution Stages
// 1. Read all coverage-total.json files
const packagesSummaryPaths = getAllPathsForPackagesSummaries();
// 2. Generate consolidated report
const currCoverageReport =
  readSummaryPerPackageAndCreateJoinedSummaryReportWithTotal(
    packagesSummaryPaths
  );
// 3. Reformat the report for visual representation
const coverageReportForVisualRepresentation =
  createCoverageReportForVisualRepresentation(currCoverageReport);
// 4. Print the report
console.table(coverageReportForVisualRepresentation);

This script will produce an easy-to-read table in the console as output:

While the current coverage report offers valuable insights, there’s potential for improvement. Specifically, it would be beneficial to contrast current data with previous reports. To achieve this, we need to implement the following improvements:

• Ensure result persistence
• Enable loading of previous results
• Introduce a function for calculating differences
• Revise the function that prepares visual representation to exhibit these differences.

Delineating Differences with Previous Results

• Let’s implement saving of the report for future usage. It can be done the following way:

function writeCoverageReportToFile(coverageReport) {
  function createDateTimeSuffix() {
    const date = new Date();
    return `${date.getFullYear()}-${
      date.getMonth() + 1
    }-${date.getDate()}_${date.getHours()}-${date.getMinutes()}`;
  }

  const dir = path.join(_dirname, "coverage");

  if (!fs.existsSync(dir)) {
    fs.mkdirSync(dir);
  }

  fs.writeFileSync(
    `coverage/coverage-total.${createDateTimeSuffix()}.json`,
    JSON.stringify(coverageReport, null, 2)
  );
}

• Loading of the previous report is also straight forward. We will pass the path to the previous report as parameter.

like this:

yarn coverage:total coverage/coverage-total.2023-5-17_15-30.json

And the implementation in the script will be the following:

// Params
const pathToPreviousReport = process.argv[2];

/**
 * Reads the coverage-summary.{XXX}.json file and returns the parsed JSON object
 * @param {*} pathToReport
 * @returns
 */
function readPreviousCoverageSummary(pathToReport) {
  if (!pathToReport) {
    console.warn("Previous coverage results were not provided.");
    return;
  }
  // Read the JSON file
  const prevCoverage = JSON.parse(fs.readFileSync(pathToReport, "utf8"));
  return prevCoverage;
}

• Now we need to calculate the difference between current and previous report. Let’s do it that way:

function creteDiffCoverageReport(currCoverage, prevCoverage = {}) {
  return Object.keys(currCoverage).reduce((summary, packageName) => {
    const currPackageCoverage = currCoverage[packageName];
    const prevPackageCoverage = prevCoverage[packageName];
    if (prevPackageCoverage) {
      const coverageKeys = ["lines", "statements", "functions", "branches"];
      coverageKeys.forEach(key => {
        const prevPct = prevPackageCoverage[key]?.pct || 0;
        const currPct = currPackageCoverage[key]?.pct || 0;

        currPackageCoverage[key] = {
          ...currPackageCoverage[key],
          pctDiff: (parseFloat(currPct) - parseFloat(prevPct)).toFixed(2),
        };
      });
    }
    return { ...summary, [packageName]: currPackageCoverage };
  }, {});
}

• And the last thing to improve is to update the output table with the difference between current and previous report.

function formatPtcWithDiff(ptc, ptcDiff) {
  return appendDiff(formatDecimal(ptc), ptcDiff && formatDecimal(ptcDiff));
}

function formatDecimal(ptc) {
  return parseFloat(ptc).toFixed(2);
}

function appendDiff(ptc, ptcDiff) {
  if (!ptcDiff || ptcDiff === ptc) {
    return ptc;
  }
  return `${ptc} (${ptcDiff > 0 ? "+" : ""}${ptcDiff}%)`;
}

/**
 * Takes the coverage report and returns an object with the
 * coverage for each package and the total coverage suitable
 * for the visual representation in a console table
 * @param {*} coverageReport
 * @returns
 * */
function createCoverageReportForVisualRepresentation(coverageReport) {
  return Object.keys(coverageReport).reduce((report, packageName) => {
    const { lines, statements, functions, branches } =
      coverageReport[packageName];
    return {
      ...report,
      [packageName]: {
        lines: formatPtcWithDiff(lines.pct, lines.pctDiff),
        statements: formatPtcWithDiff(statements.pct, statements.pctDiff),
        functions: formatPtcWithDiff(functions.pct, functions.pctDiff),
        branches: formatPtcWithDiff(branches.pct, branches.pctDiff),
      },
    };
  }, {});
}

The final report table with coverage changes will look like this:

Conclusion

In summary, this article showed how to improve the process of tracking and visualizing Jest coverage reports across monorepos. The provided script creates a handy coverage table and, while tailored to a specific Turborepo structure, can be adapted for different projects. You can find the complete script in this GitHub Gist.