test_runner: add level-based diagnostic handling for reporter

[hpatel292-seneca]

This fixes #55922

Change summary

Updated the reporter.diagnostic to accept level parameter like this

  diagnostic(nesting, loc, message, level = 'info') {
    this[kEmitMessage]('test:diagnostic', {
      __proto__: null,
      nesting,
      message,
      level,
      ...loc,
    });
  }

Then I updated #handleEvent like this

 #handleEvent({ type, data }) {
    switch (type) {
      case 'test:fail':
        if (data.details?.error?.failureType !== kSubtestsFailed) {
          ArrayPrototypePush(this.#failedTests, data);
        }
        return this.#handleTestReportEvent(type, data);
      case 'test:pass':
        return this.#handleTestReportEvent(type, data);
      case 'test:start':
        ArrayPrototypeUnshift(this.#stack, { __proto__: null, data, type });
        break;
      case 'test:stderr':
      case 'test:stdout':
        return data.message;
      case 'test:diagnostic':  // Here I added logic
        const diagnosticColor =
          reporterColorMap[data.level] || reporterColorMap['test:diagnostic'];
        return `${diagnosticColor}${indent(data.nesting)}${
          reporterUnicodeSymbolMap[type]
        }${data.message}${colors.white}\n`;
      case 'test:coverage':
        return getCoverageReport(
          indent(data.nesting),
          data.summary,
          reporterUnicodeSymbolMap['test:coverage'],
          colors.blue,
          true
        );
    }
  }

And I am Updated reporterColorMap like this

const reporterColorMap = {
  __proto__: null,
  get 'test:fail'() {
    return colors.red;
  },
  get 'test:pass'() {
    return colors.green;
  },
  get 'test:diagnostic'() {
    return colors.blue;
  },
  get info() { // Here I added logic
    return colors.blue;
  },
  get debug() { 
    return colors.gray;
  },
  get warn() { 
    return colors.yellow;
  },
  get error() { 
    return colors.red;
  },
};

and color already contain logic for this colors

I also set the reporter.diagnostic call from test.js like this (level="Error")

if (actual < threshold) {
            harness.success = false;
            process.exitCode = kGenericUserError;
            reporter.diagnostic(
              nesting,
              loc,
              `Error: ${NumberPrototypeToFixed(
                actual,
                2
              )}% ${name} coverage does not meet threshold of ${threshold}%.`,
              'error'  // Level is set to error for red color
            );
          }

Here is Demo output:

[nodejs-github-bot]

Review requested:

• @nodejs/test_runner

[hpatel292-seneca]

Hi @pmarchini, Please review and let me know if you want me to change anything.

[pmarchini]

Hey @hpatel292-seneca, I won't be able to review until Monday.
In the meantime, I suggest you take a look at the test runner output tests; we definitely need to add tests there.

I've also requested other reviews in the meantime.

[codecov]

Codecov Report

Attention: Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.

Project coverage is 87.99%. Comparing base (f270462) to head (188f357).
Report is 20 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/test_runner/reporter/utils.js	75.00%	3 Missing ⚠️
lib/internal/test_runner/test.js	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #55964      +/-   ##
==========================================
- Coverage   88.49%   87.99%   -0.51%     
==========================================
  Files         653      653              
  Lines      187728   187867     +139     
  Branches    36182    35889     -293     
==========================================
- Hits       166136   165314     -822     
- Misses      14814    15718     +904     
- Partials     6778     6835      +57     

Files with missing lines	Coverage Δ
lib/internal/test_runner/reporter/spec.js	96.26% <100.00%> (+0.07%)	⬆️
lib/internal/test_runner/tests_stream.js	91.66% <100.00%> (+0.04%)	⬆️
lib/internal/test_runner/test.js	96.97% <0.00%> (ø)
lib/internal/test_runner/reporter/utils.js	94.17% <75.00%> (-2.53%)	⬇️

... and 90 files with indirect coverage changes

---- 🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[cjihrig]

Requesting changes since this already has an approval. This also needs docs and tests.

[cjihrig]

Let's remove the debug level. The reporter stream is not a generic logger, and we have other ways (NODE_DEBUG) of adding debug output.

[hpatel292-seneca]

I will remove that. And I have a question so this CI https://github.com/nodejs/node/actions/runs/11983534043/job/33427085217?pr=55964 failed and it's for First commit message adheres to guidelines / lint-commit-message (pull_request) so should I change commit history??

[cjihrig]

test_runner: add level to diagnostics

[hpatel292-seneca]

So I am asking if I should change the commit history and force push.

[cjihrig]

Ah. Yes, please. You'll need to avoid merge commits too, as the tooling does not handle them well.

[hpatel292-seneca]

Hi @cjihrig, I pushed the change you requested and the commit history to follow the commit guidelines.

[hpatel292-seneca]

And for docs and tests could you please give more details about it? Like where the doc goes and the same for the test

[cjihrig]

The test runner docs are located in doc/api/test.md. The tests are located in test/parallel/test-runner-*.