Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,13 @@ projects/Generated
DerivedData
*.xccheckout
Build
.idea
.vs
/build/
/.idea/
/.vs/
/.vscode/
cmake-build-*
benchmark-dir
.conan/test_package/build
bazel-*
/.cache/
/compile_commands.json
21 changes: 21 additions & 0 deletions docs/reporters.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ There are four reporters built in to the single include:

* `console` writes as lines of text, formatted to a typical terminal width, with colours if a capable terminal is detected.
* `compact` similar to `console` but optimised for minimal output - each entry on one line
* `gtest` also designed for console output, but mimics the output of Google Test framework. See below for details.
* `junit` writes xml that corresponds to Ant's [junitreport](http://help.catchsoftware.com/display/ET/JUnit+Format) target. Useful for build systems that understand Junit.
Because of the way the junit format is structured the run must complete before anything is written.
* `xml` writes an xml format tailored to Catch. Unlike `junit` this is a streaming format so results are delivered progressively.
Expand All @@ -35,6 +36,26 @@ You see what reporters are available from the command line by running with `--li

By default all these reports are written to stdout, but can be redirected to a file with [`-o` or `--out`](command-line.md#sending-output-to-a-file)

### gtest reporter

The `gtest` reporter is much more verbose than `console` but it allows you to:
* See how each of your tests is run - it feels good to see a lot of green lines
* See what test is being run now - useful when some tests are slow
* See the progress is going - useful when the whole test suit is slow
* See the output of each test nested inside its scope - useful when tests print something to stdout
* Feel comfortable if you are used to Google Test

The reporter is optimized for test cases that do not have nested sections in them. A _test case_ from Catch2 is thought of as a _test case_ from gtest and each _section_ of the test case is thought of as a _test_ from gtest. The latter is referred to as simply a "test" in the output. It is assumed that code outside sections is only used for set up and tear down and is unlikely to fail the test case.

The reporter also works perfectly fine when there are nested sections, but we need to know the general definition of a "test" to understand how it works: a test is a single run of a test case. For instance:
* If there are no sections in the test case, Catch2 runs the test case once and thus there is only one test.
* If there are `N` non-nested sections in the test case, Catch2 runs it `N` times and thus there is `N` tests.
* If there are nested sections in the test case, Catch2 runs it for every leaf section and thus the number of tests depends on the structure of the sections.

Each time Catch2 enters a section the reporter prints the `[ RUN ]` line to the output. With the exception of the root section, which Catch2 implicitly creates for the body of the test case itself. It is only printed once per test case to make the output a little more compact and similar to what you'd expect. You can set verbosity to high to remove this exception and fully see how the sections stack is formed during the execution.

Note that since Catch2 is much more flexible with its sections than Google Test with its test cases and tests, there is no single "true" way to define how the output of the reporter should look. Especially if there are nested sections. To make things even more complicated, Catch2 runner does not know the structure of the sections before they are actually executed, and thus we cannot analyze this structure beforehand to e.g. collapse the output for entering multiple nested sections.

## Writing your own reporter

You can write your own custom reporter and register it with Catch.
Expand Down
18 changes: 18 additions & 0 deletions include/internal/catch_string_manip.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#include <ostream>
#include <cstring>
#include <cctype>
#include <sstream>
#include <vector>

namespace Catch {
Expand All @@ -20,6 +21,9 @@ namespace Catch {
char toLowerCh(char c) {
return static_cast<char>( std::tolower( static_cast<unsigned char>(c) ) );
}
char toUpperCh(char c) {
return static_cast<char>( std::toupper( static_cast<unsigned char>(c) ) );
}
}

bool startsWith( std::string const& s, std::string const& prefix ) {
Expand All @@ -40,11 +44,19 @@ namespace Catch {
void toLowerInPlace( std::string& s ) {
std::transform( s.begin(), s.end(), s.begin(), toLowerCh );
}
void toUpperInPlace( std::string& s ) {
std::transform( s.begin(), s.end(), s.begin(), toUpperCh );
}
std::string toLower( std::string const& s ) {
std::string lc = s;
toLowerInPlace( lc );
return lc;
}
std::string toUpper( std::string const& s ) {
std::string lc = s;
toUpperInPlace( lc );
return lc;
}
std::string trim( std::string const& str ) {
static char const* whitespaceChars = "\n\r\t ";
std::string::size_type start = str.find_first_not_of( whitespaceChars );
Expand Down Expand Up @@ -99,6 +111,12 @@ namespace Catch {
m_label( label )
{}

std::string pluralise::str() const {
std::stringstream ss;
ss << *this;
return ss.str();
}

std::ostream& operator << ( std::ostream& os, pluralise const& pluraliser ) {
os << pluraliser.m_count << ' ' << pluraliser.m_label;
if( pluraliser.m_count != 1 )
Expand Down
4 changes: 4 additions & 0 deletions include/internal/catch_string_manip.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@ namespace Catch {
bool endsWith( std::string const& s, char suffix );
bool contains( std::string const& s, std::string const& infix );
void toLowerInPlace( std::string& s );
void toUpperInPlace( std::string& s );
std::string toLower( std::string const& s );
std::string toUpper( std::string const& s );
//! Returns a new string without whitespace at the start/end
std::string trim( std::string const& str );
//! Returns a substring of the original ref without whitespace. Beware lifetimes!
Expand All @@ -34,6 +36,8 @@ namespace Catch {
struct pluralise {
pluralise( std::size_t count, std::string const& label );

std::string str() const;

friend std::ostream& operator << ( std::ostream& os, pluralise const& pluraliser );

std::size_t m_count;
Expand Down
121 changes: 121 additions & 0 deletions include/reporters/catch_reporter_bases.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

#include "../internal/catch_interfaces_reporter.h"
#include "../internal/catch_errno_guard.h"
#include "../internal/catch_text.h"
#include "catch_reporter_bases.hpp"

#include <cstring>
Expand Down Expand Up @@ -80,5 +81,125 @@ namespace Catch {
return false;
}

ConsoleAssertionPrinter::ConsoleAssertionPrinter(std::ostream& _stream, AssertionStats const& _stats, bool _printInfoMessages)
: stream(_stream),
stats(_stats),
result(_stats.assertionResult),
colour(Colour::None),
message(result.getMessage()),
messages(_stats.infoMessages),
printInfoMessages(_printInfoMessages) {
switch (result.getResultType()) {
case ResultWas::Ok:
colour = Colour::Success;
passOrFail = "PASSED";
//if( result.hasMessage() )
if (_stats.infoMessages.size() == 1)
messageLabel = "with message";
if (_stats.infoMessages.size() > 1)
messageLabel = "with messages";
break;
case ResultWas::ExpressionFailed:
if (result.isOk()) {
colour = Colour::Success;
passOrFail = "FAILED - but was ok";
} else {
colour = Colour::Error;
passOrFail = "FAILED";
}
if (_stats.infoMessages.size() == 1)
messageLabel = "with message";
if (_stats.infoMessages.size() > 1)
messageLabel = "with messages";
break;
case ResultWas::ThrewException:
colour = Colour::Error;
passOrFail = "FAILED";
messageLabel = "due to unexpected exception with ";
if (_stats.infoMessages.size() == 1)
messageLabel += "message";
if (_stats.infoMessages.size() > 1)
messageLabel += "messages";
break;
case ResultWas::FatalErrorCondition:
colour = Colour::Error;
passOrFail = "FAILED";
messageLabel = "due to a fatal error condition";
break;
case ResultWas::DidntThrowException:
colour = Colour::Error;
passOrFail = "FAILED";
messageLabel = "because no exception was thrown where one was expected";
break;
case ResultWas::Info:
messageLabel = "info";
break;
case ResultWas::Warning:
messageLabel = "warning";
break;
case ResultWas::ExplicitFailure:
passOrFail = "FAILED";
colour = Colour::Error;
if (_stats.infoMessages.size() == 1)
messageLabel = "explicitly with message";
if (_stats.infoMessages.size() > 1)
messageLabel = "explicitly with messages";
break;
// These cases are here to prevent compiler warnings
case ResultWas::Unknown:
case ResultWas::FailureBit:
case ResultWas::Exception:
passOrFail = "** internal error **";
colour = Colour::Error;
break;
}
}

void ConsoleAssertionPrinter::print() const {
printSourceInfo();
if (stats.totals.assertions.total() > 0) {
printResultType();
printOriginalExpression();
printReconstructedExpression();
} else {
stream << '\n';
}
printMessage();
}

void ConsoleAssertionPrinter::printResultType() const {
if (!passOrFail.empty()) {
Colour colourGuard(colour);
stream << passOrFail << ":\n";
}
}
void ConsoleAssertionPrinter::printOriginalExpression() const {
if (result.hasExpression()) {
Colour colourGuard(Colour::OriginalExpression);
stream << " ";
stream << result.getExpressionInMacro();
stream << '\n';
}
}
void ConsoleAssertionPrinter::printReconstructedExpression() const {
if (result.hasExpandedExpression()) {
stream << "with expansion:\n";
Colour colourGuard(Colour::ReconstructedExpression);
stream << Column(result.getExpandedExpression()).indent(2) << '\n';
}
}
void ConsoleAssertionPrinter::printMessage() const {
if (!messageLabel.empty())
stream << messageLabel << ':' << '\n';
for (auto const& msg : messages) {
// If this assertion is a warning ignore any INFO messages
if (printInfoMessages || msg.type != ResultWas::Info)
stream << Column(msg.message).indent(2) << '\n';
}
}
void ConsoleAssertionPrinter::printSourceInfo() const {
Colour colourGuard(Colour::FileName);
stream << result.getSourceInfo() << ": ";
}

} // end namespace Catch
28 changes: 28 additions & 0 deletions include/reporters/catch_reporter_bases.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#ifndef TWOBLUECUBES_CATCH_REPORTER_BASES_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_REPORTER_BASES_HPP_INCLUDED

#include "../internal/catch_console_colour.h"
#include "../internal/catch_enforce.h"
#include "../internal/catch_interfaces_reporter.h"

Expand Down Expand Up @@ -280,6 +281,33 @@ namespace Catch {
bool assertionEnded(AssertionStats const&) override;
};

// Formatter impl for ConsoleReporter
class ConsoleAssertionPrinter {
public:
ConsoleAssertionPrinter& operator= (ConsoleAssertionPrinter const&) = delete;
ConsoleAssertionPrinter(ConsoleAssertionPrinter const&) = delete;
ConsoleAssertionPrinter(std::ostream& _stream, AssertionStats const& _stats, bool _printInfoMessages);

void print() const;

private:
void printResultType() const;
void printOriginalExpression() const;
void printReconstructedExpression() const;
void printMessage() const;
void printSourceInfo() const;

std::ostream& stream;
AssertionStats const& stats;
AssertionResult const& result;
Colour::Code colour;
std::string passOrFail;
std::string messageLabel;
std::string message;
std::vector<MessageInfo> messages;
bool printInfoMessages;
};

} // end namespace Catch

#endif // TWOBLUECUBES_CATCH_REPORTER_BASES_HPP_INCLUDED
Loading