GTest Stdout capture

[EUdds]

Right now GTest prints out the stdout of all the tests it runs as it's testing. This can lead to some pretty messy testing logs. It would be beneficial to capture the stdout and stderr streams of each test and display them all separated after all tests have been completed.

[EUdds]

Some basic design requirements:

• Easily display which tests passed and which failed
• If a test failed show it's stdout/stderr after all tests have been completed
• Hide a passing test's stdout/stderr unless a command line argument has been specified
• Make it easy to look at with colors and stuff. Go nuts

[mjkpolo]

LogPrinter Preview:

GTest Event Listener which captures STDOUT and STDERR on test failure.

Requesting ideas for nice-to-haves and improvements

Sample Tests Borrowed: GTest Github
Capturing STDERR and STDOUT: stackoverflow

Sample Test Test_Development.cpp:

TEST(DevelopmentTest, Test_stdout)
{
    for (int i = 0; i < 5; i++) {
        fprintf(stdout, "Printing to stdout\n");
    }
}

TEST(DevelopmentTest, Test_stderr)
{
    for (int i = 0; i < 5; i++) {
        fprintf(stderr, "Printing to stderr\n");
    }
}

Output Using Event Listener:

*** Test DevelopmentTest.Test_stdout starting.
STDOUT: {Printing to stdout
Printing to stdout
Printing to stdout
Printing to stdout
Printing to stdout
}
STDERR: {}
*** Test DevelopmentTest.Test_stdout ending.
*** Test DevelopmentTest.Test_stderr starting.
STDOUT: {}
STDERR: {Printing to stderr
Printing to stderr
Printing to stderr
Printing to stderr
Printing to stderr
}
*** Test DevelopmentTest.Test_stderr ending.
TEST PASSED

How to Help

Comment with suggestions for functionality, or errors

[dawiggleman]

Nice work, looks good! So is this an operation that runs after the entirety of the test is completed, not simultaneously? And the stdout and stderror printouts are the consolidated messages from that singular test that was run? Is there any foreseen need to "group" the displayed stderror and stdout messages that occur "near" each other in code, rather in the two separate lists here?

[mjkpolo]

LogPrinter is an Event Listener that would be logging the prints to these streams of the tests as they run, so I think it would be simultaneous. Yep, they are the consolidated messages from that test, and each test captures a single string of everything printed from the start to end of the test. Here is the code that does it in LogPrinter.cpp:

void LogPrinter::OnTestStart(const testing::TestInfo& test_info)
{
    printf("*** Test %s.%s starting.\n",
        test_info.test_case_name(), test_info.name());
    testing::internal::CaptureStdout(); // Captures STDOUT
    testing::internal::CaptureStderr(); // Captures STDERR
}

Then, on the test ending, we have:

void LogPrinter::OnTestEnd(const testing::TestInfo& test_info)
{
    std::string stdout_output = testing::internal::GetCapturedStdout();
    std::string stderr_output = testing::internal::GetCapturedStderr();
    ...

That's the base functionality, and the grouping of STDOUT and STDERR could vary in length only from OnTestStart() to OnTestEnd() if you were thinking more of stacking STDOUT and STDERR in smaller increments of time or number of prints to the streams.

[EUdds]

Looks really good Marco - I just want to post a few things we talked about privately here for discussion among some ideas I had while reading this.

• I think adding color would be really good. You can see how to print in color here for example "TEST PASSED" could be green.
• Put a line break after the first curly bracket so that all the output starts on a new line
• If you're able to get the number of tests that have been collected at the start you can show a "Running test n / N" prompt before each test
• I agree with Shelby on possibly grouping the stdout and stderr - but I'm thinking this should be done after a failing test.
• We briefly discussed having a "verbose" option that would show output regardless of pass/fail - I think we could keep it separate in passing tests and a verbose flag was set.

Overall this is a really good start. Good job!

[mjkpolo]

UPDATE:

LogPrinter prints stdout and stderr for each test separately at the end of a unit test, only printing the logs with failed tests and the name of the failed test at the top. I'm also picking colors for fun and formatting, so please comment with suggests as to how improve the look of the printing.

[mjkpolo]

LogPrinter Pytest Copy

Copied the way pytest looks. Still using default gtest listener + LogPrinter