Add Bazel XML_OUTPUT_FILE processing to main
#2333
Conversation
`bazel test` sets the `XML_OUTPUT_FILE` environment variable which
specifies where the JUnit output of the test executable should be
written.
Picking up this variable allows catch2 to naturally integrate into
the Bazel testing ecosystem out-of-the-box.
The environment variable processing is hidden behind the
`CATCH_CONFIG_MAIN_BAZEL_XML_OUTPUT_FILE` define and it _only_
enabled in the `BUILD.bazel`.
This allows the Bazel ecosystem to create a `cc_test` target as
follows:
```
cc_test(
name = "some_test",
deps = ["@catch2//:catch2_main"],
srcs = globs("**/*_test.cpp"),
)
```
The output will be written to `bazel-testlogs/some_test/test.xml`
|
RFC: this is quite a punt to allow Catch2 to integrate more tightly into the Bazel ecosystem out-of-the-box. It's a bit overzealous about respecting Happy to create a new Bazel target ( Also happy for it to be rejected: we can add the patches downstream in the Bazel Central Registry, if needed. |
|
Does this override Catch2's default console output when run via Bazel? If so it would be nice if there was output when run with |
Codecov Report
@@ Coverage Diff @@
## devel #2333 +/- ##
=======================================
Coverage 90.96% 90.96%
=======================================
Files 151 151
Lines 7231 7232 +1
=======================================
+ Hits 6577 6578 +1
Misses 654 654 |
Yes. Changing the reporter to Catch2 doesn't support mutliple reporters. There is a proposal in #1712 which would enable both JUnit output and console output. |
There was a problem hiding this comment.
Shame Bazel didn't use properly namespaced env variable for this, it could've been enabled unconditionally. XML_OUTPUT_FILE is waaay too generic for that though.
Also this likely shouldn't be in the default-main, instead it should internally add another reporter (if the user doesn't want this behaviour, they can just not provide the define/env variable).
Oh and it obviously needs tests. We already have a bunch of separately-compiled tests in tests/ExtraTests, so you can look there at how to add new one.
| return Catch::Session().run( argc, argv ); | ||
| Catch::Session session; | ||
|
|
||
| #if defined(CATCH_CONFIG_MAIN_BAZEL_JUNIT_XML_OUTPUT_FILE) |
There was a problem hiding this comment.
That's a terrible define name. How about CATCH_CONFIG_BAZEL_SUPPORT? (Does Bazel have more of these customization env option?)
Also it needs to be documented in docs/configuration.md
There was a problem hiding this comment.
Agreed. I just sorta went for something. Happy to spin to CATCH_CONFIG_BAZEL_SUPPORT if we don't spin out another Bazel specific main.
| auto bazelOutputFile = std::getenv("XML_OUTPUT_FILE"); | ||
| if (bazelOutputFile != nullptr) { | ||
| session.configData().reporterName = "junit"; | ||
| session.configData().outputFilename = bazelOutputFile; |
There was a problem hiding this comment.
This needs to be redone with the multireporter support.
There was a problem hiding this comment.
Huh, cool. Will look into that.
Yep. I can ask upstream in Bazel world to see it could also export
Like, a whole new Bazel reporter, i.e.
Yep. Was pushing this up with minimal code to see if it got straight up rejected. I can move forward with filling out he edges. Thanks for the comments! |
No. I meant that Catch2 should internally build another reporter for Bazel when it detects the env variable, instead of having it be done in the main function. |
So either we add another to the config when it is created or in the session? Something like:
or
The former might be nicer because dumping the configuration would show the extra JUnit reporter when |
|
Either of these is fine honestly. I would probably go with inserting the new reporter as part of parsed config (I think the |
|
I recently changed how the default reporter is handled, the Bazel reporter can likely be handled in the same way: fc5552d |
|
Closing in favour of #2399 |
bazel testsets theXML_OUTPUT_FILEenvironment variable whichspecifies where the JUnit output of the test executable should be
written.
Picking up this variable allows catch2 to naturally integrate into
the Bazel testing ecosystem out-of-the-box.
The environment variable processing is hidden behind the
CATCH_CONFIG_MAIN_BAZEL_XML_OUTPUT_FILEdefine and it onlyenabled in the
BUILD.bazel.This allows the Bazel ecosystem to create a
cc_testtarget asfollows:
The output will be written to
bazel-testlogs/some_test/test.xml