Description
Some CLI output is already tested, but the most important output, that of a test run, is still untested, because it is not reproducible. In particular, the timestamps and ID of the output are always different.
- The ID is randomly generated (UUIDv4), jj uses an environment variable for setting the RNG's seed used by it to make test output reproducible.
- Timestamps on the other hand are not easy to fix, this requires some kind of abstraction between the runner and the reporter.
I've been working on this abstraction, but work is progressing slowly because of how deeply ingrained it is in the project. Once this is done, the output can be tested with a completely mocked runner for reproducibility.
Description
Some CLI output is already tested, but the most important output, that of a test run, is still untested, because it is not reproducible. In particular, the timestamps and ID of the output are always different.
I've been working on this abstraction, but work is progressing slowly because of how deeply ingrained it is in the project. Once this is done, the output can be tested with a completely mocked runner for reproducibility.