Add support for replacement macros in LogFilePath

[jrr]

When I run dotnet test at the solution level, results are placed within each test project like this:

Foo.Test/TestResults/TestResults.xml
Bar.Test/TestResults/TestResults.xml

I'd like to produce output like this:

test-results/
   Foo.Test.xml
   Bar.Test.xml

I can get part way there with dotnet test --logger "junit;LogFilePath=../test-results/foo.xml", but then all my test suites will rewrite the one foo.xml file.

Is it possible to emit filenames based on the name of the test project? (If not, consider this a feature request!)

[thzinc]

@jrr, I can confirm that the logger does not currently support writing multiple files in the way you described as desired.

I think a supporting change could be made to allow something like MSBuild-style replacement macros in the LogFilePath. (e.g., LogFilePath=../test-results/$(Source).xml) In looking at TestResultInfo, I don't know if anything beside TestCase.Source would make much sense for this feature.

I think this could be implemented in TestRunCompleteHandler by checking outputFilePath for $(Source) and grouping resultList by TestCase.Source.

For a fuller understanding of what you're trying to accomplish, what are you ultimately wanting to do with files in this different location? Could you append a && find -name TestResults.xml -exec mv {} test-results \; (or something similar) to the end of your dotnet test command? Could your use case be solved if this logger just appended to the file defined at LogFilePath instead of overwriting it for each project?

[jrr]

My use case is reporting results to CircleCI.

I'm currently working around the issue by copying/renaming the files, but am hoping to eventually avoid that.

Appending to one file would be an improvement, but then all the tests would belong to one "suite" in Circle's view.

[thzinc]

@jrr, thanks for the context. I'm going to update the title of this issue to reflect the feature change. I won't be able to pick up this enhancement in the near term, but I'm open to accepting a PR if you (or anyone else) wants to implement the replacement macro feature as described.

[viceice]

I think a runtime macro would be helpful too. We are have tests for netcoreapp2.1 and net452.
So the second run would owerwrite the first run results.

[viceice]

Or at least username, hostname, date.
This are the default values for the trx logger:

https://github.com/Microsoft/vstest/blob/b2ddbe9d19f01f5c738db4b89bcb1da16d8f486c/src/Microsoft.TestPlatform.Extensions.TrxLogger/ObjectModel/TestRun.cs#L200

[thzinc]

@viceice, regarding the target framework, do you know of a way to reliably get the target framework at runtime? Also, does the trx logger actually produce files with username, hostname, and date in the filenames?.

[viceice]

@thzinc I only know RuntimeInformation.FrameworkDescription.

Yes, the trx logger does exacly that.

[8]

Any news on this issue?

ATM this makes running dotnet test on the solution file not useful, e.g.:

dotnet test src/solution.sln --logger junit --results-directory TestResults

with the 2 test projects creates a single TestResults.xml because the first one is overwritten.

Take care,
Martin