Allow .NET Core test runner projects to test themselves

[jcansdale]

The latest build of TestDriven.Net 4.0-alpha is now available here: http://testdriven.net/download.aspx

This version is "alpha" in the sense that it's feature incomplete, not that it will take your firstborn. 😉 Please give it a try and report and issues you find. I want to make it as useful as possible.

---

This is only relevant if you're developing a .NET Core test runner (such as dotnet-test-xunit or dotnet-test-nunit). TestDriven.Net needs to locate the test runner assembly locally rather than in the set of platform assemblies.

[jcansdale]

@bradwilson I've made some tweaks to allow the development version of dotnet.test.xunit to run tests on itself. This seems to work fine when running all tests in a project (see below), but for some reason not when targeting specific tests (with -method or -class). I was wondering if you might have any idea what's going wrong or if you could take a look? (It might be something to do with the way options work and assembly loading, but that's just a guess)

Target all in project:

Target a single method:

Here is a build with the update:

TestDriven.NET-4.0.3266_Extraterrestrial_Alpha.zip

[jcansdale]

@bradwilson I'm still at a loss as to why targeting individual tests in dotnet-test-xunit.Tests doesn't work. This is a special case and it works fine for other xUnit project types. It does however work fine for the equivalent dotnet-test-nunit test project.

I've now added support for 'Test With > Debugger' on .NET Core projects. Hopefully this will make finding the problem easier. Here is a build with debugging support:

TestDriven.NET-4.0.3267_Extraterrestrial_Alpha (1).zip

[jcansdale]

@bradwilson Ah, mystery solved! Your tests are using inner classes rather than namespaces. My adapter is passing in the type names flat (without a + between nested types). This is fixable. 😃 #11

I'm curious why you use inner classes?

[bradwilson]

I use them in a couple different ways. I will use the outer class to be the class under test; I will use the inner types to either represent the method under test, or the testing context (f.e., for tests of a stack, one test context might be "with an empty stack", another might be "with a stack with one item", etc.).

[jcansdale]

I've got targeting methods and classes nested inside another class working, see below:

> dotnet-test-xunit.Tests.dll -designtime -class CommandLineTests+UnknownSwitch 

------ Test started: Assembly: dotnet-test-xunit.Tests.dll ------

xUnit.net .NET CLI test runner (64-bit .NET Core win10-x64)
  Discovering: dotnet-test-xunit.Tests (method display = Method)
  Discovered:  dotnet-test-xunit.Tests (running 1 of 73 test case)
  Starting:    dotnet-test-xunit.Tests (parallel test collections = on, max threads = 4)

  Finished:    dotnet-test-xunit.Tests

1 passed, 0 failed, 0 skipped, took 2.17 seconds (.NET Core).

...but for some reason I can't target the parent class:

> dotnet-test-xunit.Tests.dll -designtime -class CommandLineTests 

------ Test started: Assembly: dotnet-test-xunit.Tests.dll ------

xUnit.net .NET CLI test runner (64-bit .NET Core win10-x64)
  Discovering: dotnet-test-xunit.Tests (method display = Method)
  Discovered:  dotnet-test-xunit.Tests (running 0 of 73 test cases)
=== TEST EXECUTION SUMMARY ===
   dotnet-test-xunit.Tests.dll  Total: 0

From the way you describe using nested types, I imagine xUnit does support this. Do you know why it might not be working? Is there anything wrong with the arguments I'm passing?

> dotnet-test-xunit.Tests.dll -designtime -class CommandLineTests

😕

[jcansdale]

Here's a drop that shows the behavior above (inner classes work, but outer classed don't):

TestDriven.NET-3.10.0_Extraterrestrial_Beta2.zip
(ignore the version number, it's a local build)

[bradwilson]

I'm pretty sure we don't actually support that (yet) via the command line. We added specific code to our TD.net adapter to dive into child classes when asked to run a class.

[jcansdale]

We added specific code to our TD.net adapter to dive into child classes when asked to run a class.

Ah, I see what you're doing there:
https://github.com/xunit/xunit/blob/master/src/xunit.runner.tdnet/TdNetRunnerHelper.cs#L93

For the moment I could add something similar to my built-in xUnit support. Can I pass in multiple -class options?

Later on I might restore support for .tdnet files an the ITestRunner interface. I'm still working out how feasible this would be with .NET Core. So far using .NET Core's AssemblyLoadContext has been a world of pain, and I might end up having to use this.

[bradwilson]

Can I pass in multiple -class options?

Yes.

I'm not 100% sure whether your original expectation is reasonable or not. It probably is reasonable, and we should probably change the meaning of -class to include nested sub-classes always.

[jcansdale]

It looks like the NUnit runner behaves in the same way (only runs tests in the targeted class). In order to support versions on xUnit and NUnit that are already deployed, I'll need to add code to target the inner classes as well. I'll let you know when this is done.