Notes on Integration Testing with OrchardCore

[mariojsnunes]

Hello guys,

I've been playing around with unit testing with OrchardCore in the past few weeks, and I want to share some of my notes.

For those who are not familiar with UnitTesting OrchardCore, you can check out this folder:
https://github.com/OrchardCMS/OrchardCore/tree/dev/test/OrchardCore.Tests/Apis/Context
Basically it has a global SiteContext, and then a Context for each controller or module that inherits from it.

What I found:

1. You can run your recepies, create users, etc on each context initialization.

public class MyApiContext : SiteContext
{
        public override async Task InitializeAsync(PermissionsContext permissionsContext = null)
        {
            await base.InitializeAsync();

            // Creates a test scope
            if (ShellHost.TryGetSettings(TenantName, out ShellSettings shellSettings))
            {
                using (ShellScope shellScope = await ShellHost.GetScopeAsync(shellSettings))
                {
                    await shellScope.UsingAsync(async scope =>
                    {
                        // Dependencies
                        IRecipeExecutor recipeExecutor = scope.ServiceProvider.GetRequiredService<IRecipeExecutor>();
                        IEnumerable<IRecipeHarvester> recipeHarvesters = scope.ServiceProvider.GetRequiredService<IEnumerable<IRecipeHarvester>>();
                        UserManager<IUser> userManager = scope.ServiceProvider.GetRequiredService<UserManager<IUser>>();

                        IEnumerable<RecipeDescriptor>[] recipeCollections = await Task.WhenAll(recipeHarvesters.Select(x => x.HarvestRecipesAsync()));
                        IEnumerable<RecipeDescriptor> recipes = recipeCollections.SelectMany(x => x);
                        RecipeDescriptor recipe = recipes.FirstOrDefault(c => c.RecipeFileInfo.Name == "AddTestInfo.recipe.json");
                        string executionId = Guid.NewGuid().ToString("n");
                        string result = await recipeExecutor.ExecuteAsync(executionId, recipe, new { }, CancellationToken.None);

                        // Create Test User
                        User newUser = new User
                        {
                            Email = "[email protected]",
                            UserName = "TestUserAdmin",
                            EmailConfirmed = true,
                            RoleNames = new List<string>() { "Administrator" }
                        };

                        IdentityResult userResult = await userManager.CreateAsync(newUser);

                        // Execute the recipe
                        recipe = recipes.FirstOrDefault(c => c.RecipeFileInfo.Name == "AddTestArticle.recipe.json");
                        executionId = Guid.NewGuid().ToString("n");
                        result = await recipeExecutor.ExecuteAsync(executionId, recipe, new { }, CancellationToken.None);
                    });
                }
            }
        }
    }
}

2. Removing boilerplate
   As seen here, if you want to run a test, you need all this code:

using (var context = new MyApiControllerContext())
{
    // Setup
    await context.InitializeAsync();

    // Test
    using (var shellScope = await MyApiControllerContext.ShellHost.GetScopeAsync(context.TenantName))
    {
        await shellScope.UsingAsync(async scope =>
        {
            // Assert
        });
    }
}

With this generic function, we can simplify all that to just one line:

internal static async Task ExecuteTest<T>(Func<ShellScope, Task> task) where T : SiteContext, new()
{
    using (T context = new T())
    {
        await context.InitializeAsync();

        // Test
        if (ShellHost.TryGetSettings(context.TenantName, out ShellSettings shellSettings))
        {
            using (ShellScope shellScope = await ShellHost.GetScopeAsync(shellSettings))
            {
                await shellScope.UsingAsync(task);
            }
        }
    }
}

The result:

await SiteContext.ExecuteTest<MyApiContext>(async shellScope =>
{
  // Asserts
}

3. The tests are all independent of each other, an OrchardCore instance is initialized for each Test. I was afraid this would affect run time and they would take too long. That's not the case, I found each test averages 0.5 seconds on an Azure Devops Pipeline.

[Piedone]

Thank you for sharing your findings!

Just a nitpick: These tests are not really unit tests but rather integration tests. For a unit test (i.e. a test that only checks a given method) you don't need to initialize an Orchard shell.

[mariojsnunes]

Yeah thanks for that, fixed :)

[RockNHawk]

I have a similar note: WebApplicationFactory can only test by http method, could not direct call Application's component, like do some Session.Query() then direct call some method to do a complex test.
How to run test in Application instance context in Orchard Core ?
#6835